Parsl Configuration¶
TaPS provides (as of writing) two different executor configuration types for using Parsl: parsl-local and parsl-htex.
parsl-local Config¶
The parsl-local executor is the easiest way to run a benchmark using Parsl as the executor.
The parsl-local executor corresponds to the ParslLocalConfig which creates a HighThroughputExecutor configured on the local node.
For example, configuring Parsl on the local node with eight workers is easy.
python -m taps.run \
--app cholesky --app.matrix_size 100 --app.block_size 25 \
--engine.executor parsl-local --engine.executor.workers 8
parsl-htex Config¶
More advanced Parsl configurations will need to use the parsl-htex executor which corresponds to the ParslHTExConfig.
The parsl-htex executor still creates a HighThroughputExecutor, but the full configuration options are exposed (e.g., addresses, providers, launchers, etc.).
Tip
Due to the complexity of configuring the parsl-htex, a TOML configuration file should be used.
Not all Parsl configuration options will be available in the CLI parser arguments.
The following is a simple configuration that mostly defaults to the defaults set in Parsl's HighThroughputExecutor, except for max_workers_per_node and address which are specified.
| parsl-config.toml | |
|---|---|
python -m taps.run \
--app cholesky --app.matrix_size 100 --app.block_size 25 \
--config parsl-config.toml
Extra options not explicitly defined in the various sub-configs of ParslHTExConfig can still be provided and will be passed as keyword arguments to the corresponding Parsl classes.
These configuration semantics are similar to the GlobusComputeEngine which wraps Parl's HighThroughputExecutor.
Examples¶
Polaris at ALCF¶
The following configuration is an example for the Polaris GPU cluster at ALCF. The configuration is based on the Globus Compute endpoint example.