FAST-HEP keeps capabilities separate from the workflow engine.
Registries describe the capabilities available to Flow and how they are resolved.
Profiles compose registries, compiler extensions, and runtime configuration into reusable environments.
flowchart LR
Packages["<b>Packages</b><br/>provide capabilities"]:::capability
Registry["<b>Registries</b><br/>describe + resolve"]:::flow
Profile["<b>Profiles</b><br/>compose environment"]:::input
Flow["<b>Flow</b><br/>compile + execute"]:::runtime
Packages --> Registry --> Profile --> Flow
classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
A workflow selects an environment through this mechanism without needing to know which packages provide each individual capability.
Together, registries and profiles provide useful defaults while keeping capabilities replaceable and independently extensible.
A registry is a catalogue of named capabilities available to Flow. Registry entries provide the information needed to resolve those capabilities during compilation and execution.
The Carpenter registry, for example, provides capabilities including data sources, transforms, sinks, execution modifiers, product handling, and expression functions.
For operations, a registry associates a workflow-visible name with its specification and runtime implementation:
flowchart LR
Name["<b>hep.define</b><br/>workflow-visible name"]:::transform
Spec["<b>Spec</b><br/>compile-time contract"]:::flow
Impl["<b>Implementation</b><br/>runtime behaviour"]:::runtime
Name --> Spec
Name --> Impl
classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
The real Carpenter registry contains entries such as hep.define, hep.hist, and hep.di_object_mass, alongside capabilities for ROOT I/O, GPU execution modifiers, product handling, and expression functions.
This is how Flow can orchestrate capabilities supplied by other packages while remaining independent of their internal implementation.
Profiles provide reusable combinations of capabilities and configuration.
For example, the
hep
profile combines the capabilities needed for a typical HEP analysis, while the hep_debug profile builds on it by adding runtime diagnostics.
Profiles can include other profiles, allowing environments to be built through composition:
flowchart TD
Basic["<b>basic</b><br/>Flow environment"]:::flow
HEP["<b>hep</b><br/>HEP capabilities"]:::capability
Debug["<b>hep_debug</b><br/>runtime diagnostics"]:::runtime
Experiment["<b>experiment</b><br/>experiment capabilities"]:::capability
Analysis["<b>analysis</b><br/>analysis-specific capabilities"]:::capability
Basic --> HEP
HEP --> Debug
HEP --> Experiment
Experiment --> Analysis
classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
An analysis can therefore select a useful environment without needing to know which individual packages and registries provide every capability.
The same composition mechanism is available to experiments and individual analyses. They can add new capabilities or replace existing implementations while continuing to build on the standard FAST-HEP environment.
This allows customisation through composition rather than by modifying Flow or FAST-HEP packages directly.
Provenance and reproducibility: FAST-HEP is being extended to record resolved capabilities, concrete implementations, and provider package versions in the workflow provenance. This will record not only what an analysis requested, but which software components actually executed it.
This support is currently under development.
Registries and profiles are not limited to packages maintained by FAST-HEP.
Experiments, projects, and individual analyses can provide their own capabilities using the same interfaces as the standard toolkit.
For example, an extension package might provide:
Those capabilities can then be composed with existing FAST-HEP profiles.
flowchart LR
Flow["<b>Flow</b><br/>core environment"]:::flow
HEP["<b>FAST-HEP</b><br/>HEP capabilities"]:::capability
Experiment["<b>Experiment</b><br/>custom capabilities"]:::capability
Analysis["<b>Analysis</b><br/>local capabilities"]:::capability
Flow -->|compose| HEP
HEP -->|extend| Experiment
Experiment -->|extend| Analysis
classDef input fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
classDef flow fill:#e8f0fe,stroke:#3c6fbd,stroke-width:2px,color:#111111;
classDef source fill:#e7f5ff,stroke:#1c7ed6,stroke-width:2px,color:#111111;
classDef transform fill:#fff9db,stroke:#f08c00,stroke-width:2px,color:#111111;
classDef sink fill:#ebfbee,stroke:#2f9e44,stroke-width:2px,color:#111111;
classDef artifact fill:#eadcf8,stroke:#7950a3,stroke-width:2px,color:#111111;
classDef runtime fill:#fce8d5,stroke:#b56b22,stroke-width:2px,color:#111111;
classDef capability fill:#f7f7f7,stroke:#5f6368,stroke-width:2px,color:#111111;
Flow does not need to distinguish between a capability supplied by FAST-HEP and one supplied by an external package. What matters is that it satisfies the relevant contract.
Profiles and registries make extensibility part of the FAST-HEP architecture rather than an additional plugin mechanism layered on top of it.
Flow provides the compilation and runtime machinery, while the capabilities that interact with scientific data can come from FAST-HEP, experiments, external projects, or individual analyses.
Combined with the operation contracts described in Operations and specs, this allows implementations to evolve or be replaced without requiring the workflow engine itself to understand their internal details.
This page describes the role of profiles and registries in the FAST-HEP architecture.
For the current profile format, registry schema, composition rules, resource discovery, overrides, and extension interfaces, see the fasthep-flow documentation.