Pipeline Runtime
Pipeline runtime settings control how Nextflow workflows are executed — locally or on a SLURM cluster — and which tools are available.
Configuration Location
Pipeline runtime settings are at Admin → Pipeline Runtime
(/admin/pipeline-runtime), or via the config file / environment variables.
Execution Mode
| Mode | Description | Best For |
|---|---|---|
local | Run Nextflow directly on the server | Testing, small datasets |
slurm | Submit jobs to a SLURM cluster | Production, large datasets |
Local Execution
Nextflow runs as a child process on the SeqDesk server. All compute happens on the same machine.
SLURM Execution
Nextflow submits individual processes as SLURM jobs. Configure:
| Setting | Env Variable | Default | Description |
|---|---|---|---|
| SLURM Enabled | SEQDESK_SLURM_ENABLED | false | Enable SLURM |
| Queue | SEQDESK_SLURM_QUEUE | default | Partition name |
| Cores | SEQDESK_SLURM_CORES | 4 | CPUs per job |
| Memory | SEQDESK_SLURM_MEMORY | 16GB | Memory per job |
| Time Limit | SEQDESK_SLURM_TIME | 24 | Hours per job |
| Additional Options | — | — | Extra SLURM flags |
Conda Configuration
Nextflow pipelines use Conda for dependency management:
| Setting | Env Variable | Default | Description |
|---|---|---|---|
| Conda Enabled | SEQDESK_CONDA_ENABLED | false | Use Conda |
| Conda Path | SEQDESK_CONDA_PATH | /opt/conda | Installation path |
| Conda Environment | SEQDESK_CONDA_ENV | — | Environment name |
The Conda environment must have Nextflow installed or be configured to find it.
Nextflow Settings
Additional Nextflow configuration available in the admin UI:
- Nextflow path — path to the Nextflow binary
- Nextflow profile — profile name to use (e.g.,
conda,docker) - Database download paths — paths for reference databases (GTDB-Tk, etc.)
Weblog Setup
Real-time pipeline monitoring requires configuring the Nextflow weblog:
| Setting | Description |
|---|---|
| Weblog URL | Your SeqDesk URL + /api/pipelines/weblog |
| Weblog Secret | Optional authentication token |
The weblog URL is automatically included in the Nextflow execution command. See Monitoring & DAG Visualization for details.
Pipeline Run Directory
| Setting | Env Variable | Default | Description |
|---|---|---|---|
| Run Directory | SEQDESK_PIPELINE_RUN_DIR | ./pipeline_runs | Where run outputs are stored |
Each pipeline run creates a subdirectory (e.g., MAG-20240126-001/) containing
the samplesheet, scripts, logs, and outputs.
Infrastructure Validation
The Admin → Infrastructure page (/admin/data-compute) validates that your
runtime environment is correctly configured before you run any pipelines. It
tests four areas:
| Check | What It Tests |
|---|---|
| Data Path | The sequencing data directory exists and is accessible |
| Run Directory | The pipeline run directory exists and is writable |
| Conda | Conda is installed and the configured environment is available |
| Weblog | The Nextflow weblog endpoint is reachable |
Each check shows a pass/fail indicator. Fix any failures before attempting to run a pipeline.
Import Setup JSON
The infrastructure page also supports importing a complete infrastructure configuration as JSON. You can:
- Upload a JSON file or paste a configuration
- Load an example to see the expected format
- Validate (dry-run) before saving
- Save to apply the configuration
This is useful for automating setup or copying configurations between instances.
The JSON supports both flat keys (e.g., dataBasePath) and nested keys
(e.g., site.dataBasePath), as well as alternate names for flexibility.