JSON config runtime cleanup and docs pruning
- Migrate installer/update/release scripts to use runtime keys in seqdesk.config.json.
- Remove .env.example and clean README/install references to environment-file setup.
- Prune public docs and internal artifacts from this repository.
- Include test and pipeline helper cleanups from recent commits.
Configurable sequencing-tech sync and test hardening
- Admin sequencing tech settings now include a configurable registry sync URL used for reset and update checks.
- Sequencing-tech API now validates and persists sync URLs, tracks last sync time, and returns clearer update status messages.
- Added ENA API client tests for accession extraction and authentication error mapping.
- Expanded tests for pipeline manifest/package loading, generic adapter/executor paths, and parser/runtime validation.
UX refresh for studies, files, and admin forms
- Redesigned the Manage Files workflow with tabbed navigation and improved order file handling.
- Refined study and order detail pages, including updated overview/status presentation and editing flows.
- Polished dashboard surfaces (analysis, help, messages, settings) and sidebar navigation.
- Updated admin form builder, module management, sequencing technology, and study-form builder interfaces.
Installer hardening and private pipeline setup
- Add admin workflow to install private pipeline packages from Settings > Pipelines.
- Support hosted private MetaxPath package metadata while excluding private pipelines from public release tarballs.
- Installer no longer auto-runs sudo system package installs and now prints manual dependency commands.
- Ensure distribution installs have runtime Node modules, preventing next command-not-found startup failures.
- Reconfigure now skips DB push/seed by default and supports explicit reseed via --reseed-db.
- Respect configured condaPath for Miniconda install target and improve conda init shell guidance.
Study notes reliability and API validation
- Prevent unsaved notes edits from being overwritten when study data refreshes.
- Validate study update request body types and return clear 400 errors for invalid payloads.
- Improve title/description/alias normalization to avoid unsafe trim operations on non-string inputs.
Installer automation and workflow updates
- Added non-interactive installer support with --config and -y, plus reconfigure mode for in-place updates.
- Installer now syncs runtime settings and optional secrets from config/.env with safer automation defaults.
- Improved sequencing workflow UX with barcode-aware Excel import helpers and validation tooling.
- Refined dashboard and admin pages for orders, studies, and settings workflows.
SubMG coverage from metadata
- Read assembly coverage from sample/order custom fields (coverage_depth and related keys) for SubMG YAML generation.
- Use pipeline config assemblyCoverageValue/coverageValue when metadata is absent, and only then fall back to 1.
SubMG assembly coverage field fix
- Add required ASSEMBLY COVERAGE_VALUE to SubMG config YAML (default 1, configurable).
- Keep SubMG read submission flow intact while unblocking assembly step validation.
SubMG ENA compatibility fixes
- Normalize ONT platform values (including MinION/GridION/PromethION) to ENA-valid OXFORD_NANOPORE in SubMG manifests.
- Add FASTQ file path fallback between .fastq/.fastq.gz and .fq/.fq.gz during SubMG run preparation.
Fix SubMG read manifest insert size
- Write a positive INSERT_SIZE in PAIRED_END_READS manifests (default 300, configurable via pipeline config insertSize).
- Stop defaulting to --skip_checks so SubMG preflight validation runs unless explicitly disabled.
Fix SubMG metadata false-missing when MIxS keys are prefixed
- Accept legacy _mixs_ prefixed checklist keys when validating SubMG required metadata.
- Write MIxS per-sample metadata using canonical checklist keys (without _mixs_ prefix) on study create/edit save.
Show exact missing SubMG metadata fields on pipeline card
- Include missing SubMG metadata field names in the card-level Missing summary.
- Include field-level details in the card check details line for Sample metadata.
SubMG metadata visibility and edit-study save warnings
- Show exact missing SubMG sample metadata fields in pipeline validation summaries and per-sample rows.
- Add warning dialog on Edit Study Review and Save for missing required sample fields, with Save Anyway option.
SubMG checksum and ENA target improvements
- Show ENA target server (test vs production) on SubMG cards and in the run dialog.
- Add Compute and add read checksums action for missing read MD5 values, including batch processing and status feedback.
- Prevent preferred assembly UI rollback when the save succeeded but metadata refresh fails.
- Keep checksum retry path available by applying local checksum overrides only after successful SubMG validation refresh.
Assemblies view and metadata validation
- Add Assemblies page/API and optional user assembly download access control
- Validate selected-sample metadata server-side before creating/starting pipeline runs
- Support sequencing-technology-derived platform resolution and MAG run-target restrictions
Study page not found fix
- Fix study detail lookup when URLs carry stale order-linked identifiers.
- Normalize study detail/edit/metadata pages to canonical study IDs.
Order detail load fix
- Refactors /api/orders/[id] to use explicit, resilient relation loading.
- Prevents order page hard failures when some linked records are missing or out of sync.
Simulation & analysis observability
- Add raw weblog events API endpoint for pipeline runs and link it in Analysis run details.
- Improve sequencing-file simulation routes and configurable template source handling.
- Add order file inspection API and strengthen file discovery/checksum workflows.
- Refresh admin settings and dashboard sections for clearer pipeline/file operations.
MAG stability + run UI
- Fix MAG CONCOCT env by pinning compatible Python and setuptools for CONCOCT steps
- Redesign analysis run page with Live Activity/Files/Details/Health tabs
- Populate Pipeline Files size data from run artifacts and filesystem
Pipeline reliability and SubMG clarity
- Improved large DB download retry/resume behavior and partial-transfer handling
- Prevented false DB-ready status when only partial files exist
- Added manual terminal fallback commands for DB downloads in admin pipeline settings
- Added per-pipeline missing-input summaries on study pipeline cards
- Added SubMG submission scope panel with include/exclude guidance and sample-level missing input details
Pipeline DB downloads + better MAG dummy reads
- Add Download DB action for MAG pipeline with progress/status in admin pipeline settings.
- Persist downloaded GTDB DB path into pipeline config automatically.
- Improve simulated FASTQ generation to reference-based paired-end reads for more reliable MAG assembly tests.
- Bump release to v1.1.41.
Pipeline runtime stability fixes
- Force Nextflow conda creation to use --override-channels with conda-forge and bioconda (no defaults).
- Generate unique Nextflow run names per run to avoid duplicate -name collisions.
- Record weblog event timestamps at receipt time to keep Event Feed relative times accurate.
- Keep SLURM launcher honoring configured queue and slurmOptions.
SLURM submission reliability
- Fix SLURM run.sh generation so #SBATCH directives are emitted before shell commands.
- Pass configured queue and slurmOptions (e.g. --qos=broker) directly to sbatch at submit time.
- Keep Nextflow SLURM process config enabled with executor, cpus, memory, queue, time, and clusterOptions from settings.
SLURM executor defaults
- Set process.executor, cpus, memory, queue, time, and clusterOptions from app SLURM settings.
- Accept clusterOptions alias in infrastructure import JSON.
- Update infrastructure setup JSON examples with SLURM fields and qos option.
Runtime and MAG Launch Fixes
- Prefer seqdesk-pipelines env during conda auto-detect to avoid defaulting to base.
- Use runtime platform detection from conda and avoid false macOS ARM runtime checks.
- Fix MAG flag generation to skip empty path values (prevents --gtdb_db resolving to 'true').
Updater DB preservation fix
- Fix updater SQLite path resolution to match Prisma (prisma/dev.db).
- Preserve DB backup/restore paths to avoid filename collisions.
- Improve restart fallback behavior for supervised deployments.
- Show auto-seed/setup errors directly in setup status UI.
Installer seeding reliability fix
- Installer no longer prompts for data/run paths during minimal setup.
- NEXTAUTH_URL is auto-derived from selected port.
- Seed scripts no longer require runtime bcryptjs module resolution.
- Release build now includes bcryptjs in packaged node_modules.
Auto-seed database on first launch
- Auto-seed database within the app on first launch when tables exist but data is missing.
- Install scripts gracefully handle seed failures with fallback chain.
- Setup page auto-detects seeding progress and redirects to login when ready.
Fix installer seeding with Prisma 7 environments
- Pin installer Prisma CLI usage for db push and db seed to project Prisma version.
- Ensure install-dist performs seeding reliably during installation.
- Update setup page instructions (remove legacy v2 path and use correct seed command).
Fix installer login/setup loop
- Fix install-dist: run prisma db seed during installation.
- Make /setup and /api/setup/status dynamic/no-cache to avoid stale redirect behavior.
- Use no-store setup status checks on login/home.
- Default pipeline support prompt to yes in install scripts.
SLURM Bootstrap Fixes & Debug Session Info
- Harden conda/nextflow bootstrap in generated SLURM and local run scripts.
- Add fallback to conda run for nextflow when PATH lookup fails.
- Add analysis Run Health panel with failure-first diagnostics.
- Add Copy session info action with plain-text debug bundle for support.
- Always load logs for failed and non-running runs in Live Log Viewer.
Admin UX Consistency & Workflow Updates
- Redesigned admin settings and infrastructure pages with consistent layout and clearer actions.
- Improved form-builder and study-form-builder usability with stronger save/test flows.
- Refined modules configuration to better reflect field enablement behavior.
- Enhanced ENA configuration UX, validation, and submission-history robustness.
- Aligned key dashboard/admin views for better visual consistency and mobile behavior.
Pipeline Execution Fixes & Analysis UX
- Fix SLURM scripts not redirecting output to monitored log files.
- Fix SLURM log file detection using wrong directory path.
- Fix incorrect status comparison in pipeline monitor.
- Fix run number generation NaN risk from malformed data.
- Fix output resolver masking errors as success.
- Fix always-true status note condition in order updates.
- Guard against undefined runId in pipeline start flow.
- Add try-catch for JSON.parse preventing page crash on malformed data.
- Remove duplicate sync intervals on analysis detail page.
- Show Analysis page to researchers (not just admins).
- Improve analysis empty state with guidance and link to studies.
Fix SLURM Log Directives
- Fix SLURM script log output/error path directives.
Pipeline Run Directory Validation
- Prevent root (/) as pipeline run directory.
- Add validation at save and start time.
- Clear error message for misconfigured settings.
SLURM and Null-Safety Fixes
- Allow pipeline start when conda not found locally (SLURM provides it).
- Use sbatch --parsable for reliable job ID parsing.
- Handle missing user/sample data gracefully on analysis pages.
Better SLURM Diagnostics
- Capture and display SLURM stderr output when sbatch fails.
- Verify sbatch command exists before attempting submission.
- Verify run script exists before submission.
Analysis Page Fix
- Fix crash when viewing analysis runs with incomplete data.
- Add null safety for sample reads and pipeline steps.
Safe Updates (Data Preserved)
- CRITICAL: Remove dev.db from release tarball to prevent data loss on update.
- Added backup/restore of database during update process.
- Fix client-side crash when viewing studies with samples that have no reads.
Update Safety + Study Page Fix
- Robust database backup and restore during self-update to prevent data loss.
- Fix client-side crash when viewing studies with samples that have no reads.
Flexible Study Creation
- Per-sample required fields now show warnings instead of blocking.
- Added 'Create Anyway' option when required fields are missing.
- Blur active inputs before validation to commit pending values.
Safer Updates
- Strip database files from update tarball to prevent data loss.
- Load DATABASE_URL from .env for custom SQLite paths.
- Skip migrations by default for SQLite unless SEQDESK_AUTO_MIGRATE=1.
File Management + Report Preview
- Add bulk delete for files page with confirmation dialog.
- Add HTML report preview for pipeline output files.
- Improve simulated reads: realistic sequences, Illumina headers, varied quality scores.
- Fix BigInt serialization crash on analysis detail page.
- Fix stale 'Last event' timestamps for queued and pending runs.
- Add Read Files tab to study detail page.
- Refactor pipeline section with card layout and history table.
- Order Progress step 2 reflects actual file state.
Queue Tracking + Event Feed
- Persist queue status + reason for SLURM and local runs.
- Record weblog events and show an event feed in Analysis.
- Track last weblog/trace timestamps in run details.
- Detect SLURM log files and surface them in Pipeline Files.
Pipeline Run Tracking
- Improved pipeline run status tracking and log previews.
SLURM Test Button
- Added a SLURM test button in Data & Compute.
Pipeline Run Visibility
- Record last pipeline events and status sources.
- Sync trace data to finalize running jobs.
- Live refresh + clearer running state in analysis views.
Pipeline code downloads
- Add background Nextflow pull with status tracking
- Show code cache status and download actions in pipeline settings
- Expose pipeline download state via admin API
Installer Env Fallback
- Ship .env.example in release tarball
- Installer creates minimal .env when missing
Update Progress & Restart Detection
- Persist update progress and show live status in Admin Settings
- Detect installed vs running version with restart pending state
- Show app version in the sidebar
Installer and Packaging Fixes
- Include Prisma client generator in release package
- Prompt for app port during install
- Start script respects PORT or .env
- Do not ship .env or dev.db in release
Installer Improvements
- Added preflight summary and configuration review
- Optional Node.js auto-install via nvm
- Cleaner error reporting and install summary
- Include Prisma client generator in release package
- Prompt for app port during install
- Start script respects PORT or .env
- Do not ship .env or dev.db in release
Install Flow
- Added interactive install wizard for curl-based installs
- Pipeline execution settings moved to Admin > Settings > Compute
- Simplified pipeline store UI with installed-first layout
Modern UI Design
- Modernized UI with warm neutral color palette
- Redesigned sidebar with improved navigation
- Inline field validation with connected error display
- Studies page redesigned to match Orders with search/filters
- Field Help panel with AI-like shimmer effect
- Full-width pages throughout the app
- Generic manifest-driven pipeline execution system
Pipeline Package System
- Added pipeline package system with store functionality
- Install pipelines directly from Admin Settings
- MAG pipeline available as first package
Update System and Admin Improvements
- Added disk space check before updates (requires 150MB free)
- Added update banner in dashboard for facility admins
- Added rollback documentation for manual recovery
- Interactive port selection when default port is in use
- Improved update progress indicators
Self-Update System
- Added self-update functionality in Admin Settings
- Version checking against seqdesk.com
- Automatic download and installation of updates
- Database backup before updates
Initial Release
- Order management with status tracking
- Sample metadata with MIxS compliance
- nf-core pipeline integration
- ENA submission support
- Multi-user authentication with roles