Pipelines

Pipelines is a core module of the Code Ocean computational science platform, designed to help life sciences teams connect, automate, parallelize, and scale computational workflows in the cloud. It is built for computational scientists, bioinformaticians, and R&D teams who need a flexible, reproducible, and scalable environment for running complex multi-step analyses — without requiring extensive DevOps expertise.

Pipelines natively integrates Nextflow, supporting the latest DSL2 while maintaining compatibility with DSL1 for existing workflows. Users can build pipelines through a visual editor, import community-curated pipelines from nf-core, bring in pipelines from a Git repository, or write fully custom Nextflow scripts — making it accessible to both no-code users and advanced developers alike.

Key Capabilities

• Visual pipeline builder with Nextflow auto-generation: Build pipelines using a drag-and-drop visual editor while Code Ocean automatically writes and updates the underlying Nextflow script in real time. Advanced users can unlock the script and write their own custom Nextflow code, leveraging the full range of Nextflow capabilities.
• Import from nf-core: Import ready-made, community-created bioinformatics pipelines directly from nf-core with a single click. No additional setup or configuration is required — simply define your parameters and run in the cloud.
• Import from Git repositories: Bring existing pipelines into Code Ocean by importing from a Git repository or uploading pipeline files, enabling teams to quickly migrate workflows they already use.
• Per-step compute resource configuration: Define CPU, GPU, and RAM requirements individually for each step of the pipeline, allowing fine-grained control over resource allocation based on computational needs or cost constraints.
• Parallelization and scaling with AWS Batch: Code Ocean comes with a fully configured AWS Batch instance out-of-the-box. Pipelines automatically run on AWS Batch to parallelize work, and EBS autoscaling dynamically adjusts storage capacity up and down as the pipeline executes — with zero setup required.
• Real-time monitoring dashboard: A dedicated monitoring interface provides run status, resource summaries, process overviews, task-level details, logs, and file previews. This enables faster debugging and helps ensure pipeline runs remain cost-efficient.
• Collaboration and sharing: Share pipelines with individual users or groups within your organization using built-in user permissions. Teams can collaborate on a pipeline directly, or clone and adapt it for their own needs, with the option to share all associated assets.

How Pipelines Integrate with the Code Ocean Platform

• Built from Compute Capsules: Pipelines are assembled by connecting Compute Capsules — self-contained units that encapsulate code, data, environment, and results — making it straightforward to compose powerful bioinformatics workflows from reusable components.
• Deployable as no-code Apps: Pipelines can be deployed as parameterized Apps, giving non-coding team members and bench scientists access to powerful analyses without requiring any scripting knowledge.
• Lineage Graph integration: Every result produced by a Pipeline includes an automated, immutable record of its provenance. The Lineage Graph provides a visual representation of every Capsule, Pipeline, and Data asset involved in a computation, supporting full reproducibility and traceability.

Broader Platform Use Cases

• Bioinformatics pipelines: Build, configure, and monitor pipelines from scratch or import from nf-core for instant access to best-practice analysis workflows. Automatic scaling on AWS Batch requires no infrastructure setup.
• Multiomics analysis: Work with large multimodal datasets using scalable compute and storage, cached R and Python packages, and preloaded multiomics software with full lineage tracking.
• ML model development: Code Ocean supports AI, ML, deep learning, and generative AI workflows, with GPU-ready environments, MLflow integration for model tracking and lifecycle management, and built-in reproducibility.
• Imaging workflows: Process image data ranging from individual files to petabyte-scale datasets using dedicated desktop applications or custom deep learning pipelines, always with lineage and without requiring DevOps.
• Data management: Manage organizational data with FAIR-principles-compliant tooling, custom metadata, and controlled vocabularies to ensure consistency and searchability.
• Data and model provenance: Automated lineage graph generation tracks all data and results, providing a visual audit trail for every computation.

Pipelines runs on Code Ocean's cloud infrastructure with AWS Batch and EBS autoscaling built in, requiring no additional configuration. It is designed to serve teams across computational science, IT, and R&D leadership, and integrates seamlessly with the broader Code Ocean platform including Capsules, Apps, Data, Models, and the Lineage Graph.