Skip to content

on_boarding.md

Latest commit

 

History

History
236 lines (196 loc) · 33.8 KB

on_boarding.md

File metadata and controls

236 lines (196 loc) · 33.8 KB
graph LR
    Unified_Execution_Core_Unifex_["Unified Execution Core (Unifex)"]
    Persistent_Task_Graph_Hortator_Core_["Persistent Task Graph (Hortator Core)"]
    Easy_API["Easy API"]
    Bioinformatics_Tools_Integration["Bioinformatics Tools Integration"]
    Core_Data_Structures["Core Data Structures"]
    External_Environment_Management["External Environment Management"]
    IPython_Visualization["IPython Visualization"]
    Unified_Execution_Core_Unifex_ -- "executes steps from" --> Persistent_Task_Graph_Hortator_Core_
    Unified_Execution_Core_Unifex_ -- "utilizes" --> Core_Data_Structures
    Persistent_Task_Graph_Hortator_Core_ -- "manages" --> Core_Data_Structures
    Persistent_Task_Graph_Hortator_Core_ -- "integrates" --> Bioinformatics_Tools_Integration
    Easy_API -- "builds upon" --> Persistent_Task_Graph_Hortator_Core_
    Easy_API -- "generates commands for" --> Unified_Execution_Core_Unifex_
    Bioinformatics_Tools_Integration -- "uses" --> External_Environment_Management
    IPython_Visualization -- "visualizes" --> Persistent_Task_Graph_Hortator_Core_
    IPython_Visualization -- "visualizes" --> Easy_API
    click Unified_Execution_Core_Unifex_ href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/Unified Execution Core (Unifex).md" "Details"
    click Persistent_Task_Graph_Hortator_Core_ href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/Persistent Task Graph (Hortator Core).md" "Details"
    click Easy_API href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/Easy API.md" "Details"
    click Bioinformatics_Tools_Integration href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/Bioinformatics Tools Integration.md" "Details"
    click Core_Data_Structures href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/Core Data Structures.md" "Details"
    click External_Environment_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/External Environment Management.md" "Details"
    click IPython_Visualization href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//railroadtracks/IPython Visualization.md" "Details"
Loading

CodeBoardingDemoContact

Component Details

The railroadtracks project provides a robust framework for defining, managing, and executing bioinformatics workflows. It leverages a persistent task graph to track dependencies and provenance, offers a simplified high-level API for user interaction, integrates with various external bioinformatics tools, and provides visualization capabilities for workflow monitoring. The core functionality revolves around the execution of defined steps, managed by a central execution core, with strong emphasis on persistent data management and environmental dependency handling.

Unified Execution Core (Unifex)

The central runtime component responsible for executing individual steps within a workflow. It handles argument parsing, logging, model instantiation, asset management (sources and targets), and delegates the actual execution to the specific model's run method. It acts as the bridge between the abstract task definitions and the concrete execution of bioinformatics tools.

Related Classes/Methods:

Persistent Task Graph (Hortator Core)

Manages the persistent storage, retrieval, and traversal of the task graph. It defines the relationships between steps, assets, and their provenance. This component is crucial for tracking the state of the workflow, identifying dependencies, and enabling features like caching and cleaning targets. It forms the backbone for workflow persistence and integrity.

Related Classes/Methods:

Easy API

Provides a high-level, simplified interface for users to define, manage, and execute bioinformatics projects and tasks. It abstracts away the complexities of the underlying Persistent Task Graph and Unified Execution Core, offering intuitive classes for tasks, task sets, and project views. This component is the primary user-facing interface for workflow definition and interaction.

Related Classes/Methods:

Bioinformatics Tools Integration

Encapsulates the functionalities for interacting with various external bioinformatics command-line tools. This includes aligners (BWA, Star), quantifiers (Sailfish), and file utilities (Samtools). It provides a standardized way to define and execute these tools within the workflow, often including version checking and command line generation.

Related Classes/Methods:

Core Data Structures

Defines the fundamental abstract base classes and data structures used throughout the system, such as SavedEntityAbstract, File, StepAbstract, and AssetSet. These provide a common, consistent foundation for representing data, steps, and assets across different components, ensuring interoperability and type safety.

Related Classes/Methods:

External Environment Management

Handles interactions with the external system environment, including checking for the presence and version of executable software (e.g., R, BWA, Star) and managing specific package versions required by various tools. This component ensures that the necessary external dependencies for bioinformatics tools are met before execution.

Related Classes/Methods:

IPython Visualization

Responsible for generating visual representations of project and task graphs, primarily for IPython environments. It converts internal graph structures into interactive HTML or SVG formats, enabling users to graphically view and understand the workflow, task dependencies, and project status.

Related Classes/Methods:

FAQ