[Feature] Static DAG Code Generation from YAML Configs

[TheLazzziest]

Description

Would dag-factory support a CLI feature to generate static Python DAG files from YAML configs?

dagfactory render airflow_manifest.yaml templates/dbt_dag.py.jinja --output dags/generated_dag.py --namespace ns --var ...

Use case/motivation

Currently, there is a project with multiple analytical teams running dbt with Airflow, but most aren't Airflow/Python experts. They're copying nearly identical .py DAG files across repos via CICD, which creates maintenance nightmares when updates are needed. Currently, there are the following issues with this approach:

• High maintenance burden: Any DAG upgrade or fix must be applied to dozens of files
• Inconsistency risk: Code copies drift over time, making troubleshooting difficult
• Accessibility barrier: Analytical teams ask us to modify DAGs (only a few teams know how to work with Airflow)
• Auditability concerns: DAG logic is scattered across repos which is not easy to inspect when you plan a major upgrade

Why

• Centrally maintain DAG logic (one YAML, many DAGs)
• Reduce code duplication and drift
• Empower non-Python teams to manage DAGs via config
• Inspect/version-control the actual generated code

Related issues

No response

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!

[gyli]

I think DAG factory itself is already the solution to resolve your problems, by switching DAGs in py files to YAML files. Instead of creating DAG py files based on the YAML, the YAML config is already a proper DSL, and can serve as the source of truth for your DAGs. Is there any other reason you want to generate and manage the py file, instead of managing the YAML?

[Karpisek]

@gyli one use-case I can think of is to see the code within the Airflow UI for a given DAG

[gyli]

Hi @Karpisek , YAML content is already available in Airflow UI through "DAG Doc". dag-factory adds the YAML content as doc_md internally.