mdd
DocsUse CasesGitHubOpen App

Pipeline lineage a human can actually read

Auto-generated dbt and Airflow lineage graphs are unreadable hairballs; the diagram that would actually help is the curated one nobody maintains. Data and analytics teams keep a small lineage summary in the repo, and an agent keeps it current as models change — mdd gives that summary a JSON model to edit and ASCII output that renders in every code review.

A curated dbt lineage map

Sources to staging to marts to the dashboard that matters, as spliced into a dbt repo's README. The JSON is what your agent edits when a new model lands.

<!-- mdd:begin id=lineage src=docs/lineage.diagram.json -->
```

 ┌─────────────────┐          ┌─────────────────┐
 │ stripe (source) │          │ app_db (source) │
 └────────┬────────┘          └────────┬────────┘
          │                  ┌─────────┴────────┐
          │                  │                  │
  ┌───────▼──────┐   ┌───────▼───────┐   ┌──────▼─────┐
  │ stg_payments │   │ stg_customers │   │ stg_orders │
  └───────┬──────┘   └───────┬───────┘   └──────┬─────┘
          │                 ┌┘                  │
          ├─────────────────┼───────────────────┘
   ┌──────▼─────┐   ┌───────▼───────┐
   │ fct_orders │   │ dim_customers │
   └──────┬─────┘   └───────┬───────┘
          └────────┐        │
                   ├────────┘
         ┌─────────▼─────────┐
         │ Revenue dashboard │
         └───────────────────┘

```
[diagram source](docs/lineage.diagram.json)
<!-- mdd:end -->

The model your agent edits

{
  "version": 1,
  "boxes": [
    {
      "id": "stripe",
      "label": "stripe (source)",
      "row": 0,
      "connections": [
        {
          "to": "stg_payments",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "app_db",
      "label": "app_db (source)",
      "row": 0,
      "connections": [
        {
          "to": "stg_customers",
          "arrow": "to"
        },
        {
          "to": "stg_orders",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "stg_payments",
      "label": "stg_payments",
      "row": 1,
      "connections": [
        {
          "to": "fct_orders",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "stg_customers",
      "label": "stg_customers",
      "row": 1,
      "connections": [
        {
          "to": "dim_customers",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "stg_orders",
      "label": "stg_orders",
      "row": 1,
      "connections": [
        {
          "to": "fct_orders",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "fct_orders",
      "label": "fct_orders",
      "row": 2,
      "connections": [
        {
          "to": "revenue",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "dim_customers",
      "label": "dim_customers",
      "row": 2,
      "connections": [
        {
          "to": "revenue",
          "arrow": "to"
        }
      ]
    },
    {
      "id": "revenue",
      "label": "Revenue dashboard",
      "row": 3
    }
  ]
}

Your agent draws it, you review it

  1. Curate the lineage that matters — sources, staging, marts, the dashboards they feed — as a small JSON model. Your agent can draft it from the dbt DAG.
  2. Splice it into the repo README with mdd render.
  3. When a model is added or rewired, the agent edits the JSON and re-renders — the diagram diff shows exactly what changed in the pipeline.
  4. mdd check in CI keeps the map from silently drifting out of date.

A prompt to hand your agent

Keep the lineage map in README.md current. When dbt models change, update docs/lineage.diagram.json to match the DAG (curated: only sources, staging, marts, and dashboards), then run: npx mdd render --json docs/lineage.diagram.json --file README.md --id lineage.

Why mdd here

The value of a lineage summary is curation, and curation means maintenance — exactly what goes stale. mdd makes the curated map a machine-editable JSON model with a validation loop, so an agent can keep it truthful on every schema change, and the ASCII renders in the README, in diffs, and in the terminal where pipeline work happens.

More use cases