Created:        2023-05-10 Wed
Last modified:  2023-06-05 Mon

OpenLineage / Marquez

Marquez [1] [2] is an open source implementation of the OpenLineage [3] .

  • Event specification [4]

  • Integrations (automatic collection of metadata) from Spark, Airflow, dbt [5]

  • Visualization [6]

  • Metadata API [7]

  • Column-level lineage support (so far only Spark) [8]

  • AWS Big Data Blog: Automate data lineage on Amazon MWAA with OpenLineage [9]

References