dlt
dlt is an open-source library that enables you to create a data pipeline in a Python script, replacing your glue code with robust, declarative 1-liners with schema evolution. dlt is the first Python library in this space, enabling you to run pipelines anywhere python runs - no redundant orchestrators, etc.
- Can be run in unprecedented places like Airflow, Cloud Functions, GitHub Actions, and more.
- Removes complexity by integrating with your existing stack.
- Empowers data teams and collaboration by allowing you to discover or prototype in notebooks, run in cloud functions, and deploy to production.
- Rapid data exploration and prototyping with DuckDB.
- No vendor limits, dlt is forever free.
- As a relatively new tool, the community is still growing and there may be fewer learning resources available compared to more established tools.