Filter the results Item type Select All/None Link Event Project EasyForm Page Blog Post Folder Snazzy Home Page News Item Collection File New items since Yesterday Last week Last month Ever 3 items matching your search terms. Sort by relevance date (newest first) alphabetically Too Big for DAG Factories? As your infrastructure scales up, how you go about managing all DAGs in Airflow becomes very important. One method would be to create a “DAG factory,” which can churn out thousands of DAGs dynamically from a single configuration file. Located in Blog Improving Big Data: A Guide to Enhanced Pipelines Seeking to improve your big data pipeline? This blog walks you through enhancements made to a client's system using Airflow Datasets, DAG Dependencies, Azure Durable Functions, and edge cases. Learn how we added functionality and flexibility by streamlining data integration, minimizing cost increases, and creating a scalable pipeline development process. Located in Blog Data Warehousing in Microsoft Azure Data Warehousing is the process of storing and processing large amounts of data from many disparate sources. Located in Blog