Apache Airflow Bootcamp: Hands-On Workflow Automation

Source: https://www.udemy.com/course/apache-airflow-bootcamp-hands-on-workflow-automation/

What you’ll learn

• Understand what Apache Airflow is, its purpose and pros and cons of using Airflow
• Step-by-step guide to installing Airflow
• Launch and navigate the Airflow Web UI and learn about various views: DAG, Grid, Graph, Calendar, Task Duration, Code, Variable and Gantt View
• Understand what a DAG is and how to create a DAG definition file and different methods for DAG creation
• Learn about DAG Run, default_arguments, and DAG arguments and Master scheduling concepts such as depends_on_past, wait_for_downstream, catchup, and backfill
• Use the Airflow CLI for various operations and access a handy cheatsheet for quick reference
• Understand tasks, task instances and Learn the lifecycle of a task
• Master different operators including BashOperator, PostgresOperator, PythonOperator, SqliteOperator, and EmailOperator
• Implement sensors like FileSensor, SQLSensor, TimeDeltaSensor, and TimeSensor
• Apply branching logic with BranchSQLOperator, BranchPythonOperator, BranchDayOfWeekOperator, BranchDateTimeOperator, and ShortCircuitOperator
• Manage DAG dependencies and use TaskGroups ,Utilize TriggerDagRunOperator , ExternalTaskSensor and use hooks such as PostgresHook and SHook
• Manage resources with pools and task priorities
• Learn about different types of executors: SequentialExecutor and LocalExecutor and learn the Transition from SequentialExecutor to LocalExecutor
• Explore the Airflow metadata database and Manage roles and create users with different roles including admin, public, user, and operator roles
• Set and manage task-level and DAG-level SLAs and handle SLA misses
• Address issues like zombie tasks, SIGTERM, and SIGKILL errors

Requirements

• Knowledge of Python
• Rest we will cover to learn Airflow from scratch
• Familiarity with command-line interfaces.
• Understanding of database concepts is a plus but not required

It’s not my rip

Password: cms.ddpanda.org