What is the difference between Cron and Apache Airflow?

Cron is a scheduling utility in Unix-like operating systems. It is used to execute commands or scripts at specified times or intervals.

Apache Airflow is a platform for programmatically authoring, scheduling, and monitoring workflows. It allows defining workflows as directed acyclic graphs (DAGs), where each node is a task.

Main differences:

Feature	Cron	Apache Airflow
Purpose	Scheduling individual commands	Scheduling and orchestration of complex workflows
Configuration	crontab	Python scripts (DAGs)
Dependencies	None	Supports dependencies between tasks
Monitoring	Limited (system logs)	Web interface, logs, metrics
Scalability	Limited	Horizontal scaling
Testing	Manual	Programmatic testing of DAGs
Fault tolerance	Limited	Built-in mechanisms
Visualization	No	Detailed DAG and execution visualization

Example of Cron:

# Run script every Friday at 17:00
0 17 * * 5 /path/to/your/script.sh

Example of Apache Airflow (part of DAG):

# Define task
from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime

with DAG(
    dag_id='example_bash_operator',
    start_date=datetime(2023, 1, 1),
    schedule_interval=None,
    catchup=False
) as dag:
    bash_task = BashOperator(
        task_id='run_a_command',
        bash_command='echo "Hello from Airflow"',
    )