Why are sensors needed in Airflow?

Sensors in Apache Airflow are used to wait for specific conditions or events before triggering the next task in a DAG. They enable organizing task dependencies not only by schedule but also based on external factors, such as the appearance of a file, completion of another task outside Airflow, or resource availability.

An example of using a sensor is waiting for a file to appear in storage before starting data processing:

from airflow.sensors.filesystem import FileSensor

file_sensor = FileSensor(
    task_id='wait_for_file',
    filepath='/data/input/file.csv',
    poke_interval=30,  # check every 30 seconds
    timeout=600       # maximum wait of 10 minutes
)

Thus, sensors help automate and synchronize processes, ensuring the correct order of task execution based on external conditions.