The scheduler takes records from the table, two pods simultaneously took one record — how to solve the problem?

The problem of two pods simultaneously taking the same record is a classic race condition in task processing.

Solutions:

1. Use transactions with record locking (e.g., SELECT ... FOR UPDATE) so that one pod locks the record during processing.

2. Add a status field and atomically update it — for example, change the status from new to processing with a condition that the status was new. If the update succeeds, the pod has acquired the task.

Example SQL query for atomic task acquisition:

UPDATE tasks
SET status = 'processing', worker_id = :pod_id
WHERE id = (
  SELECT id FROM tasks
  WHERE status = 'new'
  LIMIT 1
  FOR UPDATE SKIP LOCKED
)
RETURNING *;

3. Use message queue mechanisms or brokers (RabbitMQ, Kafka) to distribute tasks among pods without conflicts.

4. Use unique indexes and error handling when attempting to acquire — if two pods try to update the same record, only one succeeds.

Thus, the key is to ensure the atomicity of the task acquisition operation and prevent parallel processing of the same record.