Sobes.tech
Senior

The worker runs long (2 minutes), and the transaction remains in the database if it crashes — what to do?

sobes.tech AI

Answer from AI

If a worker runs for a long time (about 2 minutes), and upon failure the transaction remains in the database, it means the transaction was not properly completed (commit/rollback), and the database holds locks.

What to do:

  1. Use a context with a timeout in Go to limit the execution time of the transaction. For example:
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

err := db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
    // operations
    return nil
})
  1. Handle errors and ensure transaction rollback in case of failures.

  2. Check database logs for hanging transactions and manually terminate them if necessary.

  3. Consider using a heartbeat or health check mechanism for the worker, so that tasks are automatically restarted if it crashes.

  4. Optimize the worker's operation to prevent long-held transactions — possibly by breaking the task into smaller parts.

Thus, it is important to control the lifespan of transactions and properly conclude them to avoid locks and hanging states.