You read events in batches and write to ClickHouse in batches. How does this work with ACK — when do you give the acknowledgment?

When reading and writing data in batches (batch) in a system with acknowledgment (ACK), it is important to correctly manage the acknowledgment timing to avoid data loss and duplicate processing.

Typically, an acknowledgment (ACK) is sent after successfully writing the entire batch to ClickHouse. That is:

1. Read a batch of events from the source.
2. Write the batch to ClickHouse.
3. If the write is successful, send an ACK for this batch.

If the write fails, do not send an ACK, and the batch will be read again (depending on the queue mechanism).

This approach guarantees that data is not lost and will not be acknowledged until it is actually written.

Here's an example:

batch := readBatch()
err := writeBatchToClickHouse(batch)
if err == nil {
    sendACK(batch)
} else {
    // log the error, possibly retry
}