How to ensure the delivery guarantee of Exactly Once (each message is written to the file exactly once)?

Guaranteeing Exactly Once delivery of a message is one of the most challenging tasks in distributed systems, especially when writing to a file. Typically, a combination of the following approaches is used:

1. Idempotency of operations — ensure that re-writing the same message does not lead to duplication. For example, store a unique message identifier and check it before writing.

2. Transactional mechanisms — use mechanisms that ensure atomicity of writes. For files, journaling (write-ahead log) or temporary files with subsequent renaming can be applied.

3. Acknowledgments and retries — the sender should receive confirmation of successful writing and only retry if no confirmation is received.

4. Use of specialized systems — for example, message brokers supporting Exactly Once (Kafka with transactions), which guarantee that a message will be processed exactly once.

A simplified Java example:

import java.io.*;
import java.util.HashSet;
import java.util.Set;

public class ExactlyOnceWriter {
    private Set<String> processedIds = new HashSet<>();
    private File file;

    public ExactlyOnceWriter(File file) {
        this.file = file;
        // load processedIds from file or database at startup
    }

    public synchronized void writeMessage(String messageId, String message) throws IOException {
        if (processedIds.contains(messageId)) {
            // Message already written
            return;
        }
        try (FileWriter fw = new FileWriter(file, true)) {
            fw.write(message + "\n");
            fw.flush();
            processedIds.add(messageId);
            // save processedIds to a reliable storage
        }
    }
}

Thus, the key is to keep track of which messages have already been processed and to use atomic write operations.