Where exactly does Elasticsearch store logs?

Elasticsearch stores logs (documents) in indexes. An index is divided into shards (which can be replicated), physically stored on cluster nodes. Each shard is an instance of the Apache Lucene search engine, which saves data on disk in the form of segments.

• Index: Logical containers for documents.
• Shard: Physical division of an index, containing a subset of documents.
• Replica: A copy of a shard, ensuring high availability and fault tolerance.
• Segment: An immutable data structure on disk within a shard, containing an inverted index and document data.

The physical location of segment data depends on Elasticsearch settings and the operating system. Usually, it is a directory specified by the path.data parameter in the elasticsearch.yml configuration file.

For example:

# elasticsearch.yml
# Path to the directory where index data (shards/segments) are stored
path.data: /var/lib/elasticsearch

Within this directory, Elasticsearch creates subdirectories for each node, index, and shard, where Lucene segment files (e.g., .cfs, .tim, .doc, .nvd, .nvm, etc.) are stored.