Sobes.tech
Senior

How would you design a table for storing high-load time series with tags (up to several million records per minute)? What key decisions about the table structure would you make?

sobes.tech AI

Answer from AI

To store high-load time series with tags (up to several million records per minute), I would choose a specialized time series database, such as TimescaleDB, InfluxDB, or ClickHouse, as they are optimized for such loads.

Main table structure solutions:

  • Sharding and partitioning by time — to quickly delete old data and speed up queries by time ranges.
  • Indexing by tags and time — for fast search and aggregation.
  • Columnar storage — for efficient compression and quick reading of necessary columns.
  • Using data compression — to reduce the volume of stored data.
  • Write optimization — batching inserts, asynchronous writing.

Example of a table structure in TimescaleDB:

CREATE TABLE metrics (
  time TIMESTAMPTZ NOT NULL,
  tag1 TEXT,
  tag2 TEXT,
  value DOUBLE PRECISION,
  PRIMARY KEY (time, tag1, tag2)
);

SELECT create_hypertable('metrics', 'time', chunk_time_interval => INTERVAL '1 day');

This approach allows for efficient storage and quick processing of large volumes of time data with many tags.

How would you design a table for storing high-load… - sobes.tech