Data Engineer

How does ORDER BY affect the ClickHouse table and what keys are better to choose?

A regular B-tree index on id is created in the records(id) table. Why will the next query not use the index? select * from records where id % 2 = 0 - GIN type index is needed for id - Indexes do not work with expressions in WHERE - % is a comparison operation, not a filter - limit and offset are mandatory for index optimization - The query accesses a numeric field, not a string field

How do you deliver DAG codes to production?

What are your expectations from a new project?

Middle+

Альфа-Банк

What types of JOINs do you know? Tell 2-3 main ones.

How did you load data from S3 into Greenplum?

Middle

Aston

What are the main Linux commands you use in the terminal?

Approximately how many uploads were there — top-level jobs, threads?

Middle

МВидео

How to create many identical tasks in Airflow in parallel (for example, to upload many identical files)?

What are the main components of a trace message (span)?

What is the danger of shuffle in Spark?

Middle

Островок

What is the Catalyst optimizer in Spark and what specific optimizations does it perform (e.g., predicate pushdown)?

Middle+

СБЕР

Do you have experience working with the Langchain library? What tasks have you solved using it?

What happens when queries are parallelized in a single-node Postgres?

How to load data from Kafka into ClickHouse via streaming for real-time reporting?

Middle

МВидео

How to remove a submodule and its related files from a project? git submodule remove <path-to-submodule> git rm --cached <path-to-submodule>; remove section from .gitmodules; git commit git clean --submodules <path> git submodule delete <path> git remove submodule <path>

Why is MergeTree needed in ClickHouse?

Middle+

Альфа-Банк

How does Hash Join work?

Middle+

СБЕР

What are the disadvantages of MPP architecture in Greenplum?

How to choose the right sharding key for even data distribution?