Data Engineer

What does Row Group in the Parquet file structure responsible for?

What Docker images have you had to build, and why?

For each row in the salaries table, output the employee's name, date, salary, and using a window function, add a column "average salary by department on this date".

How are the sorting within the window function and the final ORDER BY sorting in the query related?

What is partitioning and sharding (distribution)?

Tell an example when you used artificial intelligence to automate routine processes.

DBMS technically — a database is just a file, where do restrictions like NULL/NOT NULL come from? What are these logical abstractions for?

What additional parameters of DDL/CREATE TABLE in ClickHouse do you know, what do they affect, and in what cases are they used?

How does hash join work for tables with 1,000 and 1,000,000 rows?

What are the disadvantages of UDFs in Spark and what is their downside?

Middle

Островок

Have you configured tags in S3 on buckets or objects?

Middle+

Лига Цифровой Экономики

Tell about the basic concepts of Kafka (consumer group, partitions, topics).

Have you encountered the ClickHouse 'Too many parts' error during insertion? How did you resolve it?

What additional entities of Data Vault, besides hub, link, and satellite, have you used, such as same-as links or transactional links?

Middle+

Лига Цифровой Экономики

What file formats have you worked with?

Middle+

Лига Цифровой Экономики

Task #1 What number of records will a query return with inner, left, right, full join of two tables based on the attribute id t1 t2 | id | | id | |-----| |-----| | 1 | | 1 | | 2 | | 1 | | 3 | | 3 | | 0 | | 5 | | NULL| | NULL| | NULL| inner: left: right: full outer:

Middle

Островок

Have you stored tabular view results in S3 as Parquet or otherwise?

Middle+

Лига Цифровой Экономики

Why are you looking for a new job?

What types of physical JOINs exist in Spark?

Why are you considering new proposals now?