How were DAGs launched: by cron, datasets, triggers, or dependencies?
Data Engineer
Have you worked with data quality incidents?
What types of keys/distribution strategies exist in Greenplum?
What is YARN and why is it needed?
What groups of SQL operators do you know? What do they include?
How to parse large XML files in ZIP and load into ClickHouse using Python?
What is data partitioning and how does it help avoid full table scans?
How does MapReduce work, specifically the Reduce stage?
How have you worked with CI/CD?
What is the difference between EXPLAIN and EXPLAIN ANALYZE?
You received a task with number ABC. What is your order of actions in Git at the start of work?
When is it more advantageous to use hash partition, and when range partition?
Can you tell what we will get here in the end? Comment on each step and how it works.
Are you familiar with CTE (common table expressions)? Provide an example of its use.
SELECT * FROM table1 AS t1 LEFT JOIN table2 AS t2 ON t1.date_start > t2.date_start --DML CREATE TABLE IF NOT EXISTS Employee (id int, salary int, departmentId int); INSERT INTO Employee (id,salary,departmentId) VALUES (1, 70000, 1),(2, 94000, 1),(3, 85000, 1),(4, 80000, 2),(5, 60000, 2),(6, 62000, 3),(7, 90000, 3),(8, 94000, 3),(9, 94000, 3); with cte as( select *, rank() over(partition by departmentId order by salary desc) as max_salary from Employee ) select * from cte where max_salary = 1
Explain how Git LFS works technically and what advantages it offers compared to standard Git when working with large files.
import clickhouse_driver from airflow.hooks.base import * def get_clickhouse_client(): conn = BaseHook.get_connection("clickhouse_default") return clickhouse_driver.Client( host=conn.host, port=conn.port, user=conn.login, password=conn.password, database=conn.schema )
Have you faced conflicts within the team or with a manager? How did you handle it? Provide examples.
How did you work with analytical functions (window functions)?
What are your salary expectations?