Data Engineer
Did you do additional logging besides Airflow logs?
How do you understand the concept of keys in tables — what is it for?
Why are you looking for a new job, and what would you like to do in the future?
Why are you looking for a new job now, do you want to change your job?
Propose a solution for the task of regular incremental loading of a large table with flexible (changeable) update frequency from Postgres to Data Lake.
If we need all clients, even if they had no transactions on that date — how to display this?
Have you worked with feature stores?
LeetCode #? — In PostgreSQL, there is a table [table] with a single column id and values 1, 1, 2. Write a DELETE query that physically removes one duplicate.
Do you use AI in your current work? What tools and how exactly?
Did you have issues with time zones when reading timestamps via PXF from Parquet?
Tell us about the most significant project or task in Python over the past year.
If the query is still slow after adding several indexes - what diagnostic algorithm and what to do?
There is such a concept — transaction isolation levels. What do you know about it?
What visualization tools have you worked with and how closely have you worked with Power BI?
If the executor has 10 cores, how many Tasks will be executed in parallel at the same time?
Have you ever had to create reports, dashboards, visualizations (BI)?
import clickhouse_driver from airflow.hooks.base import * def get_clickhouse_client(): conn = BaseHook.get_connection("clickhouse_default") return clickhouse_driver.Client( host=conn.host, port=conn.port, user=conn.login, password=conn.password, database=conn.schema )
What framework did you use for writing pipelines? How did you use Airflow?
DBMS technically — a database is just a file, where do restrictions like NULL/NOT NULL come from? What are these logical abstractions for?
Tell me more about the network: what is it and why is it important?