What types of physical table joins exist (Hash Join, Merge Join, Nested Loop)?
Data Engineer
What salary are you expecting?
Have you worked with data vitrines (DWH layer)?
Расскажи про пару своих наиболее существенных задач, связанных с качеством данных.
What is CROSS JOIN? Have you worked with it?
If you join transactions with clients by client_id and date_start (without BETWEEN), what will be captured in the result?
Why are you looking for a new job now, do you want to change your job?
What types of JOINs do you know? (logical)
Is it possible to read data in parallel from a single Parquet file in Spark, or only with one core in one task?
What are the advantages and disadvantages of row-oriented and column-oriented databases? Which is more effective for grouping and aggregation?
Tell us about yourself — work experience, technology stack.
If the executor has 10 cores, how many Tasks will be executed in parallel at the same time?
Describe window functions in the most expanded form: what keywords are used and what each is responsible for.
-- Given a table named numbers with one column num (integers, possibly repeating). -- Write an SQL query that splits the numbers into two columns: -- even – even numbers (sorted in ascending order) -- odd – odd numbers (sorted in ascending order) -- The numbers should be arranged row by row: in the first row – the first even and the first odd, -- in the second row – the second even and the second odd, and so on. -- If one group has more numbers than the other, the missing places should be filled with NULL. -- Original table -- num -- --------- -- [phone] -- Result of the query -- even | odd -- -----|----- -- 2 | 1 -- 4 | 3 -- 6 | 5 -- 8 | 7 -- 10 | 9 -- NULL | 11 -- NULL | 11
Tell about your understanding of data quality — have you had experience working in this area?
How does Nested Loop Join work and what is its algorithmic complexity? What about the complexity of the other three algorithms?
What is an index? How does a clustered index differ from a non-clustered index?
What physical JOINs do you know?
Give a high-level overview of the Spark architecture: what components are there and how do they interact when executing an SQL query?
What is LEFT SEMI JOIN and LEFT ANTI JOIN in Spark SQL, have you ever used them?