What triggers the creation of a new Job in Spark, and how is a Job divided into Stages and Tasks?
Middle+
164
Data Engineer
671
0
How does Nested Loop Join work and what is its algorithmic complexity? What about the complexity of the other three algorithms?
Middle+
164
Give a high-level overview of the Spark architecture: what components are there and how do they interact when executing an SQL query?
Middle+
163
What is partitioning? What is sharding? What is replication?
Middle+
162
Given a table 'clients' (client_id, full_name, date_start, date_end) and a table 'transactions' (transaction_id, client_id, date, amount), write an SQL query to calculate the sum of transactions for each client on May 1, 2025, and output client_id, full_name, and sum.
Middle
162
If we set a filter condition on the transaction date in the WHERE clause (after the LEFT JOIN), will this cut the result based on the first table (clients)?
Middle
161
## a task on mode # Given a list of numbers. Write a function that finds the mode of this list. # find_mode([1, 2, 3, 3, 4, 5]) #> 3
Middle
161
What will the find_mode function return for the list [1, 2, 3, 3, 4, 5]? Explain step-by-step.
Middle
160
-- t1 -- num --------- -- 1 -- 2 2 null null 4 -- t2 -- num --------- -- [phone] null null 5 SELECT a.num AS a_num, b.num AS b_num FROM t1 a LEFT JOIN t2 b ON a.num = b.num;
Middle
160
Tell me where you worked as a systems analyst and what tasks you performed?
Middle
157
What is the difference between the Parquet storage format and CSV?
Middle+
157
What is the difference between partitioning and sharding?
Middle
157
Are your data quality checks performed by yourself, or do the requirements come from business/clients?
Middle
152
What methods (types) of data history storage do you know? How is history accumulated in tables?
Middle
152
In terms of work format, which is more convenient — office, hybrid, or remote? We have two offices, on Kutuzovsky and Tula.
Middle
149
Have you ever worked with the AQE (Adaptive Query Execution) optimizer in Spark? Do you know how it works?
Middle+
149
What types of physical JOINs exist in Spark?
Middle+
149
How to correctly determine the distribution field in Greenplum? What happens during Motion?
Middle
149
How to output the last full name for each client if only start_date is known (end_date is unknown)?
Middle
149
What physical mechanism JOIN will be used when joining the transactions table with the calendar table on the inequality condition (date <= date)?
Middle+
148