How would you design a scalable data loading from multiple REST APIs: from data acquisition to S3, with an interface for an analyst?
Senior
3
Data Engineer
743
0
What is procedural SQL?
Middle
3
Tell us about yourself and your experience in data engineering.
Senior
3
There are 101 objects: 100 zeros and 1 one. After sorting by score, 50 zeros come first, then one, then another 50 zeros. What are the ROC AUC and the shape of the ROC curve?
Middle+
3
What are the downsides of indexes besides the space they occupy?
Middle
3
In gradient boosting, a composition of N base algorithms has already been built. What will be the target for the new, N+1-th algorithm?
Middle+
3
Why are lazy evaluations useful?
Middle
3
What is data spill in Spark and how to avoid it if resources cannot be simply added?
Middle
3
How would you handle high-load data transformations?
Middle
3
How do the Spark launch modes differ: cluster, client, and local?
Middle
3
What are the pros and cons of implementing an API loader as a custom Airflow operator/hook compared to a separate library or container service?
Senior
3
2. There is a table t, which lists the sum of client transactions per day: - Write a query that for each row shows the client's transactions for the current and previous calendar day The final table is below:
Middle
3
What are lazy evaluations and how to understand them?
Middle
3
What typical tasks have you solved in Airflow?
Middle
3
дата, номер заказа, сумма select date, sum(amount) as sum_amount from t group by date having sum(amount) > 1000000 order by sum_amount desc limit 3
Middle
3
What are star models and Data Vault?
Middle
3
Tell us about yourself: where did you work last, what did you do, what are you looking for, and why?
Middle+
2
Where are Driver and Executors located in cluster mode and why is it used in production?
Middle
2
Where exactly does the gradient occur in the gradient boosting algorithm if the base algorithm is a simple decision tree?
Middle+
2
What components does Greenplum and Hadoop HDFS consist of?
Middle
2