Write a DAG for loading data via REST API
Middle
260
Data Engineer
743
0
Report for a logistics company You are an analyst at a logistics company that keeps track of warehouse operations. You need to compile a report on the efficiency of each warehouse. For each warehouse, calculate: • total number of operations (count_operations); • total quantity of goods processed at the warehouse (sum_quantity); • average processing time of an operation (avg_processing_time), considering only operations with a specified time (not NULL), rounded to the nearest whole number; • maximum and minimum number of goods processed in a single operation (max_quantity, min_quantity); • number of operations of each type ('delivery', 'shipment', 'transfer') in separate columns: supply_operations, shipment_operations, transfer_operations. Filter warehouses where total number of operations is more than 2 and average processing time does not exceed 60 minutes. Sort the result by warehouse ID in ascending order. Input format Table operations: • operation_id (int) — unique operation identifier • warehouse_id (int) — warehouse identifier • operation_type (text) — type of operation: 'delivery', 'shipment', 'transfer' • quantity (int) — number of goods units in the operation • operation_date (timestamp) — date and time of the operation • processing_time (int) — processing time of the operation The processing_time column may contain nulls. Output format The query should return a table with fields in the following order: • warehouse_id (int) — warehouse identifier • count_operations (int) — total number of operations at the warehouse • sum_quantity (int) — total quantity of goods processed at the warehouse • avg_processing_time (numeric) — average processing time of operations (in minutes), considering only non-NULL times, rounded to the nearest whole number • max_quantity (int) — maximum number of goods processed in a single operation • min_quantity (int) — minimum number of goods processed in a single operation • supply_operations (int) — number of 'delivery' operations • shipment_operations (int) — number of 'shipment' operations • transfer_operations (int) — number of 'transfer' operations.
Junior
260
Give an example of a typical DAG task in Airflow.
Middle+
259
What is Adaptive Query Execution (AQE)?
Middle
259
What is the difference between UNION and UNION ALL? What conditions must be met?
Middle
258
Were there any issues with OpenAPI when transitioning from Spring Boot 2 to 3?
258
What is shuffle in Spark and why is it important to minimize it?
Middle
258
# what does it output s = 'abracadabra' print(s[5][0][0][0]) print(s[::-2]) s = [1, 2, 3] print(s * 2)
Middle
258
What are the features and differences of Set from List in Python? Besides performance, what other features are there?
Middle
258
What is GIL (Global Interpreter Lock)?
Middle
257
What is the advantage of columnar storage over row-based storage?
Middle
257
When there is high memory consumption on CTE, what happens to the data?
Middle
256
How to distribute storefront data unevenly across three shards: 50% on the first and 25% on each of the remaining two? How to implement this?
Middle
256
A special sequence called even_sequence was created, which generates only even numbers. What should be substituted in place of [...], so that if the value of even_column was not specified during insertion, the value is taken from even_sequence? create table some_table( even_column [...] ); integer computed as nextval('even_sequence') integer generated always as identity (start with 2 increment by 2) integer default nextval('even_sequence') integer unique default nextval('even_sequence') integer generated by even_sequence’
Junior
256
How does Hive differ from Impala?
Middle+
256
In terms of requests, what happens to data, including what types of joins — what types of joins can we see there?
Middle
255
[name] corrected the errors and style in the text: "Hello! I am [name], you previously asked to switch to Telegram" — to make it informal, but respectful.
Junior
255
What is the concept of statistics in the context of DBMS and query optimization?
Senior
254
How can data from Greenplum and Trino be transferred to ClickHouse?
Middle
254
How to update a table in ClickHouse so that users do not notice (atomic replacement)?
Middle
254