Middle+
Tell about your recent work experience, what projects, what stack?
sobes.tech AI
Answer from AI
In my recent positions, I worked on projects related to big data processing and building ETL pipelines. In one project, we collected and processed data from various sources (log files, databases, APIs) for analytics and reporting.
The stack included:
- Apache Spark for distributed data processing
- Apache Kafka for stream messaging
- Python and SQL for scripting and querying
- Airflow for task orchestration
- Hadoop HDFS for storing large volumes of data
I also have experience working with cloud services AWS, particularly S3 and EMR for scalable data processing.