Across
- 2. Python-based framework for orchestrating workflows
- 4. Programming language used in PySpark
- 5. Component in Spark that optimizes query execution plans
- 8. Default cluster manager for Spark
- 10. Newer Airflow feature that allows event-driven DAGs
- 12. Spark engine that processes structured data
- 13. Process of executing a task in Airflow
- 14. Directed structure that defines tasks and dependencies in Airflow
- 15. File format optimized for big data processing in Spark
- 17. Component responsible for scheduling tasks in Airflow
Down
- 1. Distributed computing framework that Spark is often compared to
- 3. Process of breaking jobs into smaller execution tasks in Spark
- 5. Fast and general-purpose computation engine in Spark
- 6. Default executor in Airflow for running tasks sequentially
- 7. User interface used to monitor DAGs in Airflow
- 9. Database backend that stores Airflow metadata
- 11. Spark API for running SQL queries
- 16. Fundamental data structure in Spark