An offshoring company is looking for a Data Engineer.
Responsibilities
Design, build, and manage the complete orchestration framework for all data pipelines using Google Cloud Composer.
Author complex, dynamic, and maintainable DAGs in Python, implementing sophisticated dependency management, triggers, and scheduling logic.
Serve as the team's subject matter expert on all DAG Authoring/Best Practices, including idempotency, task modularity, backfilling, and performance tuning.
Expert-Level DAG Authoring: Demonstrable mastery of authoring complex DAGs in Python. This must include deep knowledge of Airflow operators and sensors, dynamic DAG generation, managing task dependencies and SLAs, and creating idempotent tasks.
Cloud Composer (Apache Airflow): Deep, hands‑on experience managing a production Cloud Composer environment, including configuration, security, scaling, and troubleshooting.