Responsibilities • Design, develop, and maintain scalable data pipelines using Python, PySpark , and AWS Glue, EMR, Step function. • Build and optimize distributed data processing workflows using EMR and cluster programming techniques. • Develop Airflow DAGs to orchestrate complex ETL pipelines. • Work extensively with AWS services to build end-to-end data engineering solutions. • Ensure reliability, performance, and scalability of all data workflows. • Troubleshoot production issues quickly and implement corrective actions. • Collaborate with cross-functional teams to deliver high-quality data products. • Independently drive deliverables with minimal supervision.