Design, develop, and optimize scalable data pipelines and ETL processes using Big Data frameworks (e.g., Hadoop, Spark) to ingest, transform, and store large volumes of structured and unstructured data.
Write high-performance SQL queries and Java code to support data extraction, transformation, and loading across multiple data sources and target systems.
Collaborate with cross-functional teams to define data models, data quality standards, and metadata management practices.
Implement and maintain DataOps workflows, including CI/CD pipelines, automated testing, and monitoring for data pipeline health and performance.
Ensure data security, compliance, and governance across all data assets, working closely with data stewards and security teams.
Requirements
5+ years of experience as a Data Engineer or similar role, with a strong background in Big Data technologies.