Data Pipeline Development : Develop and maintain data pipelines that extract, transform, and load (ETL) data from various sources into a centralized data storage system, such as a data warehouse or data lake.
Data Integration : Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers.
Data Transformation and Processing : Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks.
Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines.
Implement data governance in line with company standards.