As a Data Engineer, you will work closely with stakeholders, data scientists, and internal teams to develop and implement solutions.
Your key responsibilities will include building and maintaining technical platforms for advanced analytics.
You will be designing scalable, testable and maintainable data pipelines and ensuring data security and governance.
In this role, you will develop and maintain Python/PySpark libraries for data ingestion and integration with both internal and external data sources.
You will create and enhance Python/PySpark libraries for data quality validation, ensuring accuracy, consistency, and completeness in large-scale datasets.
You will design and optimize scalable data pipelines using Spark and cloud data services (AWS).
You will lead and execute data engineering projects from inception to completion, ensuring timely delivery and high quality.
You will continuously monitor, troubleshoot, and improve data pipelines and workflows to ensure optimal performance and cost-effectiveness.
You will collaborate with cross-functional teams to stay ahead of emerging trends in Generative AI and cloud-based automation.