Senior Data Engineer

Central Employment are working in partnership with a long term eCommerce Web-Print Specialist, as they look to recruit a Senior Data Engineer , joining the growing Engineering team in Newcastle. You will play a crucial role in designing, building, and maintaining our data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations. Key Responsibilities: Design, develop, and maintain scalable and efficient data pipelines. Optimize ETL processes to ensure seamless data ingestion, processing, and integration across various systems. Lead the development and maintenance of a real-time data streaming platform using tools like Apache Kafka, Databricks, Kinesis. Ensure the integration of streaming data with batch processing systems for comprehensive data management. Utilize AWS data engineering services (including S3, Redshift, Glue, Kinesis, Lambda, etc.) to build and manage our data infrastructure. Continuously optimize the platform for performance, scalability, and cost-effectiveness. Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver solutions. Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence. Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines. Implement and maintain data quality frameworks, ensuring the accuracy, consistency, and reliability of data across the platform. Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standards. Experience: 3 years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS. Strong proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java is a plus. Familiarity with Databricks and the Delta Lakehouse concept. Experience mentoring or leading junior engineers is highly desirable. ADZN1_UKTJ