Senior Data Engineer - Remote - £60k

Exciting opportunity for an experienced cloud data engineer to join an expanding data team who are using data in an exciting and advanced way. They will support your learning and development from a technical and leadership perspective as you help them design, build and optimise their data platform.

Salary & Benefits

22 days holiday, plus bank holidays
Staff discounts & Friends and Family discounts
Cycle to work scheme and Tech Scheme
Breakfast and drinks provided (if in office)
Yearly charity day supported
Summer and Christmas Parties
Perkbox membership
Maternity & Paternity leave
Employee referral schemeRole & Responsibilities

Design, develop, and maintain scalable and efficient data pipelines
Optimise ETL processes to ensure efficient data ingestion, processing, and integration across various systems
Lead development and maintenance of real-time data streaming platform using Apache Kafka, Databricks, etc.
Ensure the integration of streaming data with batch processing systems for comprehensive data management
Utilize AWS data engineering services to build and manage data infrastructure
Continuously optimize the platform for performance, scalability, and cost-effectiveness
Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver quality, deliverable solutions
Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence
Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines
Implement and maintain data quality frameworks, ensuring accuracy, consistency, and reliability of data across the platform
Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standardsWhat do I need to apply for the role

3+ years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS (cloud experience essential)
Proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java
Familiarity with Databricks and Delta Lakehouse
Experience mentoring or leading junior engineers
Deep understanding of cloud-based data architectures and best practices
Proficiency in designing, implementing, and optimizing ETL/ELT workflows
Strong database and data lake management skills
Familiarity with ML Ops practices and tools, with a desire to expand skills in this area
Excellent problem-solving abilities and a collaborative mindset

My client have very limited interview slots and they are looking to fill this vacancy by Christmas, so this process will have a rapid turnaround. I have limited slots for 1st stage interviews this week so if you're interest, get in touch ASAP with a copy of your most recent and up to date CV and email me at or you can call me on (phone number removed).

Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check.

Nigel Frank are the go-to recruiter for Power BI and Azure Data Platform roles in the UK, offering more opportunities across the country than any other. We're the proud sponsor and supporter of SQLBits, Power Platform World Tour, the London Power BI User Group, Newcastle Power BI User Group and Newcastle Data Platform and Cloud User Group. To find out more and speak confidentially about your job search or hiring needs, please contact me directly at