Senior Data Engineer

The following role is a Fixed Term Contract, with the ideal candidate starting in October and working up until Christmas 2025.

The company:

Are an organisation dedicated to building a society where everyone has a secure place to live, supportive people around them, and a meaningful purpose. They achieve this through two key approaches: delivering high-quality public services to improve lives on the frontline, and using their knowledge to drive systemic change, addressing the complex web that can trap and disempower those it was created to help. With the heart of a charity and the mindset of a business, we are uniquely positioned to address this challenging agenda.

Where You Fit In

It’s an exciting time for the charity as they embark on their first large-scale data transformation project. We are seeking a forward-thinking, dynamic problem solver to be our go-to expert in data engineering. The Data Architecture Project will centralise their siloed data assets into one ecosystem, allowing them to dive deeper into insights from their projects and databases, and to measure impact more effectively. In the future, this valuable information will be used to predict needs on the frontline of service delivery.

In this project, you will work within the Evidence and Insight Team and collaborate across the organisation, including the wider Corporate Services Team and key stakeholders from various Delivery Teams. They are a rapidly growing, agile team that ensures we stay up-to-date with the latest standards by using key tools such as Azure Data Factory, Databricks, Terraform, DevOps, and Jinja. They believe trust and transparency are key drivers of best practice, and they offer an environment built on shared ownership, unity, and emotional intelligence.

Main Duties & Accountabilities

As the Senior Data Engineer, you will play a pivotal role in designing and building our data ecosystem. Key areas of responsibility will include:

• Architecture Design:Lead the development and implementation of a dynamic, elastic, and resilient data architecture that supports current and future contracts. This role requires cross-departmental and external collaboration to identify needs and create data solutions, along with the ability to communicate and translate complex technical concepts effectively. You must understand the full data lifecycle and business needs to develop suitable solutions.

• ETL Process:Collaborate with external and internal teams to build and implement Extract, Transform, Load (ETL) processes and comprehensive pipelines using tools like Azure Data Factory, Azure Logic/Functions Apps, and Databricks. You will extract data from diverse, complex sources across the organisation, transform it into a usable format, and load it into data storage systems. Reviewing systems and suggesting optimisation solutions for organisational efficiency and security will be required.

• Data Pipeline Management:Build and manage data pipelines that are fault-tolerant, reliable, and dynamic. Automate data workflows, including data ingestion, aggregation, and ETL processing. Ensure data accuracy, integrity, security, and compliance in collaboration with the wider team and Information Security team.

• Data Modelling:Ensure business readiness from the data we ingest, with a strong knowledge of data modelling, creating queryable, efficient analytics-ready models. Work alongside analysts to speed up analysis and make our organisation AI/ML-ready.

• Cloud-Based Solutions:Serve as the expert on all things Azure, including the management of the data infrastructure, best practices, cost optimisation, and performance tuning.

• Skill Development:As an expert in data engineering, you will play a fundamental role in the data maturity journey, upskilling and supporting data professionals across the organisation to sharpen and grow their skills while enhancing an environment of skill development.

About You

You will have at least 5+ years of experience in data engineering roles and be an Azure expert. You thrive in small, collaborative teams that work quickly and maintain a high level of quality. You should have enough business acumen to understand how our work ties into the bigger picture, along with a sharp eye for details and an ability to spot inconsistencies.

Your skill set should include:

• Data Analysis and Synthesis:Ability to bring multiple data sources together in a conformed model for analysis.
• Cloud Management:Expertise in managing and optimising cloud data management systems.
• Data Modelling:Understanding of the concepts and principles of data modelling and the ability to produce relevant data models.
• Programming and Building:Proficiency in leading on design, coding, testing, and corrections, with input from others on specifications.
• Technical Understanding:Strong knowledge and understanding of technical concepts within the role and the wider landscape.
• Skill Development:Capability to lead and coach others to improve team skills.
• Collaboration:Ability to work across multiple teams and with external resources to achieve the best outcomes.
• Strategy:Understanding of how work fits within the broader organisational landscape and potential future developments.

Key Success Outcomes for This Role:

• Data is captured within one ecosystem, and systems are maintainable.
• The organisation is well-positioned for future data insights, building predictive models, and data science opportunities by Summer 2025.
• Clear and well-documented data pipelines are established and maintained.
• A data maturity framework is co-created with the Data Transformation Programme Manager and other collaborative stakeholders.
• Strong foundations of data quality, governance, and security are provided to future-proof data practices.