AI & Data Engineer

About the Role

Provide a brief overview of the role, including the team and what the day-to-day looks like.

Purpose

Our core asset is our data, and we are looking for a specialist who can not only maintain our high-standard data infrastructure while our Lead Data Engineer is on paternity leave but also accelerate our evolution into an AI-first organisation.

This role is a unique hybrid of stability and innovation. You will ensure our existing pipelines remain robust while leading the charge on AI improvements to our internal operations, systems, and client-facing products.

You will be key to helping us extract new insights, provide deeper analysis, and enable AI-driven self-service capabilities for our internal and external users.

Key facets of this role

• AI Integration & Innovation: Design and deploy AI-driven features to automate internal operations and enhance our qualitative/quantitative research assets.

• Vector Infrastructure: Build and maintain vector databases and RAG (Retrieval-Augmented Generation) pipelines to unlock the value of our unstructured data.

• Pipeline Evolution: Transform existing ETL/ELT processes into AI-ready pipelines, ensuring data quality for machine learning training and inference.

• System Maintenance: Provide interim stewardship of our core data platform, ensuring uptime and performance while the Lead Data Engineer is away.

• Technical Mentorship: Act as the internal subject matter expert, upskilling the broader team on MLOPs and AI data best practices.

• Operational AI: Implement agentic workflows or automated insights to turn raw data into "AI-driven self-service" capabilities for our global clients.

The type of person we need in this role

This role can only be done effectively by someone who:

1. Experience: 4+ years in Data Engineering, with at least 2 years focused on AI/ML implementation (LLMs, NLP, or predictive modeling).

2. AI Toolkit: Proven experience with Vector Databases (e.g., OpenSearch, CosmosDB, Milvus) and frameworks like LangChain or LlamaIndex.

3. Core Engineering: Deep proficiency in Python and PostgreSQL.

4. Big Data & Ops: Hands-on experience with Apache Spark (PySpark) and workflow orchestration (e.g., Airflow, Prefect, or Dagster).

5. Cloud & Warehouse: Extensive experience with a major cloud provider (AWS/Azure/GCP) and modern warehouses like Snowflake, Redshift, or BigQuery.

6. DevOps Mindset: Proficient with Git, CI/CD and the operationalisation of ML models (MLOps).

7. Adaptability: The ability to step into a leadership gap, manage existing priorities, and pivot quickly toward innovation.

The qualities we’re looking for

• Problem-Solver: A proactive and analytical mindset, with the ability to diagnose and solve complex data and AI/ML infrastructure challenges.

• Collaborative & Enabling: Excellent communication and interpersonal skills, with a strong desire to teach, mentor, and share expertise effectively with Data Analysts, the Senior Data Engineer, and other stakeholders.

• Detail-Oriented: Meticulous attention to data quality, integrity, and pipeline robustness.

• Adaptable: Eagerness to learn new technologies and adapt to evolving ML/AI landscapes.

• Impact-Driven: A desire to contribute directly to the success of data-driven products and business outcomes, particularly in enabling new insights and self-service capabilities.

What we offer

• Strong professional development and continued learning.

• Hybrid work environment (2 days minimum in our London office) with core hours and time flexibility.

• Enhanced pension contributions

• Annual profit share scheme

• 28 days annual leave

• Learning and development culture

• Health helplines

• Enhanced parental leave.

• Cycle to work scheme.

• Death in service insurance

About us

Source is a research-led advisory firm that helps the world’s largest professional services firms make their most important decisions.

With a wealth of independent insight, knowledge, and experience in the industry, Source delivers clear-cut direction that gives firms and their leaders the confidence to act.

As our AI & Data Engineer, you will be instrumental in enabling us to build robust, deep, and valuable data through advanced analytics and AI-driven capabilities.

Detailed role

• Build scalable data pipelines for ML and AI applications. Implement robust data ingestion strategies from diverse sources (e.g., databases, APIs, streaming services) and participate in designing efficient data transformation pipelines to prepare our qualitative and quantitative data for machine learning and AI consumption, setting standards for the team.

• Champion strategies for preparing diverse data into AI-ready features. Collaborate with Data Analysts to understand data needs for new insights and self-service tools, then lead the design and structuring of data appropriately for various AI and ML applications, guiding other engineers in these practices.

• Steer the cloud data platform's evolution to enhance AI/ML capabilities. Participate in optimising the usage of cloud-native data and ML services to ensure cost-efficiency, scalability, and high availability of the data platform, with a focus on AI/ML readiness, and advise the team on technology choices.

• Optimise data processing and ML pipelines for efficiency and scale. Proactively identify and resolve performance bottlenecks in data pipelines and ML workloads, ensuring optimal system efficiency, and sharing techniques for performance tuning with the team.

• Be the go-to expert for ML/AI data engineering best practices. Actively participate in technical discussions, conduct code reviews, and lead knowledge sharing sessions with Data Analysts, the Senior Data Engineer, and other engineering teams to foster a data-driven culture and elevate ML/AI understanding.

Diversity & Inclusion

At Source, we are committed to encouraging equality, diversity, and inclusion among our workforce, and eliminating unlawful discrimination.

We are determined to ensure that no applicant or employee receives less favourable treatment on the grounds of gender reassignment, age, disability, religion or belief, sex, sexual orientation, marital status, or race, or is disadvantaged by conditions or requirements which cannot be shown to be justifiable.

The aim is for our workforce to be truly representative of all sections of society and our customers, and for each employee to feel respected and able to give their best.