Lead Data Engineer

Lead Data Engineer

London (Hybrid) | Full-time | Up to £160K + equity

We're a fast-growing AI company in London looking for our first senior data hire to architect and lead the data infrastructure behind our AI platform, real-time pipelines, analytics, vector databases, and ML data workflows, setting the standards the whole future data team will be built on.

This is an individual contributor role: you'll be the senior technical owner of the data function, architecture, standards, and direction, not a route out of code into strategy or people management. The capability today is roughly 1.5 people, so it's started but not built; you're building it.

LLM-native experience is essential: vector retrieval, embeddings, evaluation workflows, and real-time agentic systems, not general data engineering with superficial LLM exposure.

Key Responsibilities:

• Architect and build scalable data pipelines and infrastructure for our AI and product systems
• Design data ingestion, transformation, and storage architectures across operational and AI workloads
• Develop and manage batch and real-time data pipelines
• Build and optimise vector search, retrieval, and ML data pipelines
• Own data reliability, security, and governance
• Implement monitoring, observability, and data quality frameworks
• Set the culture, standards, and hiring bar for the data function as it scales
• Partner directly with founders and product leadership on data-driven decisions

Requirements:

• 7+ years, mostly in dedicated data engineering roles
• Strong track record building data pipelines and distributed systems
• Expertise with relational databases (PostgreSQL preferred; MySQL or similar fine)
• NoSQL databases and vector databases used in modern AI systems
• Strong Python programming experience
• A habit of making, and justifying, architectural calls, not just executing them

Nice to Have:

• Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, MongoDB, Qdrant/Milvus/pgvector, Pandas/Polars, cloud infra (GCP/AWS/Azure), and time in a high-growth startup.

Why this role:

You won't be maintaining someone else's system, you'll be laying the foundation of one. If you want real ownership, direct access to founders, and the chance to build the data backbone of a company from the ground up, this is that role.