Jobs

Data Platform Manager


Job details
  • ComplyAdvantage
  • London
  • 1 month ago

What you will be doing :

The role reports to the VP Engineering of Platform and Infrastructure. You’ll be managing a team of engineers - distributed between our London and Lisbon offices - focused on the provision and support of data services for use across the organisation, including classic SQL or NoSQL databases, distributed SQL, Kafka, and Spark-based systems. Your key responsibilities will be:

  • Team Leadership: Lead, mentor, and grow a team of Data Infrastructure Engineers responsible for maintaining and improving the data systems across the organisation.
  • System Design & Architecture: Design and architect high-performance, low-latency data systems to meet the requirements of real-time financial risk detection and analysis.
  • Data Infrastructure Management: Oversee the implementation and management of SQL and NoSQL databases, distributed SQL systems, and stream processing platforms like Kafka and Apache Spark.
  • Cross-Functional Collaboration: Work closely with data scientists, software engineers, and risk management teams to understand their data requirements and deliver reliable and efficient solutions.
  • Performance Optimisation: Continuously monitor system performance, identify bottlenecks, and implement tuning strategies to ensure systems are performing at optimal levels.
  • Innovation & Best Practices: Stay up to date with the latest in data engineering, distributed systems, and database technologies, incorporating best practices into the team's workflows.
  • Data Governance & Compliance: Work closely with our Data Governance team to ensure all data systems comply with relevant regulations, privacy standards, and security protocols.
  • Project Management: Drive the execution of data infrastructure projects from concept to delivery, managing timelines, resources, and priorities effectively.

Our tech stack:

ComplyAdvantage is fully cloud-based, with a modern Kubernetes-focused tech stack. All compute workloads run in Kubernetes, with clusters in multiple regions to support the needs of our global client base. Our production services are currently based in AWS and GCP, are domain designed and event-driven or event-sourced. As such, Kafka is used widely across teams, and is seen as a core technology in our tech stack along with Yugabyte, our Distributed Database system. We also use other data tooling, such as PostgreSQL, Elasticsearch and Databricks/Spark. To tie in with our multi-cloud plans, we’ll also be developing inter-cloud failover options to be leveraged by our development teams.

Across the wider Technology team, teams build and release containerised applications to support the wide array of activities that our teams are engaged in - from developing low latency client-facing APIs, to machine learning models and data processing pipelines.

Very agile by nature, our teams operated using elaborated CI/CD pipelines. We also make heavy use of Terraform and Helm to define our infrastructure and services, and lean heavily on GitOps paradigms - production and non-production environments are defined in git and changes to these environments (both cloud infrastructure and Kubernetes applications) are managed via git.

About you:

  • Experience managing and mentoring engineering teams, with strong leadership and interpersonal skills.
  • Expertise in SQL and NoSQL databases (PostgreSQL, Cassandra, MongoDB, etc.) and distributed SQL systems (such as YugabyteDB or CockroachDB).
  • Good knowledge of Apache Kafka for real-time data streaming and Apache Spark for distributed data processing.
  • A strong understanding of cloud infrastructure (AWS, GCP, or Azure) and its data services, including storage, compute, and networking.
  • Experience with monitoring tools and performance optimization techniques for large-scale distributed systems.
  • Understanding of data governance, compliance, and security requirements for financial services (e.g., WORM storage, FedRAMP, etc.).
  • Hands-on experience with automation, infrastructure-as-code, and CI/CD pipelines.
  • Experience with data modelling, sharding and indexing strategies running databases at scale

Nice to haves:

  • Experience of working in a start-up/ scale-up environment
  • Have experience designing and developing microservices-based architectures
  • An understanding of cost and performance considerations when designing data systems architectures
  • Experience working with CI/CD tools like CircleCI, Harness, ArgoCD, etc

Benefits:

  • Equity as we want you to have a part of what we are building
  • We embrace a hybrid approach that requires employees to be in the office for two days a week. We strongly believe that this approach fosters collaboration and enables the building of meaningful relationships
  • You will also get a budget to kit out your home office
  • Opportunity to work on innovative projects with smart-minded people keen to share their knowledge and continuously improve
  • Annual learning budget to drive your performance and career development
  • Enhanced parental leave and childcare benefits
  • Medical insurance through Bupa with pre-existing conditions covered
  • Pension contribution through The Peoples Pension

About us:

ComplyAdvantage is the financial industry’s leading source of AI-driven financial crime risk data and detection technology. Our mission is to neutralise the risk of money laundering, terrorist financing, corruption, and other financial crime.

More than 1000 companies rely on us to understand the risk of who they’re doing business with through the world’s only global, real-time database of people and companies. Our solutions identify thousands of risk events daily from millions of structured and unstructured data points.

We have five global hubs in New York, London, Singapore, Lisbon and Cluj-Napoca and are backed by Goldman Sachs, Ontario Teachers, Index Ventures, and Balderton Capital.

Since 2014, we have raised over $100 million in funding, and in 2022 alone grew by over 40% to over 500 people globally. Over the next 12 months, as our revenue increases, we plan to increase to 600.

At ComplyAdvantage diversity fuels our rocket ship and our commitment to inclusion across race, gender, age, religion, identity and experience drives us forward every day. We encourage everyone to apply and aspire to consider every application fairly.

We will handle your information in accordance with our Privacy Policy. For further information, please click

Sign up for our newsletter

The latest news, articles, and resources, sent to your inbox weekly.

Similar Jobs

Data Platform Manager

What you will be doing :The role reports to the VP Engineering of Platform and Infrastructure. You’ll be managing a team of engineers - distributed between our London and Lisbon offices - focused on the provision and support of data services for use across the organisation, including classic SQL or...

ComplyAdvantage London

Product Manager - Data Warehousing | OVO

OVO Group is a leading energy technology company determined to create a world with clean, affordable energy for everyone. Since launching in 2009, they have welcomed over a million members, planted a million trees, and set their sights on helping save the planet. They are on a mission to change...

Higher - AI recruitment UK

Analytics Manager

The JobAs the Analytics Manager in the Core Analytics team at Flo, you will play a pivotal role in driving research, methodology, and analytics engineering initiatives that impact the entire company. This position requires a blend of strategic thinking, technical expertise, and leadership skills to guide two critical pillars: research...

Flo London

Product Analytics Manager

Company Description At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for job seekers and companies around the world to find a perfect match, in fair and equitable way. With over 20 brands...

Stepstone UK London

Product Analytics Manager

Company Description At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for job seekers and companies around the world to find a perfect match, in fair and equitable way. With over 20 brands...

Stepstone UK South Bank

Junior Project Manager

£250 per day (In-scope of IR35) 6-month Contract Hybrid working (1 day in the office per week) Your new organisation Our legacy spans over 2 and a half centuries, and we are entirely owned by a charitable institution based in the UK, dedicated to the advancement of science and engineering....

Hays Technology Reading