As part of this role, you'll be responsible for:
• Hands on Leadership of the Data and AI Engineering teams and ML Ops capability based in the UK and India, ensuring alignment with company standards
• Lead the evolution of our data engineering framework on Azure Databricks
• Work with the Principal AI engineer of the adaptation and evolution of our Gen AI Platform
• Provide hands on mentorship, guidance, and education for engineering, data science and ML Ops team members, promoting a high-performance culture.
• Oversee the design, implementation, and maintenance of scalable and reliable data engineering and ML Ops services and patterns on the Azure Databricks platform.
• Ensure delivery of robust and efficient data pipelines and architectures that support our data science teams
• Work with Data and Portfolio project leads to deliver data and AI related projects
• Be accountable for the delivery of data and AI engineering components of transformation and other relevant projects, ensuring timely and high-quality outcomes. Assist with estimations.
• Implement, maintain and evangelise best practices for data engineering, including data governance, security, and compliance
• Minimise consumption and storage costs across our Azure Databricks platform
The role will define, document and enforce data and ML engineering standards. It will have accountability for the performant and efficient build and operation of data components on our Azure Databricks Cloud platform. The role will be responsible for ensuring that we are utilizing the optimal software components on the platform and will own our data ingestion patterns and ML Ops practice.
You will need:
• Experience delivering data engineering solutions in cross functional environments
• Ideally 3+ years leading data engineering teams / ML Ops functions and supporting Data Science teams in cross functional environments
• Deep understanding of Azure services, including Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure Storage, and Azure ML
• Proficiency in managing and optimizing cloud resources for performance and cost-efficiency
• Extensive experience with Databricks, including Spark SQL, Delta Lake, and Databricks clusters
• Experience in deploying, monitoring, and optimizing machine learning models in a production environment
• Proficiency in programming languages such as Python, Scala, and SQL
• Excellent written, oral communication and advocacy skills, with demonstrable experience of prioritising effectively, managing diverse and multiple stakeholders
We know our people are the secret to our success. That's why we're always looking for ways to reward great work. You'll find a host of benefits designed to work for you, including:
- Company Bonus
- Private Medical
- Hybrid Working