Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Big Data Lead

Hirewand
London
7 months ago
Applications closed

Related Jobs

View all jobs

Senior Data Science Manager - Financial Planning and Analysis

Senior Data Science Manager - Financial Planning and Analytics

Data Scientist Graduate (TikTok Shop EMEA DS) - 2026 Start (BS/MS)

Senior Principal Data Scientist

Senior Data Scientist

Data Scientist - Public Sector

Job Type:

Contract

Considering making an application for this job Check all the details in this job description, and then click on Apply.Job Location:

Wimbledon , UKJob Description :For this role, senior experience of Data Engineering and building automated data pipelines on IBM Datastage & DB2, AWS and Databricks from source to operational databases through to curation layer is expected using the latest cloud modern technologies where experience of delivering complex pipelines will be significantly valuable to how to maintain and deliver world class data pipelines.Knowledge in the following areas essential:Databricks:

Expertise in managing and scaling Databricks environments for ETL, data science, and analytics use cases.AWS Cloud:

Extensive experience with AWS services such as S3, Glue, Lambda, RDS, and IAM.IBM Skills:

DB2, Datastage, Tivoli Workload Scheduler, Urban CodeProgramming Languages:

Proficiency in Python, SQL.Data Warehousing & ETL:

Experience with modern ETL frameworks and data warehousing techniques.DevOps & CI/CD:

Familiarity with DevOps practices for data engineering, including infrastructure-as-code (e.g., Terraform, CloudFormation), CI/CD pipelines, and monitoring (e.g., CloudWatch, Datadog).Familiarity with big data technologies like Apache Spark, Hadoop, or similar.ETL/ELT tools and creating common data sets across on-prem (IBMDatastage ETL) and cloud data storesLeadership & Strategy:

Lead Data Engineering team(s) in designing, developing, and maintaining highly scalable and performant data infrastructures.Customer Data Platform Development:

Architect and manage our data platforms using IBM (legacy platform) & Databricks on AWS technologies (e.g., S3, Lambda, Glacier, Glue, EventBridge, RDS) to support real-time and batch data processing needs.Data Governance & Best Practices:

Implement best practices for data governance, security, and data quality across our data platform. Ensure data is well-documented, accessible, and meets compliance standards.Pipeline Automation & Optimisation:

Drive the automation of data pipelines and workflows to improve efficiency and reliability.Team Management:

Mentor and grow a team of data engineers, ensuring alignment with business goals, delivery timelines, and technical standards.Cross Company Collaboration:

Work closely with all levels of business stakeholder including data scientists, finance analysts, MI and cross-functional teams to ensure seamless data access and integration with various tools and systems.Cloud Management:

Lead efforts to integrate and scale cloud data services on AWS, optimising costs and ensuring the resilience of the platform.Performance Monitoring: Establish monitoring and alerting solutions to ensure the high performance and availability of data pipelines and systems to ensure no impact to downstream consumers.

#J-18808-Ljbffr

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs

Why AI Careers in the UK Are Becoming More Multidisciplinary

Artificial intelligence is no longer a single-discipline pursuit. In the UK, employers increasingly want talent that can code and communicate, model and manage risk, experiment and empathise. That shift is reshaping job descriptions, training pathways & career progression. AI is touching regulated sectors, sensitive user journeys & public services — so the work now sits at the crossroads of computer science, law, ethics, psychology, linguistics & design. This isn’t a buzzword-driven change. It’s happening because real systems are deployed in the wild where people have rights, needs, habits & constraints. As models move from lab demos to products that diagnose, advise, detect fraud, personalise education or generate media, teams must align performance with accountability, safety & usability. The UK’s maturing AI ecosystem — from startups to FTSE 100s, consultancies, the public sector & universities — is responding by hiring multidisciplinary teams who can anticipate social impact as confidently as they ship features. Below, we unpack the forces behind this change, spotlight five disciplines now fused with AI roles, show what it means for UK job-seekers & employers, and map practical steps to future-proof your CV.

Jobs

AI Team Structures Explained: Who Does What in a Modern AI Department

Artificial Intelligence (AI) and Machine Learning (ML) are no longer confined to research labs and tech giants. In the UK, organisations from healthcare and finance to retail and logistics are adopting AI to solve problems, automate processes, and create new products. With this growth comes the need for well-structured teams. But what does an AI department actually look like? Who does what? And how do all the moving parts come together to deliver business value? In this guide, we’ll explain modern AI team structures, break down the responsibilities of each role, explore how teams differ in startups versus enterprises, and highlight what UK employers are looking for. Whether you’re an applicant or an employer, this article will help you understand the anatomy of a successful AI department.

Jobs

Why the UK Could Be the World’s Next AI Jobs Hub

Artificial Intelligence (AI) has rapidly moved from research labs into boardrooms, classrooms, hospitals, and homes. It is already reshaping economies and transforming industries at a scale comparable to the industrial revolution or the rise of the internet. Around the world, countries are competing fiercely to lead in AI innovation and reap its economic, social, and strategic benefits. The United Kingdom is uniquely positioned in this race. With a rich heritage in computing, world-class universities, forward-thinking government policy, and a growing ecosystem of startups and enterprises, the UK has many of the elements needed to become the world’s next AI hub. Yet competition is intense, particularly from the United States and China. Success will depend on how effectively the UK can scale its strengths, close its gaps, and seize opportunities in the years ahead. This article explores why the UK could be the world’s next global hub for artificial intelligence, what challenges it must overcome, and what this means for businesses, researchers, and job seekers.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.