Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Data Engineer

DiverseJobsMatter
Horsham
9 months ago
Applications closed

Related Jobs

View all jobs

Lead R Engineer / Data Scientist - Integrated Pest Management (IPM)

Data Scientist

Lead R Data Scientist - Sustainability

Data Science Engineer

Staff Data Scientist

Artificial Intelligence

Are you passionate about designing impactful data-driven solutions and creating cutting-edge digital experiences?

We’re seeking a Principal Data and Digital Engineer to lead the delivery of transformative solutions for our clients. You will lead the delivery of tailored solutions to our clients where you will be responsible for the design, delivery and implementation solutions, spanning from end user interfaces and innovative user experience to data driven solutions.

Sitting in the consulting division, you will be one of the drivers of our new capability framework and will lead the implementation of data and digital solutions of varying size and complexity, across several technical domains, including Analytics and Visualisation, Data Engineering and pipelining, UI and UX, Digital Twins, Simulation and/or Asset and Data Management.

In addition to the relevant technical experience in the different domains identified, you will have a depth of experience in different software engineering languages and commercial off the shelf tools across cloud and on-prem settings where you will be expected to build impactful Technical Solutions, Concept Demonstrators and Minimum Value Propositions.

Job Responsibilities

  • Build, optimise, and maintain scalable data pipelines that handle high-volume, high-velocity data in complex SaaS environments.
  • Design efficient, scalable, and flexible data models in multiple technologies to meet business requirements.
  • Analyse and consolidate business requirements and use cases to meet technical and non-technical debt.
  • Design tailored and bespoke solutions to address and mitigate challenges faced by our clients.
  • Make key decisions on build vs buy for data and digital engineering solutions and platforms.
  • Work closely with the entire business far beyond just product or technology, including driving innovation through concept demonstrators and offerings and support the sales activities, from a pre-sales perspective.
  • Carry out horizon scanning to identify emerging technology and industry trends and assess their potential impact and opportunity to become our offerings to the market.
  • Ensure solution, hosting and service architectures are subject to robust cost and effort estimation, covering design, development, transition, operation, maintenance and exit costs.

Skills Required

Data Engineering

  • Fluency in one or more of the following languages: SQL, Scala, Python, R, PySpark, Elasticsearch.
  • Data Pipeline Development: Hands-on experience in building data pipelines in AWS, Azure and/or open-source frameworks like Spark, Beam, or Airflow.
  • Experience with real time data streaming frameworks, (e.g. Kafka, Pubsub etc.).
  • Data quality/validation experience: designing metrics and KPIs to verify and automated management of data quality – experience in Purview would be very welcome.
  • Build and optimise data models, ensuring data is structured, accessible, and performant for downstream analytics and machine learning models
  • Good working experience of AWS (e.g. S3, Kinesis, Glue, Redshift, Lambda and EMR) or Azure data services (e.g. ADF, Synapse, Fabric, Azure Functions)
  • Knowledge and experience of Palantir Foundry or Gotham is a major plus.
  • Working experience in Data Lakehouse environments such as Databricks, Snowflake and/or Microsoft Fabric is a major plus.

Digital and Software Engineering

  • Experience in developing software solutions in an agile fashion employing best practices (e.g. automated testing, version control, CI/CD, containerisation, etc.).
  • Experience in designing wireframes in Figma and development of web front ends in React and/or Angular.
  • Familiarity with user testing and research methodologies.
  • Strong understanding of HTML, CSS, JavaScript, and front-end frameworks.
  • Experience in Agile methodologies (Scrum / Kanban).
  • Experience in developing Infrastructure as a Code (e.g. Terraform) is an advantage.

Core

  • Working knowledge of 1 cloud vendor (AWS, Azure) – ideally with relevant engineering certification.
  • Strong problem-solving skills, attention to detail and passion for creating business value through data.
  • Strong communication skills, in particular the ability to translate technical concepts to non-technical stakeholders.
  • Value collaboration and have experience working effectively with remote teams.
  • Willingness to mentor others and support them in achieving their goals.
  • Comfortable working in a lean, agile environment, with the ability to adapt to changing priorities and technical domains with an enthusiastic and proactive attitude.

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Why the UK Could Be the World’s Next AI Jobs Hub

Artificial Intelligence (AI) has rapidly moved from research labs into boardrooms, classrooms, hospitals, and homes. It is already reshaping economies and transforming industries at a scale comparable to the industrial revolution or the rise of the internet. Around the world, countries are competing fiercely to lead in AI innovation and reap its economic, social, and strategic benefits. The United Kingdom is uniquely positioned in this race. With a rich heritage in computing, world-class universities, forward-thinking government policy, and a growing ecosystem of startups and enterprises, the UK has many of the elements needed to become the world’s next AI hub. Yet competition is intense, particularly from the United States and China. Success will depend on how effectively the UK can scale its strengths, close its gaps, and seize opportunities in the years ahead. This article explores why the UK could be the world’s next global hub for artificial intelligence, what challenges it must overcome, and what this means for businesses, researchers, and job seekers.

The Best Free Tools & Platforms to Practise AI Skills in 2025/26

Artificial Intelligence (AI) is one of the fastest-growing career fields in the UK and worldwide. Whether you are a student exploring AI for the first time, a graduate looking to build your portfolio, or an experienced professional upskilling for career growth, having access to free tools and platforms to practise AI skills can make a huge difference. In this comprehensive guide, we’ll explore the best free resources available in 2025, covering AI coding platforms, datasets, cloud tools, no-code AI platforms, online communities, and learning hubs. These tools allow you to practise everything from machine learning models and natural language processing (NLP) to computer vision, reinforcement learning, and large language model (LLM) fine-tuning—without needing a huge budget. By the end of this article, you’ll have a clear roadmap of where to start practising your AI skills for free, how to build real-world projects, and which platforms can help you land your next AI job.

Top 10 Skills in Artificial Intelligence According to LinkedIn & Indeed Job Postings

Artificial intelligence is no longer a niche field reserved for research labs or tech giants—it has become a cornerstone of business strategy across the UK. From finance and healthcare to manufacturing and retail, employers are rapidly expanding their AI teams and competing for talent. But here’s the challenge: AI is evolving so quickly that the skills in demand today may look different from those of just a few years ago. Whether you’re a graduate looking to enter the industry, a mid-career professional pivoting into AI, or an experienced engineer wanting to stay ahead, it’s essential to know what employers are actually asking for in their job ads. That’s where platforms like LinkedIn and Indeed provide valuable insight. By analysing thousands of job postings across the UK, they reveal the most frequently requested skills and emerging trends. This article distils those findings into the Top 10 AI skills employers are prioritising in 2025—and shows you how to present them effectively on your CV, in interviews, and in your portfolio.