Data Engineer with Data Formats

Capgemini
London
1 year ago
Applications closed

Related Jobs

View all jobs

MLOps Tech Lead

MLOps Tech Lead

Climate Data Scientist

Artificial Intelligence / Machine Learning Engineer - 12 Month Fixed Term Contract

Artificial Intelligence / Machine Learning Engineer - 12 Month Fixed Term Contract

Artificial Intelligence / Machine Learning Engineer - 12 Month Fixed Term Contract

Get The Future You Want!

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.

Your Role

We are seeking a highly skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have a deep understanding of data engineering principles, data technologies, and a proven track record of designing and building complex data pipelines. This role requires strong expertise in SQL, various data formats, Python, and JavaScript to support our data-driven decision-making processes and enhance our data infrastructure.
• Architect and maintain scalable data pipelines using various programming technologies.
• Use SQL to query, transform, and process data across relational and NoSQL databases.
• Integrate data from APIs, flat files, and streaming sources for consistency and quality.
• Implement real-time data processing using Kafka or Solace.
• Manage data storage in systems and warehouses, optimizing for performance.
• Design data models and apply techniques like partitioning and indexing for efficiency.
• Handle multiple data formats (CSV, JSON, Parquet) and manage unstructured data.
• Utilize Python and JavaScript (Node.js) for data processing, automation, and ETL development.
• Leverage Microsoft technologies, Apache Spark, and Airflow for distributed computing.
• Implement DevOps tools (Jenkins, Git, Docker) for CI/CD and monitor pipeline performance.
 

Your Profile

• Bachelor’s or Master’s degree in Computer Science, Data Science, IT, or related field.
• 8+ years of experience in data engineering, architecture, or related roles.
• Proven track record in building and maintaining large-scale, complex data pipelines.
• Expertise in managing data infrastructure in high-volume environments.
• Strong foundation in designing and optimizing data systems and workflows.
 

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world while creating tangible impact for enterprises and society. It is a responsible and diverse group of 350,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market-leading capabilities in AI, cloud, and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion.

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

How to Write an AI Job Ad That Attracts the Right People

Artificial intelligence is now embedded across almost every sector of the UK economy. From fintech and healthcare to retail, defence and climate tech, organisations are competing for AI talent at an unprecedented pace. Yet despite the volume of AI job adverts online, many employers struggle to attract the right candidates. Roles are flooded with unsuitable applications, while highly capable AI professionals scroll past adverts that feel vague, inflated or disconnected from reality. In most cases, the issue isn’t a shortage of AI talent — it’s the quality of the job advert. Writing an effective AI job ad requires more care than traditional tech hiring. AI professionals are analytical, sceptical of hype and highly selective about where they apply. A poorly written advert doesn’t just fail to convert — it actively damages your credibility. This guide explains how to write an AI job ad that attracts the right people, filters out mismatches and positions your organisation as a serious employer in the AI space.

Maths for AI Jobs: The Only Topics You Actually Need (& How to Learn Them)

If you are a software engineer, data scientist or analyst looking to move into AI or you are a UK undergraduate or postgraduate in computer science, maths, engineering or a related subject applying for AI roles, the maths can feel like the biggest barrier. Job descriptions say “strong maths” or “solid fundamentals” but rarely spell out what that means day to day. The good news is you do not need a full maths degree worth of theory to start applying. For most UK roles like Machine Learning Engineer, AI Engineer, Data Scientist, Applied Scientist, NLP Engineer or Computer Vision Engineer, the maths you actually use again & again is concentrated in a handful of topics: Linear algebra essentials Probability & statistics for uncertainty & evaluation Calculus essentials for gradients & backprop Optimisation basics for training & tuning A small amount of discrete maths for practical reasoning This guide turns vague requirements into a clear checklist, a 6-week learning plan & portfolio projects that prove you can translate maths into working code.

Neurodiversity in AI Careers: Turning Different Thinking into a Superpower

The AI industry moves quickly, breaks rules & rewards people who see the world differently. That makes it a natural home for many neurodivergent people – including those with ADHD, autism & dyslexia. If you’re neurodivergent & considering a career in artificial intelligence, you might have been told your brain is “too much”, “too scattered” or “too different” for a technical field. In reality, many of the strengths that come with ADHD, autism & dyslexia map beautifully onto AI work – from spotting patterns in data to creative problem-solving & deep focus. This guide is written for AI job seekers in the UK. We’ll explore: What neurodiversity means in an AI context How ADHD, autism & dyslexia strengths match specific AI roles Practical workplace adjustments you can ask for under UK law How to talk about your neurodivergence during applications & interviews By the end, you’ll have a clearer picture of where you might thrive in AI – & how to set yourself up for success.