Jobs

Senior Data Engineer (Synapse/Databricks),12m FTC Fully Remote


Job details
  • Birmingham
  • 2 months ago
Applications closed

Leading UK Logistics Company require a Senior Data Engineer to play a pivotal role in leading the design, development and maintenance of robust data pipelines and infrastructure within their Microsoft Azure environment. Your expertise will ensure the availability, reliability and scalability of their data platform, empowering the organisation with timely and accurate information to recognise the value of their data.

Client Details

Leading UK Logistics Company

Description

Leading UK Logistics Company require a Senior Data Engineer to play a pivotal role in leading the design, development and maintenance of robust data pipelines and infrastructure within their Microsoft Azure environment. Your expertise will ensure the availability, reliability and scalability of their data platform, empowering the organisation with timely and accurate information to recognise the value of their data.

You will collaborate with the Data Science and Analytics teams to implement data models, create data integration pipelines, and optimise data workflows. This role requires a strong background in data engineering, with a solid understanding of data management principles and techniques. You will support the development of junior data engineers, providing guidance and mentorship to help them build their skills.

This is a 12 month Fixed Term Contract / Fully remote position.

Key responsibilities:

Lead the support of the Strategic Data Platform (SDP), ensuring it is robust, reliable, and cost-effective in providing high-quality data.
Design, develop, and uphold data integration processes, including the creation and execution of scalable and effective data pipelines for managing and transforming large datasets.
Optimise data storage and retrieval processes to meet performance and scalability requirements.
Ensure data quality and consistency by implementing data validation and cleansing techniques.
Collaborate with stakeholders to define requirements and integrate new data sources.
Monitor and troubleshoot data pipeline issues, identify and resolve bottlenecks, and implement performance optimisations.
Collaborate with the Business Intelligence (BI) team to define and implement data models for analysis and reporting purposes.
Maintain best practices, governance, and security in data projects.
Stay up-to-date with the latest trends and technologies in the field of data engineering, and make recommendations for improvement.
Provide guidance and mentorship to junior data engineers, supporting their development and skill-building; to ensure they become self-sufficient.
Lead the technical aspects of our data strategy, ensuring alignment with organisational goals and objectives.

Experience, Skills, and Attributes:

Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent experience).
Strong programming skills in Python, Java, or Scala, with proficiency in database management systems and excellent SQL skills.
Proven experience in designing and implementing data pipelines, ETL processes, data warehousing, and scaled data consumption patterns.
Knowledge of data integration, data modelling concepts, and familiarity with cloud data platforms and storage technologies, ideally within Azure.
Proficient in Azure data services (e.g., Synapse Analytics, SQL, Data Factory, Data Lake, Databricks, Cosmos DB) - Synapse and Databricks is essential
Experience in Agile development, branch-based source control/DevOps concepts, and gathering and analysing system requirements.
Excellent problem-solving, troubleshooting, communication, and collaboration skills.
Attention to detail and a commitment to delivering high-quality work.
Azure Data Engineer Associate/DP-203 certification or equivalent practical experience preferred.
Aptitude for learning new technologies and analytical methodologies, with experience in data visualisation tools (e.g., Power BI) and familiarity with machine learning/statistical analysis techniques being a plus.
Demonstrated ability to lead and mentor junior team members, fostering a collaborative and supportive work environment.Profile

Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent experience).
Strong programming skills in Python, Java, or Scala, with proficiency in database management systems and excellent SQL skills.
Proven experience in designing and implementing data pipelines, ETL processes, data warehousing, and scaled data consumption patterns.
Knowledge of data integration, data modelling concepts, and familiarity with cloud data platforms and storage technologies, ideally within Azure.
Proficient in Azure data services (e.g., Synapse Analytics, SQL, Data Factory, Data Lake, Databricks, Cosmos DB) - Synapse and Databricks is essential
Experience in Agile development, branch-based source control/DevOps concepts, and gathering and analysing system requirements.
Excellent problem-solving, troubleshooting, communication, and collaboration skills.
Attention to detail and a commitment to delivering high-quality work.
Azure Data Engineer Associate/DP-203 certification or equivalent practical experience preferred.
Aptitude for learning new technologies and analytical methodologies, with experience in data visualisation tools (e.g., Power BI) and familiarity with machine learning/statistical analysis techniques being a plus.
Demonstrated ability to lead and mentor junior team members, fostering a collaborative and supportive work environment.Job Offer

Opportunity to work in a complex data environment

Opportunity to work on a large scale Data & Digital modernisation programme

Sign up for our newsletter

The latest news, articles, and resources, sent to your inbox weekly.

Similar Jobs

Senior Data Engineer (Equity Only)

Job Title: Senior Data EngineerAbout Luupli:Luupli is a social media app that has equity, diversity, and equality at its heart. We believe that social media can be a force for good, and we are committed to creating a platform that maximises the value that creators and businesses can gain from...

Luupli Newcastle upon Tyne

Senior Data Engineer (Equity Only)

Job Title: Senior Data EngineerAbout Luupli:Luupli is a social media app that has equity, diversity, and equality at its heart. We believe that social media can be a force for good, and we are committed to creating a platform that maximises the value that creators and businesses can gain from...

Luupli Manchester

Senior Data Engineer (Remote) – UK Software Engineering London

At AlphaSights, we search through more than 500 million professionals working in the world today to find the small handful of experts qualified to answer our clients’ needs. They use these insights to drive amazing progress within their organisations. Our mission is to provide access to dispersed, hidden, and underutilised...

Alphasights London

Senior Data Engineer

As a Data Engineer at Dexters, you will play a vital role in developing and managing Dexters’ data integration projects, applying your expertise to seamlessly transition data from legacy systems into modern infrastructure, reporting and analytics.Salary: £60,000-£65,000 DOEHours: Monday-Friday 8.30am-5.30pmLocation:London hubs with flexibility regarding working from home.Reporting to:BI & Data...

Dexters Estate Agent Group

Senior Data Engineer

​Job reference 322235Senior Data EngineerCompetitive salary plus car allowance, healthcar, annual bonus potential,Permanent, Full time Royal Mail Farringdon3x days a week in London Office (Farringdon)Extra Benefits•Family friendly support- enhanced maternity pay, paternity leave, adoption leave and shared parental leave• Discounts and offers- there are more than 800 offers to help...

Royal Mail London

Senior Data Visualisation Engineer

​Job reference 322104Senior Data Visualisation EngineerCompetitive salary Permanent, Full time Future Walk3x days a week in London or Chesterfield OfficeExtra Benefits•Family friendly support- enhanced maternity pay, paternity leave, adoption leave and shared parental leave• Discounts and offers- there are more than 800 offers to help you save on things like...

Royal Mail Chesterfield