Lead Data Engineer, SC Clearance required, outside IR35, travel once/ twice a month to newcastle 

Role Overview:  
We are seeking a Lead Data Engineer to join our team. In this role, you will be responsible for designing, implementing, and maintaining data pipelines and frameworks that support data-driven decision-making. You will work collaboratively with cross-functional teams to ensure data integrity and accessibility while providing technical leadership to junior engineers. This position is outside IR35.

Responsibilities:  

• Design and develop robust data pipelines using Quantexa and Databricks.  
  
• Collaborate with data scientists and analysts to understand data requirements and provide solutions.  
  
• Ensure the reliability, scalability, and performance of data systems.  
  
• Lead and mentor a team of data engineers, fostering a culture of continuous improvement.  
  
• Manage data integration processes and maintain data quality standards.  
  
• Implement best practises for data governance and security.  
  
• Participate in code reviews and contribute to architectural discussions.  
  
• Maintain up-to-date documentation for data systems and processes.  
  
  Qualifications:  
  
• Proven experience as a Data Engineer, with a strong background in data pipeline development.  
  
• Expertise in Quantexa and Databricks.  
  
• Experience in leading technical teams and projects.  
  
• Strong understanding of data modelling and ETL processes.  
  
• SC clearance is required for this role.  
  
• Excellent problem-solving skills and attention to detail.  
  
• Strong communication and collaboration skills.  
  
• A degree in Computer Science, Engineering, or a related field is preferred