Data Engineer - Databricks

Nasstar
1 month ago
Applications closed

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer - Azure or Snowflake

Data Engineer - Machine Learning - Hybrid

Data Engineer - Python & Azure

We are Colibri Digital

Born in the cloud, Colibri Digital is one of the fastest-growing Cloud Consultancies in the UK with deep expertise in areas such as big data, data science, machine learning, and cloud computing. We are Premier Tier partners with AWS and our recent attainment of Gold competency with Microsoft proves our excellence in evolving cloud and data technologies. Led by industry-recognised technologists we are a dynamic team with a proven track record of delivering masterclass cloud solutions across all industry verticals.

Location:Home-based (UK Only)

Job Summary

We are seeking a number of highly skilled Data Engineers to join our team on a contract basis and play a pivotal role in designing, implementing, and managing new data technology infrastructure. You will be responsible for building and maintaining a scalable and robust cloud-based platform that consolidates existing legacy data services. The platform will enable efficient data management, automation of processes, and provide high-quality data services to our users and tooling.

Key Responsibilities

  • Design, develop, and implement data pipelines and ETL processes to ingest,
  • transform, and load data into the data platform.
  • Collaborate with the Lead Architect and Cloud Engineer to ensure the seamless
  • integration of data services within the cloud infrastructure.
  • Develop data models and schemas to support efficient data storage and retrieval.
  • Implement data quality and validation processes to ensure the accuracy and
  • consistency of data.
  • Optimize data processes and queries for performance and scalability.
  • Collaborate with business stakeholders to understand their data requirements
  • and provide insights and solutions for data-driven decision-making.
  • Work closely with data scientists to provide them with the necessary data and
  • tooling for developing revenue-generating insights and models.
  • Implement data governance and security controls to ensure data privacy and
  • compliance with regulations.
  • Stay updated with the latest data technologies and trends and make
  • recommendations for technology adoption and improvements.

Requirements

  • Proven experience as a Data Engineer, working with Azure data services.
  • Databricks experience is essential for this role.
  • Strong understanding of data modelling concepts and relational and
  • non-relational databases.
  • Proficiency in SQL and experience with data integration and ETL tools (e.g.,
  • Azure Data Factory, Azure Databricks).
  • Experience with data warehousing concepts and technologies (e.g., Azure
  • Synapse Analytics, Azure Data Lake Storage).
  • Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a
  • plus.
  • Knowledge of data governance and data security best practices.
  • Strong problem-solving skills and attention to detail.
  • Excellent communication and collaboration skills to work effectively within a team
  • environment.
  • Experience with Agile methodologies is a plus.

We welcome applications even if you don't think you match 100% of the role selection criteria

Get the latest insights and jobs direct. Sign up for our newsletter.

By subscribing you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Careers

10 Ways AI Pros Stay Inspired: Boost Creativity with Side Projects, Hackathons & More

In the rapidly evolving world of Artificial Intelligence (AI), creativity and innovation are critical. AI professionals—whether data scientists, machine learning engineers, or research scientists—must constantly rejuvenate their thinking to solve complex challenges. But how exactly do these experts stay energised and creative in their work? The answer often lies in a combination of strategic habits, side projects, hackathons, Kaggle competitions, reading the latest research, and consciously stepping out of comfort zones. This article will explore why these activities are so valuable, as well as provide actionable tips for anyone looking to spark new ideas and enrich their AI career. Below, we’ll delve into tried-and-tested strategies that AI pros employ to drive innovation, foster creativity, and maintain an inspired outlook in an industry that can be both exhilarating and daunting. Whether you’re just starting your AI journey or you’re an experienced professional aiming to sharpen your skills, these insights will help you break out of ruts, discover fresh perspectives, and bring your boldest ideas to life.

Careers

Top 10 AI Career Myths Debunked: Key Facts for Aspiring Professionals

Artificial Intelligence (AI) is one of the most dynamic and rapidly growing sectors in technology today. The lure of AI-related roles continues to draw a diverse range of job seekers—from seasoned software engineers to recent graduates in fields such as mathematics, physics, or data science. Yet, despite AI’s growing prominence and accessibility, there remains a dizzying array of myths surrounding careers in this field. From ideas about requiring near-superhuman technical prowess to assumptions that machines themselves will replace these jobs, the stories we hear sometimes do more harm than good. In reality, the AI job market offers far more opportunities than the alarmist headlines and misconceptions might suggest. Here at ArtificialIntelligenceJobs.co.uk, we witness firsthand the myriad roles, backgrounds, and success stories that drive the industry forward. In this blog post, we aim to separate fact from fiction—taking the most pervasive myths about AI careers and debunking them with clear, evidence-based insights. Whether you are an established professional considering a career pivot into data science, or a student uncertain about whether AI is the right path, this article will help you gain a realistic perspective on what AI careers entail. Let’s uncover the truth behind the most common myths and discover the actual opportunities and realities you can expect in this vibrant sector.

Jobs

Global vs. Local: Comparing the UK AI Job Market to International Landscapes

How to navigate salaries, opportunities, and work culture in AI across the UK, the US, Europe, and Asia Artificial Intelligence (AI) has evolved from a niche field of research to an integral component of modern industries—powering everything from chatbots and driverless cars to sophisticated data analytics in finance and healthcare. The job market for AI professionals is consequently booming, with thousands of new positions posted each month worldwide. In this blog post, we will explore how the UK’s AI job market compares to that of the United States, Europe, and Asia, delving into differences in job demand, salaries, and workplace culture. Additionally, we will provide insights for candidates considering remote or international opportunities. Whether you are a freshly qualified graduate in data science, an experienced machine learning engineer, or a professional from a parallel domain looking to transition into AI, understanding the global vs. local landscape can help you make an informed decision about your career trajectory. As the demand for artificial intelligence skills grows—and borders become more porous with hybrid and remote work—the possibilities for ambitious job-seekers are expanding exponentially. This article will offer a comprehensive look at the various regional markets, exploring how the UK fares in comparison to other major AI hubs. We’ll also suggest factors to consider when choosing where in the world to work, whether physically or remotely. By the end, you’ll have a clearer picture of the AI employment landscape, and you’ll be better prepared to carve out your own path.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.