Responsibilities:

• Design, deploy, and manage Databricks clusters on Google Cloud Platform (GCP), ensuring optimal performance, scalability, and reliability.
• Provide expertise in Databricks architecture, configuration, and administration, following best practices and industry standards.
• Collaborate with data engineering, data science, and analytics teams to understand their requirements and provide technical guidance and solutions.
• Optimize Databricks workloads and processes for performance, cost-efficiency, and scalability.
• Monitor and troubleshoot Databricks clusters, identifying and resolving issues proactively.
• Implement security measures and ensure compliance with data governance and privacy regulations.
• Design and implement data pipelines and workflows, integrating Databricks with other GCP services and tools.
• Stay up to date with the latest advancements in Databricks, GCP, and data engineering technologies, and evaluate their applicability to our environment.
• Mentor junior team members, providing guidance and support in Databricks administration and architecture.

Requirements:

• Minimum of 5 years of experience as a Databricks Admin/Architect, with a strong focus on Google Cloud Platform (GCP).
• Extensive experience in designing, deploying, and managing Databricks clusters, notebooks, jobs, and libraries.
• Deep understanding of Databricks architecture, configuration, and performance optimization techniques.
• Strong knowledge of GCP services, including Compute Engine, BigQuery, Cloud Storage, Dataflow, and Pub/Sub.
• Proficiency in scripting languages like Python or Scala for automation, terraform, and data engineering tasks.
• Experience with data engineering principles and tools, such as ETL, data pipelines, and data modeling.
• Strong troubleshooting and problem-solving skills, with the ability to identify and resolve complex issues in a timely manner.
• Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.
• GCP certifications, such as Google Cloud Certified - Data Engineer or Google Cloud Certified - Professional Cloud Architect, are highly desirable.