Azure Databricks Engineer

• Database

• Remote

• Permanent / Full Time

About the job:
Title – Azure Databricks Engineer
Start date: Immediate
Position Type: Full Time
Location: Remote across India
Work Time: IST with some overlap with US hours
  
Job Overview:
We are seeking a highly skilled and motivated Azure Databricks Engineer to join our dynamic team. The ideal candidate will have hands-on experience in Azure cloud services, Databricks platform, and data engineering best practices. This role will involve designing, building, and optimizing data pipelines and analytics solutions in a cloud-native environment, ensuring high availability, performance, and scalability for data-driven applications.
  
Key Responsibilities:
• Develop and implement scalable data pipelines and data engineering solutions using Azure Databricks, Azure Data Lake, and Azure Synapse Analytics.
• Optimize performance and ensure data pipelines are efficient and cost-effective by leveraging best practices for Databricks clusters, Spark, and Azure services.
• Utilize current Databricks expertise, Azure Synapse Analytics, and Azure skills to develop and optimize data solutions.
• Oversee the ingestion and migration of SQL, Hadoop, and Epic data to Databricks on Azure, prioritizing data migration of key datasets.
• Design and implement data storage solutions using Azure Blob Storage, Azure Data Lake, and Azure SQL Database.
• Apply strong SQL skills and experience in data modeling and ETL processes to ensure efficient data handling.
• Leverage experience with big data technologies and frameworks, such as Spark and Hadoop, to manage and analyze large datasets effectively
  
Skills:
• Expertise in Azure Databricks setup, management, and optimization.
• Strong understanding of Cloud architecture and big data solutions on Azure.
• Familiarity with DevOps practices and tools like Git, Jenkins, and Terraform.