Lead Data Engineer

We are seeking an experienced and highly skilled Lead Data Engineer to join our dynamic team. This pivotal role involves designing, developing, and maintaining robust and scalable data solutions, leveraging cutting-edge Azure technologies. As a Lead Data Engineer, you will be instrumental in shaping our data architecture and ensuring optimal data pipelines for analytical and operational needs.

Key Responsibilities

• Architect, develop, and optimize data pipelines using Azure Data Factory and Azure Databricks.
• Implement and manage data storage solutions with ADLS Gen2 and Delta Lake.
• Write and optimize PySpark code for data transformation and processing.
• Design and maintain complex SQL queries for data extraction and manipulation.
• Ensure data quality, integrity, and security across all data platforms.
• Collaborate with cross-functional teams to understand data requirements and deliver impactful solutions.
• Mentor junior engineers and contribute to best practices in data engineering.

Requirements

• Extensive experience with Azure Data Factory and Azure Databricks.
• Proficiency in PySpark for large-scale data processing.
• Strong command of SQL for database management and querying.
• Hands-on experience with ADLS Gen2 and Delta Lake.
• Proven ability to lead data engineering projects and initiatives.
• Excellent problem-solving skills and attention to detail.

What We Offer

• Opportunity to work on challenging and innovative data projects.
• Collaborative and supportive team environment.
• Exposure to advanced cloud data technologies.
• Competitive compensation package.