Databricks Engineer

We are seeking an experienced Databricks ETL Engineer to join our dynamic team for a 12-month contract, with potential for extension. The ideal candidate will possess expert-level proficiency in Databricks and robust experience in ETL development and data integration. This role is crucial for designing, developing, and maintaining scalable data pipelines that support our analytical and operational needs.

As a Databricks ETL Engineer, you will play a pivotal role in transforming raw data into actionable insights, ensuring data quality, and optimizing data workflows. Your expertise will be instrumental in enhancing our data infrastructure and contributing to critical business initiatives.

Key Responsibilities

• Design, develop, and maintain robust ETL processes using Databricks for various data sources.
• Implement and optimize data pipelines using PySpark for efficient data transformation and loading.
• Collaborate with data architects and business stakeholders to understand data requirements and translate them into technical solutions.
• Ensure data quality, integrity, and security across all ETL processes.
• Troubleshoot and resolve data-related issues, optimizing performance bottlenecks in existing data pipelines.
• Contribute to the continuous improvement of data engineering practices and standards.

Requirements

• Expert-level proficiency in Databricks and its associated components.
• Strong hands-on experience with ETL development and data integration techniques.
• Demonstrable strong experience with PySpark for large-scale data processing.
• Excellent SQL skills for data querying, manipulation, and analysis.
• Experience with Informatica is highly advantageous.
• Proven ability to work independently and collaboratively in a fast-paced environment.
• Bachelor's degree in Computer Science, Engineering, or a related field.

What We Offer

• An exciting opportunity to work on cutting-edge data projects.
• Collaborative and supportive team environment.
• Potential for contract extension based on performance and project needs.
• Opportunity to contribute to significant data infrastructure enhancements.