Databricks Engineer

We are seeking a highly skilled Senior Databricks Engineer to join our dynamic team. This role is crucial for developing robust, scalable data pipelines and enterprise ETL solutions, leveraging cutting-edge technologies. The ideal candidate will possess a strong background in data engineering, with a focus on delivering high-quality data solutions in a complex environment.

As a Senior Databricks Engineer, you will be instrumental in designing, building, and maintaining our data infrastructure. You will work closely with various stakeholders to understand data requirements and translate them into efficient and effective data engineering solutions. This is an excellent opportunity for an experienced professional to make a significant impact on our data capabilities.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks.
• Implement and optimize enterprise ETL solutions for various data sources.
• Collaborate with cross-functional teams to define data requirements and deliver innovative data solutions.
• Ensure data quality, integrity, and security across all data pipelines and systems.
• Utilize AWS services, specifically AWS Glue, for data integration and processing.
• Implement CI/CD practices for data solutions using Git and GitHub Actions.
• Participate in code reviews and maintain high standards of code quality.

Requirements

• 5-7+ years of experience as a Data Engineer or similar role.
• Strong proficiency in Python, SQL, and Apache Spark.
• Extensive hands-on experience with Databricks platform.
• Proven experience with AWS cloud services, particularly AWS Glue.
• Familiarity with CI/CD pipelines, Git, and GitHub Actions.
• Experience in the financial services sector is a significant plus.
• Knowledge of Delta Lake is highly desirable.

What We Offer

• Opportunity to work on challenging and impactful data engineering projects.
• A collaborative and innovative work environment.
• Exposure to advanced cloud and data technologies.
• Competitive compensation for contract work.
• Hybrid work model offering flexibility.