Databricks Engineer

We are seeking an experienced Databricks Engineer to join our dynamic team in Plano, TX. This contract position focuses on leveraging Databricks, Apache Spark, and Python to design, develop, and optimize cutting-edge data solutions. The ideal Databricks Engineer will possess a strong background in data engineering principles and be adept at building robust, scalable, and efficient data pipelines within a Lakehouse architecture.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and PySpark.
• Implement and manage ETL/ELT processes to ensure data quality and integrity.
• Work with Delta Lake for efficient data storage and processing, adhering to Lakehouse/Medallion Architecture principles.
• Develop and optimize SQL queries for data extraction, transformation, and loading.
• Perform data modeling to support analytical and reporting requirements.
• Optimize Spark jobs and workflows for performance and cost efficiency.
• Utilize Git for version control and implement CI/CD practices for automated deployments.
• Collaborate with cross-functional teams to understand data needs and deliver effective solutions.

Requirements

• Extensive experience as a Databricks Engineer with a strong understanding of the platform.
• Proficiency in Apache Spark, Python, and PySpark for data manipulation and analysis.
• Solid experience with SQL for data querying and database interactions.
• Demonstrated expertise in ETL/ELT processes and data modeling.
• Familiarity with Delta Lake and Lakehouse/Medallion Architecture concepts.
• Proven ability to perform Spark optimization and manage Databricks Jobs/Workflows.
• Experience with Git and CI/CD pipelines for source control and deployment automation.
• Excellent problem-solving skills and attention to detail.

What We Offer

• An engaging contract opportunity with a leading company.
• The chance to work on challenging and impactful data engineering projects.
• A collaborative and supportive work environment.
• Opportunity to enhance your skills in cutting-edge data technologies.
• Competitive compensation for your expertise.