Databricks Engineer

Addsource is actively seeking a highly skilled and experienced Senior Databricks Engineer to join our dynamic team. This critical role is perfect for a data engineering professional with a robust background in building and optimizing scalable data pipelines within the Databricks ecosystem. As a Senior Databricks Engineer, you will be instrumental in processing large datasets, implementing robust data transformation solutions, and ensuring the highest standards of data engineering best practices.

Key Responsibilities

• Design, develop, and optimize high-performance, scalable data pipelines using Databricks and Apache Spark.
• Build and maintain efficient ETL/ELT workflows leveraging Python, PySpark, and Scala to process vast amounts of data.
• Conduct thorough Spark job optimization, debugging, and performance tuning to ensure maximum efficiency and reliability.
• Implement robust data transformation solutions and ensure data quality and integrity across all pipelines.
• Collaborate with cross-functional teams to understand data requirements and deliver innovative data solutions.
• Adhere strictly to established coding standards and data engineering best practices throughout the development lifecycle.
• Participate in code reviews and provide constructive feedback to peers, fostering a culture of continuous improvement.
• Stay updated with the latest advancements in Databricks, Apache Spark, and general data engineering technologies.

Requirements

• Proven experience as a Senior Databricks Engineer with a strong portfolio of successfully delivered data engineering projects.
• Exceptional hands-on coding proficiency in Python, PySpark, and Scala, demonstrated through practical application.
• In-depth expertise in Databricks and Apache Spark, including architecture, optimization, and deployment best practices.
• Solid understanding of distributed computing principles and experience processing large, complex datasets.
• Familiarity with various data warehousing concepts, data modeling techniques, and ETL/ELT methodologies.
• Strong problem-solving abilities and a meticulous approach to debugging and performance tuning data pipelines.
• Candidates must be willing and able to work onsite three days per week at one of our specified locations in Edina, MN, Chicago, IL, or Irving, TX.
• A successful completion of a technical coding test is a mandatory part of our interview process, assessing practical coding skills in the required technologies.

What We Offer

• An opportunity to work on challenging and impactful data engineering projects with cutting-edge technologies.
• A collaborative and supportive work environment where innovation and continuous learning are highly encouraged.
• Competitive compensation for a contract-to-contract (C2C) engagement.
• The chance to be part of a growing team, contributing directly to the company's data strategy and success.
• Exposure to diverse industries and complex data landscapes, enhancing your professional growth.

If you are a highly motivated and technically proficient Senior Databricks Engineer ready to tackle complex data challenges, we encourage you to apply. We are looking for individuals who are passionate about data, committed to quality, and eager to make a significant impact. Join Addsource and help us build the future of data-driven solutions.