Databricks Architect

Droisys is actively seeking a highly experienced and visionary Databricks Architect to join our dynamic team in Atlanta, GA. This critical role involves leading the design and implementation of robust, scalable data solutions leveraging the full power of Databricks and its ecosystem. As a Databricks Architect, you will play a pivotal role in shaping our data strategy, driving innovation, and ensuring optimal performance of our data platforms.

We are looking for a seasoned professional with deep expertise in large-scale data architecture, performance optimization, and building advanced analytical solutions. This is an excellent opportunity for a hands-on architect to make a significant impact on complex data challenges for a 12-month contract duration.

Key Responsibilities

• Lead the architecture, design, and implementation of highly scalable and efficient data solutions on the Databricks platform.
• Design, develop, and optimize complex data pipelines using Apache Spark, PySpark, and Delta Lake to support various business needs.
• Provide technical leadership and mentorship to data engineering teams, ensuring best practices in data architecture and development are followed.
• Evaluate and recommend new data technologies and tools to enhance our data platform capabilities and drive innovation.
• Collaborate closely with data scientists, business analysts, and other stakeholders to understand data requirements and translate them into technical solutions.
• Ensure data quality, governance, and security standards are met across all data initiatives and platforms.
• Perform performance tuning and optimization of Databricks clusters, Spark jobs, and data queries to improve efficiency and reduce costs.
• Develop and maintain comprehensive documentation for data architectures, designs, and operational procedures.

Requirements

• Minimum of 12 years of extensive experience in data architecture, engineering, and platform development.
• Proven expert-level proficiency with Databricks, including significant experience in designing and implementing solutions.
• Deep expertise in Apache Spark and PySpark for large-scale data processing and transformation.
• Strong understanding and practical experience with Delta Lake for building reliable and performant data lakes.
• Demonstrated experience in data architecture principles, data modeling, ETL/ELT processes, and data pipeline optimization.
• Proficiency in cloud platforms such as AWS, Azure, or GCP, with a focus on data services.
• Excellent command of SQL and Python for data manipulation and scripting.
• Ability to work onsite in Atlanta, GA, for the full contract duration.
• Strong analytical, problem-solving, and communication skills to articulate complex technical concepts to diverse audiences.

What We Offer

• Opportunity to work on challenging and impactful data architecture projects within a leading-edge technology environment.
• A collaborative and innovative team culture focused on continuous learning and professional growth.
• Exposure to diverse industries and complex data landscapes, expanding your expertise as a Databricks Architect.
• A chance to drive significant architectural decisions and contribute to the strategic direction of data initiatives.
• Professional development opportunities to stay at the forefront of data and cloud technologies.