Azure Synapse Engineer

Catalyst IT is actively seeking a highly skilled and experienced Azure Synapse Engineer to join our dynamic team in Pune. This is a pivotal role for an expert with a minimum of 5 years of dedicated experience in designing, developing, and optimizing data solutions within the Microsoft Azure ecosystem. If you are passionate about leveraging cutting-edge cloud data technologies to build robust, scalable, and cost-efficient data platforms, we encourage you to apply. As an Azure Synapse Engineer, you will play a crucial role in our data initiatives, driving innovation and ensuring optimal performance across our data warehousing and analytics solutions.

At Catalyst IT, we pride ourselves on connecting top talent with challenging opportunities. This position is ideal for immediate joiners who are eager to make a significant impact and contribute to high-visibility projects. You will be instrumental in transforming raw data into actionable insights, working with a suite of powerful Azure services to deliver measurable business value.

Key Responsibilities

• Architect, build, and optimize complex ETL/ELT pipelines for efficient data ingestion and transformation, ensuring data integrity and availability.
• Leverage extensive expertise in Azure Synapse Analytics and Azure Data Factory to design, implement, and manage enterprise-grade data warehousing solutions.
• Develop and implement robust incremental data processing strategies to enhance performance and reduce resource consumption across data pipelines.
• Design and develop high-performance data solutions utilizing Spark and PySpark for advanced data processing, analytics, and machine learning workloads.
• Conduct thorough performance tuning, including DWU right-sizing, to ensure the optimal and cost-effective operation of Azure Synapse environments.
• Proactively identify and implement Azure cost optimization strategies across data platforms, ensuring efficient resource utilization without compromising performance.
• Manage and optimize data storage solutions, primarily working with Azure Data Lake Storage (ADLS Gen2) to handle large volumes of diverse data.
• Collaborate with cross-functional teams, including data scientists and business analysts, to understand data requirements and deliver impactful solutions.

Requirements

• Minimum of 5 years of hands-on professional experience specifically as an Azure Synapse Engineer, Data Engineer, or in a similar role with a strong focus on Azure data services.
• Demonstrated expert-level proficiency with Azure Synapse Analytics, including dedicated SQL pools, serverless SQL pools, and Spark pools.
• Extensive experience with Azure Data Factory for orchestrating and automating data pipelines.
• Solid understanding and practical experience with Azure Data Lake Storage (ADLS Gen2) for scalable and secure data storage.
• Proficiency in developing and optimizing data solutions using Spark and PySpark.
• Proven track record in designing, building, and optimizing ETL/ELT processes and data warehousing solutions.
• Experience with performance tuning, capacity planning (DWU right-sizing), and implementing cost optimization strategies within Azure environments.
• Strong analytical and problem-solving skills, with an ability to troubleshoot complex data-related issues effectively.
• Excellent communication and collaboration skills to work effectively within a team and with stakeholders.

What We Offer

• An exciting opportunity to work with cutting-edge Microsoft Azure data technologies and contribute to high-impact projects.
• A collaborative, supportive, and innovative work environment where your contributions are valued.
• Continuous learning and professional development opportunities to advance your skills and career.
• Competitive compensation package and benefits.
• A chance to be part of a forward-thinking team dedicated to excellence in data engineering.