Senior Data Engineer

Avance Services is seeking an experienced and highly skilled Senior Data Engineer to join our dynamic team for a 6-12 month contractual engagement. This hybrid role, based in India, offers a unique opportunity to design, build, and optimize critical data infrastructure for manufacturing analytics. As a Senior Data Engineer, you will be instrumental in developing robust data solutions that drive operational efficiency and informed decision-making for our clients.

We are looking for a professional with a strong background in data engineering, particularly within manufacturing environments, who can lead the development of scalable and reliable data pipelines. This role involves working with cutting-edge technologies and collaborating closely with cross-functional teams to translate complex business requirements into high-impact data solutions. If you are passionate about data, possess deep technical expertise, and thrive in a challenging, project-oriented environment, we encourage you to apply.

Key Responsibilities

• Design, develop, and maintain production-grade data pipelines for extracting, transforming, and loading critical manufacturing data, including OEE metrics, Connected Worker events, and Golden Batch analytical data.
• Architect and implement robust data models specifically tailored for manufacturing analytics, encompassing batch records, equipment telemetry, and worker activity data.
• Develop and deploy both real-time streaming and batch data processing solutions to handle high-volume operational data, such as Trio Line events.
• Build, optimize, and manage scalable data lake and data warehouse structures designed to support GMSx analytical use cases and performance requirements.
• Establish and enforce comprehensive data quality frameworks, including validation rules, anomaly detection, and data integrity checks, to ensure the reliability of manufacturing data.
• Implement advanced data governance capabilities, including data lineage tracking, metadata management, and audit trail features, to comply with stringent GxP requirements.
• Continuously monitor and optimize data pipeline performance to meet demanding production-grade throughput and low-latency requirements.
• Collaborate effectively with Sr. Techno-Functional Consultants and business stakeholders to align data architecture and engineering efforts with evolving business requirements and use case definitions.

Requirements

• Minimum of 5 years of hands-on experience in data engineering, with strong proficiency in Apache Spark, Python, and SQL.
• Proven experience with real-time data streaming technologies such as Kafka Streams, Spark Streaming, or other equivalent cloud-native streaming services.
• Demonstrable expertise with major cloud data platforms, specifically Azure Data Factory, Azure Databricks, or comparable cloud-native data orchestration and processing services.
• Solid understanding of manufacturing data models, time-series data, and patterns associated with equipment telemetry.
• Familiarity with data governance frameworks, data lineage tracking tools, and metadata management best practices.
• Practical experience in developing and implementing data quality frameworks and automated data validation pipelines.
• Working knowledge of GxP data integrity requirements, including ALCOA+ principles, electronic records, and audit trail mechanisms.
• Ability to work collaboratively in cross-functional teams and effectively translate complex data requirements from business stakeholders into technical specifications.
• Prior experience with pharmaceutical manufacturing data, including batch records, process parameters, and equipment logs, is a plus.
• Familiarity with modern lakehouse formats like Delta Lake or Apache Iceberg, and concepts such as UNS (Unified Namespace) and Manufacturing Data Fabric.
• Experience with data visualization tools such as Power BI or Grafana for creating manufacturing dashboards.

What We Offer

• An engaging contractual opportunity to work on cutting-edge data solutions within the manufacturing sector.
• Exposure to complex, high-impact projects that directly contribute to operational excellence and business intelligence.
• A collaborative and supportive hybrid work environment that fosters professional growth and skill development.
• The chance to leverage advanced data engineering techniques and technologies to solve real-world challenges.
• Opportunity to work with an experienced team and contribute significantly to client success.