GCP Data Engineer

eClerx is actively seeking an experienced GCP Data Engineer to enhance our dynamic team. In this pivotal role, you will contribute to large-scale data engineering initiatives, focusing on campaign and clickstream data. This is an excellent opportunity for a seasoned professional to leverage their expertise in cloud-based data solutions and drive significant impact.

Key Responsibilities

• Design, develop, and implement highly scalable data pipelines within the Google Cloud Platform (GCP) ecosystem, utilizing BigQuery and Airflow.
• Construct and maintain robust ETL/ELT frameworks specifically designed for processing extensive campaign and clickstream datasets.
• Proactively optimize BigQuery query performance and efficiently manage pipeline costs to ensure operational excellence.
• Automate workflows, establish comprehensive monitoring systems, and configure alerting mechanisms for critical data processing jobs.
• Guarantee that all developed data pipelines are scalable, reliable, and prepared for production deployment.
• Collaborate with cross-functional teams to understand data requirements and translate them into effective technical solutions.
• Ensure data quality, integrity, and security across all data engineering processes.

Requirements

• 6-9 years of proven experience as a Data Engineer, with a strong focus on GCP services.
• Exceptional proficiency in SQL, with a preference for BigQuery SQL syntax and optimization techniques.
• Extensive hands-on experience with Python and PySpark for data manipulation and processing.
• Demonstrable experience with core GCP services, including BigQuery and Cloud Storage.
• Practical experience with workflow orchestration tools like Airflow or Cloud Composer.
• Familiarity with CI/CD tools, Git workflows, and Jenkins for continuous integration and deployment.
• Solid understanding of ETL/ELT methodologies and fundamental data engineering concepts.
• Ability to work effectively in a fast-paced, collaborative environment.

What We Offer

• An opportunity to work on large-scale, impactful data engineering projects.
• A collaborative and innovative work environment.
• Exposure to cutting-edge GCP technologies and data solutions.
• Professional growth and development opportunities.
• A chance to contribute to the success of a leading global company.