AI Data Engineer

Are you an experienced Data Engineer passionate about building robust and scalable data solutions? Talent Drive Consulting is seeking a talented Data Engineer with expertise in GCP and Databricks for an exciting 6-month contract role, with potential for extension. This hybrid position in Bangalore offers a unique opportunity to work on critical data pipelines and contribute to cutting-edge projects leveraging cloud technologies.

We are looking for a skilled professional to join our team and drive the development of high-performance data infrastructure. If you thrive in a dynamic environment and have a strong command of Google Cloud Platform and Databricks, we encourage you to apply for this challenging Data Engineer position.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Python, and SQL on the Google Cloud Platform (GCP).
• Implement and optimize data warehousing solutions, leveraging BigQuery and Cloud Storage for efficient data management.
• Automate data workflows using orchestration tools like Airflow or Google Cloud Composer.
• Collaborate with cross-functional teams to understand data requirements and translate them into robust technical solutions.
• Ensure data quality, integrity, and reliability across all data pipelines and systems.
• Monitor and troubleshoot data processes, identifying and resolving performance bottlenecks and issues.
• Contribute to the continuous improvement of data engineering best practices and architecture.

Requirements

• 2-5 years of professional experience in data engineering or a similar role.
• Mandatory hands-on experience with Google Cloud Platform (GCP) services, including BigQuery and Cloud Storage.
• Strong proficiency in Databricks (Workflows, Notebooks, and ETL Pipelines).
• Expertise in Python and SQL for data manipulation and scripting.
• Familiarity with Azure Data Factory (ADF) is a significant plus.
• Experience with data orchestration tools such as Apache Airflow or Google Cloud Composer.
• Understanding of data warehousing concepts and best practices.
• Good to have exposure to Spark/PySpark, Git, CI/CD, Pub/Sub, or Kafka.
• Ability to work in a hybrid model and comfortable attending a mandatory final walk-in interview in Bangalore.

What We Offer

• An engaging 6-month contract opportunity with potential for extension based on project needs.
• Exposure to cutting-edge cloud data technologies and complex data engineering challenges.
• A collaborative and dynamic work environment at the forefront of data innovation.
• Opportunity to enhance your expertise in GCP, Databricks, and various data orchestration tools.
• Be part of a team that values technical excellence and continuous learning.