Data Engineer

We are seeking a highly skilled and experienced Data Engineer to join our dynamic team at IdealCrew. As a Data Engineer, you will play a crucial role in developing high-performance ETL pipelines on large datasets within the Azure cloud environment. This is an excellent opportunity for a professional with strong Python development skills and a solid understanding of relational databases to contribute to impactful projects and advance their career.

The ideal candidate will be a Data developer with extensive experience in Python and a passion for working with big data platforms. You will be instrumental in leveraging Databricks and Python to build robust data solutions, ensuring data integrity and efficient processing. This role requires a strong analytical mindset and excellent communication skills to collaborate effectively with cross-functional teams.

Key Responsibilities

• Develop and optimize high-performance ETL pipelines for large datasets in the Azure cloud.
• Utilize Databricks and Python as the primary BigData platform for data ingestion and transformation.
• Apply strong Python development skills for data manipulation, back-end API development, or framework creation.
• Design, develop, and maintain database solutions with a deep understanding of relational database concepts and SQL.
• Implement and manage ETL frameworks, with a preference for SSIS.
• Contribute to API development using Python to support various data initiatives.
• Work within an Agile environment, collaborating with team members and stakeholders.
• Ensure the reliability and scalability of data solutions.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3-10 years of professional experience as a Data Engineer or in a similar data development role.
• Proven expertise in Python development, particularly for data manipulation or back-end API/framework development.
• Strong knowledge of relational database concepts and proficiency in SQL for data inserts, updates, and selects.
• Experience with ETL frameworks/tools, with SSIS being a plus.
• Familiarity with Azure Data Factory (ADF), Spark, and Databricks is highly desirable; training can be provided.
• Excellent analytical and problem-solving skills.
• Strong verbal and written communication abilities.
• Ability to take ownership of issues and demonstrate accountability.

What We Offer

• A competitive annual salary ranging from ₹15,00,000 to ₹33,00,000.
• Opportunity to work on cutting-edge data engineering projects in the Azure cloud.
• A collaborative and supportive work environment.
• Continuous learning and development opportunities, including training in Azure Data Factory, Spark, and Databricks.
• Career growth within a rapidly expanding company.