AWS Data Engineer - Databricks

We are actively seeking an experienced and highly skilled AWS Data Engineer to join our dynamic team. This pivotal role requires strong hands-on expertise in designing, building, and managing robust, scalable data pipelines within the Amazon Web Services (AWS) ecosystem. The ideal candidate will be proficient in leveraging various AWS data services, possess excellent PySpark capabilities, and demonstrate superior SQL skills to drive efficient data solutions. As an AWS Data Engineer, you will play a crucial role in transforming raw data into actionable insights, supporting critical business functions and data-driven decision-making.

Key Responsibilities

• Design, develop, and maintain highly scalable and reliable data pipelines using a range of AWS services.
• Implement and optimize ETL/ELT processes to ingest, transform, and load large datasets into data lakes and data warehouses.
• Develop robust data processing applications and solutions using PySpark for complex data transformations and analytics.
• Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver comprehensive data solutions.
• Ensure data quality, integrity, and security across all data pipelines and storage solutions within the AWS environment.
• Monitor pipeline performance, troubleshoot issues, and implement proactive measures to ensure operational efficiency and reliability.
• Contribute to the continuous improvement of data engineering practices, tools, and infrastructure.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field.
• Minimum of 6+ years of professional experience in data engineering, with a strong focus on AWS cloud platforms.
• Demonstrated hands-on expertise with AWS data services, including AWS Lambda, AWS Step Functions, AWS Glue, and S3.
• Exceptional proficiency in PySpark for large-scale data processing and analytics.
• Advanced SQL skills for data querying, manipulation, and database management.
• Proven experience in designing, building, and optimizing complex data pipelines from various sources.
• Solid understanding of cloud-based data engineering principles, data warehousing concepts, and ETL methodologies.
• Ability to work effectively in a fast-paced environment and manage multiple priorities.

What We Offer

• An opportunity to work on challenging and impactful data engineering projects leveraging cutting-edge AWS technologies.
• A collaborative and innovative work environment that fosters continuous learning and professional growth.
• Exposure to a diverse range of industries and data challenges.
• A culture that values technical excellence and problem-solving.
• Opportunities for skill enhancement and career advancement in a rapidly evolving field.