AI Data Engineer

We are actively seeking a highly skilled and experienced Data Engineer to join our team for a challenging 6-month contract. This fully remote position is ideal for a proactive professional with a strong background in OCI, PySpark, Python, and advanced SQL, ready to dive into complex Big Data processing initiatives.

As a key member of our remote engineering team, the Data Engineer will be instrumental in designing, developing, and optimizing scalable data pipelines and solutions. This role offers an exciting opportunity to leverage cutting-edge technologies and contribute significantly to our data infrastructure, ensuring efficient and reliable data flow for critical business operations.

Key Responsibilities

• Design, build, and maintain robust, scalable, and efficient ETL/ELT pipelines for various data sources.
• Develop and optimize Big Data processing solutions using PySpark and Python within cloud environments, particularly OCI.
• Write and optimize complex SQL queries for data extraction, transformation, and loading processes.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications.
• Ensure data quality, integrity, and security across all data platforms and processes.
• Monitor, troubleshoot, and resolve issues related to data pipelines and data infrastructure.
• Implement best practices for data warehousing, data modeling, and data governance.
• Contribute to the continuous improvement of our data architecture and engineering practices.

Requirements

• Proven experience (typically 5+ years) as a Data Engineer with a strong portfolio of Big Data projects.
• Expertise in Python and PySpark for data manipulation and processing.
• Advanced proficiency in SQL, including writing complex queries, stored procedures, and performance tuning.
• Hands-on experience with Oracle Cloud Infrastructure (OCI) or other major cloud platforms like AWS or GCP.
• Solid understanding of Big Data technologies and concepts (e.g., Hadoop, Spark, Kafka).
• Experience with data warehousing, data lakes, and dimensional modeling.
• Familiarity with version control systems, preferably Git.
• Strong problem-solving skills and the ability to work independently in a remote setting.

What We Offer

• A dynamic and challenging 6-month contract opportunity with impactful projects.
• The flexibility and convenience of a fully remote work environment.
• Exposure to cutting-edge Big Data technologies and cloud infrastructure.
• A collaborative team culture focused on innovation and efficiency.
• Opportunity to significantly contribute to critical data initiatives.