Data Engineer

We are seeking an experienced Data Engineer with a strong background in data orchestration to join our dynamic team in Minneapolis, MN. As a Data Engineer, you will play a crucial role in migrating and optimizing our data workflows, ensuring robust and scalable data pipelines. This is an exciting opportunity for a professional with expertise in AutoSys and Apache Airflow to make a significant impact on our data infrastructure.

Key Responsibilities

• Lead and execute the migration of existing data orchestration workflows from AutoSys to Apache Airflow.
• Design, develop, test, and deploy Apache Airflow DAGs (Directed Acyclic Graphs) for complex data pipelines.
• Implement and manage Airflow operators, sensors, and scheduling mechanisms to ensure efficient data processing.
• Collaborate with cross-functional teams to integrate data solutions with various cloud platforms, including Azure and AWS.
• Develop and maintain modern data pipelines using technologies like Databricks, Python, and SQL.
• Ensure the reliability and performance of data systems through continuous integration/continuous delivery (CI/CD) practices, monitoring, and production support.
• Contribute to the architectural design and enhancement of our data orchestration framework.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years of professional experience in data engineering or a similar role.
• Strong hands-on experience with both AutoSys and Apache Airflow, including migration experience.
• Proficiency in Python and SQL for data manipulation and scripting.
• Demonstrated experience with REST APIs for data integration.
• Familiarity with cloud platforms such as Azure or AWS.
• Experience with Databricks and building modern data pipelines.
• Understanding of CI/CD principles, monitoring tools, and production support processes for data systems.

What We Offer

• An engaging hybrid work environment in Minneapolis, MN.
• Opportunity to work with cutting-edge data orchestration technologies.
• Collaborative and innovative team culture.
• Professional growth and development opportunities.
• Competitive compensation package.