AI Data Engineer

SquareHiring is actively seeking a highly skilled and experienced Data Engineer to join our remote team. This is a fantastic opportunity for a professional passionate about building robust and scalable data solutions using cutting-edge technologies. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our critical data infrastructure, enabling insightful analytics and data-driven decisions across the organization. If you thrive in a remote work environment and possess a strong background in big data technologies, we encourage you to apply for this pivotal Data Engineer position.

We are looking for individuals who are not only adept at solving complex data challenges but also eager to contribute to a collaborative and innovative team culture. This role offers the chance to work with a diverse set of tools and platforms within the Microsoft ecosystem, ensuring you remain at the forefront of data engineering advancements.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines and ETL processes using big data technologies to ingest, transform, and load large datasets.
• Implement and manage data storage solutions, ensuring data integrity, security, and accessibility.
• Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and translate them into technical specifications.
• Monitor data pipelines, troubleshoot issues, and ensure the reliability and performance of data systems.
• Develop and maintain comprehensive documentation for data architecture, pipelines, and processes.
• Contribute to the continuous improvement of data engineering practices, tools, and methodologies.
• Ensure compliance with data governance and quality standards across all data initiatives.

Requirements

• Proven experience as a Data Engineer with a strong understanding of data warehousing and ETL principles.
• Mandatory hands-on experience with Microsoft Synapse, Scala, Spark, and PySpark.
• Proficiency in SQL and experience with data query languages like Scope.
• Familiarity with cloud-based data storage solutions such as Cosmos DB and ADLS (Azure Data Lake Storage).
• Experience with Kusto query language and Microsoft Fabric, including OneLake, is highly desirable.
• Strong problem-solving skills and the ability to work independently in a remote setting.
• Excellent communication and collaboration skills to work effectively with cross-functional teams.
• A proactive approach to learning new technologies and adapting to evolving data landscapes.

What We Offer

• The opportunity to work on challenging and impactful data engineering projects.
• A fully remote work setup, providing flexibility and work-life balance.
• Access to and experience with leading-edge big data and cloud technologies.
• A supportive and collaborative team environment focused on innovation and growth.
• Competitive compensation for a W2 full-time role.