We are seeking an experienced Data Engineer to join our team on a full-time basis. The ideal candidate will have a strong background in designing, building, and optimizing data pipelines, with a particular focus on enabling AI-driven solutions. This role involves close collaboration with software and AI engineering teams to ensure seamless, reliable data flows and robust cloud-based data integration within Microsoft Azure.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes to support AI initiatives.
- Build and optimize data infrastructure to ensure efficient data management, processing, and storage in Azure cloud environments.
- Collaborate with AI engineers to prepare, structure, and deliver high-quality datasets for machine learning models.
- Ensure data pipelines meet security, reliability, and performance standards.
- Monitor, troubleshoot, and improve pipeline performance to guarantee timely and accurate data delivery.
- Implement data governance practices and ensure compliance with data security policies.
Required Qualifications
- Proven experience as a Data Engineer, particularly in building and maintaining complex pipelines for AI or machine learning projects.
- Strong proficiency in SQL and hands-on experience with relational databases (PostgreSQL, SQL Server) and NoSQL systems (e.g., MongoDB).
- Expertise in Azure-based data solutions such as Azure Data Factory, Azure Synapse, and Azure Databricks.
- Experience with data integration and orchestration tools (e.g., Apache Airflow).
- Proficiency in at least one programming language for data workflows (e.g., Python).
- Solid understanding of data warehousing concepts and best practices.
- Strong analytical and problem-solving abilities.
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- Proficiency in English, both written and spoken.
Desirable Qualifications
- Familiarity with machine learning concepts and data preprocessing techniques.
- Knowledge of containerization technologies such as Docker and Kubernetes for deploying data pipelines.
Job Type: Full-time