Data Engineer

We are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, ETL processes, and data integration, with hands-on experience in designing, developing, and managing data pipelines using Apache NiFi. This role will involve working closely with cross-functional teams to ensure seamless data flow, transformation, and delivery across our systems.

Key Responsibilities:

Design, develop, and maintain scalable and efficient data pipelines using Apache NiFi.
Integrate data from multiple sources, including databases, APIs, cloud storage, and streaming platforms.
Optimize data workflows for performance, reliability, and scalability.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
Troubleshoot and resolve issues related to data ingestion, transformation, and delivery.
Implement data security and governance best practices within NiFi workflows.
Monitor and maintain NiFi clusters, ensuring high availability and performance.
Document data pipelines, workflows, and processes for future reference and knowledge sharing.
Stay up-to-date with the latest trends and advancements in Apache NiFi and data engineering technologies.

Qualifications:

Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent experience).
Proven experience working with Apache NiFi to build and manage data pipelines.
Strong understanding of ETL/ELT processes and data integration concepts.
Proficiency in working with various data formats (e.g., JSON, XML, CSV) and protocols (e.g., HTTP, FTP, Kafka).
Experience with relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
Knowledge of scripting languages (e.g., Python, Bash) for automation and customization.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration skills.

Preferred Skills:

Experience with big data technologies (e.g., Hadoop, Spark).
Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).
Familiarity with data governance and compliance standards (e.g., GDPR, HIPAA).
Certification in Apache NiFi or related technologies is a plus.