Apache NiFi developer

Job Description:

Required Experience:

• 5+ years of hands-on experience with Apache NiFi, including developing, managing, and optimizing complex data flows in production environments.

• Proven experience with Cloudera NiFi (CDP Data Flow) in enterprise environments, including integration with Cloudera Manager.

• Experience migrating NiFi flows across major version upgrades with strong understanding of backward compatibility

• Strong proficiency in Groovy scripting, used for Execute Script and Invoke Scripted Processor processors.

• Solid understanding of SSH and SFTP protocols, including authentication schemes (key-based, password), session negotiation, and file permissions handling in NiFi processors (e.g., List SFTP, Fetch SFTP, Put SFTP).

• Good grasp of data encryption mechanisms, key management, and secure flow file handling using processors like Encrypt Content.

• Experience integrating NiFi with MongoDB, including reading/writing documents via processors like Get Mongo, Put Mongo, and Query Mongo.

• Experience working with Apache Kafka, including producing and consuming from Kafka topics using NiFi (Publish Kafka, Consume Kafka), and handling schema evolution with Confluent Schema Registry.

• Strong knowledge of Red Hat Enterprise Linux (RHEL) environments, including system services, filesystem permissions, log rotation, and resource tuning for JVM-based applications like NiFi.

NiFi-Specific Technical Requirements:

• In-depth knowledge of NiFi flow design principles, including proper use of queues, back pressure, prioritizes, and connection tuning.

• Mastery of controller services, including SSL Context Service, DBCP Connection Pool, and Record Reader/Record Writer services.

• Experience with Record-based processing using Avro, JSON, CSV schemas and Record processors like Convert Record, Query Record, and Lookup Record.

• Ability to debug and optimize NiFi flows using Data Provenance, bulletins, and log analysis.

• Familiarity with custom processor development in Java/Groovy (optional but preferred).

• Experience setting up secure NiFi clusters, configuring user authentication (LDAP, OIDC), TLS certificates, and access policies.

• Proficiency in parameter contexts, variable registry, and flow versioning using NiFi Registry.

• Understanding of Zero-Master clustering model, node coordination, and site-to-site protocol.

• Experience deploying and monitoring NiFi in high-availability, production-grade environments, including using Prometheus/Grafana or Cloudera Manager for metrics and alerting.

Preferred Qualifications:

• Experience working in regulated or secure environments, with strict data handling and audit requirements.

• Familiarity with DevOps workflows, including version-controlled flow templates (JSON/XML), CI/CD integration for NiFi Registry, and automated deployment strategies.

• Strong written and verbal communication skills, with ability to document flows and onboard other engineers.