Kafka Admin

Job Description – Kafka Admin

Responsibilities:

• Perform Admin functions, Setting Kafka cluster and responsible for cluster migration.
• Monitoring activities (Support team) and Hands on Monitoring tool - Confluent Control Centre, Cloudera Manager
• Deployment and Management of Confluent Kafka.
• Performance monitoring and investigation and Streams based application problem investigation.
• Container Orchestration (Kubernetes) 7. Container technologies (Docker)
• Infrastructure as code (Terraform, Concourse, Veracode)
• Service Monitoring and Graphing (Prometheus + Grafana, Splunk)
• Cluster maintenance activities – all Environments
• Adding node and removing node on prod cluster and rebalancing
• DevOps Pipeline setup - CI/CD for Kafka Ansible, Deployments, AZURE data bricks, On-premises Hadoop, Spark etc. all the data pipelines
• To work on Kafka Prerequisites of infra and network
• Confluent Kafka license configuration for the components/nodes in UAT and PROD
• Non-Prod cluster activities and migration plan - Support and adhere to the standards and make as like Production.
• Disaster recovery setup and regular failover drill runs on the cluster.
• Develop Prerequisite scripts and automation scripts for health check and monitoring the cluster.
• Automate monitoring and proactive alerts.
• App Dynamics -Integration from Kafka and Log Analyses (Splunk -Integration for the logs)
• Prepare and execute load/performance test cases.
• Automation of monitoring and ensure cluster healthy with the best practices.
• Security configuration and protocol as per the Mashreq ISG requirements
• LDAP and Cluster access strategy
• 24x7 availability will be for “Critical Production issue” marked “Severity 1” (or P1 in Confluent terms) only and NOT Sev2 or Sev3

Qualifications:

• BE or degree in Computer Science or a related technical field.
• 7+ years of experience in Confluent Kafka/Apache Kafka cluster installation and maintenance.
• Expert admin experience in Linux and distributed cluster environments.
• Good Experience in Ansible and DevOps for Admin activities.