Sr. DevOps Engineer

Job Title:Sr.DevOps Engineer

Experience: 5+yrs

Job Description:

Big Data Ingestion (Terraform | Databricks | GCP)5+yrs

Required DevOps Engineer to support a Big Data Ingestion project. The role involves automating and managing infrastructure for pipelines ingesting data from Datadog logs and Cloud SQL Server into Google BigQuery, built on Databricks and PySpark.

Responsibilities:

• 
  Build and manage GCP infrastructure (BigQuery, Cloud SQL, Cloud Storage, IAM) using Terraform.
• Automate Databricks cluster provisioning and pipeline deployments.
• Implement CI/CD pipelines for infrastructure and code.
• Set up monitoring, logging, and alerting for ingestion pipelines.
• Collaborate with Data Engineers and QA teams to ensure reliability and scalability.

Requirements:

• 
  Strong hands-on experience with Terraform and GCP services.
• Knowledge of Databricks, PySpark, and BigQuery.
• Experience with CI/CD tools (GitHub Actions, Jenkins, or Cloud Build).
• Familiarity with Datadog or other monitoring tools.
• Scripting skills in Python/Bash.

Nice to Have: Docker/Kubernetes, Airbyte/Fivetran, OAuth 2.0, cost optimization in big data environments