Data Observability Engineer

Freelancing Opportunity

Job Title: Data Engineer (Analytics & Observability)

Experience: 7+ Years

Role Summary

We are looking for a Data Engineer with strong Analytics and Pipeline Observability experience to build, monitor, and optimize end-to-end data pipelines across Azure, Databricks, and Snowflake. The role requires hands-on expertise in implementing observability using traces, metrics, and logs to ensure pipeline reliability, SLA compliance, and data quality.

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using Azure, Databricks, Snowflake, Python, and SQL.
• Monitor and troubleshoot Databricks jobs and data pipelines.
• Implement observability frameworks capturing:
• Traces – job execution flow, latency, dependencies
• Metrics – data freshness, volume, SLA adherence, runtime
• Logs – job failures, transformation errors, exceptions
• Build pipeline health dashboards and alerts using Azure Monitor / Log Analytics.
• Develop analytical dashboards in Power BI.
• Ensure data quality, governance, and performance optimization.

Required Skills

• Strong experience in Azure Data Platform
• Hands-on with Databricks & Snowflake
• Proficiency in Python & SQL
• Experience monitoring production data pipelines
• Knowledge of observability (traces, metrics, logs) in pipelines
• Experience with Power BI for reporting

Good to Have

• Azure Data Factory / Airflow
• OpenTelemetry / Prometheus
• CI/CD & Infrastructure as Code

Job Type: Freelance
Contract length: 6 months

Pay: ₹30,000.00 - ₹40,000.00 per month

Education:

• Bachelor's (Required)

Experience:

• Data Engineering: 7 years (Required)

Work Location: Remote