Data QA Automation Expert

Key Responsibilities

• Automate data validation for Big Data sources (Relational, NoSQL, Cloud, Flat Files, XML).
• Develop and maintain smoke, functional, regression, and performance tests using Python, PySpark, and Pytest/Robot.
• Design and own test automation frameworks for ETL/ELT validation, data transformation, and migration testing.
• Collaborate with cross-functional teams to align testing with data pipeline changes.
• Execute automated tests for security, architecture, visualization, and pen testing.
• Monitor and report test results using tools like Grafana, New Relic.
• Optimize CI/CD pipelines for seamless test integration and deployment.
• Document test plans, scripts, and validation strategies for compliance and traceability.

Must-Have Skills & Experience

• 4+ years in QA Automation with Big Data Technologies (PySpark, Apache Spark).
• 2–3 years building automation frameworks (Python, Pytest, Robot).
• Strong SQL and experience with Big Data Cloud Platforms.
• Hands-on ETL Validator Testing for automating ETL/ELT validation.
• BDD frameworks (Cucumber/SpecFlow) for structured test scenarios.
• Data Testing Strategies: Validation, process validation, outcome validation, code coverage.
• Performance, Security, and Migration Testing for Big Data environments.
• CI/CD and monitoring tools (Grafana, New Relic).
• Test setup and pipeline management in Agile/Scrum environments.

Behavioral Fit

• Highly technical with a keen eye for detail.
• Self-motivated, results-driven, and able to challenge processes for improvement.
• Structured, organized, and capable of multitasking in a fast-paced environment.
• Collaborative with cross-functional teams and stakeholders.

Job Type: Full-time

Pay: ₹2,000,000.00 - ₹2,500,000.00 per year

Application Question(s):

• Do you have experience automating data validation for Big Data projects?
• Have you used PySpark, Pytest, or Cucumber for test automation?
• Have you ensured data quality in ETL/ELT pipelines as part of your role?
• Have you conducted performance and security testing in Big Data environments?
• Have you worked with CI/CD pipelines for test automation?
• What is your current and expected CTC?
• What is your official notice period?

Education:

• Bachelor's (Required)

Experience:

• Data QA Automation Expert (Big Data & PySpark): 5 years (Required)

Work Location: Remote