Senior Consultant Data Analytics

We are looking for a Data Engineer with 3–4 years of experience in data engineering, data integration, or a related role.

Responsibilities

• Develop scalable ETL processes using Spark, Hadoop, and other big data technologies.

• Design, test, and maintain pipelines that transform raw data into structured formats for analysis and reporting.

• Utilize tools such as Hive, HBase, MapReduce, and Spark for both real-time and batch data processing.

• Manage and optimize relational and NoSQL databases (including Vertica) and write complex SQL queries for data manipulation.

• Automate data workflows and routine processes through shell scripting and related automation tools.

• Enhance performance, scalability, and cost efficiency across data storage, retrieval, and transformation layers.

• Partner with data analysts, data scientists, and business teams to deliver reliable, high-quality data solutions.

• Monitor ETL processes for data accuracy, troubleshoot issues, and ensure data integrity across environments.

• Maintain detailed documentation of pipelines, workflows, and processes for transparency and continuity.

Requirements:

• 3–4 years of experience in data engineering, data integration, or a similar role.

• Strong hands-on experience with the Hadoop ecosystem (HDFS, Hive, HBase, MapReduce).

• Proficiency in Apache Spark for large-scale data processing.

• Advanced SQL skills and experience working with Vertica and other relational/NoSQL databases.

• Familiarity with ETL processes, data integration, and transformation best practices.

• Proficiency in shell scripting for automating data workflows.

• Experience with data pipeline orchestration tools (e.g., Airflow, Oozie, or similar) is a plus.

• Strong problem-solving and troubleshooting skills with high attention to detail.