data engineer

Relevance Labs

<div><p><b>Experience:</b> 4-7 Years Experience</p>
<p><b>About the Role:
</b></p><p>We are seeking an experienced Data Engineer to design, implement, and optimize a global data handling and synchronization solution across multiple regions. You will work with cloud-based databases, data lakes, and distributed systems, ensuring compliance with data residency and privacy requirements (e.g., GDPR).</p>
<p><b>Key Responsibilities:</b></p>
<ul><li><ul><li><ul><li>Design and implement data pipelines for global and regional data synchronization (Azure SQL, Data Lake, etc.)</li>
<li>Develop solutions for secure handling of PII and non-PII data, ensuring compliance with GDPR and other regulations</li>
<li>Build and optimize ETL processes for anonymization, transformation, global synchronization and distribution of data</li>
<li>Collaborate with software architects and DevOps to integrate data flows with application logic and deployment pipelines</li>
<li>Set up monitoring, alerting, and documentation for data processes within the existing frameworks</li>
<li>Advise on best practices for data partitioning, replication, and schema evolution</li>
</ul></li></ul></li></ul><p><b>Requirements:</b></p>
<ul><li><ul><li><ul><li>4&ndash;7 years of experience as a Data Engineer in cloud environments (preferably Microsoft Azure)</li>
<li>Strong knowledge of Azure SQL, Data Lake, Data Factory, and related services</li>
<li>Experience with distributed data architectures and data synchronization across regions</li>
<li>Familiarity with data privacy, security, and compliance (GDPR, etc.)</li>
<li>Proficiency in Python, SQL, and ETL tools</li>
<li>Excellent problem-solving and communication skills</li>
<li>Hands-on and self-contributing</li>
</ul></li></ul></li></ul><p><b>Preferred:</b></p>
<ul><li><ul><li><ul><li>Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures</li>
<li>Knowledge of CI/CD and infrastructure-as-code (Azure DevOps, ARM/Bicep, Terraform)</li>
<li>Prior work in global/multi-region data solutions</li></ul></li></ul></li></ul></div>

Data Engineer

Accenture

<p><b>Project Role : </b>Data Engineer<br>
<b>Project Role Description : </b>Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
<br>
<b>Must have skills : </b>Microsoft Azure Databricks<br>
<b>Good to have skills : </b>NA<br>
Minimum <b>5</b> year(s) of experience is required<br>
<b>Educational Qualification : </b>15 years full time education<br>
<br>
Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to effectively migrate and deploy data across various systems. You will collaborate with cross-functional teams to understand their data needs and provide innovative solutions that enhance data accessibility and usability. Key Responsibilities: 1. Function as the Lead Data Architect for a small, simple project/proposal or as a team lead for medium/large sized project or proposal 2. Discuss specific Big data architecture and related issues with client architect/team (in area of expertise) 3. Analyze and assess the impact of the requirements on the data and its lifecycle 4. Lead Big data architecture and design medium-big Cloud based, Big Data and Analytical Solutions using Lambda architecture. 5. Breadth of experience in various client scenarios and situations 6. Experienced in Big Data Architecture-based sales and delivery 7. Thought leadership and innovation 8. Lead creation of new data assets &amp; offerings 9. Experience in handling OLTP and OLAP data workloads Technical Experience: 1. Experience working in Medallion architecture involving Delta lake house principles 2. Expert level in Designing and Architect solutions in Azure Data factory, Azure Databricks, Azure Datalake, Delta Lake implementation. 3. Expert level experience in Azure cloud technologies like PySpark, ADF, Databricks, Python, Scala and SQL. 4. Exp in one or more Real-time/batch ingestion including: Azure Delta live tables , Autoloader 5. Exp in handling medium to large Big Data implementations 6. Strong understanding of data strategy. Data Quality and Delta lake components 7. Candidate must have 7-10 years of IT experience and around 5 years of Big data experience (design + build) in Databricks 8. Architect for a medium sized client delivery project Professional Experience: 1. Should be able to drive the technology design meetings, propose technology design and architecture 2. Should have excellent client communication skills 3. Should have good analytical and problem-solving skills Educational Qualification: 1. Must have: BE/BTech/MCA 2. Good to have: ME/MTech</p>
<br>
15 years full time education

Soothsayer Analytics

<div><div><div><p>Working Hours : <b>Full Time</b></p>
<p>Locations : <b>Hyderabad</b></p>
<p>Experience : <b>4 &ndash;6 years</b></p><p><br>
</p></div>
</div><b>About the Role:
</b><p>Soothsayer Analytics is a global AI &amp; Data Science consultancy headquartered in Detroit, with a thriving delivery center in Hyderabad. We design and deploy end-to-end custom Machine Learning &amp; GenAI solutions&mdash;spanning predictive analytics, optimization, NLP, and enterprise-scale AI platforms&mdash;that help leading enterprises forecast, automate, and gain a competitive edge.
</p><p>As a Data Engineer, you will build the foundation that powers these AI systems&mdash;scalable, secure, and high-performance data pipelines.</p>
<p><b>Job Overview</b></p>
<p>
We seek a <b>Data Engineer (Mid-level)</b> with 4&ndash;6 years of hands-on experience in designing, building, and optimizing data pipelines. You will work closely with AI/ML teams to ensure data availability, quality, and performance for analytics and GenAI use cases.
</p>
<p><b>Key Responsibilities</b></p>
<p><b>Data Pipeline Development:</b></p>
<p>
&middot; Build and maintain scalable ETL/ELT pipelines for structured and unstructured data
</p>
<p>&middot; Ingest data from diverse sources (APIs, streaming, batch systems).
</p><p><b>Data Modeling &amp; Warehousing</b></p>
<p>
&middot; Design efficient data models to support analytics and AI workloads.
</p><p>&middot; Develop and optimize data warehouses/lakes using Redshift, BigQuery, Snowflake, or Delta Lake.
</p>
<p><b>Big Data &amp; Streaming
</b></p><p>&middot; Work with distributed systems like Apache Spark, Kafka, or Flink for real-time/large-scale data processing.
</p>
<p>&middot; Manage feature stores for ML pipelines
</p>
<p><b>Collaboration &amp; Best Practices</b></p>
<p>
&middot; Work closely with Data Scientists and ML Engineers to ensure high-quality training data.
</p>
<p>&middot; Implement data quality checks, observability, and governance frameworks.
</p>
<p><b>Required Skills &amp; Qualifications</b></p>
<p><b>
Education:</b>Bachelor&rsquo;s/Master&rsquo;s in Computer Science, Data Engineering, or related field.</p>
<p><b>
Experience:</b> 4&ndash;6 years in data engineering with expertise in:</p>
<p>
&middot; <b>Programming:</b> Python/Scala/Java (Python preferred).
</p>
<p>&middot; <b>Big Data &amp; Processing:</b> Apache Spark, Kafka, Hadoop.
</p>
<p>&middot; <b>Databases:</b> SQL/NoSQL (Postgres, MongoDB, Cassandra).
</p><p>&middot; <b>Data Warehousing:</b> Snowflake, Redshift, BigQuery, or similar.
</p>
<p>&middot; <b>Orchestration:</b> Airflow, Luigi, or similar.
</p>
<p>&middot; <b>Cloud Platforms:</b> AWS, Azure, or GCP (data services).
</p>
<p>&middot; <b>Version Control &amp; CI/CD:</b> Git, Jenkins, GitHub Actions.
</p>
<p>&middot; <b>MLOps/GenAI pipelines:</b> (feature engineering, embeddings, vector DBs)
</p>
<p><b>Skills Matrix</b></p>
<p>
Candidates must submit a detailed resume and fill out the following matrix:</p>
<div>
<div><div><div><p><b>Skill</b></p><br>
</div><div><p><b>Details</b></p><br>
</div><div><p><b>Skills Last Used</b></p><br>
</div><div><p><b>Experience (months)</b></p><br>
</div><div><p><b>Self-Rating (0&ndash;10)</b></p><br>
</div></div><br>
<div><div><p>Python</p><br>
</div></div><br>
<p></p><div><div><p>SQL / NoSQL</p><br>
</div></div><br>
<p></p><div><div><p>Apache Spark</p><br>
</div></div><br>
<p></p><div><div><p>Kafka</p><br>
</div></div><br>
<p></p><div><div><p>Data Warehousing (Snowflake, Redshift, etc.)<br>
</p></div></div><br>
<p></p><div><div><p>Orchestration (Airflow, Luigi, etc.)<br>
</p></div></div><br>
<p></p><div><div><p>Cloud (AWS / Azure / GCP)<br>
</p></div></div><br>
<p></p><div><div><p>Data Quality / Governance Tools<br>
</p></div></div><br>
<p></p><div><div><p>MLOps / LLMOps</p><br>
</div></div><br>
<p></p><div><div><p>GenAI Integration</p><br>
</div></div><br>
</div></div><p></p><p><b>Instructions for Candidates:</b></p>
<p>
&middot; Provide a detailed resume highlighting <b>end-to-end data engineering projects.</b>
</p>
<p>&middot; Fill out the above <b>skills matrix</b> with accurate dates, duration, and self-ratings.</p></div><p></p>

Programming.com

<p><b>Job Description:</b></p><p>Position: Data Engineer</p><p>Location: Hyderabad (WFO)</p><p>Experience Required: Minimum 1 - 5 Years</p><p>Notice Period: Immediate to 15 days</p><p>Role Overview</p><p>As a Senior Data Engineer, you will play a critical role in architecting, building, and maintaining robust data solutions for our growing organization. You will leverage your expertise to streamline data operations, ensuring data quality, security, and scalability. Working closely with cross-functional teams, you'll drive data engineering initiatives that support and enhance decision-making processes across the company.</p><p>Key Responsibilities</p><p>&#9658; Design and implement efficient data pipelines and ETL processes to manage complex data sets.</p><p>&#9658; Collaborate with data scientists, analysts, and business stakeholders to understand and implement data requirements.</p><p>&#9658; Develop and optimize data models to support reporting, analytics, and machine learning efforts.</p><p>&#9658; Ensure data integrity and compliance with industry standards and best practices.</p><p>&#9658; Monitor, troubleshoot, and optimize data workflows for performance and reliability.</p><p>Qualifications</p><p>&#9658; Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, Data Engineering, or a related field.</p><p>&#9658; Minimum 5 years of hands-on experience in data engineering roles.</p><p>&#9658; Strong problem-solving skills and ability to work in a fast-paced, team-oriented environment.</p><p>&#9658; Excellent verbal and written communication skills.</p><p>Technical Skills: Python, SQL, ETL, Apache Spark, AWS (S3, Redshift, Lambda), Data Warehousing, Big Data</p><p>Job Type: Full-time</p><p>Pay: &#8377;470,245.68 - &#8377;2,817,237.54 per year</p><p>Work Location: In person</p>

DBiz ai solutions

<div><div><b>Position: Senior Data Engineer</b></div><div></div><div><br>
We are seeking an experienced <b>Senior Data Engineer</b> to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems. In this role, you will leverage your expertise in <b>Databricks</b>, <b>Pyspark</b>, and <b>ETL design</b> to build efficient data solutions. You will also contribute to optimizing database performance and ensure that data systems meet both scalability and security standards.</div><div></div><div><b><br>
Key Responsibilities:</b></div><ul><li>
Develop, optimize, and maintain scalable <b>ETL pipelines</b> for processing large datasets using <b>Databricks</b> and <b>Pyspark</b>.</li><li>
Apply advanced techniques such as <b>chunking</b> and <b>partitioning</b> to handle large file volumes efficiently.</li><li>
Tuning and optimizing databases for better performance and storage efficiency.</li><li>
Collaborate with cross-functional teams to ensure the architecture meets business requirements and data quality standards.</li><li>
Design and implement solutions with an eye toward scalability and long-term performance.</li><li>
Work with <b>Azure</b> services, with exposure to <b>Azure Function Apps</b>, <b>Azure Data Factory</b>, and <b>Cosmos DB</b> (preferred).</li><li>
Communicate effectively with stakeholders to align on project goals and deliverables.</li></ul><div><b>
Key Skills and Experience:</b></div><ul><li><b>
Databricks</b> (advanced-level skills in data engineering workflows)</li><li><b>
Pyspark</b> (intermediate-level skills for processing big data)</li><li>
Strong <b>ETL design skills</b>, particularly in partitioning, chunking, and database tuning for large datasets</li><li><b>
Azure</b> experience (a plus, including <b>Function Apps</b>, <b>Cosmos DB</b>, and <b>Azure Data Factory</b>)</li><li>
Excellent communication skills for effective collaboration with teams and stakeholders</li></ul></div>

Data Engineer- 4

Data Engineer- 3

Data Engineer- 1

Search 10000+ Data engineer jobs near you this September. New roles added regularly across leading industries. Apply today.

10000 data engineer jobs NEAR_ME

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer- 4

Data Engineer- 3

Data Engineer- 1