data pipeline developer

<div></div><div><div><div><div><ul><div>5 - 7 Years</div>
<div></div><div>1 Opening</div>
 <div>Bangalore</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p>Role Proficiency:</p>
<p>
This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions.</p>
<p>
Outcomes:</p>
<div>
<ul><li>Act creatively to develop pipelines and applications by selecting appropriate technical options optimizing application development maintenance and performance using design patterns and reusing proven solutions.rnInterpret requirements to create optimal architecture and design developing solutions in accordance with specifications.</li></ul>
<ul><li>Document and communicate milestones/stages for end-to-end delivery.</li></ul>
<ul><li>Code adhering to best coding standards debug and test solutions to deliver best-in-class quality.</li></ul>
<ul><li>Perform performance tuning of code and align it with the appropriate infrastructure to optimize efficiency.</li></ul>
<ul><li>Validate results with user representatives integrating the overall solution seamlessly.</li></ul>
<ul><li>Develop and manage data storage solutions including relational databases NoSQL databases and data lakes.</li></ul>
<ul><li>Stay updated on the latest trends and best practices in data engineering cloud technologies and big data tools.</li></ul>
<ul><li>Influence and improve customer satisfaction through effective data solutions.</li></ul>
</div>
<p></p><p>Measures of Outcomes:</p>
<div>
<ul><li>Adherence to engineering processes and standards</li></ul>
<ul><li>Adherence to schedule / timelines</li></ul>
<ul><li>Adhere to SLAs where applicable</li></ul>
<ul><li># of defects post delivery</li></ul>
<ul><li># of non-compliance issues</li></ul>
<ul><li>Reduction of reoccurrence of known defects</li></ul>
<ul><li>Quickly turnaround production bugs</li></ul>
<ul><li>Completion of applicable technical/domain certifications</li></ul>
<ul><li>Completion of all mandatory training requirements</li></ul>
<ul><li>Efficiency improvements in data pipelines (e.g. reduced resource consumption faster run times).</li></ul>
<ul><li>Average time to detect respond to and resolve pipeline failures or data issues.</li></ul>
<ul><li>Number of data security incidents or compliance breaches.</li></ul>
</div>
<p></p><p>Outputs Expected:</p>
<p>
Code Development:</p>
<div>
<ul><li>Develop data processing code independently<br>
ensuring it meets performance and scalability requirements.</li></ul>
<ul><li>Define coding standards<br>
templates<br>
and checklists.</li></ul>
<ul><li>Review code for team members and peers.</li></ul></div><br>
<p><br>
Documentation:</p>
<div>
<ul><li>Create and review templates<br>
checklists<br>
guidelines<br>
and standards for design<br>
processes<br>
and development.</li></ul>
<ul><li>Create and review deliverable documents<br>
including design documents<br>
architecture documents<br>
infrastructure costing<br>
business requirements<br>
source-target mappings<br>
test cases<br>
and results.</li></ul></div><br>
<p><br>
Configuration:</p>
<div>
<ul><li>Define and govern the configuration management plan.</li></ul>
<ul><li>Ensure compliance within the team.</li></ul></div><br>
<p><br>
Testing:</p>
<div>
<ul><li>Review and create unit test cases<br>
scenarios<br>
and execution plans.</li></ul>
<ul><li>Review the test plan and test strategy developed by the testing team.</li></ul>
<ul><li>Provide clarifications and support to the testing team as needed.</li></ul></div><br>
<p><br>
Domain Relevance:</p>
<div>
<ul><li>Advise data engineers on the design and development of features and components<br>
demonstrating a deeper understanding of business needs.</li></ul>
<ul><li>Learn about customer domains to identify opportunities for value addition.</li></ul>
<ul><li>Complete relevant domain certifications to enhance expertise.</li></ul></div><br>
<p><br>
Project Management:</p>
<div>
<ul><li>Manage the delivery of modules effectively.</li></ul></div><br>
<p><br>
Defect Management:</p>
<div>
<ul><li>Perform root cause analysis (RCA) and mitigation of defects.</li></ul>
<ul><li>Identify defect trends and take proactive measures to improve quality.</li></ul></div><br>
<p><br>
Estimation:</p>
<div>
<ul><li>Create and provide input for effort and size estimation for projects.</li></ul></div><br>
<p><br>
Knowledge Management:</p>
<div>
<ul><li>Consume and contribute to project-related documents<br>
SharePoint<br>
libraries<br>
and client universities.</li></ul>
<ul><li>Review reusable documents created by the team.</li></ul></div><br>
<p><br>
Release Management:</p>
<div>
<ul><li>Execute and monitor the release process to ensure smooth transitions.</li></ul></div><br>
<p><br>
Design Contribution:</p>
<div>
<ul><li>Contribute to the creation of high-level design (HLD)<br>
low-level design (LLD)<br>
and system architecture for applications<br>
business components<br>
and data models.</li></ul></div><br>
<p><br>
Customer Interface:</p>
<div>
<ul><li>Clarify requirements and provide guidance to the development team.</li></ul>
<ul><li>Present design options to customers and conduct product demonstrations.</li></ul></div><br>
<p><br>
Team Management:</p>
<div>
<ul><li>Set FAST goals and provide constructive feedback.</li></ul>
<ul><li>Understand team members' aspirations and provide guidance and opportunities for growth.</li></ul>
<ul><li>Ensure team engagement in projects and initiatives.</li></ul></div><br>
<p><br>
Certifications:</p>
<div>
<ul><li>Obtain relevant domain and technology certifications to stay competitive and informed.</li></ul>
</div>
<p></p><p>Skill Examples:</p>
<div>
<ul><li>Proficiency in SQL Python or other programming languages used for data manipulation.</li></ul>
<ul><li>Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF.</li></ul>
<ul><li>Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services (e.g. AWS Glue BigQuery).</li></ul>
<ul><li>Conduct tests on data pipelines and evaluate results against data quality and performance specifications.</li></ul>
<ul><li>Experience in performance tuning of data processes.</li></ul>
<ul><li>Expertise in designing and optimizing data warehouses for cost efficiency.</li></ul>
<ul><li>Ability to apply and optimize data models for efficient storage retrieval and processing of large datasets.</li></ul>
<ul><li>Capacity to clearly explain and communicate design and development aspects to customers.</li></ul>
<ul><li>Ability to estimate time and resource requirements for developing and debugging features or components.</li></ul>
</div>
<p></p><p>Knowledge Examples:</p>
<p>
Knowledge Examples</p>
<div>
<ul><li>Knowledge of various ETL services offered by cloud providers including Apache PySpark AWS Glue GCP DataProc/DataFlow Azure ADF and ADLF.</li></ul>
<ul><li>Proficiency in SQL for analytics including windowing functions.</li></ul>
<ul><li>Understanding of data schemas and models relevant to various business contexts.</li></ul>
<ul><li>Familiarity with domain-related data and its implications.</li></ul>
<ul><li>Expertise in data warehousing optimization techniques.</li></ul>
<ul><li>Knowledge of data security concepts and best practices.</li></ul>
<ul><li>Familiarity with design patterns and frameworks in data engineering.</li></ul>
</div>
<p></p><p>Additional Comments:</p>
<p></p>
<p>Mandatory Skills Database Developer, Azure SQL Managed Instance , Stored Procedures, Triggers, Scheduling, Data Migration, Etl Developer, SQL Server, Data Pipeline Skill to Evaluate Database Developer, Stored Procedures, Triggers, Scheduling, Data Migration, Etl Developer, SQL Server, Data Pipeline Experience 6 to 8 Years Location Bengaluru Job Description Required skills and technologies - Extensive Expertise in Azure SQL Managed Instance (SQL MI). - Advanced proficiency in SQL programming and DBMS. - Writing Stored Procedures, Functions,triggers,Indexing. - Experience with database migration tools like Azure Data Factory (ADF). - Strong understanding of ETL processes and data pipelines. - Proven experience in database architecture, design, and ER modeling. - Skilled in performance tuning, query optimization, and batch processing. - Knowledge of data security, backup, recovery, and compliance standards. - Familiarity with database monitoring tools and analysis. - Strong scripting skills (PowerShell, T-SQL, or others). - Relevant certifications such as Microsoft Certified: Azure Data Fundamentals(DP 900) or Azure Database Administrator Associate (DP-300) or equivalent. Education Qualificaiton B.Tech Job Title Senior Database Developer Roles &amp; Responsibilities About the Role: - You will design, implement, and maintain database systems to support web application and services. - Your role involves optimizing database performance and ensuring data integrity and security. - You will collaborate closely with developers and other stakeholders to align database design with application requirements. About the Team: - You will be part of a dynamic and skilled team of database professionals committed to achieving excellence. - You will have opportunities to mentor junior team members and contribute to team growth through innovation. You are Responsible for: - Design and develop scalable database architecture. - Database migration from legacy system to Azure SQL Managed Instance using Azure Data Factory and other ETL tools. - Write complex SQL queries, stored procedures, and scripts for advanced data processing. - Perform database performance tuning and query optimization. - Implement data security measures, backup, and disaster recovery solutions. - Manage batch job processing, triggers, scheduled jobs and automate data workflows. - Monitor database health using Azure tools and perform root cause analysis for problems. - Collaborate on database design and implementation with developers and infrastructure teams. - Maintain data integrity and consistency during migration and operations. - Document migration plans, architecture, and technical details for project and compliance purposes. - Monitoring and tuning database performance, identifying bottlenecks, and implementing necessary improvements. - Ensuring effective backup and recovery strategies are in place to safeguard critical data. Project Details This project is a green field development of a cloud web application to support the organization services with the modern architecture. The web application will be used by external customers; hence the solution has to be efficient, scalable and tailored to organization needs.</p></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Database developer,Azure Sql managed instance,Stored Procedures,Triggers</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Lead I - Data Engineering

Amco IT Systems

<p><b>Job Title: Snowflake Developer</b></p><p><b>Location: Saudi Arabia</b></p><p><b>Experience: 3&ndash;5 years</b></p><p><b>Job Summary:</b></p><p>We&rsquo;re looking for a Snowflake Developer to build and optimize data pipelines and warehouse solutions. The ideal candidate should have hands-on experience with Snowflake architecture, strong SQL skills, and expertise in data modeling and workflow automation.</p><p><b>Key Responsibilities:</b></p><ul><li>Design and optimize Snowflake-based data pipelines.</li><li>Implement Snowpipe, Streams, Tasks, and Virtual Warehouses.</li><li>Automate workflows using Airflow or Luigi.</li><li>Integrate Snowflake with GCP and other cloud platforms.</li></ul><p><b>Requirements</b>:</p><ul><li>3&ndash;5 years&rsquo; experience in data engineering.</li><li>Strong SQL and data warehouse skills.</li><li>Proficiency in Snowflake and cloud integrations (preferably GCP).</li></ul><p>Job Type: Contract<br/>Contract length: 6 months</p><p>Ability to commute/relocate:</p><ul><li>Riyadh: Reliably commute or planning to relocate before starting work (Required)</li></ul><p>Application Question(s):</p><ul><li>What is your Nationality?</li><li>What is your notice period in days?</li><li>What is your current monthly salary?</li><li>What is your expected monthly salary in SAR?</li><li>How many years of experience do you have as a Snowflake developer?</li><li>Are you comfortable with 6 months contract (extendable)</li></ul><p>Language:</p><ul><li>Arabic (Required)</li></ul>

Snowflake Developr

<div></div><div><div><div><div><ul><div>3 - 5 Years</div>
<div></div><div>1 Opening</div>
 <div>Noida</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p>Role Proficiency:</p>
<p>
This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles.</p>
<p>
Outcomes:</p>
<div>
<ul><li>Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintain data pipelines that collect process and transform large volumes of data from various sources.</li></ul>
<ul><li>Implement ETL (Extract Transform Load) processes to facilitate efficient data movement and transformation.</li></ul>
<ul><li>Integrate data from multiple sources including databases APIs cloud services and third-party data providers.</li></ul>
<ul><li>Establish data quality checks and validation procedures to ensure data accuracy completeness and consistency.</li></ul>
<ul><li>Develop and manage data storage solutions including relational databases NoSQL databases and data lakes.</li></ul>
<ul><li>Stay updated on the latest trends and best practices in data engineering cloud technologies and big data tools.</li></ul>
</div>
<p></p><p>Measures of Outcomes:</p>
<div>
<ul><li>Adherence to engineering processes and standards</li></ul>
<ul><li>Adherence to schedule / timelines</li></ul>
<ul><li>Adhere to SLAs where applicable</li></ul>
<ul><li># of defects post delivery</li></ul>
<ul><li># of non-compliance issues</li></ul>
<ul><li>Reduction of reoccurrence of known defects</li></ul>
<ul><li>Quickly turnaround production bugs</li></ul>
<ul><li>Completion of applicable technical/domain certifications</li></ul>
<ul><li>Completion of all mandatory training requirementst</li></ul>
<ul><li>Efficiency improvements in data pipelines (e.g. reduced resource consumption faster run times).</li></ul>
<ul><li>Average time to detect respond to and resolve pipeline failures or data issues.</li></ul>
</div>
<p></p><p>Outputs Expected:</p>
<p>
Code Development:</p>
<div>
<ul><li>Develop data processing code independently<br>
ensuring it meets performance and scalability requirements.</li></ul></div><br>
<p><br>
Documentation:</p>
<div>
<ul><li>Create documentation for personal work and review deliverable documents<br>
including source-target mappings<br>
test cases<br>
and results.</li></ul></div><br>
<p><br>
Configuration:</p>
<div>
<ul><li>Follow configuration processes diligently.</li></ul></div><br>
<p><br>
Testing:</p>
<div>
<ul><li>Create and conduct unit tests for data pipelines and transformations to ensure data quality and correctness.</li></ul>
<ul><li>Validate the accuracy and performance of data processes.</li></ul></div><br>
<p><br>
Domain Relevance:</p>
<div>
<ul><li>Develop features and components with a solid understanding of the business problems being addressed for the client.</li></ul>
<ul><li>Understand data schemas in relation to domain-specific contexts<br>
such as EDI formats.</li></ul></div><br>
<p><br>
Defect Management:</p>
<div>
<ul><li>Raise<br>
fix<br>
and retest defects in accordance with project standards.</li></ul></div><br>
<p><br>
Estimation:</p>
<div>
<ul><li>Estimate time<br>
effort<br>
and resource dependencies for personal work.</li></ul></div><br>
<p><br>
Knowledge Management:</p>
<div>
<ul><li>Consume and contribute to project-related documents<br>
SharePoint<br>
libraries<br>
and client universities.</li></ul></div><br>
<p><br>
Design Understanding:</p>
<div>
<ul><li>Understand design and low-level design (LLD) and link it to requirements and user stories.</li></ul></div><br>
<p><br>
Certifications:</p>
<div>
<ul><li>Obtain relevant technology certifications to enhance skills and knowledge.</li></ul>
</div>
<p></p><p>Skill Examples:</p>
<div>
<ul><li>Proficiency in SQL Python or other programming languages utilized for data manipulation.</li></ul>
<ul><li>Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF.</li></ul>
<ul><li>Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services (e.g. AWS Glue BigQuery).</li></ul>
<ul><li>Conduct tests on data pipelines and evaluate results against data quality and performance specifications.</li></ul>
<ul><li>Experience in performance tuning data processes.</li></ul>
<ul><li>Proficiency in querying data warehouses.</li></ul>
</div>
<p></p><p>Knowledge Examples:</p>
<p>
Knowledge Examples</p>
<div>
<ul><li>Knowledge of various ETL services provided by cloud providers including Apache PySpark AWS Glue GCP DataProc/DataFlow and Azure ADF/ADLF.</li></ul>
<ul><li>Understanding of data warehousing principles and practices.</li></ul>
<ul><li>Proficiency in SQL for analytics including windowing functions.</li></ul>
<ul><li>Familiarity with data schemas and models.</li></ul>
<ul><li>Understanding of domain-related data and its implications.</li></ul>
</div>
<p></p><p>Additional Comments:</p>
<p></p>
<p>Role Proficiency: This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles. Outcomes: Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintain data pipelines that collect process and transform large volumes of data from various sources. Implement ETL (Extract Transform Load) processes to facilitate efficient data movement and transformation. Integrate data from multiple sources including databases APIs cloud services and third-party data providers. Establish data quality checks and validation procedures to ensure data accuracy completeness and consistency. Develop and manage data storage solutions including relational databases NoSQL databases and data lakes. Stay updated on the latest trends and best practices in data engineering cloud technologies and big data tools.</p></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Cloud Platform,Docker,Kubernetes</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Associate III - Data Engineering

<div></div><div><div><div><div><ul><div>3 - 5 Years</div>
<div></div><div>1 Opening</div>
 <div>Bangalore</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p>Role Proficiency:</p>
<p>
This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be adept at using ETL tools such as Informatica Glue Databricks and DataProc with coding skills in Python PySpark and SQL. Works independently and demonstrates proficiency in at least one domain related to data with a solid understanding of SCD concepts and data warehousing principles.</p>
<p>
Outcomes:</p>
<div>
<ul><li>Collaborate closely with data analysts data scientists and other stakeholders to ensure data accessibility quality and security across various data sources.rnDesign develop and maintain data pipelines that collect process and transform large volumes of data from various sources.</li></ul>
<ul><li>Implement ETL (Extract Transform Load) processes to facilitate efficient data movement and transformation.</li></ul>
<ul><li>Integrate data from multiple sources including databases APIs cloud services and third-party data providers.</li></ul>
<ul><li>Establish data quality checks and validation procedures to ensure data accuracy completeness and consistency.</li></ul>
<ul><li>Develop and manage data storage solutions including relational databases NoSQL databases and data lakes.</li></ul>
<ul><li>Stay updated on the latest trends and best practices in data engineering cloud technologies and big data tools.</li></ul>
</div>
<p></p><p>Measures of Outcomes:</p>
<div>
<ul><li>Adherence to engineering processes and standards</li></ul>
<ul><li>Adherence to schedule / timelines</li></ul>
<ul><li>Adhere to SLAs where applicable</li></ul>
<ul><li># of defects post delivery</li></ul>
<ul><li># of non-compliance issues</li></ul>
<ul><li>Reduction of reoccurrence of known defects</li></ul>
<ul><li>Quickly turnaround production bugs</li></ul>
<ul><li>Completion of applicable technical/domain certifications</li></ul>
<ul><li>Completion of all mandatory training requirementst</li></ul>
<ul><li>Efficiency improvements in data pipelines (e.g. reduced resource consumption faster run times).</li></ul>
<ul><li>Average time to detect respond to and resolve pipeline failures or data issues.</li></ul>
</div>
<p></p><p>Outputs Expected:</p>
<p>
Code Development:</p>
<div>
<ul><li>Develop data processing code independently<br>
ensuring it meets performance and scalability requirements.</li></ul></div><br>
<p><br>
Documentation:</p>
<div>
<ul><li>Create documentation for personal work and review deliverable documents<br>
including source-target mappings<br>
test cases<br>
and results.</li></ul></div><br>
<p><br>
Configuration:</p>
<div>
<ul><li>Follow configuration processes diligently.</li></ul></div><br>
<p><br>
Testing:</p>
<div>
<ul><li>Create and conduct unit tests for data pipelines and transformations to ensure data quality and correctness.</li></ul>
<ul><li>Validate the accuracy and performance of data processes.</li></ul></div><br>
<p><br>
Domain Relevance:</p>
<div>
<ul><li>Develop features and components with a solid understanding of the business problems being addressed for the client.</li></ul>
<ul><li>Understand data schemas in relation to domain-specific contexts<br>
such as EDI formats.</li></ul></div><br>
<p><br>
Defect Management:</p>
<div>
<ul><li>Raise<br>
fix<br>
and retest defects in accordance with project standards.</li></ul></div><br>
<p><br>
Estimation:</p>
<div>
<ul><li>Estimate time<br>
effort<br>
and resource dependencies for personal work.</li></ul></div><br>
<p><br>
Knowledge Management:</p>
<div>
<ul><li>Consume and contribute to project-related documents<br>
SharePoint<br>
libraries<br>
and client universities.</li></ul></div><br>
<p><br>
Design Understanding:</p>
<div>
<ul><li>Understand design and low-level design (LLD) and link it to requirements and user stories.</li></ul></div><br>
<p><br>
Certifications:</p>
<div>
<ul><li>Obtain relevant technology certifications to enhance skills and knowledge.</li></ul>
</div>
<p></p><p>Skill Examples:</p>
<div>
<ul><li>Proficiency in SQL Python or other programming languages utilized for data manipulation.</li></ul>
<ul><li>Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF.</li></ul>
<ul><li>Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services (e.g. AWS Glue BigQuery).</li></ul>
<ul><li>Conduct tests on data pipelines and evaluate results against data quality and performance specifications.</li></ul>
<ul><li>Experience in performance tuning data processes.</li></ul>
<ul><li>Proficiency in querying data warehouses.</li></ul>
</div>
<p></p><p>Knowledge Examples:</p>
<p>
Knowledge Examples</p>
<div>
<ul><li>Knowledge of various ETL services provided by cloud providers including Apache PySpark AWS Glue GCP DataProc/DataFlow and Azure ADF/ADLF.</li></ul>
<ul><li>Understanding of data warehousing principles and practices.</li></ul>
<ul><li>Proficiency in SQL for analytics including windowing functions.</li></ul>
<ul><li>Familiarity with data schemas and models.</li></ul>
<ul><li>Understanding of domain-related data and its implications.</li></ul>
</div>
<p></p><p>Additional Comments:</p>
<p></p>
<p>Job Title: Junior Data Engineer Experience: 2&ndash;4 years Employment Type: Full-time ________________________________________ Role Summary We are looking for a Junior Data Engineer who is passionate about building scalable data pipelines and working with modern data technologies. The ideal candidate should have hands-on experience in ETL development, data integration, and cloud-based data solutions&mdash;preferably on AWS. You will work closely with senior data engineers, data architects, and analysts to develop and maintain data pipelines, ensuring data is accurate, reliable, and available for business and analytics needs. This is an excellent opportunity to grow into a data engineering expert in a modern cloud ecosystem. ________________________________________ Key Responsibilities &bull; Design, develop, and maintain ETL/ELT pipelines for structured and unstructured data sources. &bull; Work with senior engineers to implement data lake and data warehouse solutions using AWS services such as S3, Glue, Redshift, and Lambda. &bull; Develop and optimize Spark / Databricks jobs for data transformation and processing. &bull; Write efficient Python or SQL scripts for data cleaning, transformation, and validation. &bull; Collaborate with data architects to implement data models and data quality frameworks. &bull; Participate in code reviews, testing, and deployment processes to maintain code quality and stability. &bull; Troubleshoot and resolve data-related issues in development and production environments. &bull; Document technical processes, data flows, and pipeline logic for maintainability and transparency. &bull; Learn and contribute to continuous improvements in automation, performance, and scalability of data systems. ________________________________________ Technical Skills and Tools Mandatory: &bull; Good understanding of ETL concepts, data warehousing, and data integration techniques. &bull; Hands-on experience with Python for data manipulation and scripting. &bull; Working knowledge of SQL and experience writing optimized queries. &bull; Familiarity with AWS data services like S3, Glue, Redshift, Athena, or equivalent cloud tools (Azure Data Factory, GCP BigQuery). &bull; Basic exposure to Apache Spark or Databricks for distributed data processing. &bull; Version control and collaboration tools: Git / GitHub / Bitbucket. Good-to-Have: &bull; Exposure to Airflow or any job orchestration tool. &bull; Knowledge of data modelling and dimensional design principles. &bull; Understanding of data quality checks, logging, and monitoring. &bull; Familiarity with DevOps / CI-CD pipelines. &bull; Awareness of streaming data concepts (Kafka, Kinesis). ________________________________________ Soft Skills &amp; Attributes &bull; Strong analytical and problem-solving skills with attention to detail. &bull; Willingness to learn and adapt quickly in a fast-paced, data-driven environment. &bull; Good communication and collaboration skills to work effectively within cross-functional teams. &bull; A proactive attitude toward automation, documentation, and best practices. &bull; Curiosity to explore emerging tools and technologies in the data ecosystem. ________________________________________ Education &amp; Experience &bull; Bachelor&rsquo;s degree in Computer Science, Information Technology, or related discipline. &bull; 2&ndash;4 years of hands-on experience in data engineering, ETL development, or data integration. &bull; Prior experience working in a cloud data environment (AWS / Azure / GCP) preferred. ________________________________________ Preferred Qualifications &bull; AWS Certified Data Practitioner or Data Engineer Associate (or equivalent certification on Azure/GCP). &bull; Exposure to modern data stack tools such as dbt, Snowflake, or Delta Lake. &bull; Experience contributing to end-to-end data pipeline projects in production environments.</p></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>ETL,Data Warehousing,AWS</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

<div><p><b>Line of Service
</b></p>Advisory
<p></p><p><b>Industry/Sector
</b></p>Technology
<p></p><p><b>Specialism
</b></p>Advisory - Other
<p></p><p><b>Management Level
</b></p>Associate
<p></p><p><b>Job Description &amp; Summary
</b></p>At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth.
<br>
<br>
In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.
<p></p><p>Driven by curiosity, you are a reliable, contributing member of a team. In our fast-paced environment, you are expected to adapt to working with a variety of clients and team members, each presenting varying challenges and scope. Every experience is an opportunity to learn and grow. You are expected to take ownership and consistently deliver quality work that drives value for our clients and success as a team. As you navigate through the Firm, you build a brand for yourself, opening doors to more opportunities.
</p><p></p><p>Examples of the skills, knowledge, and experiences you need to lead and deliver value at this level include but are not limited to:
</p><p></p><ul><li>Apply a learning mindset and take ownership for your own development.
</li><li>Appreciate diverse perspectives, needs, and feelings of others.
</li><li>Adopt habits to sustain high performance and develop your potential.
</li><li>Actively listen, ask questions to check understanding, and clearly express ideas.
</li><li>Seek, reflect, act on, and give feedback.
</li><li>Gather information from a range of sources to analyse facts and discern patterns.
</li><li>Commit to understanding how the business works and building commercial awareness.
</li><li>Learn and apply professional and technical standards (e.g. refer to specific PwC tax and audit guidance), uphold the Firm's code of conduct and independence requirements.
</li></ul><p></p><p><b>Education </b><i>(if blank, degree and/or field of study not specified)
</i></p>Degrees/Field of Study required:
<p></p>Degrees/Field of Study preferred:
<p></p><p><b>Certifications </b><i>(if blank, certifications not specified)
</i></p><p></p><p><b>Required Skills
</b></p><p></p><p><b>Optional Skills
</b></p>Accepting Feedback, Accepting Feedback, Active Listening, Agile Scalability, Amazon Web Services (AWS), Apache Airflow, Apache Hadoop, Azure Data Factory, Communication, Data Anonymization, Data Architecture, Database Administration, Database Management System (DBMS), Database Optimization, Database Security Best Practices, Databricks Unified Data Analytics Platform, Data Engineering, Data Engineering Platforms, Data Infrastructure, Data Integration, Data Lake, Data Modeling, Data Pipeline, Data Quality, Data Strategy {+ 22 more}
<p></p><p><b>Desired Languages </b><i>(If blank, desired languages not specified)
</i></p><p></p><p><b>Travel Requirements
</b></p>Not Specified
<p></p><p><b>Available for Work Visa Sponsorship?
</b></p>No
<p></p><p><b>Government Clearance Required?
</b></p>No
<p></p><p><b>Job Posting End Date</b></p></div><p></p>

ETIC, Data Engineer, Associate

Hypr4 Cloud Tech

<p>Data Infrastructure &amp; Pipeline Development: &#9679; Design, develop, and optimize scalable, efficient, and reliable data pipelines for large-scale data processing and transformation. &#9679; Manage and maintain data architecture, ensuring high availability and performance using tools like Snowflake, Dataproc, BigQuery and other cloud technologies. &#9679; Lead the integration of data sources from multiple systems, ensuring seamless data flow across various platforms.</p><p>&#9679; Build and optimize data pipelines using BigQuery, Snowflake, DBT Cloud, and Airflow.</p><p>&#9679; Expertise in Data Modelling to desing and build Data warehouses, Data Marts and Data lakes</p><p>&#9679; Manage version control and workflows with GitHub. Performance &amp; Optimization:</p><p>&#9679; Perform tuning and optimization of queries and data pipelines to ensure high-performance data systems.</p><p>&#9679; Conduct regular performance reviews and recommend improvements or optimizations for system reliability, speed, and cost-efficiency. DBT (Data Build Tool) Implementation:</p><p>&#9679; Implement and maintain DBT models for data transformation workflows.</p><p>&#9679; Collaborate with data analysts and data scientists to ensure high-quality, well-documented datasets for downstream analysis.</p><p>&#9679; Ensure the use of best practices for DBT testing, version control, and deployment. Snowflake Management:</p><p>&#9679; Architect and optimize Snowflake data warehouse environments.</p><p>&#9679; Oversee and manage Snowflake data ingestion, transformation, and storage strategies.</p><p>&#9679; Collaborate with cross-functional teams to ensure Snowflake is being utilized effectively and efficiently. Leadership &amp; Mentorship:</p><p>&#9679; Lead and mentor a team of data engineers, ensuring that best practices are followed in development and deployment of data pipelines.</p><p>&#9679; Conduct code reviews, provide feedback, and ensure the implementation of high-quality data solutions.</p><p>Preferred Skills:</p><p>&#9679; 10+ years of experience in Data Engineering with a strong focus on data warehousing, ETL pipelines, and big data technologies.</p><p>&#9679; At least 3-5 years of hands-on experience with Snowflake data warehouse or BigQuery, including setup, configuration, optimization, and maintenance.</p><p>&#9679; Proficiency in SQL for query optimization and performance tuning.</p><p>&#9679; In-depth experience with Dataproc for running large-scale data processing workflows (e.g., Spark, Hadoop).</p><p>&#9679; Expertise with DBT or any other ELT tool for data transformation and model building. Technical Skills:</p><p>&#9679; Strong experience in cloud platforms like AWS, GCP, or Azure, with a focus on data engineering tools and services.</p><p>&#9679; Proficient in programming/scripting languages such as Python, Java, or Scala for data processing.</p><p>&#9679; Experience with CI/CD pipelines and version control (Git, Jenkins, etc.).</p><p>&#9679; Knowledge of distributed computing frameworks (e.g., Spark, Hadoop) and related data processing concepts. Data Architecture &amp; Design:</p><p>&#9679; Experience with building and maintaining data warehouses and lakes.</p><p>&#9679; Strong understanding of data modeling concepts, data quality, and governance.</p><p>&#9679; Familiarity with Kafka, Airflow, or similar tools for orchestrating data workflows.</p><p>Job Types: Full-time, Permanent, Fresher</p><p>Pay: &#8377;2,540,454.62 - &#8377;2,764,456.72 per year</p><p>Work Location: In person</p>

Data lead

<div></div><div><div><div><div><ul><div>7 - 9 Years</div>
<div></div><div>1 Opening</div>
 <div>Bangalore, Hyderabad, Pune, Trivandrum</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p>Role Proficiency:</p>
<p>
This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL. This position demands independence and proficiency across various data domains. Expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required.</p>
<p>
Outcomes:</p>
<div>
<ul><li>Act creatively to develop pipelines/applications by selecting appropriate technical options optimizing application development maintenance and performance through design patterns and reusing proven solutions. Support the Project Manager in day-to-day project execution and account for the developmental activities of others.</li></ul>
<ul><li>Interpret requirements create optimal architecture and design solutions in accordance with specifications.</li></ul>
<ul><li>Document and communicate milestones/stages for end-to-end delivery.</li></ul>
<ul><li>Code using best standards debug and test solutions to ensure best-in-class quality.</li></ul>
<ul><li>Tune performance of code and align it with the appropriate infrastructure understanding cost implications of licenses and infrastructure.</li></ul>
<ul><li>Create data schemas and models effectively.</li></ul>
<ul><li>Develop and manage data storage solutions including relational databases NoSQL databases Delta Lakes and data lakes.</li></ul>
<ul><li>Validate results with user representatives integrating the overall solution.</li></ul>
<ul><li>Influence and enhance customer satisfaction and employee engagement within project teams.</li></ul>
</div>
<p></p><p>Measures of Outcomes:</p>
<div>
<ul><li>TeamOne's Adherence to engineering processes and standards</li></ul>
<ul><li>TeamOne's Adherence to schedule / timelines</li></ul>
<ul><li>TeamOne's Adhere to SLAs where applicable</li></ul>
<ul><li>TeamOne's # of defects post delivery</li></ul>
<ul><li>TeamOne's # of non-compliance issues</li></ul>
<ul><li>TeamOne's Reduction of reoccurrence of known defects</li></ul>
<ul><li>TeamOne's Quickly turnaround production bugs</li></ul>
<ul><li>Completion of applicable technical/domain certifications</li></ul>
<ul><li>Completion of all mandatory training requirementst</li></ul>
<ul><li>Efficiency improvements in data pipelines (e.g. reduced resource consumption faster run times).</li></ul>
<ul><li>TeamOne's Average time to detect respond to and resolve pipeline failures or data issues.</li></ul>
<ul><li>TeamOne's Number of data security incidents or compliance breaches.</li></ul>
</div>
<p></p><p>Outputs Expected:</p>
<p>
Code:</p>
<div>
<ul><li>Develop data processing code with guidance<br>
ensuring performance and scalability requirements are met.</li></ul>
<ul><li>Define coding standards<br>
templates<br>
and checklists.</li></ul>
<ul><li>Review code for team and peers.</li></ul></div><br>
<p><br>
Documentation:</p>
<div>
<ul><li>Create/review templates<br>
checklists<br>
guidelines<br>
and standards for design/process/development.</li></ul>
<ul><li>Create/review deliverable documents<br>
including design documents<br>
architecture documents<br>
infra costing<br>
business requirements<br>
source-target mappings<br>
test cases<br>
and results.</li></ul></div><br>
<p><br>
Configure:</p>
<div>
<ul><li>Define and govern the configuration management plan.</li></ul>
<ul><li>Ensure compliance from the team.</li></ul></div><br>
<p><br>
Test:</p>
<div>
<ul><li>Review/create unit test cases<br>
scenarios<br>
and execution.</li></ul>
<ul><li>Review test plans and strategies created by the testing team.</li></ul>
<ul><li>Provide clarifications to the testing team.</li></ul></div><br>
<p><br>
Domain Relevance:</p>
<div>
<ul><li>Advise data engineers on the design and development of features and components<br>
leveraging a deeper understanding of business needs.</li></ul>
<ul><li>Learn more about the customer domain and identify opportunities to add value.</li></ul>
<ul><li>Complete relevant domain certifications.</li></ul></div><br>
<p><br>
Manage Project:</p>
<div>
<ul><li>Support the Project Manager with project inputs.</li></ul>
<ul><li>Provide inputs on project plans or sprints as needed.</li></ul>
<ul><li>Manage the delivery of modules.</li></ul></div><br>
<p><br>
Manage Defects:</p>
<div>
<ul><li>Perform defect root cause analysis (RCA) and mitigation.</li></ul>
<ul><li>Identify defect trends and implement proactive measures to improve quality.</li></ul></div><br>
<p><br>
Estimate:</p>
<div>
<ul><li>Create and provide input for effort and size estimation<br>
and plan resources for projects.</li></ul></div><br>
<p><br>
Manage Knowledge:</p>
<div>
<ul><li>Consume and contribute to project-related documents<br>
SharePoint<br>
libraries<br>
and client universities.</li></ul>
<ul><li>Review reusable documents created by the team.</li></ul></div><br>
<p><br>
Release:</p>
<div>
<ul><li>Execute and monitor the release process.</li></ul></div><br>
<p><br>
Design:</p>
<div>
<ul><li>Contribute to the creation of design (HLD<br>
LLD<br>
SAD)/architecture for applications<br>
business components<br>
and data models.</li></ul></div><br>
<p><br>
Interface with Customer:</p>
<div>
<ul><li>Clarify requirements and provide guidance to the Development Team.</li></ul>
<ul><li>Present design options to customers.</li></ul>
<ul><li>Conduct product demos.</li></ul>
<ul><li>Collaborate closely with customer architects to finalize designs.</li></ul></div><br>
<p><br>
Manage Team:</p>
<div>
<ul><li>Set FAST goals and provide feedback.</li></ul>
<ul><li>Understand team members' aspirations and provide guidance and opportunities.</li></ul>
<ul><li>Ensure team members are upskilled.</li></ul>
<ul><li>Engage the team in projects.</li></ul>
<ul><li>Proactively identify attrition risks and collaborate with BSE on retention measures.</li></ul></div><br>
<p><br>
Certifications:</p>
<div>
<ul><li>Obtain relevant domain and technology certifications.</li></ul>
</div>
<p></p><p>Skill Examples:</p>
<div>
<ul><li>Proficiency in SQL Python or other programming languages used for data manipulation.</li></ul>
<ul><li>Experience with ETL tools such as Apache Airflow Talend Informatica AWS Glue Dataproc and Azure ADF.</li></ul>
<ul><li>Hands-on experience with cloud platforms like AWS Azure or Google Cloud particularly with data-related services (e.g. AWS Glue BigQuery).</li></ul>
<ul><li>Conduct tests on data pipelines and evaluate results against data quality and performance specifications.</li></ul>
<ul><li>Experience in performance tuning.</li></ul>
<ul><li>Experience in data warehouse design and cost improvements.</li></ul>
<ul><li>Apply and optimize data models for efficient storage retrieval and processing of large datasets.</li></ul>
<ul><li>Communicate and explain design/development aspects to customers.</li></ul>
<ul><li>Estimate time and resource requirements for developing/debugging features/components.</li></ul>
<ul><li>Participate in RFP responses and solutioning.</li></ul>
<ul><li>Mentor team members and guide them in relevant upskilling and certification.</li></ul>
</div>
<p></p><p>Knowledge Examples:</p>
<p>
Knowledge Examples</p>
<div>
<ul><li>Knowledge of various ETL services used by cloud providers including Apache PySpark AWS Glue GCP DataProc/Dataflow Azure ADF and ADLF.</li></ul>
<ul><li>Proficient in SQL for analytics and windowing functions.</li></ul>
<ul><li>Understanding of data schemas and models.</li></ul>
<ul><li>Familiarity with domain-related data.</li></ul>
<ul><li>Knowledge of data warehouse optimization techniques.</li></ul>
<ul><li>Understanding of data security concepts.</li></ul>
<ul><li>Awareness of patterns frameworks and automation practices.</li></ul>
</div>
<p></p><p>Additional Comments:</p>
<p></p>
<p>Responsibilities: &bull; Design, configure, and deploy Amazon Connect contact center solutions. &bull; Integrate Amazon Connect with other AWS services (Lambda, Lex, S3, DynamoDB, etc.). &bull; Develop and maintain custom contact flows, routing profiles, and agent hierarchies. &bull; Implement real-time and historical reporting using AWS tools and SQL queries. &bull; Collaborate with DevOps and engineering teams to ensure seamless deployment and monitoring. &bull; Troubleshoot and optimize existing Amazon Connect implementations. &bull; Ensure compliance with security and data privacy standards. &bull; Provide technical documentation and training to internal stakeholders. Required Skills &amp; Qualifications: Amazon Connect Expertise (Expanded): &bull; Build and optimize complex contact flows using Amazon Connect&rsquo;s drag-and-drop interface. &bull; Implement dynamic routing logic based on customer inputs, agent availability, and business rules. &bull; Configure whisper prompts, queue hold messages, and error handling flows. &bull; Define agent hierarchies, routing profiles, and user permissions. &bull; Set up queues, hours of operation, and escalation paths. &bull; Monitor agent performance and optimize staffing using historical and real-time metrics. &bull; Connect Amazon Connect with AWS Lambda for serverless backend logic. &bull; Use Amazon Lex for conversational IVR and chatbot capabilities. &bull; Store and retrieve customer data using Amazon DynamoDB or S3 during interactions. &bull; Customize and extend reporting using Amazon Connect&rsquo;s Contact Trace Records (CTR). &bull; Use CloudWatch for monitoring, ing, and logging of contact center metrics. &bull; Implement role-based access control using AWS IAM. &bull; Ensure encryption of data in transit and at rest. &bull; Maintain compliance with industry standards (e.g., HIPAA, PCI-DSS) where applicable. &bull; Integrate with CRMs like ServiceNow. &bull; Use APIs and webhooks to connect with external systems for ticketing, customer data, and notifications. &bull; Diagnose call quality issues, latency, and flow errors. AWS Development Skills: &bull; Proficiency in AWS Lambda (Node.js or Python). &bull; Experience with API Gateway, CloudFormation, IAM, S3, DynamoDB, and CloudWatch. &bull; Familiarity with AWS SDKs and CLI tools. &bull; Understanding of serverless architecture and event-driven design. SQL &amp; Data Handling: &bull; Ability to write basic to intermediate SQL queries. &bull; Experience with relational databases (e.g., PostgreSQL). &bull; Understanding of data modeling and reporting. &bull; Experience with Dynamodb General Skills: &bull; Strong problem-solving and analytical skills. &bull; Excellent communication and documentation abilities. &bull; Ability to work independently and in a team environment. &bull; Agile/Scrum experience is a plus. Preferred Qualifications: &bull; AWS Certified Solutions Architect or Developer. &bull; Experience with Amazon Lex, Polly, and Transcribe. &bull; Knowledge of contact center KPIs and analytics.</p></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Aws Services,amazon connect,Sql,Analytical</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Lead II - Data Engineering

Search 10000+ Data pipeline developer jobs near you this October. New roles added regularly across leading industries. Apply today.

10000 data pipeline developer jobs NEAR_ME

Lead I - Data Engineering

Snowflake Developr

Associate III - Data Engineering

Associate III - Data Engineering

ETIC, Data Engineer, Associate

Data lead

Lead II - Data Engineering