azure data engineer

Redlinux

Freelancing OpportunityJob Title: Python Data EngineerExperience: 5+ YearsJob Summary:We are looking for an experienced Python Data Engineer with deep expertise in Azure Data Factory (ADF), ETL processes, and Azure Functions. The ideal candidate will handle ADF pipeline corrections, enhancements, and integrate Azure Functions effectively within ADF workflows to deliver optimized, scalable, and reliable data solutions.Key Responsibilities:<ul><li>Design, build, and enhance ETL workflows in Azure Data Factory (ADF) for seamless data ingestion, transformation, and loading.</li><li>Develop and maintain Python scripts for complex data transformations, data quality checks, and automation tasks.</li><li>Integrate and optimize Azure Functions within ADF pipelines for advanced data processing and orchestration.</li><li>Identify and resolve ADF pipeline issues, performance bottlenecks, and deployment challenges.</li><li>Troubleshoot and fix space utilization and performance issues in Azure Functions and data pipelines.</li><li>Work with Azure Storage, Data Lake, Key Vault, Synapse Analytics, and other Azure services as part of end-to-end data workflows.</li><li>Ensure high performance, reliability, and reusability of all ADF components and Python-based utilities.</li><li>Collaborate with architects, analysts, and DevOps teams for version control, CI/CD, and production support.</li></ul>Required Skills &amp; Experience:<ul><li>5+ years of hands-on experience in Data Engineering or ETL development.</li><li>Strong expertise in Azure Data Factory (ADF) &mdash; building, debugging, and enhancing data pipelines.</li><li>Proven proficiency in Python programming, including complex data manipulation, exception handling, and performance tuning.</li><li>Experience integrating and invoking Azure Functions within ADF pipelines.</li><li>Solid understanding of ETL concepts, data flow orchestration, and dependency management.</li><li>Working knowledge of Azure services &mdash; Blob Storage, Data Lake, Key Vault, and Synapse Analytics.</li><li>Good SQL skills for querying, validation, and performance optimization.</li><li>Familiarity with Azure DevOps / Git for version control and CI/CD.</li></ul>Preferred Qualifications:<ul><li>Experience with Databricks, PySpark, or ADF Mapping Data Flows.</li><li>Understanding of event-driven data architectures and serverless design patterns.</li><li>Excellent analytical, debugging, and problem-solving skills.</li><li>Strong communication and teamwork abilities in a collaborative environment.</li></ul>Job Types: Part-time, Freelance Contract length: 12 monthsPay: &#8377;30,000.00 - &#8377;40,000.00 per monthBenefits:<ul><li>Work from home</li></ul>Experience:<ul><li>total work: 6 years (Required)</li></ul>Shift availability:<ul><li>Night Shift (Required)</li><li>Overnight Shift (Required)</li></ul>Work Location: Remote

Azure Data Engineer

Fiftyfive technologies

<div></div><div><h3 class="jobSectionHeader">About the Role:</h3><div>
We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Azure Data Services, particularly Azure Databricks. The ideal candidate will be responsible for building robust, scalable, and high-performance data solutions to support business analytics and insights. Databricks certification is highly preferred.</div><h3 class="jobSectionHeader">
Key Responsibilities:</h3><ul><li>
Design, develop, and maintain data pipelines and ETL/ELT workflows using Azure Data Factory and Azure Databricks.</li><li>
Build and optimize data lake and data warehouse architectures on Azure.</li><li>
Implement efficient data ingestion, transformation, and cleansing processes from diverse sources.</li><li>
Develop and tune PySpark/SparkSQL code for large-scale data processing.</li><li>
Ensure data quality, lineage, and security compliance across all layers.</li><li>
Collaborate with data analysts, scientists, and business stakeholders to enable data-driven decision-making.</li><li>
Optimize cost and performance of data solutions in the Azure cloud environment.</li></ul><h3 class="jobSectionHeader">
Technical Skills Required:</h3><ul><li>
Mandatory:</li><li>
Azure Databricks (hands-on, advanced level) &ndash; Databricks Certification preferred</li><li>
Azure Data Factory (ADF)</li><li>
Azure Data Lake Storage (ADLS)</li><li>
Azure Synapse Analytics / SQL Data Warehouse</li><li>
Proficiency in PySpark, SQL, and data modeling</li><li>
Good to Have:</li><li>
Experience with Delta Lake, Power BI, or Azure DevOps CI/CD pipelines</li><li>
Scripting experience in Python or Scala</li><li>
Exposure to data governance, monitoring, and optimization</li></ul><h3 class="jobSectionHeader">
Qualifications:</h3><ul><li>
Bachelor&rsquo;s or Master&rsquo;s in Computer Science, Information Technology, or a related field.</li><li>
4&ndash;7 years of proven experience in data engineering, specifically within the Azure ecosystem.</li><li>
Azure Databricks certification is a strong plus.</li></ul><h3 class="jobSectionHeader">
What We&rsquo;re Looking For:</h3><ul><li>
Excellent analytical and problem-solving mindset.</li><li>
Strong communication and collaboration skills.</li><li>
Self-motivated and capable of delivering in fast-paced environments.</li></ul><div>
Let me know if we have any queries in this</div><div>
 Everyone As discussed in morning first complete your sourcing for our existing positions and create a pipeline, then start working on this</div></div><div></div>

PCS Gulf

PCS Gulf excels in IT and financial consulting, driven by the belief that optimal solutions arise from a diverse network of experts collaborating across IT, Finance, and business strategy. Our seasoned team remains industry leaders, staying updated on the latest trends. We provide innovative, cutting-edge solutions tailored to clients' needs, drawing from vast industry exposure. Collaborating closely, we craft comprehensive solutions addressing all aspects of business. With a stellar track record and proficiency, we empower clients, from startups to large corporations, to realize their full potential. At PCS Gulf, our unwavering dedication is to deliver tangible results. 
 Responsibilities: <ul>
 <li>You are detailed reviewing and analysing structured, semi-structured and unstructured data sources for quality, completeness, and business value. </li>
 <li>You design, architect, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences. </li>
 <li>You participate in early state design and feature definition activities. </li>
 <li>Responsible for implementing robust data pipeline using Microsoft,Databricks Stack. </li>
 <li>Responsible for creating reusable and scalable data pipelines. </li>
 <li>You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products. </li>
</ul> Requirements: 
 <ul>
 <li>Advanced working knowledge and experience with relational and non-relational databases </li>
 <li>Experience building and optimizing Big Data data pipelines, architectures, and data sets. </li>
 <li>Strong experience in Azure Datafactory,Databricks, Eventhub, Python,PySpark ,Azure Synapse and SQL </li>
 <li>Azure Devops experience to deploy the ADF pipelines. </li>
 <li> Minimum 3-5 years of experience. </li>
 <li> Knowledge/Experience with Azure cloud stack. </li>
</ul>

Confiz

<div>This role requires a professional with a deep background in data engineering and expertise in designing and implementing cloud-based data warehousing solutions using Azure Synapse. The ideal candidate will be a strategic thinker with a proven record of accomplishment of delivering scalable and efficient data solutions to meet complex business requirements.
Responsibilities:
<ul><li>Design and implement highly scalable, high performance data warehousing solutions in Azure Synapse.</li>
<li>Analyze and migrate existing data systems to Azure Cloud, optimizing data flow and collection to improve data accuracy and utility.</li>
<li>Contribute to data architecture design and modeling initiatives, ensuring alignment with business needs and compliance with data governance standards.</li>
<li>Consult with stakeholders to define data requirements, deliverables, and timelines, providing technical leadership and guidance throughout project lifecycles.</li>
<li>Implement data security and privacy policies in line with industry best practices and organizational standards.</li>
<li>Optimize data integration pipelines for performance and scalability.</li>
<li>Develop and maintain documentation regarding data architectures, procedures, and solutions.</li>
</ul>
Qualifications
<ul><li>Bachelor&rsquo;s or master's degree in computer science, Engineering, Information Technology, or a related field.</li>
<li>5+ years of experience in data engineering with a strong focus on cloud-based solutions.</li>
<li>Expertise in Azure Synapse Analytics, including data integration, management, and security.</li>
<li>Proficient with SQL, Python, PySpark, and other scripting languages commonly used in data engineering.</li>
<li>Familiar with Azure IaC (Infrastructure as Code) and CI/CD automation using ARM, Bicep or Terraform</li>
<li>Deep knowledge of ELT (Extract, Load, Transform) and ETL (Extract, Transform, Load) processes and experience with Azure Data Factory.</li>
<li>Strong understanding of data modeling (Data Vault 2.0 experience or certification preferred), data warehousing, and data lakes.</li>
<li>Proven consulting or advisory roles in the design and deployment of large-scale data solutions in Azure environments.</li>
<li>Experience working directly with clients or stakeholders in a consulting capacity, translating business needs into technical specifications and actionable plans.</li>
<li>Demonstrated ability to contribute to project leadership, coordinate across client functional groups, and manage client relationships.</li>
<li>Microsoft Certified: Azure Data Engineer Associate preferrable</li>
<li>Microsoft Certified: Azure Solutions Architect Expert preferrable</li>
<li>Certified Data Vault 2.0 Practitioner (CDVP2) preferrable</li>
</ul>
We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Azure Data Engineer - Pyspark

Trekrecruit LLC

Key Skills<ul><li>Proficiency in AWS/Azure with hands-on experience in deploying and managing scalable infrastructure.</li><li>Strong knowledge of Infrastructure as Code (IaC) tools such as Terraform, AWS CloudFormation</li><li>Expertise in containerization and orchestration (Docker, Kubernetes, Helm, Openshift, EKS/AKS).</li><li>Experience building and maintaining CI/CD pipelines using Jenkins, GitHub Actions or Azure DevOps.</li><li>Scripting and automation skills with Python, Bash, or PowerShell.</li><li>Familiarity with configuration management tools (Ansible, Chef, or Puppet).</li><li>Experience in monitoring, logging, and observability using tools like Prometheus, Grafana, ELK/EFK, Datadog, or CloudWatch.</li><li>Strong understanding of networking concepts (VPCs, firewalls, DNS, load balancers, VPNs).</li><li>Solid Linux/Unix system administration and troubleshooting skills.</li><li>Knowledge of cloud security practices (IAM, secrets management, compliance frameworks).</li><li>Ability to work effectively in Agile/DevOps environments with cross-functional teams.</li><li>Strong problem-solving, documentation, and communication skills.</li></ul>Job Type: Contractual / Temporary Contract length: 12 monthsPay: &#8377;517,059.61 - &#8377;900,000.00 per yearBenefits:<ul><li>Health insurance</li><li>Provident Fund</li><li>Work from home</li></ul>

Grant Thornton

<div></div><ul><div>7 - 10 Years</div>
<div></div><div>1 Opening</div>
 <div>Bengaluru</div></ul>
 <div><h3 class="jobSectionHeader">Role description</h3>
<div>Key Responsibilities:
<ul>
<li>Design, develop, and deploy AI/ML models using Microsoft Azure AI services (e.g., Azure Machine Learning, Cognitive Services, OpenAI on Azure) and have experience on Agentic Frameworks.</li>
<li>Collaborate with stakeholders to translate business challenges into AI-based solutions.</li>
<li>Lead end-to-end AI projects: from data collection and preprocessing to model training, evaluation, and deployment.</li>
<li>Integrate open-source frameworks (e.g., TensorFlow, PyTorch, Scikit-learn) into Microsoft cloud environments.</li>
<li>Optimize AI pipelines using MLOps tools (e.g., Azure ML Pipelines, GitHub Actions, MLflow).</li>
<li>Develop data science models for predictive analytics, NLP, computer vision, or generative AI applications.</li>
<li>Champion responsible AI practices, data governance, model fairness, and transparency.</li>
<li>Mentor junior data scientists and engineers; conduct code reviews and knowledge-sharing sessions.</li>
</ul></div>
<h3 class="jobSectionHeader">Skills</h3>
<div>Required Qualifications:
<ul>
<li>10+ years of experience in AI, machine learning, or data science roles.</li>
<li>Strong experience with Microsoft AI/ML stack: Azure Machine Learning, Azure Cognitive Services, Power BI with AI, Azure Synapse Analytics.</li>
<li>Proficiency in open-source AI frameworks: TensorFlow, PyTorch, Hugging Face, Scikit-learn.</li>
<li>Strong programming skills in Python (preferred), R, or C#.</li>
<li>Deep understanding of data modeling, statistical analysis, and algorithm development.</li>
<li>Experience with cloud-native AI deployments and containerization (Docker, Kubernetes).</li>
<li>Hands-on knowledge of MLOps, CI/CD pipelines, and AI lifecycle management.</li>
<li>Experience working in agile and cross-functional teams.</li>
<li>Excellent communication, leadership, and presentation skills.</li>
</ul>
Preferred Qualifications:
<ul>
<li>Microsoft certifications such as:</li>
<ul><li>Azure AI Engineer Associate</li>
<li>Azure Data Scientist Associate</li>
<li>Azure Solutions Architect Expert</li>
</ul><li>Experience integrating AI into Microsoft Power Platform and Dynamics 365.</li>
<li>Exposure to generative AI (e.g., OpenAI, GPT models on Azure, LLM fine-tuning).</li>
<li>Experience in multi-cloud or hybrid-cloud AI architecture.</li>
<li>Familiarity with open-source data versioning and pipeline tools (e.g., DVC, Kedro, Prefect).</li></ul></div></div>
 <div><h3 class="jobSectionHeader">About Grant Thornton INDUS</h3>
<div>&lsquo;Grant Thornton INDUS&rsquo; comprises GT U.S. Shared Services Center India Pvt Ltd and Grant Thornton U.S. Knowledge and Capability Center India Pvt Ltd. Grant Thornton INDUS is the shared services center supporting the operations of Grant Thornton LLP, the U.S. member firm of Grant Thornton International Ltd. Established in 2012, Grant Thornton INDUS employs professionals across a wide range of disciplines including Tax, Audit, Advisory, and other operational functions. What sets us apart isn&rsquo;t just what we do &ndash; it&rsquo;s how we do it. We support and enable the firm&rsquo;s purpose of making business more personal and building trust into every result. We&rsquo;re collaborators &ndash; obsessed with quality and ready for anything &ndash; who understand the value of strong relationships. Our professionals are well integrated to seamlessly support the U.S. engagement teams, help increase Grant Thornton&rsquo;s access to a wide talent pool, and improve operational efficiencies. Empowered people, bold leadership, and distinctive client service are imbibed in the culture at Grant Thornton INDUS. We are a transparent, competitive, and excellence-driven firm that offers an opportunity to be part of something significant. In addition, professionals at Grant Thornton INDUS serve communities in India through inspirational and generous services to give back to the communities they work in. Grant Thornton INDUS has its offices in two locations in India &ndash; Bengaluru and Kolkata</div></div>

Azure Data Engineer - Lead Consultant

FICUSOFT TECHNOLOGIES PVT LTD

<ul><li>Builds deep understanding of the Microsoft Security business, technology, and customers.</li><li>Translate qualitative and quantitative signals from customers into effective insights that drive business and product planning</li><li>Implements scalable data models, data pipelines, data storage, management, and transformation solutions.</li><li>Ensures data quality and compliance.</li><li>Partners with teams across Microsoft &amp; within Identity division to drive customer opportunities and revenue growth.</li><li>Communicates key outcomes with stakeholders and leadership.</li></ul>Job Type: Full-timePay: From &#8377;100.00 per hourBenefits:<ul><li>Health insurance</li></ul>Experience:<ul><li>total work: 1 year (Preferred)</li></ul>Work Location: In person

Azure Data Engineer Trainee

Search 10000+ Azure data engineer jobs near you this October. New roles added regularly across leading industries. Apply today.

10000 azure data engineer jobs NEAR_ME

Azure Data Engineer

Azure Data Engineer

Azure Data Engineer

Azure Data Engineer - Pyspark

Azure Data Engineer

Azure Data Engineer - Lead Consultant

Azure Data Engineer Trainee