databricks engineer

Sierra Support Centre Pvt. Ltd

<p><b>Job Title:</b> Databricks Engineer<br/><b>Shift:</b> Night Shift<br/><b>Experience:</b> Minimum 8+ Years<br/><b>Location:</b> Coimbatore or Chennai<br/><b>Notice Period:</b> Immediate or Max 30 Days</p><p><b>Skills Needed:</b></p><ul><li>Strong Azure Databricks</li></ul><ul><li>Strong Azure Data Engineer</li></ul><p>Job Types: Full-time, Permanent</p><p>Pay: ₹1,000,000.00 - ₹2,000,000.00 per year</p><p>Application Question(s):</p><ul><li>what is your total work experience in Azure Data?</li><li>Are you fine to work in Night Shift?</li><li>Are you fine to work from Office</li></ul><p>Work Location: In person</p>

Databricks Engineer

APPTESTIFY GLOBAL SERVICES PVT LTD

<p><b>Role: Databricks Engineer</b></p><p><b>Location:</b> Remote<br/><b>Experience:</b> 5+ Years</p><p><b>About the Role:</b></p><p>We are looking for an experienced <b>Databricks Engineer</b> with strong expertise in big data processing and analytics. The ideal candidate will work closely with data engineers, data scientists, and business stakeholders to design, develop, and optimize scalable data solutions using Databricks.</p><p><b>Key Responsibilities:</b></p><ul><li>Design, develop, and maintain scalable data pipelines using <b>Databricks</b></li><li>Work with <b>Apache Spark (PySpark/Scala)</b> for large-scale data processing</li><li>Optimize Databricks jobs for performance, cost, and reliability</li><li>Integrate data from multiple sources (structured &amp; unstructured)</li><li>Implement data transformation, cleansing, and validation processes</li><li>Collaborate with cross-functional teams to deliver data-driven solutions</li><li>Ensure data quality, security, and governance best practices</li><li>Troubleshoot and resolve data pipeline and performance issues</li></ul><p><b>Required Skills &amp; Qualifications:</b></p><ul><li>5+ years of experience in <b>Data Engineering / Big Data</b> roles</li><li>Strong hands-on experience with <b>Databricks</b></li><li>Proficiency in <b>Apache Spark, PySpark, or Scala</b></li><li>Experience with <b>SQL</b> and data modeling concepts</li><li>Familiarity with cloud platforms (<b>Azure / AWS / GCP</b>)</li><li>Experience working with <b>ETL/ELT pipelines</b></li><li>Knowledge of data lakes, Delta Lake, and big data ecosystems</li><li>Good understanding of data warehousing and analytics concepts</li></ul><p><b>Nice to Have:</b></p><ul><li>Databricks certifications</li><li>Experience with <b>Delta Live Tables</b> and <b>Unity Catalog</b></li><li>Exposure to CI/CD for data pipelines</li><li>Experience supporting analytics and ML workloads</li></ul><p>Job Types: Full-time, Contractual / Temporary<br/>Contract length: 12 months</p><p>Pay: ₹70,000.00 - ₹80,000.00 per month</p><p>Work Location: Remote</p>

Databricks Engineer/5+ years/Remote

Tata Consultancy Services (TCS)

<div></div><div><div><div><div><p><b>Must Have:</b></p>
<div>
<ul><li><b>Hands-on Experience in ETL t</b>esting with knowledge on Data Warehouse, Data Lake ETL Testing application</li></ul>
<ul><li>Solid expertise in working with ETL / Databricks testing, Informatica tools (IDMC)</li></ul>
<ul><li>Experience with big data technologies (Spark, PySpark, Delta Lake).</li></ul>
<ul><li>Knowledge of data modeling, data quality frameworks, and BI tools</li></ul>
<ul><li>Good Knowledge in Test data articulation
</li></ul></div><p>Good to have ETL &amp; SQL work experience, Databricks, Teradata.</p>
<p><b>Worked on design and development of any digital dashboard preferred</b></p>
<p>Experience with Datawarehouse Automation testing, SQL and test data creation.</p>
<div><ul><li>Good experience in Agile Software Testing Lifecycle</li></ul>
<ul><li>Should have worked on data warehouse projects with good exposure to database technologies like Hadoop, Teradata, Snowflake.</li></ul></div><p><br>
</p></div></div></div><div><div><div><b>Location</b></div>
<div>Hyderabad</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>387014</div>
</div><div><div><b>Desired Skills</b></div>
<div>Python | Spark | ETL Testing</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF TECHNOLOGY</div></div></div><div></div>

Databricks Test Engineer

<div></div><div><div><div><div><div><b>Role:</b> <b>Databricks Engineer</b></div>
<div></div>
<div><b>Must-Have:</b></div>
<div><b>
1. Azure Data Bricks (Python)</b></div>
<div><b>
2. Pyspark</b></div>
<div><b>
3. ADLS</b></div>
<div></div>
<div><b>Responsibility:</b></div>
<ul>
<li>Designing and building Databricks data pipelines is required.</li>
<li>Experience in transforming the data using Azure Databricks &amp; Delta lake.</li>
<li>Hands-on experience implementing Big Data solutions in a cloud ecosystem, including Data/Delta Lakes.</li>
<li>SQL experience, specifically to write complex, highly optimized queries across large volumes of data.</li>
<li>Experience in Azure DevOps CI/CD &ndash; Stories, Bugs, and Issue management.</li>
<li>Implement Integration solution in Microsoft Azure.</li>
<li>Performs data analysis required to troubleshoot data related issues and assist in the resolut</li></ul>
</div></div></div><div><div><div><b>Location</b></div>
<div>Hyderabad</div>
</div><div><div><b>Job Function</b></div>
<div>IT INFRASTRUCTURE SERVICES</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>385636</div>
</div><div><div><b>Desired Skills</b></div>
<div>Azure | MySql
</div></div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF TECHNOLOGY</div></div></div><div></div>

Data Engineer

Cigres Technologies

<p></p><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Bengaluru, Karnataka, India</div></div><div><div>
Job Type</div></div><div><div>
Full Time</div></div></div></div></div></div></div><div></div><div><div><div><div><div><div><div><div><div><div>
About the Role</div></div><div><div>
Mandatory Skills<br>
Databricks Engineer, Power Bi Developer, Dax Queries, SQL, Data Warehousing, Power Automate<br>
<br>
Skill to Evaluate<br>
Databricks Engineer, Power Bi Developer, Dax Queries, SQL, Data Warehousing, Power Automate<br>
<br>
Experience<br>
5 to 9 Years<br>
Key Responsibilities:<br>
<br>
<ul><li>Design and implement scalable data processing solutions using Databricks, including ETL processes that efficiently handle large data volumes.</li><li>Develop interactive dashboards and reports using Power BI to facilitate real-time data visualization and insights for stakeholders.</li><li>Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs and objectives.</li><li>Optimize data storage and retrieval processes to enhance performance and reduce latency.</li><li>Ensure data quality and integrity by implementing best practices for data governance and validation.</li><li>Mentor junior data engineers and contribute to knowledge sharing within the team.</li><li>Stay updated with the latest trends and technologies in data engineering, and evaluate their applicability to the organization's needs.</li></ul><br>
Skills and Tools Required:<br>
<br>
<ul><li>Proficiency in Databricks and experience with Spark for big data processing.</li><li>Strong experience with Power BI, particularly in building reports and dashboards using DAX.</li><li>Solid programming skills in Python or Scala for developing data pipelines.</li><li>Knowledge of SQL and experience with relational databases and data warehousing concepts.</li></ul>
- Familiarity with cloud platforms - Azure, particularly in relation to data storage and processing.<br>
<ul><li>Understanding of data modeling, ETL processes, and data integration methodologies.</li><li>Excellent problem-solving skills and attention to detail.</li><li>Strong communication and collaboration abilities to work effectively in a team-oriented environment.</li></ul><br>
Skill Area Tools &amp; Technologies<br>
Data Engineering Databricks, Apache Spark, Delta Lake<br>
Programming Python, SQL, PySpark<br>
BI &amp; Visualization Power BI, DAX, Power Query<br>
Cloud Platforms Azure<br>
DevOps &amp; CI/CD Git, Azure DevOps<br>
Data Governance Unity Catalog, Data Lake Security<br>
<br>
Preferred Qualifications:<br>
<br>
<ul><li>Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field.</li><li>Previous experience in a similar data engineering role, particularly within a corporate environment.</li><li>Knowledge of additional BI tools and data visualization platforms is a plus.</li></ul></div></div><div><div>
Requirements</div></div><div></div><div><div><div><div><div><div><div><div><div><div><div>
Mandatory Skills</div></div><div><div>
Databricks Engineer, Power Bi Developer, Dax Queries, SQL, Data Warehousing, Power Automate</div></div><div></div><div><div><br>
Skill to Evaluate</div></div><div><div>
Databricks Engineer, Power Bi Developer, Dax Queries, SQL, Data Warehousing, Power Automate</div></div><div></div><div><div><br>
Experience</div></div><div><div>
5 to 9 Years</div></div><div><div>
Key Responsibilities:</div></div><div></div><div><ul><li><br>
Design and implement scalable data processing solutions using Databricks, including ETL processes that efficiently handle large data volumes.</li></ul></div><div><ul><li>Develop interactive dashboards and reports using Power BI to facilitate real-time data visualization and insights for stakeholders.</li></ul></div><div><ul><li>Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs and objectives.</li></ul></div><div><ul><li>Optimize data storage and retrieval processes to enhance performance and reduce latency.</li></ul></div><div><ul><li>Ensure data quality and integrity by implementing best practices for data governance and validation.</li></ul></div><div><ul><li>Mentor junior data engineers and contribute to knowledge sharing within the team.</li></ul></div><div><ul><li>Stay updated with the latest trends and technologies in data engineering, and evaluate their applicability to the organization's needs.</li></ul></div><div></div><div><div><br>
Skills and Tools Required:</div></div><div></div><div><ul><li><br>
Proficiency in Databricks and experience with Spark for big data processing.</li></ul></div><div><ul><li>Strong experience with Power BI, particularly in building reports and dashboards using DAX.</li></ul></div><div><ul><li>Solid programming skills in Python or Scala for developing data pipelines.</li></ul></div><div><ul><li>Knowledge of SQL and experience with relational databases and data warehousing concepts.</li></ul></div><div><div>
- Familiarity with cloud platforms - Azure, particularly in relation to data storage and processing.</div></div><div><ul><li>Understanding of data modeling, ETL processes, and data integration methodologies.</li></ul></div><div><ul><li>Excellent problem-solving skills and attention to detail.</li></ul></div><div><ul><li>Strong communication and collaboration abilities to work effectively in a team-oriented environment.</li></ul></div><div></div><div><div><br>
Skill Area Tools &amp; Technologies</div></div><div><div>
Data Engineering Databricks, Apache Spark, Delta Lake</div></div><div><div>
Programming Python, SQL, PySpark</div></div><div><div>
BI &amp; Visualization Power BI, DAX, Power Query</div></div><div><div>
Cloud Platforms Azure</div></div><div><div>
DevOps &amp; CI/CD Git, Azure DevOps</div></div><div><div>
Data Governance Unity Catalog, Data Lake Security</div></div><div></div><div><div><br>
Preferred Qualifications:</div></div><div></div><div><ul><li><br>
Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field.</li></ul></div><div><ul><li>Previous experience in a similar data engineering role, particularly within a corporate environment.</li></ul></div><div><ul><li>Knowledge of additional BI tools and data visualization platforms is a plus.</li></ul></div></div></div></div></div></div></div></div></div></div><div><div>
About the Company</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div></div>

Senior Data Engineer with Data Bricks

<div></div><div><div><div><div><ul><div>5 - 7 Years</div>
<div></div><div>2 Openings</div>
 <div>Pune</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p><b>Job Title: Senior Data Engineer &ndash; Azure &amp; Databricks</b></p>
<h3 class="jobSectionHeader"><b>
Overview</b></h3>
<p>
We are seeking a highly skilled <b>Senior Data Engineer</b> with <b>5+ years of hands-on experience</b> in building scalable, high-performance data solutions within the <b>Azure ecosystem</b>. The ideal candidate will have strong expertise in <b>Databricks, Apache Spark (Java), modern data lake architectures</b>, and end-to-end ETL pipeline development. This role involves designing, developing, and optimizing data systems that support business-critical analytics and operational use cases.</p>
<p><b>
Key Responsibilities</b></p>
<ul>
<li><p>Design, develop, and maintain <b>scalable data pipelines</b> using Azure services and Databricks.</p>
</li>
<li><p>Build efficient <b>data ingestion and transformation workflows</b> leveraging Databricks, Spark, and Delta Lake.</p>
</li>
<li><p>Develop, enhance, and optimize <b>ETL pipelines using Java, Spark, and SQL</b>.</p>
</li>
<li><p>Implement industry-best practices for <b>data modeling, data quality, governance, and pipeline performance</b>.</p>
</li>
<li><p>Work with relational and lakehouse data platforms, including <b>PostgreSQL and Delta Lake</b>.</p>
</li>
<li><p>Collaborate with cross-functional teams to ensure smooth execution, deployment, and support of production data workloads.</p>
</li>
<li><p>Troubleshoot performance issues and optimize workloads for reliability and scalability.</p>
</li>
</ul><p><b>Required Skills &amp; Experience</b></p>
<ul>
<li><p><b>5+ years</b> of hands-on experience in <b>Data Engineering</b>.</p>
</li>
<li><p>Strong working knowledge of the <b>Azure ecosystem</b>, specifically:</p>
<ul>
<li><p>Azure Databricks</p>
</li>
<li><p>Azure Data Lake Storage (ADLS)</p>
</li>
<li><p>Azure Functions</p>
</li>
<li><p>Azure Data Factory (preferred but not mandatory)</p>
</li>
</ul></li><li><p>Expertise in <b>Apache Spark with Java (Spark Core, Spark SQL, Spark Streaming)</b>.</p>
</li>
<li><p>Advanced <b>SQL</b> skills including performance tuning and complex query optimization.</p>
</li>
<li><p>Strong understanding of <b>ETL/ELT methodologies</b>, data warehousing concepts, and lakehouse architectures.</p>
</li>
<li><p>Experience with <b>Delta Lake features</b> such as schema evolution, ACID transactions, and time travel.</p>
</li>
<li><p>Hands-on experience with <b>PostgreSQL</b> or similar relational databases.</p>
</li>
<li><p>Excellent analytical, problem-solving, and debugging skills.</p>
</li>
<li><p>Strong communication skills with the ability to work with technical and non-technical stakeholders.</p>
</li>
<li><p>Self-motivated, proactive, and comfortable working in <b>fast-paced environments</b>.</p>
</li>
</ul><p><b>Preferred Skills</b></p>
<ul>
<li><p>Experience with <b>CI/CD tools</b> such as Azure DevOps and Git.</p>
</li>
<li><p>Knowledge of <b>Infrastructure-as-Code (IaC)</b> tools like Terraform.</p>
</li>
<li><p>Exposure to <b>streaming technologies</b> such as Kafka or Azure Event Hub.</p>
</li>
<li><p>Familiarity with <b>data governance tools</b> such as Azure Purview.</p>
</li>
</ul></div><h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Azure Cloud,Java,Pyspark</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Lead I - Data Engineering

Valtech Group

<div><p>Why Valtech? We&rsquo;re advisors, visionaries, creative and techies. We embrace all things digital. We talk to each other. We have fun. We love our clients. We&rsquo;re looking ahead &bull; We are global</p></div>
<br><p></p><div><p>Join our innovative, fast-growing data team at the forefront of cloud data architecture on Microsoft Azure. We're building scalable, secure, and modern data platforms using cutting-edge Azure services and Databricks unified analytics platform. If you're passionate about creating high-performance data infrastructure and solving complex big data challenges in a cloud-native environment, this is the perfect opportunity for you.</p><p>
As a Senior Data Engineer specializing in Azure and Databricks, you will architect and implement enterprise-grade cloud-native data solutions on the Microsoft Azure ecosystem. This is a hands-on engineering role with significant architectural influence, where you'll work extensively with Azure Data Factory, Databricks, Delta Lake, and other modern data tools to create efficient, maintainable, and scalable data pipelines using medallion architecture and lakehouse patterns.</p><p><b>
What you do</b><br>
<br>
4&ndash;7 years of hands-on experience in data engineering with strong focus on cloud data platforms. Proven experience with big data technologies and distributed computing</p><p>
Technical Expertise<br>
Azure Mastery: Deep expertise in Azure data services including Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure SQL Database<br>
Databricks Proficiency: Advanced proficiency in Databricks including cluster management, notebook development, and Delta Lake<br>
Big Data Processing: Strong experience with Apache Spark (PySpark, Scala), distributed computing concepts, and performance optimization<br>
Programming: Proficiency in Python, Scala, and SQL for data processing and transformation<br>
<br>
Data Architecture<br>
Experience with lakehouse architecture, medallion architecture, and modern data warehouse design patterns<br>
Streaming Technologies: Knowledge of real-time data processing using Azure Event Hubs, Kafka, and Spark Streaming<br>
Cloud Platforms: Deep understanding of Microsoft Azure ecosystem and native data services</p><p>
Data Management &amp; DevOps<br>
Data Governance: Understanding of data governance frameworks, Unity Catalog, and data quality practices<br>
DevOps: Experience with CI/CD practices, Git-based workflows, and Infrastructure as Code<br>
Orchestration: Knowledge of data orchestration tools like Azure Data Factory, Airflow, or similar platforms<br>
Security: Understanding of Azure security best practices, RBAC, and data encryption</p><p>
Advanced Technologies<br>
Experience with Azure Machine Learning and MLOps practices<br>
Knowledge of containerization (Docker, Kubernetes) and Azure Container Instances<br>
Experience with Power BI for data visualization and reporting<br>
Understanding of data mesh architecture and domain-driven design principles<br>
Experience with Azure DevOps for CI/CD pipeline management<br>
Familiarity with monitoring tools like Azure Monitor, Application Insights, and Databricks monitoring</p><p>
Specialized Skills<br>
Experience with real-time analytics and complex event processing<br>
Knowledge of graph databases and Azure Cosmos DB<br>
Understanding of data lake optimization techniques and performance tuning<br>
Experience with multi-cloud or hybrid cloud architectures</p><p>
Technology Stack<br>
Core Azure Services<br>
Data Storage: Azure Data Lake Storage Gen2, Azure Blob Storage, Azure SQL Database<br>
Data Processing: Databricks, Azure Synapse Analytics, Azure Data Factory<br>
Streaming: Azure Event Hubs, Azure Stream Analytics, Kafka<br>
Analytics: Databricks SQL, Azure Analysis Services, Power BI<br>
Machine Learning: Azure Machine Learning, MLflow, Databricks ML</p><p>
Big Data &amp; Processing<br>
Compute: Apache Spark (PySpark, Scala), Databricks Runtime<br>
Data Formats: Delta Lake, Parquet, JSON, Avro<br>
Architecture: Medallion Architecture, Lakehouse, Data Mesh<br>
Orchestration: Azure Data Factory, Airflow, Azure Logic Apps</p><p>
Development &amp; DevOps<br>
Programming: Python, Scala, SQL, PowerShell<br>
Version Control: Git, Azure DevOps, GitHub<br>
Infrastructure: ARM Templates, Terraform, Azure Resource Manager<br>
Monitoring: Azure Monitor, Application Insights, Databricks monitoring</p><p>
Security &amp; Governance<br>
Security: Azure Active Directory, Key Vault, RBAC<br>
Governance: Unity Catalog, Azure Purview, Data Lineage<br>
Compliance: GDPR, data encryption, access controls</p><p>
Certifications<br>
Microsoft Azure Data Engineer Associate (DP-203)<br>
Databricks Certified Associate Developer for Apache Spark<br>
Databricks Certified Professional Data Engineer<br>
Azure Solutions Architect or Azure Data Fundamentals certifications</p><p><b>
What we ask</b></p><p>
Data Architecture &amp; Engineering<br>
Design and implement enterprise-scale, robust data solutions using Azure and Databricks<br>
Architect scalable ELT/ETL pipelines using Azure Data Factory, Azure Synapse Analytics, and Databricks<br>
Build automated data ingestion processes from various sources including streaming and batch data<br>
Develop and maintain data transformation workflows using PySpark, Scala, and SQL on Databricks<br>
<br>
Data Platform &amp; Optimization<br>
Implement lakehouse architecture using Delta Lake and Databricks Delta tables<br>
Design and optimize Databricks clusters for performance, cost management, and resource utilization<br>
Apply medallion architecture (Bronze, Silver, Gold) for data processing and transformation<br>
Optimize Spark jobs and queries for maximum performance and cost efficiency<br>
<br>
Big Data &amp; Analytics<br>
Build streaming data pipelines using Azure Event Hubs, Kafka, and Databricks Structured Streaming<br>
Implement machine learning pipelines using MLflow and Databricks ML capabilities<br>
Design and maintain data models for analytical workloads and real-time processing<br>
Work with large-scale datasets and implement partitioning strategies for optimal performance<br>
<br>
Quality &amp; Governance<br>
Develop comprehensive data quality tests and monitoring using Great Expectations and custom frameworks<br>
Implement data governance, lineage, and security policies within Azure and Databricks<br>
Create and maintain comprehensive data documentation and lineage tracking using Unity Catalog<br>
Build data validation and testing frameworks for proactive data monitoring<br>
<br>
DevOps &amp; Automation<br>
Build and maintain CI/CD pipelines for Databricks notebooks and data workflows<br>
Develop custom Python/Scala scripts for data processing, manipulation, and automation tasks<br>
Implement Infrastructure as Code (IaC) using ARM templates or Terraform for Azure resources<br>
Work with orchestration tools for pipeline scheduling and dependency management<br>
<br>
Collaboration &amp; Analytics<br>
Collaborate with data scientists, analysts, and ML engineers to support advanced analytics use cases<br>
Enable self-service analytics capabilities using Databricks SQL and Azure Analytics services<br>
Work closely with stakeholders to understand data requirements and deliver scalable solutions</p><p><b>
What we offer</b></p><p>
You&rsquo;ll join an international network of data professionals within our organisation. We support continuous development through our dedicated Academy. If you're looking to push the boundaries of innovation and creativity in a culture that values freedom and responsibility, we encourage you to apply.</p><p><i>
At Valtech, we&rsquo;re here to engineer experiences that work and reach every single person. To do this, we are proactive about creating workplaces that work for every person at Valtech. Our goal is to create an equitable workplace which gives people from all backgrounds the support they need to thrive, grow and meet their goals (whatever they may be). You can find out more about what we&rsquo;re doing to create a Valtech for everyone </i><i>here</i><i>.</i></p><div><i>
Please do not worry if you do not meet all of the criteria or if you have some gaps in your CV. We&rsquo;d love to hear from you and see if you&rsquo;re our next member of the Valtech team!</i></div></div>

Azure & Databricks - Senior Data Engineer

Search 6077+ Databricks engineer jobs near you this January. New roles added regularly across leading industries. Apply today.