big data lead

ixceed

<p><b>Role: Data Lead (Scala+Spark)</b></p><p><b>Location: Chennai</b></p><p><b>Mode: Hybrid</b></p><p><b>Type: Permanent</b></p><p><b>Job Description:</b></p><p>Responsibilities:</p><ul><li>Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.</li><li>Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency.</li><li>Optimize Spark applications for performance and cost-efficiency within the Databricks environment.</li><li>Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning.</li><li>Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions.</li><li>Implement data governance and security best practices within Databricks.</li><li>Troubleshoot and resolve data-related issues, ensuring data availability and reliability.</li><li>Stay updated with the latest advancements in Spark, Scala, Databricks, and related big data technologies.</li></ul><p>Required Skills and Experience:</p><ul><li>Proven experience as a Data Engineer with a strong focus on big data technologies.</li><li>Expertise in the Scala programming language for data processing and Spark application development.</li><li>In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark Core.</li><li>Proficiency in using Databricks platform features, including notebooks, jobs, workflows, and Unity Catalog.</li><li>Experience with Delta Lake and its capabilities for building data lakes.</li><li>Strong understanding of data warehousing concepts, data modeling, and relational databases.</li><li>Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.</li><li>Experience with version control systems like Git.</li><li>Excellent problem-solving and analytical skills.</li><li>Ability to work independently and as part of a team.</li></ul><p>Preferred Qualifications (Optional):</p><ul><li>Experience with other big data technologies like Kafka, Flink, or Hadoop ecosystem components.</li><li>Knowledge of data visualization tools.</li><li>Understanding of DevOps principles and CI/CD pipelines for data engineering.</li><li>Relevant certifications in Spark or Databrick</li></ul><p>Job Type: Permanent</p><p>Pay: &#8377;3,500,000.00 - &#8377;4,500,000.00 per year</p><p>Application Question(s):</p><ul><li>Immediate Joiner</li></ul><p>Experience:</p><ul><li>Design and build ETL: 3 years (Required)</li><li>Scala: 4 years (Required)</li><li>Spark: 4 years (Required)</li><li>Databricks: 3 years (Required)</li><li>Team Lead: 2 years (Required)</li><li>Big data architecture: 2 years (Required)</li><li>Data Engineering: 3 years (Required)</li></ul><p>Work Location: In person</p>

Data Lead (Scala+Spark)

Prodapt Solutions

<div>Overview:<div></div><p><br>
We are looking for a highly skilled Big Data Engineer with extensive experience in Spark and Scala to join our team. The ideal candidate will play a crucial role in designing, developing, and optimizing large-scale data processing systems. You will work closely with data scientists, analysts, and other stakeholders to deliver high-quality data solutions.</p>
Responsibilities:
<p><b>
Key Responsibilities: -</b></p><ul><li>
Design, develop, and maintain scalable data pipelines using Apache Spark and Scala.</li><li>
Collaborate with cross-functional teams to understand data requirements and deliver data solutions that meet business needs.</li><li>
Optimize Spark jobs for performance and cost-efficiency in a distributed computing environment.</li><li>
Implement best practices for data modeling, ETL processes, and data governance.</li><li>
Monitor and troubleshoot data processing workflows to ensure data integrity and availability.</li><li>
Work with cloud platforms (AWS, Azure, or GCP) to implement big data solutions.</li><li>
Stay up to date with industry trends and emerging technologies in big data and analytics.</li></ul><div><div>
</div></div><br>
<div></div>
Requirements:
<ul><li>7-10 years of experience in Big Data technologies, with a strong focus on Apache Spark and Scala.</li><li>
Proficiency in data processing frameworks (Hadoop, Spark) and languages (Scala, Java).</li><li>
Experience with data warehousing solutions (Snowflake, Redshift, etc.) and SQL.</li><li>
Knowledge of data modeling, ETL processes, and data visualization tools (Tableau, Power BI).</li><li>
Familiarity with cloud services (AWS, Azure, Google Cloud) and containerization (Docker, Kubernetes).</li><li>
Strong analytical skills and the ability to work with large datasets.</li><li>
Excellent communication and teamwork skills.</li></ul></div><div></div>

BigData- Spark/Scala

Hypr4 Cloud Tech

<p>Data Infrastructure &amp; Pipeline Development: &#9679; Design, develop, and optimize scalable, efficient, and reliable data pipelines for large-scale data processing and transformation. &#9679; Manage and maintain data architecture, ensuring high availability and performance using tools like Snowflake, Dataproc, BigQuery and other cloud technologies. &#9679; Lead the integration of data sources from multiple systems, ensuring seamless data flow across various platforms.</p><p>&#9679; Build and optimize data pipelines using BigQuery, Snowflake, DBT Cloud, and Airflow.</p><p>&#9679; Expertise in Data Modelling to desing and build Data warehouses, Data Marts and Data lakes</p><p>&#9679; Manage version control and workflows with GitHub. Performance &amp; Optimization:</p><p>&#9679; Perform tuning and optimization of queries and data pipelines to ensure high-performance data systems.</p><p>&#9679; Conduct regular performance reviews and recommend improvements or optimizations for system reliability, speed, and cost-efficiency. DBT (Data Build Tool) Implementation:</p><p>&#9679; Implement and maintain DBT models for data transformation workflows.</p><p>&#9679; Collaborate with data analysts and data scientists to ensure high-quality, well-documented datasets for downstream analysis.</p><p>&#9679; Ensure the use of best practices for DBT testing, version control, and deployment. Snowflake Management:</p><p>&#9679; Architect and optimize Snowflake data warehouse environments.</p><p>&#9679; Oversee and manage Snowflake data ingestion, transformation, and storage strategies.</p><p>&#9679; Collaborate with cross-functional teams to ensure Snowflake is being utilized effectively and efficiently. Leadership &amp; Mentorship:</p><p>&#9679; Lead and mentor a team of data engineers, ensuring that best practices are followed in development and deployment of data pipelines.</p><p>&#9679; Conduct code reviews, provide feedback, and ensure the implementation of high-quality data solutions.</p><p>Preferred Skills:</p><p>&#9679; 10+ years of experience in Data Engineering with a strong focus on data warehousing, ETL pipelines, and big data technologies.</p><p>&#9679; At least 3-5 years of hands-on experience with Snowflake data warehouse or BigQuery, including setup, configuration, optimization, and maintenance.</p><p>&#9679; Proficiency in SQL for query optimization and performance tuning.</p><p>&#9679; In-depth experience with Dataproc for running large-scale data processing workflows (e.g., Spark, Hadoop).</p><p>&#9679; Expertise with DBT or any other ELT tool for data transformation and model building. Technical Skills:</p><p>&#9679; Strong experience in cloud platforms like AWS, GCP, or Azure, with a focus on data engineering tools and services.</p><p>&#9679; Proficient in programming/scripting languages such as Python, Java, or Scala for data processing.</p><p>&#9679; Experience with CI/CD pipelines and version control (Git, Jenkins, etc.).</p><p>&#9679; Knowledge of distributed computing frameworks (e.g., Spark, Hadoop) and related data processing concepts. Data Architecture &amp; Design:</p><p>&#9679; Experience with building and maintaining data warehouses and lakes.</p><p>&#9679; Strong understanding of data modeling concepts, data quality, and governance.</p><p>&#9679; Familiarity with Kafka, Airflow, or similar tools for orchestrating data workflows.</p><p>Job Types: Full-time, Permanent, Fresher</p><p>Pay: &#8377;2,540,454.62 - &#8377;2,764,456.72 per year</p><p>Work Location: In person</p>

Data lead

Rebert Technologies Pvt.Ltd

<p>Position: <b>Data Lead EdTech Department Rebert Technologies Pvt. Ltd.</b></p><p><b>Location:</b> Part Time (Pune preferred)</p><p><b>Type:</b> Part -Time | <b>Role:</b> Founding Team</p><p><b>About Rebert Technologies</b></p><p>Rebert Technologies is a <b>software solutions company</b> driven by innovation and technology. We specialize in developing intelligent digital solutions that simplify business operations and enhance performance.</p><p>We are now <b>launching our EdTech division</b>, focused on building a data-driven education ecosystem that bridges the gap between learning and employability. Our goal is to empower students and institutions through technology, analytics, and measurable outcomes.</p><p><b>About the Role</b></p><p>We are looking for a <b>Data Lead</b> to spearhead our analytics and data systems within the EdTech division. You will be responsible for building data pipelines, creating dashboards, analyzing trends, and turning numbers into actionable insights. This role involves close collaboration with our product, marketing, and academic teams to make data-driven decisions that directly influence student outcomes and company growth.</p><p>As one of the early team members, you will play a key role in shaping the foundation of our data culture and have the opportunity to lead a team as we scale.</p><p><b>Key Responsibilities</b></p><ul><li>Design and manage end-to-end data pipelines, dashboards, and reports.</li><li>Analyze learner, marketing, and performance data to extract key insights.</li><li>Support product and business teams with data-backed recommendations.</li><li>Maintain data accuracy, integrity, and accessibility across systems.</li><li>Develop scalable analytics systems and mentor future data team members.</li></ul><p><b>Required Skills</b></p><ul><li>Strong in <b>Excel / Google Sheets</b>, <b>SQL</b>, and <b>Power BI / Tableau / Google Data Studio</b>.</li><li>Knowledge of <b>data visualization, automation, and analytics</b>.</li><li>Familiarity with <b>Python / R</b> is a plus.</li><li>Excellent analytical, communication, and problem-solving skills.</li><li>Prior exposure to <b>EdTech</b> or analytics in the education sector preferred.</li></ul><p><b>Expectations:</b></p><ul><li>Self-motivated, analytical, and outcome-oriented.</li><li>Thrive in a startup environment with ownership and autonomy.</li><li>Passionate about data, learning, and innovation.</li></ul><p><b>Compensation</b></p><ul><li><b>&#8377;10,000 per month</b> (based on experience &amp; performance).</li></ul><p>Job Type: Part-time</p><p>Pay: &#8377;8,000.00 - &#8377;10,000.00 per month</p><p>Expected hours: 16 per week</p><p>Benefits:</p><ul><li>Flexible schedule</li><li>Work from home</li></ul><p>Education:</p><ul><li>Master's (Preferred)</li></ul><p>Work Location: Remote</p>

Data Lead

Proservartner

<p><b>Role:</b> Data Benchmarking &amp; Analytics Lead</p><p><b>Location:</b> Dubai / Hybrid (with flexibility based on client requirements)</p><p><b>Day Rate:</b> USD 600&ndash;700</p><p><b>Contract:</b> Project-based, expected Nov/Dec start</p><p>Proservartner is a global consulting firm specialising in business transformation, shared services, outsourcing, and digital enablement. We partner with leading organisations to design and deliver practical, high-impact solutions across industries.</p><p>Our approach is hands-on and collaborative: we combine strategic expertise with operational know-how to ensure clients see tangible results. With deep capability in finance, HR, supply chain, and customer experience transformation, we are trusted advisors to multinational firms across EMEA, Asia, and beyond.</p><p><b>About the Project</b></p><p>We are supporting a major client engagement in Dubai, delivering a multi-country Compensation &amp; Rewards Benchmarking project across the media and investment sectors.</p><p>The project covers:</p><ul><li>Market benchmarking and data analytics across GCC and global comparators</li><li>Compensation survey design, normalisation, and analysis</li><li>Sector-specific insights into reward structures and allowances</li><li>Change &amp; implementation support for new total rewards frameworks</li></ul><p>This is a high-profile engagement requiring experienced contractors to deliver alongside Proservartner&rsquo;s leadership team.</p><p><b>Role Overview &ndash; Data Benchmarking &amp; Analytics Lead</b></p><p>The Data Benchmarking &amp; Analytics Lead will manage survey data, job architecture, and analytics across the multi-country compensation benchmarking project. You will ensure that data is normalized, insightful, and presented in a client-ready format.</p><p>Key Responsibilities</p><ul><li>Manage compensation survey data collection, cleaning, and normalization</li><li>Develop job architecture frameworks and maintain accurate mapping</li><li>Build advanced Excel models, dashboards, and visualizations</li><li>Triangulate multiple data sources to generate actionable insights</li><li>Collaborate with SMEs and senior consultants to produce client-ready deliverables</li></ul><p><b>Candidate Requirements</b></p><ul><li>8&ndash;10 years&rsquo; experience in compensation analytics, benchmarking, or HR data analysis</li><li>Strong Excel modeling, data visualization, and survey analytics skills</li><li>Experience in job architecture and market survey analysis</li><li>Ability to interpret complex datasets and present insights clearly</li><li>Excellent stakeholder communication and reporting skills</li></ul><p>Job Type: Full-time</p><p>Pay: AED800.00 - AED1,000.00 per day</p>

Data Benchmarking & Analytics Lead

Antons

<p><strong>Key Responsibilities:</strong></p>
<ul>
 <li>Own the end-to-end product lifecycle for digital banking offeringsideation, design, development, launch, performance tracking, and ongoing optimization.</li>
 <li>Shape and deliver the digital strategy and product roadmap, balancing long-term vision with near-term execution.</li>
 <li>Drive digital transformation by integrating emerging technologies (AI, machine learning, open banking APIs, blockchain) into new and existing products.</li>
 <li>Lead cross-functional teams and collaborate with stakeholders in IT, marketing, compliance, risk, and operations to ensure seamless delivery of digital initiatives.</li>
 <li>Conduct market research, competitor analysis, and customer feedback sessions to identify new opportunities and areas for improvement in digital banking products.</li>
 <li>Oversee the implementation of user-centric design principles, ensuring top-tier customer experience across all digital touchpoints.</li>
 <li>Monitor product performance through data analytics and KPIs, and make data-driven recommendations for enhancements.</li>
 <li>Ensure products comply with all regulatory standards and internal governance frameworks.</li>
</ul>
<p> br&gt;
</p>

Digital Banking Products Lead

<div></div><div><div><div><div><ul><div>5 - 7 Years</div>
<div></div><div>3 Openings</div>
 <div>Bangalore</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p>Role Proficiency:</p>
<p>
Provide expertise on data analysis techniques using software tools. Under supervision streamline business processes.</p>
<p>
Outcomes:</p>
<div>
<ul><li> Design and manage the reporting environment; which include data sources security and metadata.</li></ul>
<ul><li> Provide technical expertise on data storage structures data mining and data cleansing.</li></ul>
<ul><li> Support the data warehouse in identifying and revising reporting requirements.</li></ul>
<ul><li> Support initiatives for data integrity and normalization.</li></ul>
<ul><li> Assess tests and implement new or upgraded software. Assist with strategic decisions on new systems. Generate reports from single or multiple systems.</li></ul>
<ul><li> Troubleshoot the reporting database environment and associated reports.</li></ul>
<ul><li> Identify and recommend new ways to streamline business processes</li></ul>
<ul><li> Illustrate data graphically and translate complex findings into written text.</li></ul>
<ul><li> Locate results to help clients make better decisions. Solicit feedback from clients and build solutions based on feedback.</li></ul>
<ul><li> Train end users on new reports and dashboards.</li></ul>
<ul><li>Set FAST goals and provide feedback on FAST goals of repartees</li></ul>
</div>
<p></p><p>Measures of Outcomes:</p>
<div>
<ul><li> Quality - number of review comments on codes written</li></ul>
<ul><li> Data consistency and data quality.</li></ul>
<ul><li> Number of medium to large custom application data models designed and implemented</li></ul>
<ul><li> Illustrates data graphically; translates complex findings into written text.</li></ul>
<ul><li> Number of results located to help clients make informed decisions.</li></ul>
<ul><li> Number of business processes changed due to vital analysis.</li></ul>
<ul><li> Number of Business Intelligent Dashboards developed</li></ul>
<ul><li> Number of productivity standards defined for project</li></ul>
<ul><li>Number of mandatory trainings completed</li></ul>
</div>
<p></p><p>Outputs Expected:</p>
<p>
Determine Specific Data needs:</p>
<div>
<ul><li>Work with departmental managers to outline the specific data needs for each business method analysis project</li></ul></div><br>
<p><br>
Critical business insights:</p>
<div>
<ul><li>Mines the business&rsquo;s database in search of critical business insights; communicates findings to relevant departments.</li></ul></div><br>
<p><br>
Code:</p>
<div>
<ul><li>Creates efficient and reusable SQL code meant for the improvement<br>
manipulation<br>
and analysis of data.</li></ul>
<ul><li>Creates efficient and reusable code. Follows coding best practices.</li></ul></div><br>
<p><br>
Create/Validate Data Models:</p>
<div>
<ul><li>Builds statistical models; diagnoses<br>
validates<br>
and improves the performance of these models over time.</li></ul></div><br>
<p><br>
Predictive analytics:</p>
<div>
<ul><li>Seeks to determine likely outcomes by detecting tendencies in descriptive and diagnostic analysis</li></ul></div><br>
<p><br>
Prescriptive analytics:</p>
<div>
<ul><li>Attempts to identify what business action to take</li></ul></div><br>
<p><br>
Code Versioning:</p>
<div>
<ul><li>Organize and manage the changes and revisions to code. Use a version control tool for example git<br>
bitbucket. etc.</li></ul></div><br>
<p><br>
Create Reports:</p>
<div>
<ul><li>Create reports depicting the trends and behaviours from analyzed data</li></ul></div><br>
<p><br>
Document:</p>
<div>
<ul><li>Create documentation for worked performed. Additionally<br>
perform peer reviews of documentation of others' work</li></ul></div><br>
<p><br>
Manage knowledge:</p>
<div>
<ul><li>Consume and contribute to project related documents<br>
share point<br>
libraries and client universities</li></ul></div><br>
<p><br>
Status Reporting:</p>
<div>
<ul><li>Report status of tasks assigned</li></ul>
<ul><li>Comply with project related reporting standards and processes</li></ul>
</div>
<p></p><p>Skill Examples:</p>
<div>
<ul><li> Analytical Skills: Ability to work with large amounts of data: facts figures and number crunching.</li></ul>
<ul><li> Communication Skills: Communicate effectively with a diverse population at various organization levels with the right level of detail.</li></ul>
<ul><li> Critical Thinking: Data Analysts must review numbers trends and data to come up with original conclusions based on the findings.</li></ul>
<ul><li> Presentation Skills - facilitates reports and oral presentations to senior colleagues</li></ul>
<ul><li> Strong meeting facilitation skills as well as presentation skills.</li></ul>
<ul><li> Attention to Detail: Vigilant in the analysis to determine accurate conclusions.</li></ul>
<ul><li> Mathematical Skills to estimate numerical data.</li></ul>
<ul><li> Work in a team environment</li></ul>
<ul><li> Proactively ask for and offer help</li></ul>
</div>
<p></p><p>Knowledge Examples:</p>
<p>
Knowledge Examples</p>
<div>
<ul><li> Database languages such as SQL</li></ul>
<ul><li> Programming language such as R or Python</li></ul>
<ul><li> Analytical tools and languages such as SAS &amp; Mahout.</li></ul>
<ul><li> Proficiency in MATLAB.</li></ul>
<ul><li> Data visualization software such as Tableau or Qlik.</li></ul>
<ul><li> Proficient in mathematics and calculations.</li></ul>
<ul><li> Efficiently with spreadsheet tools such as Microsoft Excel or Google Sheets</li></ul>
<ul><li> DBMS</li></ul>
<ul><li> Operating Systems and software platforms</li></ul>
<ul><li>Knowledge regarding customer domain and sub domain where problem is solved</li></ul>
</div>
<p></p><p>Additional Comments:</p>
<p></p>
<p>1. Experience in Python stored and executed in Foundry transforms. 2. Strong Foundry AIP logic &ndash; this is the GenAI portion making calls out to a model and can vary by source 3. Experience in Prompting &ndash; working with the business to understand how to instruct the LLM based on what the goals are for source 4. Good in Foundry Workshop &ndash; building the User Interface. Strong proficiency in Python, especially for Foundry transforms Experience with Palantir Foundry AIP and integrating GenAI models Understanding of prompting techniques and LLM-based workflows Experience with Foundry Workshop (UI development) Ability to work closely with business stakeholders to translate requirements into Foundry-based solutions</p></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Python,Palantir Foundry,Foundry Workshop</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Lead I - Data Science

Search 10000+ Big data lead jobs near you this October. New roles added regularly across leading industries. Apply today.