data scientist

data engineer (pyspark) Jobs in Bangalore India

data engineer (pyspark)

data scientist project manager Jobs in Bangalore India

data scientist project manager

data scientist-artificial intelligence Jobs in Bangalore India

data scientist-artificial intelligence

data strategy consultant Jobs in Bangalore India

data strategy consultant

data engineer pyspark

Virtusa

<div>
 
 About The Role
 
 
 
 We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality and availability across the organization. This role requires a strong background in big data ecosystems, cloud-native tools, and advanced data processing techniques.
 
 
 The ideal candidate has hands-on experience with data ingestion, transformation, and optimization on the Cloudera Data Platform, along with a proven track record of implementing data engineering best practices. You will work closely with other data engineers to build solutions that drive impactful business insights.
 
 
 
 Responsibilities
 
 
 
 Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
 
 
 Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
 
 
 Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
 
 
 Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
 
 
 Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
 
 
 Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
 
 
 Education and Experience
 
 
 Bachelors or Masters degree in Computer Science, Data Engineering, Information Systems, or a related field.
 
 
 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
 
 
 Technical Skills
 
 
 PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
 
 
 Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
 
 
 Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
 
 
 Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
 
 
 Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
 
 
 Scripting and Automation: Strong scripting skills in Linux.
</div>

Data Engineer (Pyspark)

MKS Inc.

<div>
 
 A Day in Your Life at MKS:
 
 
 
 As a
 
 Technical Data Compliance Engineer/Analyst
 
 at MKS Global Capability Center Bangalore, you will partner with R&amp;D teams across MKS PSD Business Units to manage technical data of complex high-precision positioning systems to ensure design documentation in compliance with MKS standards and international regulations.
 
 
 
 You Will Make an Impact By:
 
 
 
 <ul>
 <li>
 Technical data management of electromechanical parts and assemblies, including regular communication with suppliers and R&amp;D teams.
 </li>
 <li>
 Design quality and regulatory compliance (RoHS, REACH, etc.)
 </li>
 <li>
 Manage, generate, and implement ECO documentation changes. ERP system of record data entry and maintenance.
 </li>
 <li>
 CAD/manufacturing drawing review.
 
 
 </li>
 </ul>
 
 Skills You Bring:
 
 
 
 <ul>
 <li>
 3D CAD experience with SolidWorks and/or AutoCAD with fundamental drafting proficiency and knowledge of ANSI Y14.5 GD&amp;T
 </li>
 <li>
 Working knowledge of safety and regulatory compliance for mechanical and electrical hardware
 </li>
 <li>
 Experience with mechanical and electrical hardware designs, product BOMs, and Engineering Change Order process, and new product introduction.
 </li>
 <li>
 Experience with ERP systems and data entry (SAP preferred)
 </li>
 <li>
 Excellent verbal and written communication skills (French speaking preferred)
 
 
 </li>
 </ul>
</div>

Mechanical Engineer Analyst

<div>
 We are seeking a skilled Python Engineer to join our dynamic team. The ideal candidate will be responsible for developing robust web frameworks, building backend code to efficiently serve datasets, and performing data transformations to support our data-driven applications. This role requires a deep understanding of Python, web development frameworks, data processing, and database management
 
 
 Core Python: Be proficient in the concepts and features of the Python language , experienced with web frameworks like Flask and FastAPI for developing RESTful APIs, and skilled in asynchronous programming using asyncio. Experience with handling concurrent connections and optimizing IO operations. Should have a deep understanding of microservices architecture and be capable of writing clean, efficient, and maintainable code.
 
 
 Security Best Practices and Requirements
 
 
 Understanding of security practices for microservices, including authentication, authorization, and encryption. Experience with securing APIs using OAuth, JWT, and API gateways.
 
 
 Web Framework Development
 
 
 Design, develop, and maintain scalable web frameworks using Python.
 
 
 Collaborate with front-end developers to integrate user-facing elements with server-side logic.
 
 
 Ensure the web frameworks are secure, performant, and maintainable.
 
 
 Have a sound knowledge of one or more web frameworks that are based on Python, such as Django, Flask, Pyramid, etc. Should have the know how to use the framework's features, such as routing, templating, database integration, authentication, testing.
 
 
 Object Relational Mappers (ORMs): Develop backend systems to serve datasets through APIs and other interfaces.
 
 
 Implement data access layers, integrating with various databases and data storage solutions.
 
 
 Optimize backend code for performance, scalability, and reliability.
 
 
 Be able to use ORMs to interact with databases using Python objects. ORMs abstract away the low-level SQL queries and provide a high-level interface to manipulate data. Some popular ORMs for Python are SQL Alchemy, Django ORM, etc. Should know how to define models, perform CRUD operations, query data, handle transactions.
 
 
 Data Transformations and Database Management: Design and maintain database schemas to support application requirements.
 
 
 Optimize database performance and ensure efficient data retrieval.
 
 
 Implement and manage data storage solutions, both relational and NoSQL.
 
 
 Design and implement data transformation pipelines to process and prepare data for analysis.
 
 
 Utilize ETL (Extract, Transform, Load) processes to clean, aggregate, and enrich data.
 
 
 Ensure data integrity, quality, and consistency across all transformation processes.
 
 
 Other Skills:Should also have general skills that are fundamental to any developer, such as:
 
 
 Debugging and testing: Should be able to use tools and techniques to find and fix errors in code, such as logging, breakpoints, unit testing, integration testing
</div>

Python Developer

Straive

<div>
 
 
 About the Role
 
 
 
 Straive is looking for a talented and driven
 
 Consultant / Data Scientist / GenAI Engineer
 
 to join our
 
 Analytics &amp; GenAI delivery team
 
 . In this role, you will work under the guidance of the Senior Project Manager / Engagement Manager to design, develop, and deploy advanced AI/ML and Generative AI solutions for global enterprise clients. You will be part of a high-performing team, collaborating with both onshore and offshore members to build scalable, production-grade AI systems.
 
 
 
 
 
 This role is ideal for candidates from
 
 premier engineering institutes
 
 with
 
 2&ndash;3 years of relevant experience
 
 in Python development, LLM integration, and RAG workflows, along with a passion for solving complex problems in real-world business contexts.
 
 
 
 
 
 
 Key Responsibilities
 
 
 <ul>
 <li>
 Develop and maintain
 
 Python-based
 
 applications, AI/ML models, and data processing pipelines for GenAI projects.
 </li>
 <li>
 Implement
 
 Large Language Model (LLM)
 
 integrations, including
 
 Retrieval-Augmented Generation (RAG)
 
 pipelines and embedding-based search solutions.
 </li>
 <li>
 Build data ingestion and transformation workflows, working with structured and unstructured datasets.
 </li>
 <li>
 Optimize AI model performance through
 
 prompt engineering
 
 , fine-tuning, and evaluation techniques.
 </li>
 <li>
 Collaborate closely with senior team members to translate business requirements into technical solutions.
 </li>
 <li>
 Integrate AI solutions with
 
 vector databases
 
 (e.g., Cosmos DB, Pinecone, ChromaDB) and API-driven applications.
 </li>
 <li>
 (Optional) Contribute to
 
 cloud-native deployments
 
 and
 
 Azure architecture
 
 &ndash;based solutions, including containerization, CI/CD, and basic MLOps workflows.
 </li>
 <li>
 Document workflows, maintain code repositories, and follow Agile development practices.
 </li>
 </ul>
 
 
 
 
 
 Required Qualifications
 
 
 <ul>
 <li>
 
 2&ndash;3 years
 
 of relevant experience in AI/ML development, preferably in enterprise projects.
 </li>
 <li>
 Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, Data Science, AI/ML, or related field from a
 
 premier engineering institute
 
 .
 </li>
 <li>
 Proficiency in
 
 Python
 
 programming and familiarity with relevant libraries (e.g., LangChain, Hugging Face, Pandas, NumPy).
 </li>
 <li>
 Hands-on experience implementing
 
 RAG pipelines
 
 , embeddings, and vector search solutions.
 </li>
 <li>
 Understanding of
 
 LLM architectures
 
 and integration patterns.
 </li>
 <li>
 Working knowledge of
 
 SQL
 
 and data processing best practices.
 </li>
 <li>
 Basic knowledge of
 
 cloud DevOps concepts
 
 , preferably with
 
 Azure
 
 (AWS/GCP experience is also acceptable).
 </li>
 </ul>
 
 
 
 
 Preferred Skills
 
 
 <ul>
 <li>
 Exposure to
 
 agentic AI frameworks
 
 such as LangGraph, Semantic Kernel, or similar.
 </li>
 <li>
 Familiarity with ML model lifecycle management and deployment workflows.
 </li>
 <li>
 Prior experience working with cross-border teams and Agile environments.
 </li>
 </ul>
 
 
</div>

Data Scientist

MakeMyTrip

<div>
 
 
 Role:
 
 Lead Data Engineer
 
 
 
 Location:
 
 Bangalore
 
 
 
 Years of Experience:
 
 6 -8 Years
 
 
 
 About the function:
 
 
 
 At MakeMyTrip (MMT), technology is at the heart of everything we do. As a leading player in the travel industry, we leverage cutting-edge solutions like AI, machine learning, and cloud infrastructure to create seamless and innovative travel experiences. We prioritize a tech-first mindset, where our engineering teams lead the charge in transforming the way people travel.
 
 
 Our culture encourages collaboration, creativity, and thought leadership, empowering engineers to challenge the status quo and develop impactful, game-changing solutions. At MMT, you'll work with the latest technologies, continually push boundaries, and have the chance to shape the future of travel.
 
 
 Join us and become part of a forward-thinking, tech-driven organization that thrives on innovation and continuous learning.
 
 
 
 
 
 
 About the team:
 
 
 
 MakeMyTrip as India&rsquo;s leading online travel company and provides petabytes of raw data which is helpful for business growth, analytical and machine learning needs.
 
 
 
 
 
 Data Platform Team is a horizontal function at MakeMyTrip to support various LOBs (Flights, Hotels, Holidays, Ground) and works heavily on streaming datasets which powers personalized experiences for every customer from recommendations to in-location engagement.
 
 
 
 
 
 Our team's key responsibilities are:
 
 <ol>
 <li>
 Design, construct, and maintain robust data systems and architectures
 </li>
 <li>
 Develop and optimize data capture, storage, processing, serving, and querying platforms
 </li>
 <li>
 Create data products for personalization, recommendation, customer segmentation, and intelligence
 </li>
 <li>
 Enhance our Measurement platform for A/B experimentation
 </li>
 <li>
 Contribute to our Feature Store, an internal unified data analytics platform
 </li>
 <li>
 Participate in the development of our next-generation Travel Planner using Generative AI and Multi-Agent frameworks.
 </li>
 <li>
 Implement and optimize data solutions for travel-specific use cases, such as:
 </li>
 <li>
 Analyzing cross-city travel patterns to extend trip recommendations
 </li>
 <li>
 Identifying correlations between hotel bookings in different areas to suggest complementary destinations
 </li>
 </ol>
 
 
 
 
 Required Skills and Experience:
 
 <ul>
 <li>
 Extensive experience working with large data sets with hands-on technology skills to design and build robust data architecture
 </li>
 <li>
 At least 6+ years of hands-on experience in Spark/BigData Tech stack
 </li>
 <li>
 Expertise in stream processing engines like Spark Structured Streaming or Apache Flink
 </li>
 <li>
 Analytical processing on Big Data using Spark
 </li>
 <li>
 At least 4+ years of experience in Scala. Experience with Python will be a plus.
 </li>
 <li>
 Hands-on administration, configuration management, monitoring, performance tuning of Spark workloads, Distributed platforms, and JVM based systems
 </li>
 <li>
 At least 2+ years of cloud deployment experience &ndash; AWS | Azure | Google Cloud Platform
 </li>
 <li>
 2 or more product deployments of big data technologies &ndash; Business Data Lake, NoSQL databases etc
 </li>
 <li>
 Awareness and decision-making ability to choose among various big data, NoSQL, analytics tools and technologies
 </li>
 <li>
 Should have experience in architecting and implementing domain centric big data solutions
 </li>
 <li>
 Ability to frame architectural decisions and provide technology leadership &amp; direction
 </li>
 <li>
 Excellent problem solving, hands-on engineering, and communication skills
 </li>
 </ul>
 
 
 
 
 
 
 
 At MakeMyTrip, we're committed to innovation and excellence in the travel industry. Join us in shaping the future of travel through data-driven solutions and advanced technologies. If you're passionate about leveraging data to create exceptional travel experiences, we want to hear from you!
 
</div>

Lead Data  Engineer

Pocket FM

<div>
 
 Senior Software Engineer-Backend
 
 
 Location: Bengaluru
 
 
 Experience: 3&ndash;5 years
 
 
 
 
 
 About Pocket FM
 
 
 Pocket FM, founded in 2018, is India&rsquo;s leading audio storytelling platform, transforming the way millions consume stories. Offering high-quality serialized content across genres such as Romance, Drama, Thriller, Fantasy, Sci-Fi, and Mythology in eight languages, Pocket FM has built a strong global presence with over 200 million listeners worldwide. With users spending an average of 120 minutes daily on the platform, it has emerged as one of the fastest-growing audio platforms, rapidly expanding its reach across the US, Europe, LATAM, and Southeast Asia.
 
 
 
 
 
 Our unique model combines free listening with micropayments for premium content, powering strong business growth. In FY25, we reached an ARR of INR 2,000 crore, with over 100,000 hours of content on the platform. We're also at the forefront of innovation, leveraging AI-generated content to scale efficiently.
 
 
 
 
 
 Role Overview:
 
 
 We are hiring for a Senior Software Engineer &ndash; Backend who will be responsible for building high-performance APIs, scalable microservices, and robust infrastructure powering our audio platform.
 
 
 This is a high-ownership role that requires strong technical skills, problem-solving abilities, and a mindset to work in a fast-paced product environment.
 
 
 
 
 
 Key Responsibilities:
 
 <ul>
 <li>
 Design, build, and maintain scalable backend services using Python or Golang
 </li>
 <li>
 Develop and manage RESTful APIs for mobile/web platforms and internal tools
 </li>
 <li>
 Collaborate closely with product, design, and analytics teams to deliver features at scale
 </li>
 <li>
 Optimize performance, ensure high availability, and handle large-scale traffic
 </li>
 <li>
 Write clean, modular, and testable code adhering to software best practices
 </li>
 <li>
 Lead debugging efforts, production issue resolution, and post-mortem analysis
 </li>
 <li>
 Participate in code reviews and continuously drive technical improvements
 </li>
 </ul>
 
 
 
 
 Skills &amp; Qualifications:
 
 <ul>
 <li>
 3&ndash;5 years of backend development experience in Python or Golang
 </li>
 <li>
 Strong understanding of data structures, algorithms, OOP, operating systems, and networking
 </li>
 <li>
 Proven experience developing and managing REST APIs
 </li>
 <li>
 Proficient in working with databases like PostgreSQL, MongoDB, and Redis
 </li>
 <li>
 Hands-on experience with cloud platforms &ndash; AWS, GCP, or Azure
 </li>
 <li>
 Experience with Docker or other container technologies
 </li>
 <li>
 Familiarity with CI/CD pipelines and deployment best practices
 </li>
 <li>
 Experience with monitoring and logging tools like Prometheus, Grafana, or ELK stack
 </li>
 <li>
 Experience with Git and version control workflows
 </li>
 <li>
 Contributions to open-source projects or a strong personal backend project portfolio
 </li>
 <li>
 Bachelor&rsquo;s degree in Computer Science, Engineering, or a related technical discipline
 </li>
 <li>
 Strong communication skills and a proactive, ownership-driven mindset
 </li>
 </ul>
 
 
 
 
 You can get more updates, insights and everything behind the scenes at Pocket FM here -
 
 Pocket FM
 
 
</div>

Senior Software Engineer - Backend

<div>
 
 Introduction
 
 
 
 We are seeking a software engineer to be part of security product development in Z systems.
 
 
 <ul>
 <li>
 Z Security product has polyglot development environment with focus on Operators framework for microservices deployment. Microservice(s) are using more than one runtime. Java, Python, nodeJS, Javascript , docker, Kubernetes and GO { for operators} are part of development cycle.
 </li>
 <li>
 The role consists of understanding product requirement, baseline, decomposition and putting it through execution cycle [Development &amp; delivery]
 
 
 </li>
 </ul>
 
 Your Role And Responsibilities
 
 
 
 <ul>
 <li>
 As a member of the product development team, you will be responsible for the decomposition of requirements into technical WorkItems, their mapping to development process and drive the quality of deliverables
 </li>
 <li>
 Strong overview of the infrastructure {z, z/Linux, zCX, LPAR} needed for driving development lifecycle of the offering. Managing the desired level of automation for delivery process. Sound development experience in K8 lifecycle.
 </li>
 <li>
 Should be able to code in Python for the back-end and JavaScript [variant] for the front-end.
 </li>
 <li>
 Sound knowledge of data layer which is not limited to databases alone. It includes nature of data store, abstraction layer [driver(s) &amp; SQL CLI], various integration as well as needed experience for DAO layer.
 </li>
 <li>
 The ideal candidate would have a strong technical aptitude and a flair for picking up new skills
 </li>
 <li>
 The role provides ample opportunity for learning and career growth and is only constrained by the individual&rsquo;s appetite for learning
 
 
 </li>
 </ul>
 
 Preferred Education
 
 
 
 Master's Degree
 
 
 
 Required Technical And Professional Expertise
 
 
 
 <ul>
 <li>
 8+ years of IT experience
 </li>
 <li>
 At least last 3 - 4years in Cloud/OCP/Z based environment
 </li>
 <li>
 Experience conceptualizing Cloud deployments and/OR OCP based product development lifecycle
 </li>
 <li>
 Strong foundational knowledge of containerized deployments on Docker, Kubernetes, RedHat OpenShift and RedHat Enterprise Linux
 </li>
 <li>
 Ability to demonstrate full-stack outlook for development/change process
 </li>
 <li>
 The candidate should have a passion for continuous improvement in building knowledge base both technically and professionally
 
 
 </li>
 </ul>
 
 Preferred Technical And Professional Experience
 
 
 
 <ul>
 <li>
 Certifications in Cloud development , architecture &amp; support areas will be an advantage
 </li>
 <li>
 Security / cyber security development expertise and AI certifications
 </li>
 <li>
 Prior experience with development and automation of delivery
 </li>
 <li>
 Strong technical and functional skills to leverage and drive the team collaboration
 </li>
 <li>
 Proactive posture on key identified areas of product development cycle.
 </li>
 </ul>
</div>

Software Developer

Explore 53+ Data engineer pyspark in Bangalore India this November. Find roles across top companies and apply now to boost your career.