data scientist

data engineer (pyspark) Jobs in Bangalore

data engineer (pyspark)

data scientist project manager Jobs in Bangalore

data scientist project manager

data scientist-advanced analytics Jobs in Bangalore

data scientist-advanced analytics

data scientist-artificial intelligence Jobs in Bangalore

data scientist-artificial intelligence

data engineer pyspark

Virtusa

<div>
 
 About The Role
 
 
 
 We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality and availability across the organization. This role requires a strong background in big data ecosystems, cloud-native tools, and advanced data processing techniques.
 
 
 The ideal candidate has hands-on experience with data ingestion, transformation, and optimization on the Cloudera Data Platform, along with a proven track record of implementing data engineering best practices. You will work closely with other data engineers to build solutions that drive impactful business insights.
 
 
 
 Responsibilities
 
 
 
 Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
 
 
 Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
 
 
 Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
 
 
 Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
 
 
 Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
 
 
 Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
 
 
 Education and Experience
 
 
 Bachelors or Masters degree in Computer Science, Data Engineering, Information Systems, or a related field.
 
 
 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
 
 
 Technical Skills
 
 
 PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
 
 
 Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
 
 
 Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
 
 
 Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
 
 
 Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
 
 
 Scripting and Automation: Strong scripting skills in Linux.
</div>

Data Engineer (Pyspark)

<div>
 
 Introduction
 
 
 
 In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
 
 
 
 Your Role And Responsibilities
 
 
 
 <ul>
 <li>
 As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing.
 </li>
 <li>
 Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets.
 </li>
 <li>
 In this role, your responsibilities may include:
 </li>
 <li>
 Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques
 </li>
 <li>
 Designing and implementing various enterprise search applications such as Elasticsearch and Splunk for client requirements
 </li>
 <li>
 Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviours.
 </li>
 <li>
 Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modelling results
 
 
 </li>
 </ul>
 
 Preferred Education
 
 
 
 Master's Degree
 
 
 
 Required Technical And Professional Expertise
 
 
 
 <ul>
 <li>
 Proof of Concept (POC) Development: Develop POCs to validate and showcase the feasibility and effectiveness of the proposed AI solutions.
 </li>
 <li>
 Help in showcasing the ability of Gen AI code assistant to refactor/rewrite and document code from one language to another
 </li>
 <li>
 Document solution architectures, design decisions, implementation details, and lessons learned.
 </li>
 <li>
 Stay up to date with the latest trends and advancements in AI, foundation models, and large language models.
 </li>
 <li>
 Evaluate emerging technologies, tools, and frameworks to assess their potential impact on solution design and implementation
 
 
 </li>
 </ul>
 
 Preferred Technical And Professional Experience
 
 
 
 <ul>
 <li>
 Experience and working knowledge in COBOL &amp; JAVA would be preferred
 </li>
 <li>
 Having experience in Code generation, code matching &amp; code translation leveraging LLM capabilities would be a Big plus
 </li>
 <li>
 Demonstrate a growth mindset to understand clients' business processes and challenges
 </li>
 </ul>
</div>

Data Scientist-Advanced Analytics

MKS Inc.

<div>
 
 A Day in Your Life at MKS:
 
 
 
 As a
 
 Technical Data Compliance Engineer/Analyst
 
 at MKS Global Capability Center Bangalore, you will partner with R&amp;D teams across MKS PSD Business Units to manage technical data of complex high-precision positioning systems to ensure design documentation in compliance with MKS standards and international regulations.
 
 
 
 You Will Make an Impact By:
 
 
 
 <ul>
 <li>
 Technical data management of electromechanical parts and assemblies, including regular communication with suppliers and R&amp;D teams.
 </li>
 <li>
 Design quality and regulatory compliance (RoHS, REACH, etc.)
 </li>
 <li>
 Manage, generate, and implement ECO documentation changes. ERP system of record data entry and maintenance.
 </li>
 <li>
 CAD/manufacturing drawing review.
 
 
 </li>
 </ul>
 
 Skills You Bring:
 
 
 
 <ul>
 <li>
 3D CAD experience with SolidWorks and/or AutoCAD with fundamental drafting proficiency and knowledge of ANSI Y14.5 GD&amp;T
 </li>
 <li>
 Working knowledge of safety and regulatory compliance for mechanical and electrical hardware
 </li>
 <li>
 Experience with mechanical and electrical hardware designs, product BOMs, and Engineering Change Order process, and new product introduction.
 </li>
 <li>
 Experience with ERP systems and data entry (SAP preferred)
 </li>
 <li>
 Excellent verbal and written communication skills (French speaking preferred)
 
 
 </li>
 </ul>
</div>

Mechanical Engineer Analyst

<div>
 We are seeking a skilled Python Engineer to join our dynamic team. The ideal candidate will be responsible for developing robust web frameworks, building backend code to efficiently serve datasets, and performing data transformations to support our data-driven applications. This role requires a deep understanding of Python, web development frameworks, data processing, and database management
 
 
 Core Python: Be proficient in the concepts and features of the Python language , experienced with web frameworks like Flask and FastAPI for developing RESTful APIs, and skilled in asynchronous programming using asyncio. Experience with handling concurrent connections and optimizing IO operations. Should have a deep understanding of microservices architecture and be capable of writing clean, efficient, and maintainable code.
 
 
 Security Best Practices and Requirements
 
 
 Understanding of security practices for microservices, including authentication, authorization, and encryption. Experience with securing APIs using OAuth, JWT, and API gateways.
 
 
 Web Framework Development
 
 
 Design, develop, and maintain scalable web frameworks using Python.
 
 
 Collaborate with front-end developers to integrate user-facing elements with server-side logic.
 
 
 Ensure the web frameworks are secure, performant, and maintainable.
 
 
 Have a sound knowledge of one or more web frameworks that are based on Python, such as Django, Flask, Pyramid, etc. Should have the know how to use the framework's features, such as routing, templating, database integration, authentication, testing.
 
 
 Object Relational Mappers (ORMs): Develop backend systems to serve datasets through APIs and other interfaces.
 
 
 Implement data access layers, integrating with various databases and data storage solutions.
 
 
 Optimize backend code for performance, scalability, and reliability.
 
 
 Be able to use ORMs to interact with databases using Python objects. ORMs abstract away the low-level SQL queries and provide a high-level interface to manipulate data. Some popular ORMs for Python are SQL Alchemy, Django ORM, etc. Should know how to define models, perform CRUD operations, query data, handle transactions.
 
 
 Data Transformations and Database Management: Design and maintain database schemas to support application requirements.
 
 
 Optimize database performance and ensure efficient data retrieval.
 
 
 Implement and manage data storage solutions, both relational and NoSQL.
 
 
 Design and implement data transformation pipelines to process and prepare data for analysis.
 
 
 Utilize ETL (Extract, Transform, Load) processes to clean, aggregate, and enrich data.
 
 
 Ensure data integrity, quality, and consistency across all transformation processes.
 
 
 Other Skills:Should also have general skills that are fundamental to any developer, such as:
 
 
 Debugging and testing: Should be able to use tools and techniques to find and fix errors in code, such as logging, breakpoints, unit testing, integration testing
</div>

Python Developer

Straive

<div>
 
 
 About the Role
 
 
 
 Straive is looking for a talented and driven
 
 Consultant / Data Scientist / GenAI Engineer
 
 to join our
 
 Analytics &amp; GenAI delivery team
 
 . In this role, you will work under the guidance of the Senior Project Manager / Engagement Manager to design, develop, and deploy advanced AI/ML and Generative AI solutions for global enterprise clients. You will be part of a high-performing team, collaborating with both onshore and offshore members to build scalable, production-grade AI systems.
 
 
 
 
 
 This role is ideal for candidates from
 
 premier engineering institutes
 
 with
 
 2&ndash;3 years of relevant experience
 
 in Python development, LLM integration, and RAG workflows, along with a passion for solving complex problems in real-world business contexts.
 
 
 
 
 
 
 Key Responsibilities
 
 
 <ul>
 <li>
 Develop and maintain
 
 Python-based
 
 applications, AI/ML models, and data processing pipelines for GenAI projects.
 </li>
 <li>
 Implement
 
 Large Language Model (LLM)
 
 integrations, including
 
 Retrieval-Augmented Generation (RAG)
 
 pipelines and embedding-based search solutions.
 </li>
 <li>
 Build data ingestion and transformation workflows, working with structured and unstructured datasets.
 </li>
 <li>
 Optimize AI model performance through
 
 prompt engineering
 
 , fine-tuning, and evaluation techniques.
 </li>
 <li>
 Collaborate closely with senior team members to translate business requirements into technical solutions.
 </li>
 <li>
 Integrate AI solutions with
 
 vector databases
 
 (e.g., Cosmos DB, Pinecone, ChromaDB) and API-driven applications.
 </li>
 <li>
 (Optional) Contribute to
 
 cloud-native deployments
 
 and
 
 Azure architecture
 
 &ndash;based solutions, including containerization, CI/CD, and basic MLOps workflows.
 </li>
 <li>
 Document workflows, maintain code repositories, and follow Agile development practices.
 </li>
 </ul>
 
 
 
 
 
 Required Qualifications
 
 
 <ul>
 <li>
 
 2&ndash;3 years
 
 of relevant experience in AI/ML development, preferably in enterprise projects.
 </li>
 <li>
 Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, Data Science, AI/ML, or related field from a
 
 premier engineering institute
 
 .
 </li>
 <li>
 Proficiency in
 
 Python
 
 programming and familiarity with relevant libraries (e.g., LangChain, Hugging Face, Pandas, NumPy).
 </li>
 <li>
 Hands-on experience implementing
 
 RAG pipelines
 
 , embeddings, and vector search solutions.
 </li>
 <li>
 Understanding of
 
 LLM architectures
 
 and integration patterns.
 </li>
 <li>
 Working knowledge of
 
 SQL
 
 and data processing best practices.
 </li>
 <li>
 Basic knowledge of
 
 cloud DevOps concepts
 
 , preferably with
 
 Azure
 
 (AWS/GCP experience is also acceptable).
 </li>
 </ul>
 
 
 
 
 Preferred Skills
 
 
 <ul>
 <li>
 Exposure to
 
 agentic AI frameworks
 
 such as LangGraph, Semantic Kernel, or similar.
 </li>
 <li>
 Familiarity with ML model lifecycle management and deployment workflows.
 </li>
 <li>
 Prior experience working with cross-border teams and Agile environments.
 </li>
 </ul>
 
 
</div>

Data Scientist

MakeMyTrip

<div>
 
 
 Role:
 
 Lead Data Engineer
 
 
 
 Location:
 
 Bangalore
 
 
 
 Years of Experience:
 
 6 -8 Years
 
 
 
 About the function:
 
 
 
 At MakeMyTrip (MMT), technology is at the heart of everything we do. As a leading player in the travel industry, we leverage cutting-edge solutions like AI, machine learning, and cloud infrastructure to create seamless and innovative travel experiences. We prioritize a tech-first mindset, where our engineering teams lead the charge in transforming the way people travel.
 
 
 Our culture encourages collaboration, creativity, and thought leadership, empowering engineers to challenge the status quo and develop impactful, game-changing solutions. At MMT, you'll work with the latest technologies, continually push boundaries, and have the chance to shape the future of travel.
 
 
 Join us and become part of a forward-thinking, tech-driven organization that thrives on innovation and continuous learning.
 
 
 
 
 
 
 About the team:
 
 
 
 MakeMyTrip as India&rsquo;s leading online travel company and provides petabytes of raw data which is helpful for business growth, analytical and machine learning needs.
 
 
 
 
 
 Data Platform Team is a horizontal function at MakeMyTrip to support various LOBs (Flights, Hotels, Holidays, Ground) and works heavily on streaming datasets which powers personalized experiences for every customer from recommendations to in-location engagement.
 
 
 
 
 
 Our team's key responsibilities are:
 
 <ol>
 <li>
 Design, construct, and maintain robust data systems and architectures
 </li>
 <li>
 Develop and optimize data capture, storage, processing, serving, and querying platforms
 </li>
 <li>
 Create data products for personalization, recommendation, customer segmentation, and intelligence
 </li>
 <li>
 Enhance our Measurement platform for A/B experimentation
 </li>
 <li>
 Contribute to our Feature Store, an internal unified data analytics platform
 </li>
 <li>
 Participate in the development of our next-generation Travel Planner using Generative AI and Multi-Agent frameworks.
 </li>
 <li>
 Implement and optimize data solutions for travel-specific use cases, such as:
 </li>
 <li>
 Analyzing cross-city travel patterns to extend trip recommendations
 </li>
 <li>
 Identifying correlations between hotel bookings in different areas to suggest complementary destinations
 </li>
 </ol>
 
 
 
 
 Required Skills and Experience:
 
 <ul>
 <li>
 Extensive experience working with large data sets with hands-on technology skills to design and build robust data architecture
 </li>
 <li>
 At least 6+ years of hands-on experience in Spark/BigData Tech stack
 </li>
 <li>
 Expertise in stream processing engines like Spark Structured Streaming or Apache Flink
 </li>
 <li>
 Analytical processing on Big Data using Spark
 </li>
 <li>
 At least 4+ years of experience in Scala. Experience with Python will be a plus.
 </li>
 <li>
 Hands-on administration, configuration management, monitoring, performance tuning of Spark workloads, Distributed platforms, and JVM based systems
 </li>
 <li>
 At least 2+ years of cloud deployment experience &ndash; AWS | Azure | Google Cloud Platform
 </li>
 <li>
 2 or more product deployments of big data technologies &ndash; Business Data Lake, NoSQL databases etc
 </li>
 <li>
 Awareness and decision-making ability to choose among various big data, NoSQL, analytics tools and technologies
 </li>
 <li>
 Should have experience in architecting and implementing domain centric big data solutions
 </li>
 <li>
 Ability to frame architectural decisions and provide technology leadership &amp; direction
 </li>
 <li>
 Excellent problem solving, hands-on engineering, and communication skills
 </li>
 </ul>
 
 
 
 
 
 
 
 At MakeMyTrip, we're committed to innovation and excellence in the travel industry. Join us in shaping the future of travel through data-driven solutions and advanced technologies. If you're passionate about leveraging data to create exceptional travel experiences, we want to hear from you!
 
</div>

Lead Data  Engineer

Pocket FM

<div>
 
 Senior Software Engineer-Backend
 
 
 Location: Bengaluru
 
 
 Experience: 3&ndash;5 years
 
 
 
 
 
 About Pocket FM
 
 
 Pocket FM, founded in 2018, is India&rsquo;s leading audio storytelling platform, transforming the way millions consume stories. Offering high-quality serialized content across genres such as Romance, Drama, Thriller, Fantasy, Sci-Fi, and Mythology in eight languages, Pocket FM has built a strong global presence with over 200 million listeners worldwide. With users spending an average of 120 minutes daily on the platform, it has emerged as one of the fastest-growing audio platforms, rapidly expanding its reach across the US, Europe, LATAM, and Southeast Asia.
 
 
 
 
 
 Our unique model combines free listening with micropayments for premium content, powering strong business growth. In FY25, we reached an ARR of INR 2,000 crore, with over 100,000 hours of content on the platform. We're also at the forefront of innovation, leveraging AI-generated content to scale efficiently.
 
 
 
 
 
 Role Overview:
 
 
 We are hiring for a Senior Software Engineer &ndash; Backend who will be responsible for building high-performance APIs, scalable microservices, and robust infrastructure powering our audio platform.
 
 
 This is a high-ownership role that requires strong technical skills, problem-solving abilities, and a mindset to work in a fast-paced product environment.
 
 
 
 
 
 Key Responsibilities:
 
 <ul>
 <li>
 Design, build, and maintain scalable backend services using Python or Golang
 </li>
 <li>
 Develop and manage RESTful APIs for mobile/web platforms and internal tools
 </li>
 <li>
 Collaborate closely with product, design, and analytics teams to deliver features at scale
 </li>
 <li>
 Optimize performance, ensure high availability, and handle large-scale traffic
 </li>
 <li>
 Write clean, modular, and testable code adhering to software best practices
 </li>
 <li>
 Lead debugging efforts, production issue resolution, and post-mortem analysis
 </li>
 <li>
 Participate in code reviews and continuously drive technical improvements
 </li>
 </ul>
 
 
 
 
 Skills &amp; Qualifications:
 
 <ul>
 <li>
 3&ndash;5 years of backend development experience in Python or Golang
 </li>
 <li>
 Strong understanding of data structures, algorithms, OOP, operating systems, and networking
 </li>
 <li>
 Proven experience developing and managing REST APIs
 </li>
 <li>
 Proficient in working with databases like PostgreSQL, MongoDB, and Redis
 </li>
 <li>
 Hands-on experience with cloud platforms &ndash; AWS, GCP, or Azure
 </li>
 <li>
 Experience with Docker or other container technologies
 </li>
 <li>
 Familiarity with CI/CD pipelines and deployment best practices
 </li>
 <li>
 Experience with monitoring and logging tools like Prometheus, Grafana, or ELK stack
 </li>
 <li>
 Experience with Git and version control workflows
 </li>
 <li>
 Contributions to open-source projects or a strong personal backend project portfolio
 </li>
 <li>
 Bachelor&rsquo;s degree in Computer Science, Engineering, or a related technical discipline
 </li>
 <li>
 Strong communication skills and a proactive, ownership-driven mindset
 </li>
 </ul>
 
 
 
 
 You can get more updates, insights and everything behind the scenes at Pocket FM here -
 
 Pocket FM
 
 
</div>

Senior Software Engineer - Backend

Explore 65+ Data engineer pyspark in Bangalore this November. Find roles across top companies and apply now to boost your career.