pyspark developer

Cognizant Technology Solutions

<div><h3 class="jobSectionHeader">Skill: Pyspark</h3><h3 class="jobSectionHeader">
Experience: 6 to 9 years</h3><h3 class="jobSectionHeader">
Location: AIA Kochi</h3><h3 class="jobSectionHeader">
Responsibilities</h3><ul><li>
Develop and maintain scalable data pipelines using Python and PySpark.</li><li>
Collaborate with data engineers and data scientists to understand and fulfill data processing needs.</li><li>
Optimize and troubleshoot existing PySpark applications for performance improvements.</li><li>
Write clean, efficient, and well-documented code following best practices.</li><li>
Participate in design and code reviews.</li><li>
Develop and implement ETL processes to extract, transform, and load data.</li><li>
Ensure data integrity and quality throughout the data lifecycle.</li><li>
Stay current with the latest industry trends and technologies in big data and cloud computing.</li></ul></div>

Pyspark Developer

<div>Line of Service
Advisory
Industry/Sector
Not Applicable
Specialism
Data, Analytics &amp; AI
Management Level
Senior Associate
Job Description &amp; Summary
At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth.
 
 
In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.
Why PWC
At PwC, you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes for our clients and communities. This purpose-led and values-driven work, powered by technology in an environment that drives innovation, will enable you to make a tangible impact in the real world. We reward your contributions, support your wellbeing, and offer inclusive benefits, flexibility programmes and mentorship that will help you thrive in work and life. Together, we grow, learn, care, collaborate, and create a future of infinite experiences for each other. Learn more about us .
At PwC, we believe in providing equal employment opportunities, without any discrimination on the grounds of gender, ethnic background, age, disability, marital status, sexual orientation, pregnancy, gender identity or expression, religion or other beliefs, perceived differences and status protected by law. We strive to create an environment where each one of our people can bring their true selves and contribute to their personal growth and the firm&rsquo;s growth. To enable this, we have zero tolerance for any discrimination and harassment based on the above considerations.
Responsibilities:
&middot; Total Experience: 3 to 8 years &middot; Languages: Scala/Python 3.x/Pyspark &middot; Ability to design and build Python-based code generation framework and runtime engine &middot; Spark SQL, Spark Streaming in Python &middot; Hands on experience in writing PySpark codes
Mandatory skill sets:
PySpark, Python, Hadoop
Preferred skill sets:
PySpark, Python, Hadoop
Years of experience required:
3-8 Years
Education qualification:
BE, B.Tech, MCA, M.Tech
Education (if blank, degree and/or field of study not specified)
Degrees/Field of Study required: Bachelor of Engineering
Degrees/Field of Study preferred:
Certifications (if blank, certifications not specified)
Required Skills
PySpark
Optional Skills
Accepting Feedback, Accepting Feedback, Active Listening, Agile Scalability, Amazon Web Services (AWS), Analytical Thinking, Apache Airflow, Apache Hadoop, Azure Data Factory, Communication, Creativity, Data Anonymization, Data Architecture, Database Administration, Database Management System (DBMS), Database Optimization, Database Security Best Practices, Databricks Unified Data Analytics Platform, Data Engineering, Data Engineering Platforms, Data Infrastructure, Data Integration, Data Lake, Data Modeling, Data Pipeline {+ 27 more}
Desired Languages (If blank, desired languages not specified)
Travel Requirements
Not Specified
Available for Work Visa Sponsorship?
No
Government Clearance Required?
No
Job Posting End Date</div>

IN-Senior Associate_PySpark Developer_Data & Analytics_Advisory_PAN India

<div>Skill: Pyspark
Experience: 6 to 13 years
Location: AIA Group - Bhubaneswar
Job description :
<ul><li>Develop and maintain scalable data pipelines using Python and PySpark.</li><li>
Collaborate with data engineers and data scientists to understand and fulfill data processing needs.</li><li>
Optimize and troubleshoot existing PySpark applications for performance improvements.</li><li>
Write clean, efficient, and well-documented code following best practices.</li><li>
Participate in design and code reviews.</li><li>
Develop and implement ETL processes to extract, transform, and load data.</li><li>
Ensure data integrity and quality throughout the data lifecycle.</li><li>
Stay current with the latest industry trends and technologies in big data and cloud computing.</li></ul>
Qualifications<ul><li>
Proven experience as a Python Developer with expertise in PySpark.</li><li>
Strong understanding of big data technologies and frameworks.</li><li>
Experience with distributed computing and parallel processing.</li><li>
Proficiency in SQL and experience with database systems.</li><li>
Solid understanding of data engineering concepts and best practices.</li><li>
Ability to work in a fast-paced environment and handle multiple projects simultaneously.</li><li>
Excellent problem-solving and debugging skills.</li><li>
Strong communication and collaboration abilities.</li></ul></div>

Pyspark

ixceed

Role: Snowflake DeveloperLocation: BangaloreMode: HybridType: ContractJob Description:<ul><li>Minimum 6 &ndash; 10 years of work experience with at least two project implementations leveraging Snowflake - either migration from traditional DWHs or a fresh implementation</li><li>Expert-level knowledge in SQL, SnowSQL, query optimization, performance tuning etc.</li><li>Data Modeling: Knowledge of data modeling principles and techniques.</li><li>Solid work experience in AWS/ AZURE cloud platforms</li><li>Expertise in Snowflake concepts like setting up Data Masking, Tagging, SnowPipe, RBAC controls, virtual warehouse sizing, zero copy clone, data sharing, Snow pipe, Streams &amp;Tasks, Time travel, Secure Data sharing, DMF, iceberg tables, Dynamic Tables, Micro Partition, Query Profiler, etc.</li><li>Good to have Snowflake's advanced concepts on Cortex AI features, search mechanism, etc.</li><li>Cloud best Practice adoption, strategy development</li><li>In-depth knowledge of Snowflake Virtual Warehouse, Monitoring Warehouse, Table Structures, Views, Materialized Views, Cloning, etc.</li><li>Knowledge on Integration (ETL) and Data Ingestion, Data Conversion, and Data Migration.</li><li>Technically strong in Python, PySpark</li><li>Exposure to tools like Jira, Confluence, and Agile practices</li></ul>Job Type: Contractual / Temporary Contract length: 12 monthsPay: &#8377;10,000.00 - &#8377;13,000.00 per dayExperience:<ul><li>Snowflake : 5 years (Required)</li><li>SQL: 4 years (Required)</li><li>AWS/ AZURE : 3 years (Required)</li><li>Python, Pyspark: 3 years (Required)</li></ul>Work Location: In person

Snowflake Data Developer

Fluxtek solutions

Job Description:<ul><li>Job Title: App Developer</li><li>Rounds of interviews: 2</li><li>Degree Requirement: Bachelors</li><li>Work Authorization: USC &ndash; Must be eligible to obtain Public Trust Clearance.</li><li>Work Location: Remote</li><li>Placement Type: Staffing</li><li>Tech Screen: N/A</li></ul>Candidate Qualifications: (10+ Years&rsquo; Experience) Technical Skills: REST API Development P4 - Expert Python Software Development P4 - ExpertRole Description:Responsible for developing and/or engineering the end-to-end features of a system, from user experience to backend code. Use development skills to deliver innovative solutions that help our clients improve the services they provide. Leverage new technologies that can be applied to solve challenging business problems with a cloud first and agile mindset.Experience Required: ~3 years with Python REST API FrameworkPreferred: Streamlit, FAST API, Pyspark, SwagJob Type: Full-timeWork Location: Remote

App Developer

Quadrabay Verification Services

Position SummaryWe are seeking a skilled Data Engineer with strong expertise in building and managing ETL/ELT pipelines using PySpark within an AWS cloud environment. The ideal candidate will be responsible for designing, implementing, and maintaining scalable data workflows to support multiple data sources, while ensuring seamless integration with Power BI for reconciliation, reporting, and advanced analytics.Key Responsibilities&middot; Data Pipeline Developmento Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.o Implement data ingestion from multiple data sources (Finance, CRM, Payment Gateways, etc.) into AWS Data Lake/Warehouse.o Ensure efficient data partitioning, schema management, and optimization for large datasets.&middot; Cloud &amp; Infrastructure (AWS)o Leverage AWS services (S3, Glue, EMR, Redshift, Lambda, Step Functions) for data engineering workflows.o Implement data security, access control, and compliance in line with organizational standards.o Automate deployments using CI/CD pipelines.&middot; Data Integration &amp; Reportingo Prepare curated datasets for Power BI dashboards and reports.o Enable reconciliation processes by providing granular, auditable datasets.o Collaborate with BI developers to ensure data accuracy, integrity, and performance in reporting.&middot; Collaboration &amp; Processo Work closely with Business Analysts, Data Scientists, and ERP functional teams to understand requirements.o Participate in data governance initiatives, ensuring metadata, lineage, and quality controls are implemented.o Provide support for ad-hoc analytics requests from stakeholders.Required Skills &amp; Qualifications&middot; Technical Skillso Strong programming in Python with deep knowledge of PySpark.o Proven experience in ETL/ELT pipeline development in distributed environments.o Expertise in AWS data services (S3, Glue, EMR, Redshift, Athena, Lambda, Kinesis).o Proficiency in SQL (analytical queries, optimization, tuning).o Familiarity with Power BI integration (Direct Query, Dataflows, Gateways).&middot; Data Engineering Practiceso Experience with data lakes, warehouses, and star/snowflake schema design.o Knowledge of data quality frameworks, reconciliation processes, and auditability controls.o Hands-on experience with workflow orchestration tools.&middot; Soft Skillso Strong problem-solving and analytical skills.o Ability to collaborate across IT, business, and analytics teams.o Clear communication for technical and non-technical stakeholders.Preferred Qualifications&middot; Experience with ERP data models (Finance, CRM - Reconciliation).&middot; Exposure to machine learning data preparation (feature engineering, training/test datasets).&middot; Certification(s):o AWS Certified Data Analytics &ndash; Specialtyo Databricks/Spark Developer Certificationo Microsoft Power BI Data Analyst AssociateJob Type: Full-timeAbility to commute/relocate:<ul><li>Ajman: Reliably commute or planning to relocate before starting work (Preferred)</li></ul>Application Question(s):<ul><li>Availability to start immediately </li><li>Visa Status</li></ul>Experience:<ul><li>Data Engineer: 3 years (Preferred)</li></ul>Language:<ul><li>English (Preferred)</li></ul>

Data Engineer

Enterprise64

<div>Overview 
Enterprise64 is a U.S.-based technology company that creates digital solutions for startups to Fortune 500 companies in the United States, Europe, and the United Arab Emirates. We are growing rapidly across all our global offices and have career opportunities for tech professionals who are looking to work in a fast-growing company, work in a fun environment, be challenged, grow, and fast-track their careers.
Job Overview 
We are hiring a Senior Python Backend Developer to join our dynamic team (Contract Role). The ideal candidate will have strong expertise in backend development, API design, and database optimization, ensuring high-performance, scalable, and secure applications. You will collaborate with cross-functional teams to build efficient, maintainable, and reliable backend solutions that drive our innovative products.
Key Responsibilities<ul><li>
Design, develop, and maintain ETL pipelines for large-scale data processing and integration.</li><li>
Work with AWS Glue, PySpark, and Apache Spark to process structured and unstructured data efficiently.</li><li>
Build and optimize data ingestion workflows from multiple sources into data lakes and warehouses.</li><li>
Ensure data quality, consistency, and reliability through validation and monitoring.</li><li>
Collaborate with cross-functional teams, including data scientists, analysts, and backend engineers, to deliver scalable data solutions.</li><li>
Implement and manage data transformation and schema evolution using AWS services (Glue, S3, Lambda, Redshift, etc.).</li><li>
Optimize Spark jobs for performance and cost efficiency.</li><li>
Develop and maintain metadata management and data cataloging in AWS Glue.</li><li>
Ensure adherence to data security, compliance, and governance best practices.</li><li>
Use Python for automation, data manipulation, and integration tasks.</li><li>
Implement CI/CD pipelines for data workflows and maintain infrastructure-as-code (IaC) using tools like Terraform or CloudFormation (a plus).</li></ul> 
Preferred Qualifications<ul><li>
Strong expertise in Python and PySpark for large-scale data processing.</li><li>
Hands-on experience with AWS Glue, AWS Lambda, S3, and Redshift.</li><li>
Solid understanding of ETL frameworks, data modeling, and data warehousing concepts.</li><li>
Experience with Apache Spark and distributed computing principles.</li><li>
Familiarity with workflow orchestration tools (e.g., Airflow, Step Functions).</li><li>
Knowledge of SQL and NoSQL databases for data querying and optimization.</li><li>
Strong problem-solving and debugging skills for data-intensive applications.</li><li>
Experience working in agile environments and collaborating with cross-functional teams.</li></ul></div>

Senior Python Developer - Contractor

Search 3244+ Pyspark developer jobs near you this October. New roles added regularly across leading industries. Apply today.