PySpark Developer (Microsoft Fabric ETL)

Technisoft Consulting,Inc is Looking an experienced PySpark Developer with strong Python and ETL development skills to join our team. The ideal candidate will have hands-on experience in building scalable data pipelines, transforming complex datasets, and working with Microsoft Fabric for enterprise data solutions.

This role requires someone who can design, implement, and optimize ETL workflows using PySpark in Microsoft Fabric, ensuring data quality, reliability, and performance.

Responsibilities :

• Design and develop ETL processes using PySpark in Microsoft Fabric.
• Write efficient, reusable, and scalable Python code for data extraction, transformation, and loading.
• Integrate data from multiple sources (structured/unstructured) into Fabric Lakehouse environments.
• Optimize ETL pipelines for performance, reliability, and cost efficiency.
• Collaborate with data engineers, analysts, and business stakeholders to understand data requirements.
• Implement best practices for data quality, validation, and governance.
• Troubleshoot issues, debug ETL jobs, and ensure smooth operations in production.
• Document solutions, workflows, and processes.

Requirements :

• 2–4 years of experience in Python ETL development (preferably enterprise-scale).
• Strong working knowledge of PySpark (DataFrames, transformations, window functions, optimizations).
• Hands-on experience with Microsoft Fabric (Lakehouse, Notebooks, Delta tables, Dataflows).
• Strong SQL skills for data querying, profiling, and validation.
• Experience with data modeling concepts (Star Schema, Dimensional Modeling, Slowly Changing Dimensions).
• Familiarity with version control (Git), CI/CD pipelines, and Agile development practices.
• Strong problem-solving skills and ability to work independently with minimal supervision.

Preferred Skills :

• Experience with Azure Data Factory or other orchestration tools.
• Familiarity with Python beyond PySpark (for automation, API integration, or ML workflows).
• Knowledge of Spark SQL and performance optimization techniques in distributed systems.
• Exposure to CI/CD pipelines for BI and data engineering solutions.
• Understanding of business domains such as finance, sales, or operations.

What We Offer:

• Opportunity to work with a talented team and gain hands-on experience.
• Professional growth and mentorship.
• Competitive salary and benefits.
• A collaborative and learning-focused work environment.

Location: Onsite, Egerton Road, Lahore

Salary: Market competitive + Bonus and other benefits

Interested candidates can share resumes at Abdul@technisoftconsulting.com

Job Type: Full-time

Pay: Rs150,000.00 - Rs170,000.00 per month

Work Location: In person