Python Backend Engineer + Web scraper

Position: Associate Tech Specialist / Tech Specialist

Experience: 6 to 8 Years

Location: Remote

Work timing: 2 PM to 11 PM

Contact Duration: 6 Months (Extendable)

PF & BGV Mandatory

Roles and Responsibilities:

* Lead the design, architecture, and implementation of scalable and maintainable web scraping solutions using the Scrapy framework, integrated with tools such as Kafka, Zookeeper, and Redis

* Develop and maintain web crawlers to automate data extraction from various sources, ensuring alignment with user and application requirements

* Research, design, and implement automation strategies across multiple platforms, tools, and technologies to optimize business processes

* Monitor, troubleshoot, and resolve issues affecting the performance, reliability, and stability of scraping systems and automation tools

* Serve as a Subject Matter Expert (SME) for automation systems, providing guidance and support to internal teams

* Analyze and validate extracted data to ensure accuracy, integrity, and compliance with client's data standards

* Define, implement, and enforce data requirements, standards, and best practices to ensure consistent and efficient operations

* Collaborate with stakeholders and end users to define technical requirements, business goals, and alternative solutions for data collection and reporting

* Create, manage, and document reports, processes, policies, and project plans, including risk assessments and goal tracking

* Conduct code reviews, enforce coding standards, and provide technical leadership and mentorship to development team members

* Proactively identify and mitigate technical risks, recommending improvements in technologies, tools, and processes

* Drive the adoption of modern development tools, frameworks, and best practices

* Contribute to strategic planning related to automation initiatives and product development

* Ensure clear, thorough communication and documentation across teams to support knowledge sharing and training

Minimum Qualifications:

* Bachelor’s degree in Computer Science, Software Engineering, or related field.

* 5+ years of professional software development experience.

* Strong proficiency in HTML, XML, XPath, XSLT, and Regular Expressions for data extraction and transformation

* Hands-on experience with Visual Studio

* Strong proficiency in Python

* Some experience with C# .NET

* Solid experience with MS SQL Server, with strong skills in SQL querying and data analysis

* Experience with web scraping, particularly using the Scrapy framework integrated with Kafka, Zookeeper, and Redis

* Experience with .NET automation tools such as Selenium

* Understanding of CAPTCHA-solving services and working with proxy services

* Experience working in a Linux environment is a plus

* Highly self-motivated and detail-oriented, with a proactive, goal-driven mindset

* Strong team player with dependable work habits and well-developed interpersonal skills

* Excellent verbal and written communication skills

* Demonstrates willingness and flexibility to adapt schedule when necessary to meet client needs.

Core Technical Skills- Python (strong proficiency)

1. HTML, XML, XPath, XSLT, Regex (for data extraction)

2. Visual Studio

3. C# .NET (some experience)

4. MS SQL Server (SQL querying & analysis)

5. Web Scraping Framework: Scrapy (integrated with Kafka, Zookeeper, Redis)

6. Automation Tools: Selenium (.NET automation tools)

Job Types: Freelance, Internship
Contract length: 6 months

Pay: ₹80,000.00 per month

Work Location: Remote