Python Web Scraping Engineer

3 weeks ago

BangaloreHyderabad, India Whizz HR Full time

We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team.

The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python.

As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts, extracting data from various sources, and delivering high-quality datasets for analysis and :

Web Scraping Development :


- Design, develop, and maintain web scraping scripts and automation tools using Python.

- Write efficient and scalable code to extract data from websites, APIs, databases, and other online sources.

Data Extraction :


- Implement data extraction techniques to retrieve structured and unstructured data from HTML pages, JSON responses, XML documents, and other formats.

- Handle pagination, AJAX requests, and dynamic content rendering.

Data Transformation :


- Clean, preprocess, and transform raw data into usable formats for analysis and visualization.

- Perform data validation, deduplication, normalization, and enrichment as needed.

Proxy Rotation and CAPTCHA Handling :


- Implement strategies for proxy rotation and CAPTCHA handling to bypass rate limits, IP blocks, and other anti-scraping measures.

- Use proxy servers, user-agent rotation, and CAPTCHA solvers to ensure uninterrupted scraping.

Scalability and Performance Optimization :


- Optimize web scraping scripts for performance, scalability, and efficiency.

- Implement asynchronous and parallel processing techniques to improve speed and throughput.

Error Handling and Logging :


- Handle errors, exceptions, and edge cases gracefully in web scraping scripts.

- Implement logging and monitoring solutions to track scraping activities, detect anomalies, and troubleshoot issues.

Compliance and Legal Considerations :


- Ensure compliance with website terms of service, copyright laws, and data privacy regulations while scraping data.

- Respect robots txt rules and obtain necessary permissions for scraping restricted or sensitive content.

Documentation and Collaboration :


- Document technical specifications, data schemas, and scraping workflows.

- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders.

Qualifications :

- Educational Qualifications: Bachelor's/Master's

- 4 to 6 years of experience in web scraping and data extraction using Python.

- Proficiency in web scraping libraries and frameworks such as BeautifulSoup, Scrapy, Selenium, or Requests.

- Strong understanding of HTML, CSS, XPath, JSON, and regular expressions for data parsing and extraction.

- Experience with database systems such as MySQL, PostgreSQL, MongoDB, or SQLite for storing and managing scraped data.

- Familiarity with proxy servers, CAPTCHA solving services, and IP rotation techniques for bypassing anti-scraping measures.

- Excellent problem-solving, analytical, and communication skills.

(ref:hirist.tech)

  • Python Web Scraping Engineer

    2 months ago

    Bangalore/Hyderabad, Karnataka, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python Web Scraping Engineer

    3 weeks ago

    Bangalore/Hyderabad, Karnataka, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python Web Scraping Engineer

    2 months ago

    Hyderabad, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python Web Scraping Engineer

    2 months ago

    hyderabad, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python Web Scraping Engineer

    3 weeks ago

    Hyderabad, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python Web Scraping Engineer

    1 week ago

    hyderabad, India Whizz HR Full time

    We are seeking a highly skilled and experienced Python Web Scraping Engineer with 4 to 6 years of experience to join our team. The ideal candidate will have a strong background in web scraping, data extraction, and automation using Python. As a Python Web Scraping Engineer, you will be responsible for developing and maintaining web scraping scripts,...

  • Python-Web Scraping

    2 months ago

    bangalore, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.Python, Shell scriptingWeb Scraping, Regex Good ExperienceRabbitMQ, AWS Message...

  • Python-Web Scraping

    2 months ago

    bangalore, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software Engineer Job Opportunity for Python + Web Scraping!! We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills. Python, Shell scripting Web Scraping, Regex Good Experience RabbitMQ, AWS...

  • GMWARE - Python Engineer - Web Scraping & Crawling

    2 weeks ago

    Bangalore, India GM WARE Full time

    Python Engineer - Web Scraping & Crawling Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set : - Develop and maintain service that extracts...

  • GMWARE - Python Engineer - Web Scraping & Crawling

    6 days ago

    Bangalore, Karnataka, India GM WARE Full time

    Python Engineer - Web Scraping & Crawling Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set : - Develop and maintain service that extracts websites...

  • GMWARE - Python Developer - Web Scraping & Crawling

    4 weeks ago

    Bangalore, India GM WARE Full time

    Role : Python Engineer - Web Scraping & Crawling Python Web scraping Engineer : - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities : -...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 weeks ago

    Bangalore, Karnataka, India GM WARE Full time

    Role : Python Engineer - Web Scraping & CrawlingPython Web scraping Engineer :- Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities :- Develop...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 weeks ago

    Bangalore, India GM WARE Full time

    Role : Python Engineer - Web Scraping & Crawling Python Web scraping Engineer : - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities : -...

  • GMWARE - Python Developer - Web Scraping & Crawling

    4 weeks ago

    Bangalore, Karnataka, India GM WARE Full time

    Role : Python Engineer - Web Scraping & CrawlingPython Web scraping Engineer :- Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities :- Develop...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 weeks ago

    Bangalore, Karnataka, India GM WARE Full time

    Job Description :Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines. Responsibilities : - Develop and maintain service that extracts...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 weeks ago

    Bangalore, India GM WARE Full time

    Job Description :Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines. Responsibilities : - Develop and maintain service that extracts...

  • Lead Software Engineer | Python

    3 weeks ago

    bangalore, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    4 weeks ago

    bangalore, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer

    2 months ago

    bangalore, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    2 weeks ago

    Bangalore City, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...