Web Scraping Expert

16 hours ago

Bengaluru Karnataka, India FullStackTechies Full time

AIMLEAP is Hiring:
**Job Title**: Web Scraping Expert (Scrapy & Python)

**Company**: AIMLEAP

**CTC**: 3 LPA - 5 LPA ( **Depends on interview performance** )

**Department**: Data Engineering/Web Development

**Summary**:
We are seeking a talented and results-oriented Web Scraping Expert to join our team. You will be responsible for developing and maintaining robust web scraping pipelines using Scrapy, a high-performance Python framework. You will leverage your expertise in Python and web scraping techniques to extract valuable data from various online sources in a reliable and scalable manner.

**Responsibilities**:

- Design, develop, and maintain web scraping projects using Scrapy, including:

- Crawling websites to extract specific data points
- Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures
- Employing data extraction techniques with CSS selectors and XPath
- Processing and cleaning scraped data using Python libraries (e.g., Pandas, NumPy)
- Storing extracted data in appropriate formats (e.g., CSV, JSON, databases)
- Collaborate with data engineers and analysts to identify data needs and define scraping requirements
- Write well-documented, maintainable, and efficient Scrapy code
- Integrate scraping pipelines with other Python frameworks and tools as needed
- Stay up-to-date with the latest web scraping trends and best practices
- Troubleshoot and address challenges related to rate limiting, authentication, and website changes
- Implement caching strategies for efficient data retrieval and reduced website load
- Consider potential ethical and legal implications of web scraping

**Qualifications**:

- Proven experience with web scraping techniques and tools (Scrapy preferred)
- Strong proficiency in Python programming
- Knowledge of HTML and CSS for navigating website structures
- Experience with data extraction methods (CSS selectors, XPath)
- Familiarity with data cleaning and manipulation libraries (Pandas, NumPy)
- Understanding of distributed crawling concepts for large-scale data extraction
- Excellent problem-solving and analytical skills
- Ability to work independently and as part of a team
- Strong communication and documentation skills

**Salary**: ₹300,000.00 - ₹600,000.00 per year

**Benefits**:

- Health insurance
- Work from home

Schedule:

- Day shift

Supplemental pay types:

- Yearly bonus

Application Question(s):

- Years of experience in web scraping
- Do you have experience in Scrapy
- Current CTC
- Notice period (in days)
- Experience in Middlewares.py,settings.py,pipelines.py

**Experience**:

- total work: 2 years (preferred)

Work Location: Remote

  • Web Scraping Specialist

    1 day ago

    Bangalore, Karnataka, India Hypersonix Full time

    Position Overview We are seeking a highly skilled Web Scraping Architect to join our team The successful candidate will be responsible for designing implementing and maintaining web scraping processes to gather data from various online sources efficiently and accurately As a Web Scraping Specialist you will play a crucial role in collecting data for...

  • Gmware - Web Scraping Developer

    4 weeks ago

    Bengaluru, India GM WARE Full time

    Job Description :Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated knowledge experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Roles and Responsibilities :- Develop and maintain service that extracts websites...

  • Python Automation and Web Scraping Specialist

    1 week ago

    Bengaluru, India Foresiet Full time

    Company DescriptionForesiet is an AI-enabled SaaS-based Cybersecurity company that provides a comprehensive solution for Digital Risk Prevention. Leveraging the Cyber Digital Investigator platform, Foresiet proactively detects, monitors, and secures identity, data, and asset threats. Our unique combination of Human Intelligence (HUMINT) and Applied Research...

  • Python automation and web scraping specialist

    6 days ago

    Bengaluru, India Foresiet Full time

    Company DescriptionForesiet is an AI-enabled Saa S-based Cybersecurity company that provides a comprehensive solution for Digital Risk Prevention. Leveraging the Cyber Digital Investigator platform, Foresiet proactively detects, monitors, and secures identity, data, and asset threats. Our unique combination of Human Intelligence (HUMINT) and Applied Research...

  • Python Developer Web Scraping

    1 week ago

    Bengaluru, Karnataka, India Gmware Full time ₹ 2,00,000 - ₹ 6,00,000 per year

    Web Scraping engineerPython web scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.ResponsibilitiesDevelop and maintain a service that extracts website data using...

  • Python Developers Web Scraping, Crawling

    2 days ago

    Bengaluru, Delhi, Mumbai, NCR, India Hatchtra Innotech Full time ₹ 2,50,000 - ₹ 12,00,000 per year

    Were Hiring: Python Developers Web Scraping, Crawling & Automation Experts(40 Openings | Yrs Experience | Multiple Levels & Roles)Location: Mumbai / Pune / Chandigarh / Bengaluru / Gurugram / Noida / New Delhi (WFO, Hybrid, or Remote Based on RoleIndustry: Fortune 500 Client Projects (Staffing via Hatchtra Innotech Pvt. Ltd.) Employment Type: Full-Time |...

  • Web Scraping Specialist

    4 days ago

    Bengaluru, Karnataka, India Foresiet Technology Full time

    Building and maintaining advanced web scraping and automation scripts across platforms (LinkedIn, Facebook, Twitter, Discord). Handle CAPTCHA, proxy, IP rotation, and anti-blocking. Ensure deployable, scalable, and monitored solutions using Docker.

  • Web Data Scraping Intern

    2 weeks ago

    Bengaluru, Karnataka, India Stylumia Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionStylumia, founded in 2015, aims to revolutionize retail strategy and decision-making through advanced intelligence, significantly reducing wastage from unsold products. Utilizing predictive and prescriptive analytics, Stylumia enables brands to anticipate market trends and stay ahead. With features like demand science, real-time consumer...

  • Python Automation and Web Scraping Specialist

    1 week ago

    Bengaluru, India Foresiet Full time

    Company DescriptionForesiet is an AI-enabled SaaS-based Cybersecurity company that provides a comprehensive solution for Digital Risk Prevention. Leveraging the Cyber Digital Investigator platform, Foresiet proactively detects, monitors, and secures identity, data, and asset threats. Our unique combination of Human Intelligence (HUMINT) and Applied Research...

  • Python automation and web scraping specialist

    2 weeks ago

    Bengaluru, India Foresiet Full time

    Company DescriptionForesiet is an AI-enabled Saa S-based Cybersecurity company that provides a comprehensive solution for Digital Risk Prevention. Leveraging the Cyber Digital Investigator platform, Foresiet proactively detects, monitors, and secures identity, data, and asset threats. Our unique combination of Human Intelligence (HUMINT) and Applied Research...