Web Scraping Expert

6 days ago

Bengaluru Karnataka, India FullStackTechies Full time

AIMLEAP is Hiring:
**Job Title**: Web Scraping Expert (Scrapy & Python)

**Company**: AIMLEAP

**CTC**: 3 LPA - 5 LPA ( **Depends on interview performance** )

**Department**: Data Engineering/Web Development

**Summary**:
We are seeking a talented and results-oriented Web Scraping Expert to join our team. You will be responsible for developing and maintaining robust web scraping pipelines using Scrapy, a high-performance Python framework. You will leverage your expertise in Python and web scraping techniques to extract valuable data from various online sources in a reliable and scalable manner.

**Responsibilities**:

- Design, develop, and maintain web scraping projects using Scrapy, including:

- Crawling websites to extract specific data points
- Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures
- Employing data extraction techniques with CSS selectors and XPath
- Processing and cleaning scraped data using Python libraries (e.g., Pandas, NumPy)
- Storing extracted data in appropriate formats (e.g., CSV, JSON, databases)
- Collaborate with data engineers and analysts to identify data needs and define scraping requirements
- Write well-documented, maintainable, and efficient Scrapy code
- Integrate scraping pipelines with other Python frameworks and tools as needed
- Stay up-to-date with the latest web scraping trends and best practices
- Troubleshoot and address challenges related to rate limiting, authentication, and website changes
- Implement caching strategies for efficient data retrieval and reduced website load
- Consider potential ethical and legal implications of web scraping

**Qualifications**:

- Proven experience with web scraping techniques and tools (Scrapy preferred)
- Strong proficiency in Python programming
- Knowledge of HTML and CSS for navigating website structures
- Experience with data extraction methods (CSS selectors, XPath)
- Familiarity with data cleaning and manipulation libraries (Pandas, NumPy)
- Understanding of distributed crawling concepts for large-scale data extraction
- Excellent problem-solving and analytical skills
- Ability to work independently and as part of a team
- Strong communication and documentation skills

**Salary**: ₹300,000.00 - ₹600,000.00 per year

**Benefits**:

- Health insurance
- Work from home

Schedule:

- Day shift

Supplemental pay types:

- Yearly bonus

Application Question(s):

- Years of experience in web scraping
- Do you have experience in Scrapy
- Current CTC
- Notice period (in days)
- Experience in Middlewares.py,settings.py,pipelines.py

**Experience**:

- total work: 2 years (preferred)

Work Location: Remote

  • Web Scraping Expert

    1 week ago

    Bengaluru, India FullStackTechies Full time

    AIMLEAP is Hiring: **Web Scraping Expert - Python Trainer** Experience: 3 to 7 Years Location: Bangalore (Work from Home) No of positions: 2 Industry: Automation/IT-Software Employment Type**:Full Time/Part Time/Consulting** Educational Qualification: B.Tech / B.E / MCA / BCA/MBA Salary/CTC: 4 lacs to 8 lacs INR per annum Notice Period: Immediate - 1...

  • Web Scraping Expert

    2 months ago

    Bengaluru, India FullStackTechies Full time

    AIMLEAP is Hiring: **Web Scraping Expert - Python Trainer** Experience: 3 to 7 Years Location: Bangalore (Work from Home) No of positions: 2 Work Timing**:8:30AM to 1:00 PM** Industry: Automation/IT-Software Employment Type**:Full Time/Part Time/Consulting** Educational Qualification: B.Tech / B.E / MCA / BCA/MBA Salary/CTC: 4 lacs to 8 lacs INR per...

  • Python-Web Scraping

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.Python, Shell scriptingWeb Scraping, Regex Good ExperienceRabbitMQ, AWS Message...

  • Python-Web Scraping

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.Python, Shell scriptingWeb Scraping, Regex Good ExperienceRabbitMQ, AWS Message...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 19 years of experience in offering custom market research...

  • Lead Software Engineer

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 19 years of experience in offering custom market research...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 weeks ago

    Bengaluru, India GM WARE Full time

    Role : Python Engineer - Web Scraping & CrawlingPython Web scraping Engineer :- Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities :- Develop...

  • ▷ Immediate Start: Python-Web Scraping

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.- Python, Shell scripting- Web Scraping, Regex Good Experience- RabbitMQ, AWS...

  • DataWeave - Node js Engineer Web Automation and Web Scraping

    3 weeks ago

    Bengaluru, Karnataka, India timesjobs Full time

    # ## Engineer (Web Automation and Web Scraping)About DataWeaveDataWeave Digital Commerce capabilities are fundamental to winning today's shoppers. The pandemic has accelerated eCommerce adoption across the globe with multiple options for consumers to research, compare and buy products. Solutions that provide actionable data-driven insights have become...

  • GMWARE - Python Developer - Web Scraping & Crawling

    1 week ago

    Bengaluru, India GM WARE Full time

    Job Description :Python Lead Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines. Responsibilities : - Develop and maintain service that extracts...

  • Lead Software Engineer

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research: We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights. Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market...

  • Lead Software Engineer | Python

    3 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer

    2 months ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    3 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    3 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research: We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights. Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market...

  • Lead Software Engineer | Python

    2 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    2 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research: We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights. Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market...

  • Lead Software Engineer | Python

    2 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...