Web Scraping Expert

2 months ago

Bengaluru Karnataka, India FullStackTechies Full time

AIMLEAP is Hiring:
**Job Title**: Web Scraping Expert (Scrapy & Python)

**Company**: AIMLEAP

**CTC**: 3 LPA - 5 LPA ( **Depends on interview performance** )

**Department**: Data Engineering/Web Development

**Summary**:
We are seeking a talented and results-oriented Web Scraping Expert to join our team. You will be responsible for developing and maintaining robust web scraping pipelines using Scrapy, a high-performance Python framework. You will leverage your expertise in Python and web scraping techniques to extract valuable data from various online sources in a reliable and scalable manner.

**Responsibilities**:

- Design, develop, and maintain web scraping projects using Scrapy, including:

- Crawling websites to extract specific data points
- Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures
- Employing data extraction techniques with CSS selectors and XPath
- Processing and cleaning scraped data using Python libraries (e.g., Pandas, NumPy)
- Storing extracted data in appropriate formats (e.g., CSV, JSON, databases)
- Collaborate with data engineers and analysts to identify data needs and define scraping requirements
- Write well-documented, maintainable, and efficient Scrapy code
- Integrate scraping pipelines with other Python frameworks and tools as needed
- Stay up-to-date with the latest web scraping trends and best practices
- Troubleshoot and address challenges related to rate limiting, authentication, and website changes
- Implement caching strategies for efficient data retrieval and reduced website load
- Consider potential ethical and legal implications of web scraping

**Qualifications**:

- Proven experience with web scraping techniques and tools (Scrapy preferred)
- Strong proficiency in Python programming
- Knowledge of HTML and CSS for navigating website structures
- Experience with data extraction methods (CSS selectors, XPath)
- Familiarity with data cleaning and manipulation libraries (Pandas, NumPy)
- Understanding of distributed crawling concepts for large-scale data extraction
- Excellent problem-solving and analytical skills
- Ability to work independently and as part of a team
- Strong communication and documentation skills

**Salary**: ₹300,000.00 - ₹600,000.00 per year

**Benefits**:

- Health insurance
- Work from home

Schedule:

- Day shift

Supplemental pay types:

- Yearly bonus

Application Question(s):

- Years of experience in web scraping
- Do you have experience in Scrapy
- Current CTC
- Notice period (in days)
- Experience in Middlewares.py,settings.py,pipelines.py

**Experience**:

- total work: 2 years (preferred)

Work Location: Remote

  • Web Scraping Expert

    4 weeks ago

    Bengaluru, India FullStackTechies Full time

    AIMLEAP is Hiring: **Web Scraping Expert - Python Trainer** Experience: 3 to 7 Years Location: Bangalore (Work from Home) No of positions: 2 Work Timing**:8:30AM to 1:00 PM** Industry: Automation/IT-Software Employment Type**:Full Time/Part Time/Consulting** Educational Qualification: B.Tech / B.E / MCA / BCA/MBA Salary/CTC: 4 lacs to 8 lacs INR per...

  • Python-Web Scraping

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.Python, Shell scriptingWeb Scraping, Regex Good ExperienceRabbitMQ, AWS Message...

  • Python-Web Scraping

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.Python, Shell scriptingWeb Scraping, Regex Good ExperienceRabbitMQ, AWS Message...

  • Lead Software Engineer

    4 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 19 years of experience in offering custom market research...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 19 years of experience in offering custom market research...

  • Lead Software Engineer

    4 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • GMWARE - Python Developer - Web Scraping & Crawling

    3 days ago

    Bengaluru, India GM WARE Full time

    Role : Python Engineer - Web Scraping & CrawlingPython Web scraping Engineer :- Web Scraping will be responsible for efficient web scraping/web crawling and parsing. - The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set deadlines.Responsibilities :- Develop...

  • ▷ Immediate Start: Python-Web Scraping

    4 weeks ago

    Bengaluru, India Infiniti Research Ltd. Full time

    Designation: Junior Software Engineer/ Senior Software Engineer/ Lead Software EngineerJob Opportunity for Python + Web Scraping!!We are currently looking for Python Developer + Web Scraping with 8+ years of work experience with good communication and strong technical skills.- Python, Shell scripting- Web Scraping, Regex Good Experience- RabbitMQ, AWS...

  • DataWeave - Node js Engineer Web Automation and Web Scraping

    5 days ago

    Bengaluru, Karnataka, India timesjobs Full time

    # ## Engineer (Web Automation and Web Scraping)About DataWeaveDataWeave Digital Commerce capabilities are fundamental to winning today's shoppers. The pandemic has accelerated eCommerce adoption across the globe with multiple options for consumers to research, compare and buy products. Solutions that provide actionable data-driven insights have become...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research: We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights. Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market...

  • Lead Software Engineer

    1 month ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    3 days ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research: We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights. Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market...

  • Lead Software Engineer | Python

    4 days ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • Lead Software Engineer | Python

    3 days ago

    Bengaluru, India Infiniti Research Ltd. Full time

    About Infiniti Research:We are a global leader in next-gen custom market research and market intelligence services. We enable clients in more than a dozen industries to tackle challenges and navigate complexities by offering timely, incisive market insights.Founded in 2003, Infiniti Research has over 20 years of experience in offering custom market research...

  • GMWARE - Python Engineer - Web Scraping/Beautiful Soup

    3 weeks ago

    Bengaluru, India GM WARE Full time

    Role : Python Engineer - Web Scraping & Crawling Python Engineer - Web Scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction along with the ability to communicate effectively and adhere to set : Develop and maintain service that extracts...

  • Web Scraping Python Trainer

    1 month ago

    Bengaluru, India FullStackTechies Full time

    AIMLEAP is Hiring: **Web Scraping Python Trainer** Experience: 3 to 7 Years Location: Bangalore (Work from Home) No of positions: 2 Industry: Automation/IT-Software Employment Type**:Full Time/Part Time/Consulting** Educational Qualification: B.Tech / B.E / MCA / BCA/MBA Salary/CTC: 4 lacs to 8 lacs INR per annum Notice Period: Immediate - 1 Week JOB...

  • Scrapy Framework Expert

    4 weeks ago

    Bengaluru, Karnataka, India FullStackTechies Full time

    AIMLEAP is Hiring: **Web Scraping Specialist - WFH (2+ Years)** Experience: 2+ Years Location: Work From Home/India **Salary**: 3 to 6 INR LPA No of positions: 2 No Industry: Automation/Data Processing Notice Period**:Immediate - 2 weeks** JOB DESCRIPTION: We are seeking a Web Scraping Specialist(Scrapy/Beautifulsoup)proficient in Python,...

  • Web Scraper

    3 weeks ago

    Bengaluru, India Sustainability Economics Full time

    Position Title:  Web Scraper Location: Bengaluru, Karnataka About the Company: Sustainability Economics is a global organisation targeting the markets of US, Japan, EU, UK, Canada and Singapore. As a pioneer, our focus is on the Lifecycle Management of E2E Net-Zero transitions while leveraging our automation-first approach. We offer our collaborative...

  • Web Scraper

    3 weeks ago

    Bengaluru, India Sustainability Economics Full time

    Position Title: Web Scraper Location: Bengaluru, KarnatakaAbout the Company:Sustainability Economics is a global organisation targeting the markets of US, Japan, EU, UK, Canada and Singapore. As a pioneer, our focus is on the Lifecycle Management of E2E Net-Zero transitions while leveraging our automation-first approach. We offer our collaborative services...