Junior Web Crawling Engineer

3 weeks ago

Bhavnagar, Gujarat, India Forage AI Full time

We are seeking a Junior Web Crawling Engineer who will be responsible for building and maintaining web crawlers , extracting valuable insights from the web, and ensuring data quality. The ideal candidate will have strong Python programming skills and experience in web scraping frameworks, browser automation tools, and handling anti-scraping mechanisms.

Key Responsibilities:

  • Maintain and enhance existing web scraping and data crawling projects.
  • Develop and refine crawlers using Python-based tools and frameworks.
  • Utilize browser automation tools (e.g., Playwright, Selenium) to handle dynamic content.
  • Clean, validate, and integrate extracted data into downstream storage systems.
  • Implement and manage solutions for anti-bot measures (CAPTCHAs, IP rotation, etc.).
  • Optimize crawling efficiency and ensure compliance with web crawling best practices.
  • Collaborate with cross-functional teams to improve data acquisition strategies.

Required Skills & Qualifications :

  • Proficiency in Python and 2 years of work experience of web scraping frameworks (especially Scrapy).
  • Strong knowledge of browser automation tools such as Playwright or Selenium.
  • Solid understanding of HTML, CSS, and selector languages (XPath/CSS).
  • Experience in handling anti-scraping challenges and ensuring robust data extraction.
  • Familiarity with distributed scraping techniques and data pipelines.
  • Ability to troubleshoot and optimize web crawlers for performance and reliability.
  • Strong analytical and problem-solving skills with attention to detail.
  • Excellent communication and inter-personal skills.

Other Infrastructure Requirements

Since this is a completely work-from-home position, you will also require the following -

● High-speed internet connectivity for video calls and efficient work.

● Capable business-grade computer (e.g., modern processor, 8 GB+ of RAM, and

no other obstacles to interrupted, efficient work).

● Headphones with clear audio quality.

  • AI-Powered Junior SEO Specialist

    6 days ago

    Bhavnagar, Gujarat, India beBeeAi Full time ₹ 9,00,000 - ₹ 12,00,000

    Junior SEO Specialist – AI Job DescriptionAs a Junior SEO Specialist, you will be responsible for optimizing web pages to improve their visibility and ranking performance. You will work with our team to develop and implement effective SEO strategies that drive business results.Develop and execute comprehensive SEO plans to improve website health and search...

  • Junior Automation Engineer

    4 days ago

    Bhavnagar, Gujarat, India beBeeAutomation Full time ₹ 10,00,000 - ₹ 15,00,000

    Join a cutting-edge developer platform and take the first step in your career as an automation engineer.About the RoleThis entry-level position offers hands-on experience with Appium Pro certification, building web UIs, and testing mobile applications. You will be working with both Android and iOS devices, using your technical skills to dig into device...

  • Junior Exim Executive

    7 days ago

    Bhavnagar, Gujarat, India SKAPS Industries Full time

    At SKAPS as a Junior Exim Executive, you will be directly communicating with India staff at different locations & Exim-related works, which is beneficial for one's personal as well as professional growth.Job duties and responsibilities include;Ø Can communicate internally with staff through email or on the phone.Ø Import, Export & domestic related worksØ...

  • Web Interface Specialist

    4 days ago

    Bhavnagar, Gujarat, India beBeeFrontend Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Front-End Web DeveloperWe are seeking a seasoned professional to lead our front-end development team.Design and maintain front-end architecture using HTML, CSS, and BootstrapMentor junior developers and conduct code reviewsTranslate high-fidelity UI/UX designs into functional web componentsEvaluate and integrate new tools and librariesKey...

  • Innovative Web Solutions Specialist

    1 week ago

    Bhavnagar, Gujarat, India beBeeWebDeveloper Full time ₹ 8,00,000 - ₹ 15,00,000

    Junior Web Developer OpportunityWe are seeking an experienced Junior Web Developer to join our dynamic team. This is a fantastic opportunity for someone looking to grow their skills in web development and contribute to diverse projects.About the Role:The ideal candidate will have around 2 years of experience working with WordPress, WooCommerce, and...

  • Junior Data Engineering Expert

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Role OverviewAt our organization, we are looking for a highly skilled and self-driven Junior Data Scientist to join our team.The ideal candidate will have 3-4 years of experience in designing, implementing, and maintaining robust data science workflows that enable scalable, repeatable, and automated model training and deployment.The successful candidate will...

  • Senior Web Developer

    20 hours ago

    Bhavnagar, Gujarat, India beBeeSearchEngineOptimization Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Web Developer Search Engine OptimizationThe role of a Web Developer SEO at our organization is to design, develop, and maintain websites that are optimized for search engines. This involves utilizing various tools and techniques to improve the website's visibility, ranking, and overall user experience.Key Responsibilities:Developing and...

  • Technical Lead

    4 days ago

    Bhavnagar, Gujarat, India beBeeSenior Full time ₹ 12,00,000 - ₹ 15,00,000

    Job DescriptionAs a senior full-stack developer, you will be responsible for designing, developing, and maintaining web applications using PHP, WordPress, and Laravel.Key responsibilities include:Ensuring good architectural design and directionDesigning, developing, coding, testing, and debugging featuresPerforming integration and testing before going...

  • Highly Skilled Web Automation Test Engineer Wanted

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeAutomation Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Web Automation Test Engineer – Selenium with PythonLocation: Hyderabad, IndiaExperience: Minimum 5 YearsJob Summary:We are seeking an experienced Web Automation Test Engineer to join our team. The ideal candidate should have a solid understanding of web application testing, test automation strategies, and agile methodologies.You will be...

  • High Performance Web Applications Engineer

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a talented Full Stack Developer to join our team. This role involves designing, building and maintaining end-to-end web applications using modern JavaScript frameworks such as Angular, React or Vue.js, with expertise in Node.js, PHP or Python.The ideal candidate should have at least 5 years of experience in frontend and backend technologies,...