Web Crawling Engineering Team Lead

2 days ago

Jodhpur, India beBeeDataEngineering Full time

As a Data Engineering Manager – Web Crawling, you will lead and mentor data engineering and web crawling teams to ensure high-quality delivery and adherence to best practices.The role involves leading and mentoring data engineering and web crawling teams to deliver high-quality solutions. Key responsibilities include:Architecting and implementing scalable data pipelines that support high-volume data ingestion, transformation, and storage.Building and maintaining robust crawling systems using modern frameworks, handling IP rotation, throttling, and dynamic content extraction.Establishing pipeline orchestration using Airflow, Celery, or similar distributed processing technologies.In this role, you will also be responsible for collaborating with product, engineering, and analytics teams to translate data requirements into scalable technical solutions. This includes:Developing monitoring, logging, and performance metrics to ensure high availability and reliability of data systems.Overseeing cloud-based deployments, cost optimization, and infrastructure improvements on AWS/GCP/Azure.Integrating AI agents or LLM-based automation for tasks such as error resolution, data validation, enrichment, and adaptive crawling.

  • Expert Web Data Extractor

    2 days ago

    Jodhpur, India beBeeWebCrawling Full time

    Web Crawling SpecialistThis role is focused on developing and maintaining efficient web crawling solutions. As a Web Crawling Specialist, you will be responsible for designing and implementing crawlers that can extract valuable insights from the web.The ideal candidate has strong Python skills, experience with web scraping frameworks and browser automation...

  • Web Development Intern

    2 weeks ago

    Jodhpur, Rajasthan, India WEB FARM HOUSE Full time ₹ 2,00,000 - ₹ 4,00,000 per year

    Company DescriptionWeb Farm House is a full-service IT company that develops and improves professional websites for clients across various industries, including advertising, communications, fashion, finance, e-commerce, law, medical and health, construction, insurance, and real estate. The company's key strength lies in its ability to provide custom-made web...

  • QA Team Lead Position

    2 days ago

    Jodhpur, India beBeeLeadership Full time

    Quality Assurance Leadership OpportunityA key position within our development team, responsible for managing and improving the quality of online commerce services. This role focuses on a high-traffic platform offering a chance to enhance your QA leadership skills in a global environment.This opportunity is part of a global team with both onsite and offshore...

  • Chief Web Engineer

    3 days ago

    Jodhpur, India beBeeFrontend Full time

    Web Development OpportunityPocket FM is an audio entertainment platform that powers storytelling through audiobooks, podcasts across multiple languages.We are seeking a highly skilled Frontend Developer to design and build high-performance web applications that reach millions of users daily.Collaborate with product managers, designers, and backend engineers...

  • Web Developer and AI Engineer

    4 weeks ago

    Jodhpur, India CopperScreen Full time

    Web Developer & AI Automation Specialist | Remote (India)Location: Fully Remote (India)Experience: 3 to 5 years of experienceMust have Skills: Wordpress, Webflow, PythonAbout CopperScreenCopperScreen is a boutique B2B digital agency based in Sydney, Australia, specialising in performance marketing for B2B SaaS companies. We work with innovative startups and...

  • Data Engineering Lead

    15 hours ago

    Jodhpur, India beBeeTechnicalArchitect Full time

    Job Summary:We are seeking a skilled technical professional to join our team as a Technical Architect with hands-on coding experience. The successful candidate will have expertise in data engineering, including modelling and managing multi-dimensional entities and graphs, as well as knowledge of graph databases and data lake technologies.The ideal candidate...

  • AI Engineering Team Lead

    11 hours ago

    Jodhpur, India beBeeEngineering Full time

    Unlock the full potential of AI by joining our team as an experienced AI Engineering Lead.Analyze complex business problems and design innovative AI solutions, encompassing architecture, approach selection, and technical feasibility assessmentDevelop and deploy scalable production-ready AI applications using cutting-edge frameworks and libraries while...

  • Lead Quality Assurance Engineer

    20 hours ago

    Jodhpur, India Backgammon Galaxy Full time

    Lead QA Engineer (Remote) Company: Backgammon Galaxy (backgammongalaxy.com) Location: Remote (HQ in Cyprus) | Full-time Backgammon Galaxy is the leading web and mobile platform for playing backgammon, with a community of 150,000+ players worldwide. We’ve been growing for four years and are now looking for a Lead QA Engineer to help us scale further. Our...

  • Senior Technical Team Lead

    3 days ago

    Jodhpur, India beBeeTechnical Full time

    Job Title: Engineering ManagerThe ideal candidate will inspire their team to achieve performance metrics by strategizing, project management, and staff management.Engineering Management & Leadership:Directly lead multiple software engineering teams, fostering a collaborative culture and continuous learning.Define high engineering standards, best practices,...

  • Senior Web Application Developer

    2 days ago

    Jodhpur, India beBeeFrontend Full time

    Lead Frontend DeveloperWe are seeking an experienced Lead Frontend Developer to spearhead the development of front-end and back-end components using React/Next.js and Node.js.The ideal candidate will have a proven track record of delivering high-quality software solutions, collaborating with cross-functional teams, and driving technical innovation.Bachelor's...