Senior Data Engineering Lead

12 hours ago


Vadodara, Gujarat, India beBeeDataScraper Full time ₹ 15,00,000 - ₹ 20,00,000
Job Title

We are seeking an experienced Data Scraping Engineer Leader.

This role requires deep technical knowledge in advanced scraping techniques, reverse engineering, automation, and leadership skills to drive a team towards success.

  • Design and develop scalable data scraping solutions using tools like Scrapy and Python libraries.
  • Lead and mentor a team of developers, managing project timelines and deliverables.
  • Implement blocking and captcha-solving techniques to bypass scraping restrictions.
  • Conduct source code reverse engineering and automate web and app interactions.
  • Manage proxies, IP rotation, and SSL unpinning to ensure effective scraping.
  • Maintain and improve API integrations and data pipelines.
  • Ensure code quality through version control, error handling, and documentation.
  • Collaborate with cross-functional teams for project planning and execution.
Requirements
  • Data Scraping: 4+ years experience in the IT industry.
  • Leadership: Minimum 2 years leading a team of developers.
  • Scraping Tools: Scrapy, Threading, requests, web automation.
  • Technical Proficiency:
  1. Advanced Python programming.
  2. Captcha solving and blocking handling.
  3. Source code reverse engineering.
  4. Proxy management & IP rotation.
  5. App automation, SSL Unpin, Frida.
  6. API Management, Version Control Systems.
  7. Error Handling, SQL, MongoDB, Pandas.
Leadership Skills:
  • Basic project management.
  • Moderate documentation.
  • Team handling.
  • Pressure management.
  • Flexibility and adaptability.
  • High accountability.

  • Senior Data Engineer

    2 weeks ago


    Vadodara, Gujarat, India Betterbeans Consultants Pvt Ltd Full time

    Role : Senior Data EngineerExperience : 8-10 years' experienceSkill Set Required : Azure Data FactoryAzure DatabricksPythonSQL, PL/SQLSSISJob Role/Job Description : Senior Data Engineer will be responsible for designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server (SSIS, SSAS, UC4 Atomic)Collaborate with...


  • Vadodara, Gujarat, India beBeebigdataengineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Cloud Data Engineering Leadership OpportunityWe are seeking an experienced Engineering Manager to lead our team of data engineers in designing and delivering scalable data pipelines and analytics platforms. With a strong background in Big Data ecosystems and Microsoft Azure, you will drive business growth through innovative solutions.


  • Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000

    Job Title:">Senior Data Engineer">This is a highly sought-after role for professionals with a strong background in big data engineering and analytics. As a senior data engineer, you will be responsible for designing, developing, and implementing large-scale data processing systems that drive business insights.">Required Skills and Qualifications:">5+ years...


  • Vadodara, Gujarat, India beBeeData Full time ₹ 30,00,000 - ₹ 40,00,000

    Data Scientist PositionAs a senior data scientist, you will lead end-to-end initiatives that leverage data, analytics, and artificial intelligence technologies to extract insights, build predictive models, and develop AI-powered solutions.Develop business understanding, prepare data, create models, evaluate them, and deploy projects.Analyze findings, draw...

  • Lead Data Analyst

    1 day ago


    Vadodara, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    About the RoleWe are seeking a skilled professional to fill a Data Science position. The ideal candidate will have expertise in data analytics, with the ability to write code to automate hypothesis generation and insights from statistical data.Key Responsibilities:Write code to extract valuable insights from large datasetsAutomate common data science tasks...


  • Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job DescriptionAs a senior data engineer, you will be responsible for designing and developing scalable data pipelines that meet the needs of our clients.Responsibilities:Design, develop, and maintain ETL (Extract, Transform, Load) pipelines using FiveTran, DBT Cloud, and custom frameworks to efficiently move and transform data from various sources into our...


  • Vadodara, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    We are seeking an experienced Lead Data Engineer to join our team.This is a senior technical role with high expectations. The ideal candidate must be extremely hands-on, technically strong in Azure and Databricks, and capable of delivering high-quality solutions with speed and precision.Key ResponsibilitiesDesign, develop, and optimize data pipelines using...

  • Data Engineer

    6 days ago


    Vadodara, Gujarat, India Allata Full time US$ 1,50,000 - US$ 2,00,000 per year

    Allata is a global consulting and technology services firm with offices in the US, India, and Argentina. We help organizations accelerate growth, drive innovation, and solve complex challenges by combining strategy, design, and advanced technology. Our expertise covers defining business vision, optimizing processes, and creating engaging digital experiences....

  • Senior Data Engineer

    3 weeks ago


    Vadodara, Gujarat, India myGwork Full time

    This job is with Charles River, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.At Charles River, we are passionate about improving the quality of people's lives. When you join our global family, you will help create healthier lives for millions of...


  • Vadodara, Gujarat, India beBeeDataIntegration Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Integration SpecialistWe are seeking a skilled Senior Data Integration Specialist to join our team. This is an exciting opportunity for someone who is passionate about data engineering and integration.About the RoleThis position involves designing, building, and maintaining scalable data pipelines using Azure Data Factory, Azure Synapse...