Senior Data Engineer Lead

6 days ago

Thāne, Maharashtra, India beBeeData Full time ₹ 19,45,600 - ₹ 24,14,400

Job Opportunity: Senior Data Engineer Lead

About the Role

We are seeking an experienced Senior Data Engineer Lead to join our team. As a key member of our engineering team, you will be responsible for designing and developing scalable data scraping solutions, leading and mentoring a team of developers, and implementing advanced blocking and captcha-solving techniques.

Key Responsibilities
  • Design and develop scalable data scraping solutions using Scrapy, Threading, and requests.
  • Lead and mentor a team of developers, managing project timelines and deliverables.
  • Implement advanced blocking and captcha-solving techniques to bypass scraping restrictions.
  • Conduct source code reverse engineering and automate web and app interactions.
  • Maintain and improve API integrations and data pipelines.
Requirements
  • Data Scraping: Minimum 4 years in IT scraping industry with experience in working with large datasets.
  • Leadership: Minimum 2 years leading a team of 5+ developers with strong leadership skills.
  • Scraping Tools: Proficiency in Scrapy, Threading, requests, and web automation tools.
  • Technical Proficiency:
    • Advanced Python programming skills.
    • Captcha solving and blocking handling expertise.
    • Source reverse engineering skills.
    • Proxy management and IP rotation expertise.
    • App automation and SSL Unpin skills.
    • API Management and Version Control Systems skills.
    • Error Handling, SQL, MongoDB, and Pandas skills.
  • Leadership Skills:
    • Basic project management skills.
    • Moderate documentation skills.
    • Team handling skills.
    • Pressure management skills.
    • Flexibility and adaptability skills.
    • High accountability skills.
    Preferred Qualifications
    • Experience with Linux operating systems.
    • Knowledge of Appium, Fiddler, and Burp Suite tools.

  • Azure Data Engineering Test Lead

    4 days ago

    Thāne, Maharashtra, India beBeeTestLeader Full time ₹ 15,00,000 - ₹ 25,00,000

    Role Overview">This is a full-time remote opportunity for a Senior Test Lead/Manager in Azure Data Engineering.">The Senior Test Lead/Manager will oversee the entire testing lifecycle, developing test strategies, coordinating with various teams, managing test plans, and ensuring the quality of Azure Data solutions.">Responsibilities">Experience in Azure Data...

  • Distinguished Technical Lead in Data Engineering

    2 days ago

    Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Description">We are seeking a skilled Senior Data Engineer to lead the design, development, and maintenance of data pipelines and data warehouse solutions on Google Cloud Platform (GCP).This role plays a critical part in ensuring seamless data flow across systems, enabling data-driven decision-making.The successful candidate will have expertise in GCP,...

  • Data Engineering Lead

    4 days ago

    Thāne, Maharashtra, India DMart - Avenue Supermarts Ltd Full time

    About the organization:DMart is one of India's leading retail chains, serving millions of customers across 425+ stores and e-commerce channels throughout India. Our core objective is to offer customers good products at great value. We focus on everyday low pricing, seamless shopping experiences, and data-driven decision making. As we scale our data analytics...

  • Data Engineering Lead

    4 days ago

    Thāne, Maharashtra, India DMart - Avenue Supermarts Ltd Full time

    About the organization: DMart is one of India's leading retail chains, serving millions of customers across 425+ stores and e-commerce channels throughout India. Our core objective is to offer customers good products at great value. We focus on everyday low pricing, seamless shopping experiences, and data-driven decision making. As we scale our data...

  • Data Engineering Lead

    1 day ago

    Thāne, Maharashtra, India DMart - Avenue Supermarts Ltd Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About the organization:DMart is one of India's leading retail chains, serving millions of customers across 425+ stores and e-commerce channels throughout India. Our core objective is to offer customers good products at great value. We focus on everyday low pricing, seamless shopping experiences, and data-driven decision making. As we scale our data analytics...

  • Senior Data Engineer Position

    5 days ago

    Thāne, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data EngineerJob OverviewWe are seeking a highly skilled and motivated Senior Data Engineer to lead the design, development, and maintenance of our modern data platform.This role is ideal for a seasoned professional with deep technical expertise in ETL pipeline design, data modeling, and data infrastructure who excels in a fast-paced,...

  • Data Engineer Lead

    6 days ago

    Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job DescriptionWe are looking for a highly skilled professional to lead the design, construction, and optimization of large-scale data processing systems.This role involves collaborating closely with data scientists, analysts, and other engineers to ensure the availability, reliability, and efficiency of our Databricks solution.The ideal candidate will have...

  • Senior Data Analyst Lead

    4 days ago

    Thāne, Maharashtra, India beBeeAnalytics Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Analyst LeadAt the core of our organization, we strive to drive operational efficiency, customer experience, and business outcomes through strategic oversight of data-driven initiatives. We are seeking a seasoned professional with expertise in leading cross-functional teams and managing complex projects.Responsibilities:

  • Senior Cloud Data Engineering Specialist

    4 days ago

    Thāne, Maharashtra, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    Job DescriptionWe are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation.This role will play a pivotal part in...

  • Senior Engineer, Data Engineering

    5 days ago

    Thāne, Maharashtra, India Envu Full time

    Data Engineer TYPE: Permanent We are seeking a Data Engineer with strong analytical and technical expertise to support our data-driven transformation. You will be responsible for building and maintaining data infrastructure and pipelines, enabling robust analytics and machine learning initiatives across multiple departments This role requires hands-on...