Senior Data Engineer

5 hours ago

Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

Job Title: Senior Data Engineer

Location: Flexible Work Location (FWR)

Experience Level: Senior (4+ years)

Employment Type: Full-time

Job Summary

  • We are seeking a highly skilled and experienced Senior Data Engineer to lead our data scraping projects. The ideal candidate will have a minimum of 4 years of hands-on experience in IT scraping, with at least 2 years leading a team of 5+ developers.

Key Responsibilities:

  • Design and develop scalable data scraping solutions using Scrapy and Python libraries.
  • Lead and mentor a team of 5+ developers, managing project timelines and deliverables.
  • Implement advanced blocking and captcha-solving techniques to bypass scraping restrictions.
  • Conduct source code reverse engineering and automate web and app interactions.
  • Manage proxies, IP rotation, and SSL unpinning to ensure effective scraping.
  • Maintain and improve API integrations and data pipelines.
  • Ensure code quality through effective version control, error handling, and documentation.
  • Collaborate with cross-functional teams for project planning and execution.
  • Monitor performance and provide solutions under high-pressure environments.

Required Skills and Experience:

Data Scraping:

  • Minimum 4 years in IT scraping industry

Leadership:

  • Minimum 2 years leading a team of 5+ developers

Scraping Tools:

  • Scrapy, Threading, requests, web automation

Technical Proficiency:

  • Advanced Python
  • Captcha solving and blocking handling
  • Source reverse engineering
  • Proxy management & IP rotation
  • App automation, SSL Unpin, Frida
  • API Management, Version Control Systems
  • Error Handling, SQL, MongoDB, Pandas

Leadership Skills:

  • Basic project management
  • Moderate documentation
  • Team handling
  • Pressure management
  • Flexibility and adaptability
  • High accountability

Preferred (Good to Have):

  • Experience with Linux
  • Knowledge of Appium, Fiddler, Burp Suite

  • Data Scientist/ML Engineer

    1 week ago

    Vijayawada, Andhra Pradesh, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...

  • Senior Data Engineering Strategist

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data Engineering Strategist A senior data engineering strategist is required to lead and manage a team of experienced data engineers. The ideal candidate should have a strong background in enterprise-level data engineering roles with expertise in OLAP design, data warehousing concepts, and Snowflake. Key Responsibilities: Leverage...

  • Senior Data Engineer

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About the RoleAt our organization, we are seeking a skilled Senior Software Engineer/Technical Specialist to join our team. This role is ideal for an individual with expertise in data engineering, including experience with Azure Data Factory (ADF), Databricks, and Synapse Analytics.As a Senior Software Engineer/Technical Specialist, you will be responsible...

  • Senior Data Engineering Expert

    7 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time US$ 90,000 - US$ 1,20,000

    Job Title: Senior Data EngineerWe are looking for a skilled data engineer to join our team.The ideal candidate will have 5+ years of experience in ETL development and data engineering.In this role, you will be responsible for developing and maintaining data pipelines using FiveTran, DBT Cloud, and custom frameworks.You will work closely with the QC team to...

  • Lead Data Engineer

    10 hours ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time US$ 8,00,000 - US$ 12,00,000

    Senior Data ArchitectWe are seeking a Senior Data Architect to lead the design and implementation of our data engineering solutions.In this role, you will be responsible for designing, building, and maintaining robust data pipelines that integrate multiple sources of data. You will work closely with cross-functional teams to ensure that data from various...

  • Senior Data Engineering Specialist

    16 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data Engineering SpecialistJob DescriptionWe are seeking a skilled and experienced Data Engineer to join our team. As a key member of our organization, you will be responsible for designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in distributed data processing and...

  • Senior Data Engineering Specialist

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    About us:We are an ecommerce focused end-to-end data analytics firm assisting enterprises & brands in data-driven decision making to maximize business value. Our suite of work spans extraction, transformation, visualization & analysis of data delivered via industry-leading products, solutions & services. Our flagship product is a cloud-based ETL tool. We...

  • Senior Cloud-Native Data Engineering Specialist

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Engineer - Cloud Native ExpertAbout the Role:We are seeking a seasoned Senior Data Engineer to lead our cloud-native data engineering initiatives. The ideal candidate will have in-depth experience with Databricks and AWS, as well as expertise in designing and developing scalable data pipelines.Key Responsibilities:Design and develop...

  • Senior Data Developer

    5 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 12,00,000 - ₹ 15,00,000

    Job Title: Senior Data DeveloperAbout the Role:We are seeking a skilled data developer to join our team. As a senior data developer, you will be responsible for developing and maintaining scalable data pipelines to support increasing data volume and complexity.Main Responsibilities:Design and implement efficient data pipelines using various tools and...

  • Big Data Engineering Expert

    7 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job Title: Senior Big Data EngineerAbout the RoleWe are seeking a highly skilled Senior Big Data Engineer to join our team. The ideal candidate will have hands-on experience with AWS, Python, and PySpark.Key ResponsibilitiesDesign, develop, and deploy scalable big data architectures using AWS services such as Amazon EMR, Amazon Athena, and AWS Glue.Work...