Senior Data Engineer

15 hours ago

Varanasi, Uttar Pradesh, India beBeeScraping Full time ₹ 15,00,000 - ₹ 20,00,000

We are seeking an experienced software engineer with expertise in web data extraction and strong skills in automation.

Job Overview

The ideal candidate will design, develop, and optimize large-scale data scraping solutions that power business decision-making.

Key Responsibilities:
  • Design, develop, and maintain scalable data extraction frameworks using Python libraries.
  • Work with additional tools such as BeautifulSoup, Selenium, Playwright, Requests, etc.
  • Implement robust error handling, data parsing, and data storage mechanisms (JSON, CSV, SQL/NoSQL databases).
  • Build and optimize asynchronous data extraction workflows and handle multithreading/multiprocessing.
  • Collaborate with product managers, QA, and DevOps teams to ensure timely delivery.
  • Research and adopt new data technologies to improve performance, scalability, and efficiency.
Requirements:
  • 2+ years of experience in Python development with expertise in data extraction.
  • Proficiency with automation libraries such as Playwright or Selenium.
  • Experience with REST APIs, asynchronous programming, and concurrency.
  • Familiarity with databases (SQL/NoSQL) and cloud-based data pipelines.
  • Strong problem-solving skills and ability to meet deadlines in an Agile environment.
Preferred Qualifications:
  • Knowledge of DevOps tools such as Docker, GitHub Actions, or CI/CD pipelines.

You will have the opportunity to work on exciting projects, collaborate with a talented team, and grow your career in a fast-paced and dynamic environment.

  • Senior IoT Data Engineer

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior IoT Data EngineerWe are seeking an experienced Senior IoT Data Engineer with hands-on coding expertise in Python and PySpark. The successful candidate will have a strong understanding of IoT systems, especially in refrigeration units, HVAC systems, controllers, and sensors.Key Responsibilities: Implement an application for a large retailer...

  • Data Engineering Lead

    3 days ago

    Varanasi, Uttar Pradesh, India beBeedata Full time ₹ 15,91,059 - ₹ 21,47,717

    Job Title: Senior Data EngineerWe are seeking a highly skilled and experienced Informatica developer to lead our data engineering team.

  • Lead Data Solutions Engineer

    14 hours ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineering Full time US$ 1,80,000 - US$ 2,50,000

    Our Senior Data Engineering team is seeking a seasoned professional to design and optimize data solutions. The ideal candidate will be instrumental in designing and maintaining scalable data pipelines using Snowflake, Matillion, and Python.The role requires deep expertise in ETL development, data modeling, and financial data domains, particularly within...

  • Senior Cloud Data Architect

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeAzure Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Key Role: Senior Cloud Data ArchitectWe're seeking an exceptional Senior Cloud Data Architect to lead our data engineering efforts.Data Engineering Expertise:Azure and Databricks development, optimization, and implementationData pipelines, ETL workflows, and Lakehouse architectureCloud-based technologies and services integrationDatabricks Asset Bundles...

  • Senior Data Engineer

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 9,00,000 - ₹ 14,00,000

    Job DescriptionWe are seeking a skilled professional to lead our Data Engineering efforts.The ideal candidate will have a strong background in Big Data technologies and experience working with AWS services.This is an exciting opportunity for a talented individual to join our team and contribute to the development of innovative data solutions.Required Skills...

  • Senior Data Engineer Position

    24 minutes ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Big Data Engineer Role SummaryThe role requires a highly skilled Big Data Engineer with expertise in Scala, Spark, Python, NiFi, and Apache Kafka.Key responsibilities include designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.Additionally, the ideal candidate will have hands-on experience in streaming...

  • Senior Cloud Data Migration Specialist

    28 minutes ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We're looking for a highly skilled and experienced Senior Data Engineer to join our team.About the Role:As a Senior Data Engineer, you will be responsible for designing, developing, and implementing complex data pipelines for migrating data from various on-premise sources to cloud platforms. You will serve as a subject matter expert in project core...

  • Senior Big Data Platform Engineer

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    About This RoleWe seek a seasoned Engineering Manager to lead our data engineering team in designing and delivering scalable data pipelines and analytics platforms.This is an exceptional opportunity for a strategic leader with deep expertise in Big Data, Azure, and Databricks to make a lasting impact on our organization's data infrastructure.Lead and Grow a...

  • Senior Data Pipeline Specialist

    17 hours ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 27,00,000

    Meet the requirements for a senior data engineer position to create and maintain ETL/ELT pipelines in our organization.Key ResponsibilitiesPipeline Development: Design, build, and manage efficient ETL/ELT pipelines using AWS services like Lambda, DMS, and Glue.Data Transformation: Implement complex business logic and data transformations utilizing Snowflake...

  • Senior Backend Engineer

    5 days ago

    Varanasi, Uttar Pradesh, India Rayole Software Full time

    We're Hiring: Senior Backend Engineer Location: RemoteEmployment Type: Full-TimeExperience Level: SeniorAre you an experienced Backend Engineer with a passion for building scalable, high-performance applications? We're looking for a highly skilled Senior Backend Engineer with extensive expertise in Python, Django, Next.js, Node.js, React, and web...