Senior Large Scale Data Specialist

7 days ago

Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 16,00,000 - ₹ 20,10,000

Job Title: Data Engineer

We are seeking a talented Data Architect to build and maintain large-scale data systems.

This role involves designing scalable pipelines, working with real-time data streams, and collaborating with cross-functional teams to deliver high-quality solutions.

Key Responsibilities:

  • Design, build, and optimize complex data pipelines and ETL workflows.
  • Manage and process massive datasets using Spark, PySpark, and SQL.
  • Build and maintain efficient real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure data quality, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements:

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

  • Senior Project Manager

    4 hours ago

    Kanpur, Uttar Pradesh, India beBeeImplementation Full time ₹ 18,00,000 - ₹ 20,00,000

    Project Implementation SpecialistThis role involves coordinating with cross-functional teams to plan and execute large-scale implementations. The successful candidate will ensure end-to-end planning, coordinated communication, and effective project management across multiple teams.Main Responsibilities:Plan and coordinate project timelines, release planning,...

  • Senior Data Operations Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDatabaseProfessional Full time ₹ 15,00,000 - ₹ 25,00,000

    Key Roles and ResponsibilitiesWe are seeking a senior-level data professional to support large-scale data operations across ~200 servers. The role combines data analysis, architecture, and database administration responsibilities to improve data accessibility, performance, and retention.Main Duties:Review and interpret data across multiple server...

  • Principal Big Data Specialist

    13 hours ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    As a senior data engineer, you will be responsible for designing and implementing large-scale data processing systems to support advanced analytics and business intelligence.The ideal candidate will have experience with Databricks, strong programming skills in languages such as Python, Java, or Scala, and expertise in big data technologies like Hadoop,...

  • Data Architectural Specialist

    5 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    **Data Architectural Specialist**This role involves designing, developing and maintaining large-scale data systems using SQL, Azure Data Factory, Data Bricks, MS Synapse, Python, or other relevant tools.The primary responsibilities include:Data IntegrationData ModellingTo be successful in this position, you will need to have:A degree in Computer Science,...

  • Data Engineering Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistAbout the RoleWe are seeking a highly skilled Data Engineering Specialist to join our team. As a key member of our data infrastructure team, you will design, develop, and maintain large-scale data pipelines using Python and AWS services.Key ResponsibilitiesData Pipeline Development: Develop and implement data pipelines...

  • Senior Data Engineering Lead

    6 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe seek an experienced data engineer to lead the design and development of large-scale data processing systems that enable advanced analytics and business intelligence.As a senior data engineer, you will be responsible for architecting and implementing scalable and robust data pipelines that collect, process, and store large volumes of...

  • Senior Data Processing Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Big Data Engineer RoleWe are looking for a skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing and developing large-scale data processing systems.The ideal candidate will have experience with PySpark, Hive, Spark, HBase, DQ tools, and Agile Scrum methodologies. They should also have exposure to data...

  • Data Engineering Specialist

    10 hours ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time US$ 9,00,000 - US$ 12,00,000

    Job Title: Data Engineering SpecialistAbout the Role:We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will have a strong background in distributed data processing and storage, as well as experience with cloud-based technologies.Key Responsibilities:- Design, develop, and maintain large-scale data engineering systems-...

  • Data Engineering Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Engineering SpecialistWe are seeking a highly skilled data engineering specialist to join our team. This role involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.Key Responsibilities:Design and develop efficient data processing systems using Scala, Spark, and Python.Develop...

  • Senior Big Data Specialist

    5 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking an experienced Data Engineer to join our team.Job DescriptionA Data Engineer with 5+ years of experience is required to design, develop, and maintain large-scale data systems. The ideal candidate will have strong hands-on experience with Trino (PrestoSQL) in production environments.Key ResponsibilitiesDesign, develop, and deploy scalable data...