Lead Big Data Innovator

3 days ago

Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 15,00,000
Job Overview

This is an exciting opportunity to work as a Data Engineer, driving data-driven initiatives forward. The role involves building scalable pipelines, working with streaming data, and collaborating with cross-functional teams.

Key Responsibilities:
  • Data Pipeline Development: Design, build, and optimize data pipelines and ETL workflows to ensure efficient data processing.
  • Big Data Management: Manage and process large datasets using Spark, PySpark, and SQL to drive business insights.
  • Streaming Applications: Build and maintain real-time streaming applications with Spark Streaming/Kafka for immediate business impact.
  • AI/ML Integration: Collaborate with data scientists and product teams to integrate AI/ML models into production, enhancing our data-driven decision-making capabilities.
  • Pipeline Optimization: Ensure data quality, scalability, and performance in all pipelines to meet evolving business needs.
  • Cloud Deployment: Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks) for seamless scalability.
Requirements:
  • Education: Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Experience: Minimum 4 years of professional experience as a Data Engineer, with strong expertise in Python, Spark, PySpark.
  • Technical Skills: Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution, knowledge of data ingestion tools (Kafka, Flume, Kinesis), data formats (Parquet, Avro, ORC), proficiency in Airflow/Prefect for scheduling and orchestration, familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Deployment Experience: Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Testing and CI/CD: Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).
About the Role

This role requires strong analytical, problem-solving, and communication skills to drive success in this Data Engineer position.

  • Senior Big Data Architect

    4 hours ago

    Bareilly, Uttar Pradesh, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Big Data Solutions ArchitectWe are seeking a highly skilled big data solutions architect with expertise in designing and implementing scalable, high-performance data pipelines.The ideal candidate will have strong knowledge of AWS cloud services, Databricks, and big data frameworks.Main Responsibilities:Design, develop, and maintain complex data pipelines...

  • Big Data Expert

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Description:We are seeking a skilled Data Engineer to join our team. The ideal candidate will have experience working with Azure Databricks and PySpark.The successful candidate will be responsible for designing, implementing, and maintaining large-scale data processing systems using Azure Databricks and PySpark.Key Responsibilities:Design and develop...

  • Big Data Specialist

    15 hours ago

    Bareilly, Uttar Pradesh, India beBeeEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Big Data Specialist We are seeking an experienced data engineer to join our team. This individual will be responsible for designing, building, and maintaining large-scale data systems. The ideal candidate will have hands-on experience with distributed computing frameworks and a proven track record of delivering high-quality solutions. Key...

  • Highly Skilled Big Data Engineer Wanted

    23 hours ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Job Overview\Helpers are looking for a highly skilled engineer to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.\

  • Principal Data Engineer

    3 weeks ago

    Bareilly, Uttar Pradesh, India Optum Full time

    As a Principal Software Engineer for Data, the person will lead the design and implementation of scalable, secure, and high-performance data pipelines across that involves healthcare clinical data, using modern big data and cloud technologies (Azure, Databricks, and Spark), ensuring alignment with UnitedHealth Group's data governance standards. This role...

  • Senior Data Engineering Manager

    20 hours ago

    Bareilly, Uttar Pradesh, India beBeeTechnicalLead Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Platform LeadSigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting.We collaborate with leading data-rich organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges.The role of a Big Data Platform Lead...

  • Data Innovation Specialist

    3 days ago

    Bareilly, Uttar Pradesh, India beBeeEngineer Full time ₹ 5,000 - ₹ 15,000

    As a Full Stack Engineer, you will play a pivotal role in shaping the future of Edtech with innovative AI-powered systems.The mission is to empower educators and learners alike by creating intelligent, autonomous agents that collaborate seamlessly across industries.We operate at the intersection of creativity, data, and relentless innovation.Key...

  • Advanced Data Scientist

    2 days ago

    Bareilly, Uttar Pradesh, India beBeeMachine Full time ₹ 25,00,000 - ₹ 35,00,000

    Job Opportunity: Advanced Data ScientistAbout the Position:We are seeking an experienced professional in machine learning engineering to lead our data science team. The ideal candidate will have expertise in designing and developing complex machine learning models, integrating them into big data processing pipelines, and evaluating their performance.Key...

  • Principal Data Architect

    23 hours ago

    Bareilly, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a highly skilled professional to lead our data engineering efforts and drive business growth through innovative solutions.As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines that support the increasing demands of our organization. You will work closely with...

  • Chief Data Innovation Officer

    2 days ago

    Bareilly, Uttar Pradesh, India beBeeDataScience Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Science LeaderWe are seeking a highly experienced and visionary leader to shape our data science initiatives. This role will be instrumental in defining our data strategy, driving innovation in AI/ML, and building scalable data solutions that directly impact our business.