Big Data Pipeline Specialist

1 day ago


Kolkata, West Bengal, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Senior Data Engineer for Real-Time & Big Query

We're seeking an experienced Senior Data Engineer to join our team in India. This is a remote opportunity for a specialist with 10+ years of experience to work on challenging projects for industries like Energy, Oil & Gas, Healthcare, and Finance.

This role will be at the forefront of our data solutions, responsible for designing, developing, and maintaining scalable data pipelines for both batch and real-time processing using GCP tools.

The ideal candidate will have deep expertise in GCP services including Big Query, Cloud Dataflow (Apache Beam), Cloud Run, Cloud Functions, Pub/Sub, and Cloud Composer (Apache Airflow).

Key Responsibilities:
  • Designing and developing large-scale data pipelines for batch and real-time processing.
  • Constructing complex datasets to meet diverse business needs.
  • Translating business requirements into technical specifications.
  • Leveraging various GCP services to build integrated systems.
  • Orchestrating workflows using Cloud Composer (Apache Airflow).
Required Skills:
  • Proficiency in Python and SQL.
  • Experience with structured ERP solutions like SAP, Oracle EBS, and Salesforce.
  • A strong understanding of different file formats (CSV, JSON, XML, Parquet, ORC, Avro).
  • Knowledge of Data Lake concepts.
  • Experience with API development and consumption (REST, SOAP).
Benefits:
  • Opportunity to work on challenging projects with a talented team.
  • Remote working arrangement.
  • Flexible shift: 3:00 PM to 12:00 AM IST.
  • Immediate to 15 days notice period.


  • Kolkata, West Bengal, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OpportunitySr. Data Architect - Big Data SpecialistWe are seeking a highly skilled and experienced Senior Data Architect to join our team as a Big Data Specialist. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing large-scale data processing systems using Big Data technologies.You will lead...

  • Big Data Specialist

    9 hours ago


    Kolkata, West Bengal, India beBeeDataiku Full time ₹ 1,80,00,000 - ₹ 2,30,00,000

    Job DescriptionWe are seeking a seasoned Big Data Platform specialist to join our team.This individual will be responsible for managing Big Data Platform components including Hadoop, Hive, HBase, and BigSQL. They will also support technical and application teams with data copies across environments, data cleanup, query tuning, etc.The ideal candidate will...

  • Big Data Engineer Lead

    10 hours ago


    Kolkata, West Bengal, India beBeeBigDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a visionary Data Architecture Specialist to spearhead the development of our Enterprise TruRisk Platform.As a key member of our engineering team, you'll be responsible for designing, developing, and supporting a highly scalable, distributed SaaS-based Security Risk Prioritization product.This cutting-edge solution leverages big...

  • Big Data Innovator

    3 days ago


    Kolkata, West Bengal, India beBeeGcpEngineer Full time ₹ 18,00,000 - ₹ 30,00,000

    Data Engineer Role">This is an exciting opportunity to work as a Data Engineer in our organization. We are looking for someone who can design, develop, and optimize data pipelines using GCP BigQuery, Dataflow, and Apache Airflow to support large-scale data analytics. As a Data Engineer, you will be responsible for utilizing the Big Data Hadoop ecosystem to...


  • Kolkata, West Bengal, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,50,00,000

    Cloud Data EngineerAzure Databricks engineers design, implement, and manage scalable big data processing pipelines on the cloud.Key Responsibilities:Develop and maintain large-scale ETL processes using Azure Databricks.Collaborate with cross-functional teams to integrate data pipelines with business applications.Ensure data quality, security, and compliance...

  • Big Data Specialist

    1 week ago


    Kolkata, West Bengal, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Data Scientist PositionWe are seeking a highly skilled data scientist to join our team and contribute to the development of cutting-edge solutions.Key Responsibilities:Develop and implement advanced data processing and analytics capabilities using PySpark and Hive.Design and implement scalable data ingestion pipelines from disparate sources onto Big Data...


  • Kolkata, West Bengal, India beBeeData Full time ₹ 0 - ₹ 1

    Big Data Specialist\We are seeking a skilled Big Data Specialist to drive data innovation and growth across our global operations.\The ideal candidate will have at least 5-8 years of experience in big data engineering with strong expertise in:\Key Technologies:\Palantir Foundry / WorkshopPySparkTypescript\About the Role:\You will leverage your expertise in...

  • Big Data Specialist

    4 days ago


    Kolkata, West Bengal, India beBeeDataEngineer Full time US$ 8,00,000 - US$ 12,00,000

    Job Title: Big Data Specialist About the Role:We are seeking a highly skilled professional to join our team as a Data Engineer. As a Data Engineer, you will be responsible for designing and developing scalable data architectures, ensuring high-performance data processing, and optimizing data quality. Key Responsibilities:Design and develop scalable data...


  • Kolkata, West Bengal, India Peoplemint Consultant LLP Full time

    Job Summary :We are seeking an experienced Big Data Platform Engineer to join our banking technology team. The ideal candidate will be responsible for designing, building, and maintaining scalable big data platforms to support advanced analytics, reporting, and regulatory compliance. This role demands strong technical acumen in distributed systems, data...


  • Kolkata, West Bengal, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a highly skilled professional to lead the design and optimization of our distributed data pipelines, infrastructure, and tools that power insights across our business.You will collaborate closely with software developers, architects, analysts, and data scientists to ensure smooth, secure, and efficient data delivery across multiple...