Data Pipeline Specialist

1 day ago

Vadodara, Gujarat, India beBeeEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

**Data Engineering Role**

As a skilled Data Engineer, you will design and build robust data pipelines to process high-volume financial data.

**Key Responsibilities:**

  • Design and develop end-to-end data pipelines for stocks, crypto, and other financial datasets.
  • Integrate third-party APIs and data feeds into internal systems.
  • Develop and optimize data ingestion, storage, and transformation workflows (batch and real-time).
  • Ensure data quality, consistency, and reliability across all pipelines.
  • Collaborate with data scientists, analysts, and engineers to provide clean, structured, and scalable datasets.
  • Monitor, troubleshoot, and optimize pipeline performance.
  • Implement ETL/ELT best practices, data governance, and security protocols.
  • Contribute to the scalability and automation of our data infrastructure.

**Requirements:**

  • Proven experience as a Data Engineer (preferably in financial or crypto domains).
  • Strong expertise in Python, SQL, and distributed data systems.
  • Hands-on experience with data pipeline tools (e.g., Apache Spark, Kafka, Airflow, Flink, Prefect).
  • Experience with cloud platforms (AWS, GCP, or Azure) and data warehousing (Snowflake, BigQuery, Redshift, etc.).
  • Knowledge of API integrations and handling real-time streaming data.
  • Familiarity with databases (relational and NoSQL) and data modeling.
  • Solid understanding of stocks, cryptocurrencies, and financial data structures (preferred).
  • Strong problem-solving skills with the ability to handle large-scale data challenges.

**About the Role**

This is an exciting opportunity to work on cutting-edge data engineering projects. You will have the chance to design and implement innovative solutions to complex problems.

**What We Offer**

We offer a dynamic and collaborative work environment, competitive compensation, and opportunities for professional growth and development.

  • Data Pipeline Specialist

    2 days ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    As a seasoned data professional, you will be instrumental in shaping our organization's data landscape.The ideal candidate will have a strong background in designing and implementing scalable data pipelines using Snowflake, Fivetran, DBT, and Python. Experience with complex SQL query writing, including joins, CTEs, window functions, and performance tuning is...

  • Data Pipeline Developer

    3 days ago

    Vadodara, Gujarat, India beBeeDataPipeline Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled data pipeline developer to join our team. This role involves designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.Key Responsibilities:Design, develop, and maintain data pipelines using IBM DataStage and AWS Glue/Lambda for ingestion from varied sources like...

  • Chief Data Pipelines Architect

    3 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    Job Title: Data EngineerWe are seeking a skilled Data Engineer to develop and maintain ETL/ELT data pipelines. The successful candidate will be responsible for designing, developing, and maintaining data pipelines using AWS services such as Lambda, DMS, and Glue.Key Responsibilities:Design and develop data pipelines to ingest data from various sources into...

  • Data Integration Specialist

    1 day ago

    Vadodara, Gujarat, India beBeeAzure Full time ₹ 80,00,000 - ₹ 1,00,00,000

    Job Description:We are seeking an experienced Azure Data Factory (ADF) Specialist to support our data integration and ETL needs. The ideal candidate will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure smooth, scalable, and secure data flows across multiple sources and destinations.Design, build, and...

  • Senior ETL Developer

    1 day ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 80,00,000 - ₹ 1,50,00,000

    As a senior ETL developer, you will be responsible for designing, developing and maintaining complex data pipelines using IBM DataStage. The ideal candidate will have 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage.Required skills include hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based...

  • Chief Data Pipeline Architect

    22 hours ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerThe Role:We are seeking a highly skilled Data Engineer with deep expertise in Microsoft Azure to design, build, and optimize scalable data pipelines and Lakehouse architectures.You will work closely with cross-functional teams to provide thought leadership and deliver robust, secure, and high-performance data solutions that...

  • Data Specialist

    3 days ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 20,39,900 - ₹ 25,49,950

    We are seeking a detail-oriented Data Specialist to design, build and manage scalable data pipelines and enterprise data solutions.This role will be crucial in supporting analytics, reporting and data science initiatives by delivering high-quality, reliable and performant data systems.Key Responsibilities:Data Pipeline Development: Design, develop and manage...

  • Data Engineering Specialist

    7 days ago

    Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Opportunity: Data Engineering SpecialistAt our organization, we are currently seeking an experienced data engineering specialist to lead the design, development, and optimization of ETL pipelines. The ideal candidate will possess strong programming skills in Python and Scala, with hands-on experience working with Spark (PySpark & Scala Spark)....

  • Data Transformation Specialist

    1 day ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Be at the forefront of innovation as a Data Transformation Specialist!About this RoleWe are seeking an exceptional Data Transformation Specialist to join our team. As a key member of our data warehousing group, you will be responsible for designing, developing, and optimizing data transformation processes using Snowflake.This role requires strong analytical...

  • Data Integration Specialist

    1 day ago

    Vadodara, Gujarat, India beBeeDataIntegration Full time ₹ 1,59,00,000 - ₹ 1,99,00,000

    Job Title: Data Integration SpecialistWe are seeking a highly skilled professional to lead the integration of data from various sources and systems, ensuring seamless and efficient data flow.About the Role:Develop and implement ETL (Extract, Transform, Load) processes using scripting languages such as Python, Shell, and PowerShell.Utilize scheduling tools...