Data Pipeline Specialist

2 days ago

Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 20,00,000
Big Data Engineer

We are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.

Key Responsibilities:
  • Design, develop, and manage end-to-end data pipelines for stocks, crypto, and other financial datasets.
  • Integrate third-party APIs and data feeds into internal systems.
  • Build and optimize data ingestion, storage, and transformation workflows (batch and real-time).
  • Ensure data quality, consistency, and reliability across all pipelines.
  • Collaborate with data scientists, analysts, and backend engineers to provide clean, structured, and scalable datasets.
  • Monitor, troubleshoot, and optimize pipeline performance.
  • Implement ETL/ELT best practices, data governance, and security protocols.
  • Contribute to the scalability and automation of our data infrastructure.
Requirements:
  • Proven experience as a Big Data Engineer / Data Engineer (preferably in financial or crypto domains).
  • Strong expertise in Python, SQL, and distributed data systems.
  • Hands-on experience with data pipeline tools (e.g., Apache Spark, Kafka, Airflow, Flink, Prefect).
  • Experience with cloud platforms (AWS, GCP, or Azure) and data warehousing (Snowflake, BigQuery, Redshift, etc.).
  • Knowledge of API integrations and handling real-time streaming data.
  • Familiarity with databases (relational and NoSQL) and data modeling.
  • Solid understanding of stocks, cryptocurrencies, and financial data structures (preferred).
  • Strong problem-solving skills with the ability to handle large-scale data challenges.

  • Data Pipeline Specialist

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 17,50,000

    About the RoleWe're seeking a highly skilled Data Engineer to build robust pipelines that transform large, messy technical datasets into high-quality structured data.

  • Data Pipeline Specialist

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OpportunityWe are seeking a skilled professional to take on the role of Data Engineer, responsible for designing, developing, and maintaining data pipelines that ingest data from various sources into our enterprise-wide data warehouse system.Utilize cutting-edge technologies like AWS services (AWS Lambda, AWS DMS, and AWS Glue) for data ingestion and...

  • Pipeline Intervention Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeePipeline Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Title: Pipeline Intervention SpecialistThe pipeline industry is seeking a highly skilled professional to conduct hot tapping and pipeline isolation services on-site.Key Responsibilities:Conduct hot tapping and pipeline isolation services in accordance with industry standards and regulations.Collaborate with project teams to ensure safe and efficient...

  • Data Pipeline Engineer

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeETL Full time ₹ 10,00,000 - ₹ 15,00,000

    Senior ETL DeveloperWe are seeking an accomplished Senior ETL Developer to join our Data Engineering team. The ideal candidate will have a proven track record of designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.The role involves collaborating with architects, business analysts, and data...

  • Data Pipeline Architect

    7 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the Role:The ideal candidate will have a solid foundation in data engineering with a minimum of 7 years of experience in development on data-centric projects.Mandatory Skills:Expertise in working with real-time data and Kafka framework, including kSQL/Mirror Maker etc.Proficiency in at least one programming language: Groovy or Java.Thorough...

  • High-Performance Data Pipeline Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About Us We're looking for a talented Data Engineer to help us scale and optimize our data processing capabilities. Role Overview You will design, build, and maintain high-performance data pipelines that process terabytes of data. Key Responsibilities Design and implement scalable batch processing systems using Python and big data technologies Optimize...

  • Data Engineering Specialist

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistAbout the RoleWe are seeking a highly skilled Data Engineering Specialist to join our team. As a key member of our data infrastructure team, you will design, develop, and maintain large-scale data pipelines using Python and AWS services.Key ResponsibilitiesData Pipeline Development: Develop and implement data pipelines...

  • Data Ingestion Specialist

    6 days ago

    Kanpur, Uttar Pradesh, India beBeeDataIngestion Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title:We are seeking a talented Data Ingestion Specialist to design and develop data pipelines for integrating multiple sources into Databricks.The ideal candidate will have hands-on experience with automation, infrastructure management, and scalable data workflows.Design and develop efficient data ingestion pipelines for integrating various data sources...

  • High-Performance Data Pipelines Expert

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Are you a data engineering expert looking for a new challenge? This is an excellent opportunity to leverage your skills and experience in designing, developing, and maintaining scalable and high-performance data pipelines using Databricks and PySpark.About the RoleThis is a Senior Data Engineer position where you will be working on the design, development,...

  • Strategic Data Architect

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data SpecialistWe are seeking a skilled Senior Data Specialist to design, develop, and maintain robust data pipelines, data warehouses, and analytics platforms that power our digital marketing platform. As a key member of our data infrastructure team, you will transform raw marketing data into actionable insights that drive business growth.Key...