High Volume Data Engineer

6 days ago

Hubli, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 20,00,000
Big Data Developer

We are seeking a highly skilled engineer to design, build and maintain robust data pipelines that handle high-volume financial data.

Key Responsibilities
  • Design, develop and manage end-to-end data pipelines for stocks, crypto and other financial datasets.
  • Integrate third-party APIs and data feeds into internal systems.
  • Build and optimize data ingestion, storage and transformation workflows (batch and real-time).
  • Ensure data quality, consistency and reliability across all pipelines.
  • Collaborate with data scientists, analysts and backend engineers to provide clean, structured and scalable datasets.
  • Monitor, troubleshoot and optimize pipeline performance.
  • Implement ETL/ELT best practices, data governance and security protocols.
  • Contribute to the scalability and automation of our data infrastructure.

Requirements:

  • Proven experience as a Big Data Engineer / Data Engineer (preferably in financial or crypto domains).
  • Strong expertise in Python, SQL and distributed data systems.
  • Hands-on experience with data pipeline tools (e.g. Apache Spark, Kafka, Airflow, Flink, Prefect).
  • Experience with cloud platforms (AWS, GCP or Azure) and data warehousing (Snowflake, BigQuery, Redshift, etc.).
  • Knowledge of API integrations and handling real-time streaming data.
  • Familiarity with databases (relational and NoSQL) and data modeling.
  • Solid understanding of stocks, cryptocurrencies and financial data structures (preferred).
  • Strong problem-solving skills with the ability to handle large-scale data challenges.

  • High-Volume System Optimizer

    9 hours ago

    Hubli, Karnataka, India beBeePerformance Full time US$ 90,123 - US$ 1,23,456

    Optimizing Application Performance is a key focus for any technology-enabled organization.Key ResponsibilitiesThe ideal candidate will be responsible for designing, developing, and executing performance tests to ensure that applications meet non-functional requirements and scale to handle large volumes of traffic.They will also identify and troubleshoot...

  • High-Performance Backend Engineer

    2 days ago

    Hubli, Karnataka, India beBeeBackend Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job SummaryAs a skilled Backend Developer, you will play a crucial role in designing and implementing high-performance RESTful APIs using Node.js and MongoDB. You will work closely with frontend teams and DevOps to ensure seamless API integrations and deployments.Key Responsibilities:API Development: Design, develop, optimize, and maintain high-performance...

  • Key Data Engineering Expert

    1 day ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Our organization is seeking a highly skilled and experienced Data Engineering expert to assume key responsibilities.The ideal candidate will have:4+ years of hands-on experience with Big Data technologies – including Spark, Hadoop, and Hive;Strong analytical and problem-solving skills, with the ability to think critically;Excellent communication and...

  • Large Scale Data Engineer

    1 week ago

    Hubli, Karnataka, India beBeeDataEngineer Full time US$ 1,80,000 - US$ 2,10,000

    We seek a seasoned professional to lead our data engineering efforts. As a key member of our team, you will be responsible for designing and developing large-scale data processing systems that drive business intelligence.Key ResponsibilitiesDesign and develop scalable and robust data pipelines that collect, process, and store large volumes of structured and...

  • High-Performance Data Architect

    3 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    Job Title: Databricks EngineerWe are seeking a seasoned data engineer to build and optimize scalable, high-performance data solutions.About the RoleThe ideal candidate possesses hands-on experience in production environments and a deep understanding of modern data architecture principles.Key Responsibilities:Design and develop robust data pipelines and ETL...

  • Data Pipeline Specialist

    7 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    As a Data Engineer, you will play a pivotal role in crafting and implementing scalable data pipelines to support our growing data needs. Your expertise in Bigdata, Pyspark, Python, Hadoop/HDFS, Spark, and GCP/any cloud will enable you to collaborate with analytics and business teams to refine data models and foster data-driven decision making across the...

  • High-Level Data Architect

    12 hours ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Job Opportunity for Senior Data EngineerWe are seeking an experienced Senior Data Engineer to lead the design and implementation of scalable data pipelines on AWS.Tech Requirements:Proficiency in programming languages such as Python and/or Scala is required.A strong understanding of Apache Spark for big data processing on AWS cloud is essential.Expertise in...

  • Senior Data Architect

    9 hours ago

    Hubli, Karnataka, India beBeeDataModeler Full time US$ 2,00,000 - US$ 2,50,000

    Job Title: Data Engineering and Modeling ExpertWe are seeking a seasoned data professional to lead our data management team in developing innovative solutions for complex business problems.The ideal candidate will possess expertise in designing scalable data capture and transformation processes, as well as proficiency in entity-relational, data vault, and...

  • Chief Data Architect

    7 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data EngineerWe are seeking a highly skilled and motivated data engineer to design, develop, and maintain scalable data pipelines that process and transform large volumes of structured and unstructured data.The ideal candidate will have a strong background in data engineering, cloud technologies, and data architecture, and will collaborate closely...

  • Ai Security Engineer

    5 days ago

    Hubli, Karnataka, India NTT DATA Full time

    Req ID 338875 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a AI Security Engineer - Shared Security Services Engineering to join our team in Banaglore Karn taka IN-KA India ...