Senior Data Pipeline Architect

19 hours ago

Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

We are seeking a highly skilled Data Engineer to join our team. This role involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.

Key Responsibilities:

  1. Design, develop, and optimize large-scale data pipelines and distributed data processing systems.
  2. Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.
  3. Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.
  4. Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.
  5. Collaborate with cross-functional teams to design scalable cloud-based data architectures.
  6. Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.
  7. Build monitoring and alerting systems leveraging Splunk or equivalent tools.
  8. Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.
  9. Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills:

  • Scala – Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark (PySpark) and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi / Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
  • Good understanding of Cloud Architecture.

Preferred Skills:

  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive, responsive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.
  • Hands-on experience with CI/CD tools (Git, Jenkins).
  • Strong grasp of software engineering concepts, data modeling, and optimization techniques.

Why this Role?

This is an excellent opportunity to work with a talented team and contribute to the development of innovative data solutions. If you have a passion for data engineering and want to take your career to the next level, we encourage you to apply.

  • Chief Data Pipeline Architect

    18 hours ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking an accomplished Senior Data Engineer to spearhead the development, maintenance, and optimization of our data infrastructure. This critical role requires a seasoned professional with expertise in designing and implementing large-scale data pipelines, leveraging cloud services, and ensuring seamless integration with...

  • Data Pipeline Architect

    15 hours ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time US$ 90,000 - US$ 1,20,000

    Job OverviewThis role focuses on designing, developing and maintaining data pipelines using Python and AWS services. The ideal candidate will have experience in building scalable and efficient data systems.Developing and implementing data processing solutions using AWS services (S3, Redshift, Lambda, Glue)Ensuring data quality and integrity throughout the...

  • Senior Data Engineer Architect

    12 hours ago

    Belgaum, Karnataka, India beBeeSoftware Full time ₹ 1,50,00,000 - ₹ 2,10,00,000

    Job Title: Senior Software Architect - Data EngineeringJob SummaryWe are seeking an experienced Senior Software Architect to lead our data engineering efforts. This individual will be responsible for designing and building a scalable data pipeline using Apache Beam and Databricks.About the RoleThis is a critical position that requires a deep understanding of...

  • Chief Data Pipeline Architect

    3 days ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 25,00,000

    Job OverviewAs a data engineer, you will be responsible for designing and building scalable data pipelines to ingest, process, and store data.The ideal candidate will have expertise in data integration, processing, and storage, as well as knowledge of data optimization architecture, data processes, and data flow.You will work closely with cross-functional...

  • Building Scalable Clinical Data Pipelines

    1 day ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer - Clinical Data Pipeline LeadAbout the RoleWe're seeking a highly experienced Senior Data Engineer to lead the design and operation of our AI-powered applications' backbone. This critical role involves creating scalable pipelines that process clinical data, retrieve medical documents, and deliver validated information to machine learning...

  • Data Engineering Specialist

    15 hours ago

    Belgaum, Karnataka, India beBeeDataEngineering Full time US$ 15,00,000 - US$ 20,00,000

    Job DescriptionWe are seeking a seasoned Data Engineering Specialist to join our team. As a key member of our data engineering group, you will play a pivotal role in designing, building, and maintaining our data pipeline architecture.Our ideal candidate is an experienced professional with a passion for data engineering, a strong track record of delivering...

  • Expert Data Architect

    1 day ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 13,20,000 - ₹ 2,01,60,000

    Data Architect SpecialistWe are seeking an experienced Data Engineer to join our organization. The ideal candidate should have strong expertise in building and automating data pipelines, managing multi-environment deployments, and working with modern DevOps/configuration management tools.

  • Senior Data Pipeline Specialist

    1 day ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    ETL Developer Job DescriptionWe are seeking a talented ETL Developer to join our data engineering team. As an ETL Developer, you will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key Responsibilities:Design, develop, and maintain ETL...

  • Data Pipeline Specialist

    15 hours ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Overview">The role of a Data Engineer involves designing and developing scalable data pipelines to support the organization's growing data needs. This will involve collaborating with analytics and business teams to improve data models, making data more accessible and driving informed decision-making across the organization.Key Responsibilities">Develops...

  • Data Architect

    1 day ago

    Belgaum, Karnataka, India beBeeBusiness Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    Senior Business Intelligence DeveloperThis role is responsible for designing and developing advanced data warehousing solutions using columnar databases. The successful candidate will have a strong background in ETL processes, proficiency in SQL, and experience with SSIS packages.The ideal candidate will be able to work closely with our team of solution...