Chief Data Pipeline Architect

2 days ago

India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

Data Engineering Position

We are seeking an experienced Data Engineer to design and build scalable data pipelines, manage large datasets using Spark and SQL, and deploy real-time streaming applications.

  • Key Responsibilities:
    • Design and optimize data pipelines and ETL workflows.
    • Manage and process large datasets using Spark, PySpark, and SQL.
    • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
    • Collaborate with data scientists and product teams to integrate AI/ML models into production.
    • Ensure data quality, scalability, and performance in all pipelines.
    • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
    • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

About the Role

This role involves building scalable pipelines, working with streaming data, and collaborating with data scientists and business stakeholders to deliver high-quality solutions.

  • Chief Data Pipeline Architect

    1 day ago

    India beBeeData Full time US$ 1,50,000 - US$ 1,75,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer with expertise in designing, building, and operating scalable data pipelines that support real-time clinical workflows.The ideal candidate will have 5+ years of experience building and maintaining production-grade data pipelines, strong background in workflow orchestration and...

  • Chief Data Architect

    3 days ago

    India beBeeDataScience Full time ₹ 1,50,00,000 - ₹ 2,40,00,000

    Job Title: Chief Data ArchitectAbout the Role:- Develops and implements business strategies that leverage data-driven insights to drive growth and profitability.- Oversees, monitors, and evaluates operational performance by directing the preparation of operating plans and supporting strategic management of client engagements.Main Responsibilities:•...

  • Chief Data Architect

    2 days ago

    India beBeeDataSystems Full time US$ 90,000 - US$ 1,20,000

    Job Opportunity: Data Systems ArchitectOverview:Our organization is seeking a skilled Data Systems Architect to design, build, and optimize scalable data pipelines using Snowflake, AWS (Lambda, Glue), DBT, and SQL. The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence...

  • Data Pipeline Architect

    17 hours ago

    India beBeeAzure Full time US$ 1,04,000 - US$ 1,30,878

    Azure Data Factory Specialist We are seeking an experienced Azure Data Factory specialist to join our team. The ideal candidate will have strong hands-on expertise in designing, building, and maintaining scalable and secure data pipelines using Azure ADF. Design, build, and maintain Azure Data Factory pipelines for ETL and ELT processes. Integrate data...

  • Chief Data Architect

    2 days ago

    India beBeeEngineering Full time ₹ 15,00,000 - ₹ 30,00,000

    Job Title: Azure Data Factory EngineerWe are seeking a seasoned data architect to spearhead the development and maintenance of data & analytics solutions using Azure Data Engineering technologies with a strong focus on performance, scalability, and reliability.Key Responsibilities:Design, develop, and support data pipelines using Azure Data Factory,...

  • Data Pipeline Specialist

    1 day ago

    India beBeeDeveloper Full time US$ 1,04,000 - US$ 1,30,878

    ETL Data Engineer - Scalable Pipeline DeveloperWe are seeking an experienced ETL data engineer to build and maintain scalable data pipelines using IBM DataStage and AWS Glue. As a key member of our data engineering team, you will work collaboratively with architects, business analysts, and data modelers to ensure timely and accurate delivery of critical data...

  • Enterprise / Platform Architect

    6 days ago

    India NTT DATA Full time

    **Req ID**: 327606 We are currently seeking a Enterprise / Platform Architect to join our team in Bnagalore, Karnātaka (IN-KA), India (IN). **Job Description: ServiceNow Technical Architect** **Experience Level**: 8+ years in ServiceNow development & solutioning **Domain Expertise**: ITSM, ITOM, ITAM, ServiceNow Architecture, Performance...

  • Data Architect

    1 day ago

    India beBeeDataAnalysis Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data ArchitectWe are seeking a highly skilled Data Architect to design and implement data processing pipelines that meet functional requirements.Candidates must have experience in designing, implementing, and maintaining data models to support reporting and analytics needs of the business.The ideal candidate will be able to develop and optimize...

  • Enterprise Data Visionary

    18 hours ago

    India beBeeData Architect Full time US$ 1,04,000 - US$ 1,30,878

    We are seeking an experienced data architect to manage and oversee all activities that define client business data assets. The role requires discovering and analyzing data and analytics requirements, and applying logical and analytical thinking to deliver innovative database solutions.">Design and implement data integration solutions using Informatica...

  • Highly Skilled Data Pipelines Architect

    2 days ago

    India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878

    Senior Data EngineerAbout the RoleWe are seeking an experienced Senior Data Engineer with in-depth expertise in building data pipelines and integrating healthcare data. You will report to the Head of AI Engineering and be responsible for designing, building, and operating the backbone of our AI-powered applications. This is a critical role where you will...