Chief Data Pipeline Architect

2 weeks ago

ballari, India beBeeData Full time

Senior Data EngineerWe are building the future of healthcare analytics. Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.This is a fully remote role, open to candidates based in Europe or India, with periodic team gatherings in Mountain View, California.ResponsibilitiesDesign, build, and maintain scalable ETL pipelines using Python (Pandas, PySpark) and SQL, orchestrated with Airflow (MWAA).Develop and maintain the SAIVA Data Lake/Lakehouse on AWS, ensuring quality, governance, scalability, and accessibility.Run and optimize distributed data processing jobs with Spark on AWS EMR and/or EKS.Implement batch and streaming ingestion frameworks (APIs, databases, files, event streams).Enforce validation and quality checks to ensure reliable analytics and ML readiness.Monitor and troubleshoot pipelines with CloudWatch, integrating observability tools like Grafana, Prometheus, or Datadog.Automate infrastructure provisioning with Terraform, following AWS best practices.Manage SQL Server, PostgreSQL, and Snowflake integrations into the Lakehouse.Participate in an on-call rotation to support pipeline health and resolve incidents quickly.Requirements5+ years in data engineering, ETL pipeline development, or data platform roles (flexible for exceptional candidates).Experience designing and operating data lakes or Lakehouse architectures on AWS (S3, Glue, Lake Formation, Delta Lake, Iceberg).Strong SQL skills with PostgreSQL, SQL Server, and at least one AWS cloud warehouse (Snowflake or Redshift).Proficiency in Python (Pandas, PySpark); Scala or Java a plus.Hands-on with Spark on AWS EMR and/or EKS for distributed processing.Strong background in Airflow (MWAA) for workflow orchestration.Expertise with AWS services: S3, Glue, Lambda, Athena, Step Functions, ECS, CloudWatch.Proficiency with Terraform for IaC; familiarity with Docker, ECS, and CI/CD pipelines.Experience building monitoring, validation, and alerting into pipelines with CloudWatch, Grafana, Prometheus, or Datadog.Strong communication skills and ability to collaborate with data scientists, analysts, and product teams.A track record of delivering production-ready, scalable AWS pipelines, not just prototypes.

  • Chief Mobility Data Architect

    2 weeks ago

    ballari, India beBeeData Full time

    As a senior solutions architect in the mobility data ecosystem, your key role is to design technical architectures for mobility projects. Ensuring day-one success requires aligning every workflow to meet client requirements and ensure operational readiness.You will translate client needs into detailed workflows, schemas, annotation guidelines, and tool...

  • Chief Data Architect

    2 weeks ago

    ballari, India beBeeDataEngineering Full time

    Azure Data Engineer Role OverviewAs a skilled Azure Data Engineer, you will be responsible for designing and developing data platforms using various Azure services such as Databricks, Data Factory, Data Lake Storage, Event Hub, and others.The role involves building efficient data pipelines, collaborating with stakeholders, ensuring data security, and staying...

  • Chief Data Architect

    1 week ago

    ballari, India beBeeDataSpecialist Full time

    Data SpecialistThe Data Specialist evaluates learners' technical proficiency in data engineering, analytics, and cloud data platforms. To excel in this role, you will be responsible for:Designing and executing quality audit frameworks for professional training programs.Conducting structured technical interviews to evaluate learners' proficiency in SQL, data...

  • Data Pipeline Specialist

    7 days ago

    ballari, India beBeeDataSupport Full time

    Job OverviewAs a Data Support Engineer, you will be responsible for monitoring data support issues and resolving them within defined guidelines or escalating them to the right person.The team oversees critical data pipelines using cutting-edge support applications. Any issues or breaks in data pipelines are addressed promptly within established guidelines or...

  • Strategic Data Architect

    2 weeks ago

    ballari, India beBeeArchitect Full time

    Senior Solutions ArchitectRole OverviewThis is a highly technical position requiring 12–15 years of experience, proven leadership in data architecture, deep ETL/analytics expertise, and familiarity with education data ecosystems or similarly regulated verticals.The Senior Solutions Architect will lead the design and delivery of enterprise data & analytics...

  • AI Data Architect

    5 days ago

    ballari, India beBeeData Full time

    Job Title:Advanced Data ArchitectWe are seeking a highly skilled data architect to join our team. The successful candidate will be responsible for designing and implementing cutting-edge data solutions using machine learning algorithms.Key Responsibilities:Develop and maintain AI/ML features for an open-source Observability Platform built on Grafana and...

  • Data Systems Architect

    2 weeks ago

    ballari, India beBeeData Full time

    Job Title: Data Systems ArchitectData systems architects design, develop and implement large-scale data architectures to support business growth. They have a deep understanding of data infrastructure, including databases, data warehouses, and data pipelines.They are responsible for ensuring the scalability, reliability, and performance of data systems, and...

  • Senior Data Architect

    2 weeks ago

    ballari, India beBeeDataEngineer Full time

    About usWe are seeking a skilled data pipeline architect to design, build and optimize scalable data solutions using modern cloud-based frameworks.Develop robust data models and schemas for efficient data warehousing and business intelligence purposesImplement complex transformations and ETL/ELT processes leveraging Snowflake, Databricks and...

  • Cloud Data Architect

    2 weeks ago

    ballari, India beBeeCloudDataEngineer Full time

    Leverage your technical expertise as a Cloud Data Engineer to drive data-driven decisions within our organization.We are seeking an experienced professional with a strong background in cloud-native ETL tools, including AWS DMS, AWS Glue, Kafka, Azure Data Factory, and GCP Dataflow.The ideal candidate will have extensive experience designing, implementing,...

  • Data Pipeline Specialist

    2 weeks ago

    ballari, India beBeeDataEngineer Full time

    We are seeking a skilled professional to spearhead the design, development and maintenance of scalable data pipelines.Key responsibilities include designing, developing and optimizing ETL/ELT pipelines using Python, Pyspark and AWS Glue.The ideal candidate will be responsible for implementing data ingestion, transformation and integration from diverse...