Masterful Data Pipelines Architect

1 week ago

rajahmundry, India beBeeDataEngineer Full time

We are building the future of healthcare analytics.Job SummaryOur team is seeking a Senior Data Engineer to design, build, and scale robust data pipelines for nationwide analytics and machine learning systems.Design scalable ETL pipelines using Python (Pandas, PySpark) and SQL, orchestrated with Airflow (MWAA).Maintain the SAIVA Data Lake/Lakehouse on AWS ensuring quality, governance, scalability and accessibility.Run and optimize distributed data processing jobs with Spark on AWS EMR and/or EKS.Implement batch and streaming ingestion frameworks APIs databases files event streams.Enforce validation and quality checks for reliable analytics and ML readiness.Monitor and troubleshoot pipelines with CloudWatch integrating observability tools like Grafana Prometheus or Datadog.Automate infrastructure provisioning with Terraform following AWS best practices.Manage SQL Server PostgreSQL and Snowflake integrations into the Lakehouse.Participate in an on-call rotation to support pipeline health and resolve incidents quickly.RequirementsRequires 5+ years experience in data engineering ETL pipeline development or data platform roles.Experience designing operating data lakes or Lakehouse architectures on AWS S3 Glue Lake Formation Delta Lake Iceberg.Strong SQL skills with PostgreSQL SQL Server and at least one AWS cloud warehouse Snowflake or Redshift.Proficient in Python Pandas PySpark Scala or Java a plus.Hands-on with Spark on AWS EMR and/or EKS for distributed processing.Strong background in Airflow MWAA for workflow orchestration.Expertise with AWS services S3 Glue Lambda Athena Step Functions ECS CloudWatch.Proficient with Terraform for IaC familiarity with Docker ECS and CI/CD pipelines.Experience building monitoring validation and alerting into pipelines with CloudWatch Grafana Prometheus or Datadog.Strong communication skills collaboration with data scientists analysts and product teams.A track record of delivering production-ready scalable AWS pipelines not just prototypes.Why This Role MattersThis role enables you to work fully remotely from Europe or India with occasional team gatherings in Mountain View, California.

  • Data Pipeline Architect

    7 days ago

    rajahmundry, India beBeeDataEngineer Full time

    Senior Data Engineering PositionWe are seeking an experienced Senior Data Engineer to design, develop and maintain large-scale data pipelines using Google Cloud Platform (GCP) services.Responsibilities:Design, develop, test and deploy scalable data processing solutionsCollaborate with cross-functional teams to meet business objectivesRequirements:5+ years of...

  • **Senior Data Pipeline Architect

    1 week ago

    rajahmundry, India beBeeData Full time

    **Job Opportunity:** We are looking for a skilled professional to lead the design and implementation of scalable data pipelines. As a key member of our team, you will be responsible for ingesting, transforming, and activating customer data.Key Responsibilities:Develop and orchestrate workflows using Apache Airflow and SparkLeverage AWS services to manage...

  • Master Data Architect

    6 days ago

    rajahmundry, India beBeeEnterprise Full time

    Job Title: Master Data ArchitectAbout the Role:Design, develop, and deploy Unified Namespace (UNS) architectures using HighByte Intelligence Hub.Expertise in ISA-95 standards, hierarchical data modeling for OT/IT integration, and industrial communication protocols.Proven ability to design scalable and flexible integration solutions across multiple sites and...

  • Data Pipeline Validator

    2 weeks ago

    rajahmundry, India beBeeData Full time

    Business Intelligence Tester RoleWe are seeking a highly skilled Business Intelligence (BI) tester to join our team. As a BI tester, you will be responsible for ensuring the accuracy, performance, and reliability of data pipelines, reports, dashboards, and analytics solutions.The ideal candidate will have strong experience in data validation, SQL, ETL...

  • Data Architect

    1 week ago

    rajahmundry, India beBeeData Full time

    As a Data Architect, you will play a pivotal role in crafting and implementing large-scale data pipelines.Your focus will be on developing data integration strategies, ensuring seamless data exchange between systems, and guaranteeing robust data quality and reliability.The ideal candidate will have 2+ years of experience in designing and building scalable...

  • Enterprise Cloud Data Architect

    1 week ago

    rajahmundry, India beBeeDataEngineer Full time

    Cloud Data Architects design and build enterprise data systems using cloud-native technologies. This freelance opportunity is ideal for experienced engineers looking to work on large-scale, long-term projects.Key Responsibilities:Design and implement scalable, reliable data pipelines using cloud-native tools and frameworks.Develop ETL/ELT workflows for...

  • Senior Data Solutions Architect

    6 days ago

    rajahmundry, India beBeeData Full time

    Data Architect PositionWe are seeking an experienced Data Architect to design and build scalable and reliable data infrastructure from the ground up. Key Responsibilities:Designing and implementing robust ETL/ELT pipelines to process and transform large datasets efficiently.Owning the data platform, including data warehousing, data lakes, and associated...

  • Enterprise Data Architect

    2 weeks ago

    rajahmundry, India beBeeData Full time

    Key Data ArchitectWe are seeking a highly skilled Key Data Architect to drive innovation within our data-driven organization.Job Summary:We are looking for an experienced architect who will lead the design and development of our big data infrastructure, ensuring scalable solutions that meet business needs.Responsibilities:Data Architecture: Design and...

  • Senior Data Pipelines Developer

    7 days ago

    rajahmundry, India beBeeDataEngineer Full time

    Job TitleWe are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our data team.Job Description:The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines that drive business insights and operational efficiency.Design, develop, and...

  • Senior Data Architect

    2 weeks ago

    rajahmundry, India beBeeData Full time

    About the RoleWe're seeking a seasoned data architect to spearhead the development of our big data ecosystem. The ideal candidate will have extensive hands-on experience in designing, building, and optimizing large-scale data ingestion and transformation pipelines.You will be responsible for crafting robust data acquisition frameworks that support real-time...