Lead Data Pipeline Architect

1 week ago

kollam, India beBeeDataEngineer Full time

Senior Data Engineer Job DescriptionWe are seeking a seasoned Senior Data Engineer to architect, build, and own data pipelines that power our large language model development.Your primary mission is to create scalable, automated systems transforming massive raw datasets into pristine model-ready formats.Key Responsibilities:Design, develop, and maintain robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets.Implement rigorous data cleaning, deduplication, filtering, and normalization strategies.Define and enforce data quality standards ensuring the highest integrity for model training.Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks.Collaborate with AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle.Continuously optimize data processing workflows for speed, cost, and reliability.Required Qualifications:8+ years of professional experience in data engineering, data processing, or backend software engineering.Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars).Proven experience building and maintaining large-scale data pipelines.Benefits:Opportunity to lead cutting-edge AI and ML projects.Collaborative and innovative team culture.Competitive compensation with continuous learning opportunities.

  • Lead Data Architect

    7 days ago

    kollam, India beBeeData Full time

    Job Title: Enterprise Data Architect LeadWe are seeking an exceptional senior-level data architect to lead the design and delivery of enterprise data solutions. The ideal candidate will possess extensive experience in data engineering, analytics, and architecture roles.Key responsibilities include defining end-to-end data architectures, designing and...

  • Data Landscape Designer

    1 week ago

    kollam, India beBeeData Architect Full time

    Solution Designer & ImplementerThis position encompasses the oversight of entire data and analytics architecture for a company.The ideal candidate will be responsible for defining end-to-end data architecture, leading design and implementation of ETL/ELT pipelines, and driving integration patterns and APIs for seamless interoperability.Key qualifications...

  • Chief Data Pipeline Architect

    7 days ago

    kollam, India beBeeDataEngineer Full time

    Job Overview We are seeking a skilled Data Engineer to join our global engineering team for an international enterprise organization. As a Data Engineer, you will be responsible for designing and developing data pipelines using Palantir Foundry and PySpark, as well as implementing data governance practices using TypeScript. This is a long-term full-time...

  • Data Pipeline Specialist

    1 week ago

    kollam, India beBeeDataPipeline Full time

    Job OverviewA Data Pipeline Specialist builds data pipelines for enterprise search applications using ADF and Databricks.Integrates feeds according to requirement.Reviews business requirements and designs pipelines that load data into Azure Data Lakes or Azure Data warehouses, Azure Synapse, Azure Databricks or cloud services.Develops pipelines between...

  • Data Warehouse Cloud Architect

    3 days ago

    kollam, India beBeeCloud Full time

    We are currently seeking an experienced Senior Cloud Architect to lead the design and development of a modern, cloud-native data warehouse on AWS. This role is critical to building a scalable, secure, and efficient data platform that supports analytics, reporting, and AI use cases across the organization.Key Requirements:A minimum of 7 years of experience in...

  • Data Pipeline Tester

    7 days ago

    kollam, India beBeeData Full time

    ETL Quality Assurance SpecialistThe position requires end-to-end testing of data pipelines across various environments.Validate source-to-target mappings, complex data transformations, and Slowly Changing Dimension (SCD) logic.Test Informatica IDMC/IICS Control Center for Integration (CDI) mappings, taskflows, and ingestion patterns.You will also create test...

  • Lead Data Infrastructure Architect

    1 week ago

    kollam, India beBeeData Full time

    Job OverviewWe are seeking a highly skilled and experienced Principal Data Engineer to lead our data infrastructure development.Key ResponsibilitiesDesigning and implementing robust ETL/ELT pipelines to process and transform large datasets efficiently.Owning the data platform, including data warehousing, data lakes, and associated processing...

  • Expert Data Architect

    3 days ago

    kollam, India beBeeDataEngineer Full time

    Job Title: Expert Data ArchitectJob Summary:We seek a highly skilled data architect to design and build scalable data lakehouse solutions that enable analytics, AI, and MLOps across multiple clients.About the Role:Design lakehouse architectures using Azure Databricks, Delta Lake, or Iceberg.Build efficient PySpark pipelines for batch and streaming...

  • Chief Analytics Pipeline Architect

    9 hours ago

    kollam, India beBeeDataEngineer Full time

    Senior Data EngineerWe are pioneering the development of cutting-edge healthcare analytics platforms. Our mission is to create robust data pipelines that empower nationwide analytics and support our machine learning systems.This role offers a unique blend of remote work flexibility and collaboration opportunities, with team gatherings in California. We...

  • Senior Cloud Data Architect

    1 week ago

    kollam, India beBeeDataArchitect Full time

    Job Title:A Data Architect with a focus on building scalable, secure data platforms.We are seeking an experienced Data Architect to lead our team in designing and implementing a modern data platform on AWS. You will play a key role in transitioning from legacy systems to a cloud-native architecture using technologies like Apache Iceberg, AWS Glue, Redshift,...