Data Pipeline Architect

5 days ago

Malappuram, India beBeeDataEngineer Full time

Senior Data EngineerWe are seeking an accomplished Senior Data Engineer to spearhead the development of data pipelines that power our large language model (LLM) platform. As a senior contributor, you will be responsible for designing, building, and owning scalable systems that transform massive datasets into pristine formats.Your primary objective is to architect and implement automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets. You will also be tasked with implementing robust data cleaning, deduplication, filtering, and normalization strategies to ensure the highest integrity for model training.Design & Build: Develop and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets.Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies to ensure data quality standards.Data Transformation: Efficiently structure and format diverse datasets for consumption by LLM training frameworks.Collaboration: Work closely with AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle.Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.

  • Data Pipeline Architect

    2 weeks ago

    Malappuram, India beBeeAzure Full time

    Job Opportunity: Experienced professionals are sought after to leverage their expertise in designing and developing ETL/ELT pipelines, optimizing data workflows, and ensuring robust data observability and streaming.Key Responsibilities:Design, develop, and maintain scalable data pipelines using cloud-based tools like Azure Data Factory (ADF) and Databricks...

  • Chief Data Pipeline Architect

    2 weeks ago

    Malappuram, India beBeeDataEngineer Full time

    Data Engineering LeadWe are seeking an experienced Data Engineer to lead the design and implementation of scalable data pipelines for ingesting, transforming, and activating customer data. The successful candidate will develop and orchestrate workflows using Apache Airflow and Spark.Leveraging AWS services, they will manage and optimize large-scale data...

  • Senior Data Architect

    7 days ago

    Malappuram, India beBeeDataEngineer Full time

    Senior Data Architect PositionAs a seasoned professional, you will play a pivotal role in designing and developing scalable data pipelines to ingest information from diverse sources. Key responsibilities include collaborating with stakeholders to gather data requirements, translating business needs into technical specifications, and ensuring data...

  • Data Pipeline Specialist

    7 days ago

    malappuram, India beBeeDataSupport Full time

    Overview of the Data Support Engineer role:The position involves monitoring and resolving data support issues within established guidelines. This includes collaborating with clients and internal teams, working independently while maintaining trust and communication.Key responsibilities include:Monitoring data pipelines using cutting-edge toolsFixing issues...

  • Senior Cloud Data Solutions Architect

    2 weeks ago

    Malappuram, India beBeeData Full time

    Senior Data Architect OpportunityJob Title: Senior Data ArchitectLocation: Flexible, Hyderabad, Pune, Jaipur Experience required: 12+ yearsJob Description:We are seeking an experienced senior data architect to lead the design and implementation of robust, scalable, and high-performance data architectures that support our enterprise data initiatives.The ideal...

  • Leading Cloud Data Architect

    2 weeks ago

    Malappuram, India beBeeDataEngineer Full time

    Establishing a robust data infrastructure on the cloud requires strategic leadership from a Senior Data Engineer.This professional will be responsible for architecting and optimizing data pipelines and workflows, leveraging technologies such as Apache Iceberg, AWS Glue, Redshift, and Atlan for governance. The goal is to create scalable platforms that support...

  • Sr. Azure Data Architect

    4 weeks ago

    Malappuram, India Whatjobs IN C2 Full time

    Job Description: Job Title: Azure Data Architect Location: Hyderabad, Pune, Jaipur Experience required: 12+ years Role Overview: We are seeking a highly experienced senior data architect to design and implement robust, scalable, and high-performance data architectures that support our enterprise data initiatives. The ideal candidate should bring strong...

  • Enterprise-Grade Data Solutions Architect

    2 weeks ago

    Malappuram, India beBeeDataEngineer Full time

    Job OverviewWe are seeking a Lead Data Engineer to design, build, and manage enterprise-grade data pipelines in Microsoft Fabric and Azure Data Factory (ADF). This customer-facing role involves partnering directly with clients to translate business needs into scalable data solutions.We need someone who can design, develop, and optimize metadata-driven data...

  • Data Pipeline Quality Assurance Professional

    2 weeks ago

    Malappuram, India beBeeEtl Full time

    Job Description:We are seeking an experienced ETL Specialist to join our team. The ideal candidate will have strong hands-on experience in ETL/Data Pipeline Testing and API Testing.The successful candidate will have a good understanding of data flows, strong SQL skills, and familiarity with Informatica tools such as IDMC/IICS, CDI, and MDM SaaS.ETL / Data...

  • Enterprise Data Architect Opportunity

    5 days ago

    Malappuram, India beBeeData Full time

    Lead Enterprise Data ArchitectJob Summary:Snowflake data architecture requires defining and leading enterprise-grade architectures.This role demands expertise in cloud platforms, data modeling, governance, integration frameworks, and advanced analytics ecosystems.Defining scalable, secure, and high-performance architectures aligned with business...