Data Pipeline Specialist

7 days ago

nashik, India beBeeData Full time

Job Title: Senior Data ArchitectLocation: Remote / HybridEmployment Type: Contract/ FreelanceRole SummaryWe are seeking a seasoned Senior Data Architect to design, build, and own scalable data pipelines that power our large language model (LLM) development.Your primary mission is to create automated systems that transform massive raw datasets into pristine model-ready formats. As a senior individual contributor, you will be the team's expert on data ingestion, processing, and quality for all AI training.You will thrive on solving large-scale data challenges and enjoy working at the intersection of data engineering and machine learning.Key Responsibilities:Architect & Build: Design, develop, and own robust ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets.Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies.Data Transformation: Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks.Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle.Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.Required Skills and Qualifications8+ years of professional experience in data engineering, data processing, or backend software engineering.Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars).Proven experience building and maintaining large-scale data pipelines.Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing).Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale.Excellent problem-solving skills and a meticulous attention to detail.Strong communication and collaboration skills, with experience working in a team environment.BenefitsOpportunity to lead cutting-edge AI and ML projects.Collaborative and innovative team culture.Competitive compensation with continuous learning opportunities.

  • Data Pipeline Specialist

    7 days ago

    nashik, India beBeeDataSupport Full time

    Job Opportunity:The role of a Data Support Engineer is critical in ensuring the high availability and stability of our data pipelines. This position requires an individual with strong analytical skills to identify and troubleshoot complex technical issues.Key Responsibilities:Act as the escalation point for data ingestion, consumption, and pipeline...

  • ETL Pipeline Specialist

    5 days ago

    nashik, India beBeeDataAnalysis Full time

    Job OverviewThe ETL Pipeline Specialist role is designed to deliver end-to-end testing of complex data transformations, ensuring accurate and reliable source-to-target mappings. This involves leveraging Informatica tools, such as IDMC/IICS, CDI, and MDM SaaS, to validate data flows and enhance business intelligence.Main Responsibilities:Data Transformation...

  • Principal Data Pipeline Specialist

    3 days ago

    nashik, India beBeeData Full time

    Job OverviewWe are seeking a seasoned Data Engineer with extensive hands-on expertise in crafting scalable data pipelines and cloud-based solutions.Design, develop, and optimize large-scale data processing systems and ETL workflowsCreate efficient and reusable Python scripts for data manipulation and analysisWrite complex SQL queries for data extraction,...

  • Senior Data Pipeline Specialist

    7 days ago

    nashik, India beBeeDataEngineer Full time

    A seasoned data professional is sought to lead the design and implementation of secure, scalable data pipelines. Key Responsibilities- Collaborate with cross-functional teams to develop and optimize complex data processing workflows.- Design and implement robust security measures for sensitive data assets.- Develop and maintain high-quality documentation for...

  • Chief Data Pipeline Architect

    6 days ago

    nashik, India beBeeDataEngineering Full time

    Job Title: Chief Data Pipeline ArchitectJob Summary:We are seeking an experienced and skilled Chief Data Pipeline Architect to design, develop, and optimize scalable data pipelines and cloud-based data solutions using Python, ETL/ELT processes, and AWS cloud services such as S3, Glue, Lambda, Redshift, Kinesis, and DynamoDB.Key Responsibilities:Design and...

  • Senior Data Infrastructure Specialist

    3 days ago

    nashik, India beBeeData Full time

    Job Title: Senior Data Infrastructure SpecialistWe are seeking an experienced professional to join our engineering organization as a senior data infrastructure specialist. This role involves designing and maintaining high-performance data pipelines and databases across multiple platforms.Key Responsibilities:Data Architecture and Maintenance: Design and...

  • Real-Time Data Ingestion Specialist

    3 days ago

    nashik, India beBeeDataIngestion Full time

    Job TitleA real-time data ingestion and processing specialist is needed to design and build cloud-based pipelines using Apache NiFi and GCP services.The ideal candidate will create scalable and high-throughput data pipelines, integrate them with the GCP ecosystem, and ensure low-latency data movement across layers.Design and implement real-time data...

  • AI Data Engineer

    3 hours ago

    nashik, India beBeeDataEngineering Full time

    Data Engineering SpecialistWe are seeking a highly skilled Data Engineering Specialist to join our team.Implement large language model (LLM) workflows using LangChain or similar technologies.Design and develop scalable data pipelines using Python and Azure tech stack.Collaborate with stakeholders to understand data requirements and design effective...

  • Building Healthcare Analytics Pipelines

    6 days ago

    nashik, India beBeeDataEngineer Full time

    As a key member of our team, you will be responsible for building the future of healthcare analytics. We are designing robust data pipelines that power nationwide analytics and support our machine learning systems.This role requires remote work with periodic team gatherings in Mountain View, California. You will work independently to design and build...

  • Data Insights Specialist

    4 days ago

    nashik, India beBeeData Full time

    Job Description:The role of a Business Intelligence Specialist involves supporting client business operations to ensure timely and high-quality deliverables.Effectively communicate client business issues, operating rules, data, and standard procedures to stakeholders.Execute and improve predefined operational processes in collaboration with senior team...