Senior Data Engineer

4 weeks ago

Vellore, India MyRemoteTeam Inc Full time

About UsMyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.Position: Senior Data Engineer (Python Coder)Location: India ( Remote )Work Commitment: 40 Hrs / Week (full-time)Contract Duration: 3 - 6 MonthsClient: Wipro ( Google ) BGV: YESRole: Senior Data Engineer (Python Coder)Exp: Min. 8 Years Role SummaryWe are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning. Key ResponsibilitiesArchitect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets.Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training.Data Transformation: Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks.Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle.Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs. Required Qualifications8+ years of professional experience in data engineering, data processing, or backend software engineering.Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars).Proven experience building and maintaining large-scale data pipelines.Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing).Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale.Excellent problem-solving skills and a meticulous attention to detail.Strong communication and collaboration skills, with experience working in a team environment. Preferred Qualifications (Nice-to-Haves)Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family).Strong experience with big data frameworks like Apache Spark or Ray.Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers).Familiarity with ML frameworks like PyTorch or TensorFlow.Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services.

  • Senior Data Engineer

    6 days ago

    vellore, India beBeeData Full time

    For an international project in Chennai, we are seeking a senior-level data engineer.The successful candidate will be responsible for developing and managing data pipelines using Azure Data Factory and Databricks.Key Responsibilities:Design and implement scalable data pipelines to support enterprise search applicationsDevelop and maintain ADF and Databricks...

  • Data Engineering Leadership Role

    6 days ago

    vellore, India beBeeData Full time

    Senior Data Engineer PositionWe are looking for a highly skilled Senior Data Engineer to lead our data engineering team in designing, developing, and maintaining robust data acquisition frameworks that support large-scale analytics and operational data systems.The ideal candidate will have extensive hands-on experience in building and optimizing data...

  • Data Engineer

    4 weeks ago

    Vellore, India HISH IT SERVICES Full time

    We have a new urgent GCP Data Engineer opportunity open to support a migration initiative from Teradata to Cerebro (BigQuery). This role requires a hands-on developer who can collaborate closely with our data and reporting teams to ensure smooth repointing and validation of Power BI reports. Request Details: - Title: GCP Data Engineer - Seniority: Mid to...

  • Senior Data Platform Strategist

    5 days ago

    vellore, India beBeeSolution Full time

    Job Title: Senior Solutions Architect - DataWe are seeking a highly experienced and skilled Senior Solutions Architect to lead the design and delivery of data platforms across Clinical, Commercial, and Medical Affairs domains.This is an exciting opportunity for a professional with expertise in cloud-based solutions, data engineering, and architecture to join...

  • Senior Data Scientist

    6 days ago

    vellore, India beBeeGraphAnalytics Full time

    Senior AI/ML Cloud Engineer – Graph Analytics PlatformAs a senior leader in our graph analytics team, you will be responsible for architecting and implementing distributed graph computing solutions processing billions of entities and relationships.The role requires a deep understanding of high-performance computing, scalable architecture, and distributed...

  • Senior Enterprise Data Architect

    5 days ago

    vellore, India beBeeData Full time

    Senior Data Solutions ArchitectWe are seeking a senior-level data solutions architect to lead the design and delivery of our enterprise data and analytics solutions.The ideal candidate will have 12-15 years of experience in data engineering, analytics, or data architecture roles with progressive ownership. They will also have hands-on experience building and...

  • Senior Data Solutions Specialist

    2 days ago

    vellore, India beBeeData Full time

    Job Title: Data Solutions EngineerWe are seeking a Senior Data Solutions Engineer to design, build and scale robust data solutions on Microsoft Azure.This role involves owning modern data pipelines and models that power analytics and reporting across the business.To be successful in this position you will need 5+ years of professional experience in data...

  • Senior Cloud Data Specialist

    6 days ago

    vellore, India beBeeDataEngineer Full time

    Job Summary:We are seeking an experienced Senior Data Engineer to build a modern data platform on AWS. The ideal candidate will have hands-on experience with legacy and modern data stacks, including Apache Iceberg, AWS Glue, Redshift, and Atlan for governance.Key ResponsibilitiesDesign, develop, and optimize data pipelines and ETL/ELT workflows on...

  • Senior Data Engineer

    2 weeks ago

    Vellore, India Arenema Full time

    Location: India (remote – Bangalore/Karnataka area preferred) Type: Full-time contractor / employee Urgency: Position to be filled ASAP About the role You will be a core member of the team building a data platform that maps economic, advertising and real-estate actors using public/open data sources (social networks, marketplaces, registers, press) for...

  • Data Architecturer

    6 days ago

    vellore, India beBeeOracle Full time

    Senior Oracle Data Engineer Job OpportunityAs a senior data engineer, you will design, develop and optimize end-to-end data pipelines using Oracle Data Integrator (ODI) to create robust and scalable data architecture.Create high-performance data processing by configuring ODI load plans, packages and interfaces.Develop and maintain metadata layers, subject...