Senior Data Engineer
2 days ago
About UsMyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent. We empower businesses by providing world-class software engineers, operations support, and infrastructure to help them grow faster and better.Position: Senior Data Engineer (Python Coder)Location: India ( Remote )Work Commitment: 40 Hrs / Week (full-time)Contract Duration: 3 - 6 MonthsClient: Wipro ( Google ) BGV: YESRole: Senior Data Engineer (Python Coder)Exp: Min. 8 Years Role SummaryWe are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning. Key ResponsibilitiesArchitect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets.Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training.Data Transformation: Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks.Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle.Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs. Required Qualifications8+ years of professional experience in data engineering, data processing, or backend software engineering.Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars).Proven experience building and maintaining large-scale data pipelines.Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing).Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale.Excellent problem-solving skills and a meticulous attention to detail.Strong communication and collaboration skills, with experience working in a team environment. Preferred Qualifications (Nice-to-Haves)Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family).Strong experience with big data frameworks like Apache Spark or Ray.Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers).Familiarity with ML frameworks like PyTorch or TensorFlow.Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services.
-
Senior DevOps Engineer
2 weeks ago
bangalore, India Crest Data Full timeCompany Overview: Crest Data is the global leading provider of Data Analytics, Security, DevOps, Cloud Solutions, Software integrations, Analytics, and security-based technological services. With a clientele that includes several Fortune 500 corporations and some of the innovative Silicon Valley Startups. Company URL : Job Location - Ahmedabad Job...
-
Senior DevOps Engineer
1 week ago
bangalore, India Crest Data Full timeCompany Overview: Crest Data is the global leading provider of Data Analytics, Security, DevOps, Cloud Solutions, Software integrations, Analytics, and security-based technological services. With a clientele that includes several Fortune 500 corporations and some of the innovative Silicon Valley Startups. Company URL : Job Location - Ahmedabad Job...
-
Senior DevOps Engineer
1 week ago
bangalore, India Crest Data Full timeCompany Overview: Crest Data is the global leading provider of Data Analytics, Security, DevOps, Cloud Solutions, Software integrations, Analytics, and security-based technological services. With a clientele that includes several Fortune 500 corporations and some of the innovative Silicon Valley Startups. Company URL : Job Location - Ahmedabad Job...
-
Senior DevOps Engineer
2 weeks ago
bangalore, India Crest Data Full timeCompany Overview:Crest Data is the global leading provider of Data Analytics, Security, DevOps, Cloud Solutions, Software integrations, Analytics, and security-based technological services. With a clientele that includes several Fortune 500 corporations and some of the innovative Silicon Valley Startups.Company URL: http://www.crestdata.aiJob Location -...
-
Senior Data Scientist
2 days ago
bangalore, India Enable Data Incorporated Full timeEnable Data Incorporated is looking for a talented and experienced Senior Data Scientist to join our innovative team. In this role, you will be responsible for driving data science initiatives, developing predictive models, and providing insights that guide key business decisions. The ideal candidate has a strong background in statistical analysis, machine...
-
Senior Data Engineer
3 weeks ago
bangalore, India AIQU Full timeWe are hiring for Senior Data Engineer to join one of our major clients based out of KSA. Job Details: Role: Senior Data Engineer Work Location: Remote Employment Type: Contract – 12 months & extendable Role Summary: The Senior Data Engineer plays a lead role in designing, building, optimizing, and governing data pipelines and architectures that power...
-
Senior Data Engineer
7 days ago
bangalore, India Elements HR Services Pvt Ltd Full timeSenior Data Engineer Title Senior Data Engineer Department ISSLocation BengaluruReports To Data Engineers – LeadLevel 4,5,6 We share a commitment to making things better for clients and each other. We continually explore new technology and different ways of working to put our clients first. So, bring your boldest ideas to our (name of team/business area)...
-
Senior Data Engineer
6 days ago
Bangalore, India Elements HR Services Pvt Ltd Full timeSenior Data Engineer Title Senior Data Engineer Department ISS Location Bengaluru Reports To Data Engineers – Lead Level 4,5,6 We share a commitment to making things better for clients and each other. We continually explore new technology and different ways of working to put our clients first. So, bring your boldest ideas to our (name of team/business...
-
Senior Data Engineer
1 day ago
bangalore, India Tata Consultancy Services Full timeRole**: Senior Data EngineerRequired Technical Skill Set: Senior Data EngineerDesired Experience Range: 7 - 12 yrsNotice Period: Immediate to 90Days onlyLocation of Requirement: BangaloreWe are currently planning to do a Virtual InterviewJob Description:Primary Skills (Java, Python, etc.)Python, Scala/Java, Spark, SQL, Snowflake, BDD, Databricks, AWS Key...
-
Senior Data Engineer
2 weeks ago
bangalore, India MeghGen Technologies Full timeCompany DescriptionMeghGen Technologies is a trusted technology solutions partner specializing in data engineering, cloud-native applications, and cloud migrations. We design and implement end-to-end cloud solutions to help businesses modernize infrastructure, scale applications, and harness the power of their data. Our expertise extends to AI-powered...
