Data Engineer

12 hours ago

Anand, India Aceolution Full time

Job Title: Data Engineer – Python Expert(Freelance Role) Location: Remote / Hybrid Employment Type: Contract/ Freelance Role Summary We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning. Key Responsibilities Architect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets. Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training. Data Transformation : Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks. Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle. Optimization: Continuously optimize data processing workflows for speed, cost, and reliability. ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs. Required Qualifications 8+ years of professional experience in data engineering, data processing, or backend software engineering. Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars). Proven experience building and maintaining large-scale data pipelines. Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing). Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale. Excellent problem-solving skills and a meticulous attention to detail. Strong communication and collaboration skills, with experience working in a team environment. Preferred Qualifications (Nice-to-Haves) Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family). Strong experience with big data frameworks like Apache Spark or Ray. Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers). Familiarity with ML frameworks like PyTorch or TensorFlow. Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services. Why Join Us Opportunity to lead cutting-edge AI and ML projects. Collaborative and innovative team culture. Competitive compensation with continuous learning opportunities. If you are interested, please share your updated CV to along with your expected rate per hour.

  • Data engineer

    7 days ago

    Anand, India Philodesign Technologies Inc Full time

    Job Title: Data Engineer Experience: 6–8 Years Work Mode: Remote (9 AM – 5 PM EST) Employment Type: Full-time Notice Period: Immediate Joiner About the Role We are seeking an experienced Data Engineer to design, develop, and implement robust data exchange and processing solutions using Azure technologies. The ideal candidate will have strong hands-on...

  • Data engineer

    7 days ago

    Anand, India Canopus Infosystems - A CMMI Level 3 Company Full time

    Position: Data Engineer  Experience: 6 Months to 3 Years  Location: Remote  Joining: Immediate Joiners Preferred  About the Role:   We are looking for a skilled Data Engineer with strong Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization. The ideal candidate should be capable of building data...

  • Principal data engineer

    1 week ago

    Anand, India K2 Partnering Solutions Full time

    Job Title: Principal Data/App Engineer, Celonis Work Location: Bengaluru, Karnataka Job Type: Full-time/Permanent with the Client directly Job Details Our Client’s Center for Process Excellence team is on a mission to improve business outcomes and employee experiences by driving step-change improvements in critical enterprise processes, and the technology...

  • Data engineer

    3 weeks ago

    Anand, India IntraEdge Full time

    Job Title: Data EngineerLocation: (Remote)Experience: (5+ years)Employment Type: Full-timeJob Summary:We are seeking a skilled Data Engineer with hands-on experience in Snowflake, AWS (Lambda, Glue), DBT, and SQL to design, build, and optimize scalable data pipelines. The ideal candidate will be responsible for enabling seamless data integration,...

  • Data Governance Engineer

    1 day ago

    Anand, India Tixy Tech Full time

    Location: Hyderabad, Bangalore, Chennai, Coimbatore, Pune, Kochi About the Role: Seeking a Data Governance Engineer with hands-on experience in IBM Cloud Pak for Data (CP4D) — particularly with Information Governance Catalog (IGC), Information Knowledge Catalog (IKC), and Manta Lineage tools.The role focuses on metadata management, data lineage, and...

  • Pyspark data engineer

    4 weeks ago

    Anand, India EXTRAGIG Full time

    ???? Contract Assistant – Data Engineer Support (Remote, EST Hours) ???????? Start Date: Sept 10, 2025⏳ Duration: 6 months (extendable)???? Pay: $1,000/month???? Work Hours: 8:00 AM – 5:30 PM ESTWe’re looking for a Contract Assistant to support a Py Spark Data Engineer with daily activities. This is a remote contract role (not formal employment).What...

  • Azure data engineer

    1 week ago

    Anand, India Tata Consultancy Services Full time

    Greetings from Tata Consultancy Services!! We are hiring for Data Engineer Required Skillset: Python, Pyspark, Azure Function Python, Pyspark, Data bricks Python, Pyspark, Azure Function, Data Bricks Python, Fast API, Flask Experience : 5+years Location : Azure, ADF, Databricks, Python Job Description: Expertise in Python programming language and frameworks...

  • Engineering manager

    3 weeks ago

    Anand, India Coinbase Full time

    Ready to be pushed beyond what you think you’re capable of?At Coinbase, our mission is to increase economic freedom in the world. It’s a massive, ambitious opportunity that demands the best of us, every day, as we build the emerging onchain platform — and with it, the future global financial system.To achieve our mission, we’re seeking a very...

  • Project Manager – Data Engineering

    6 days ago

    Anand, India Whatjobs IN C2 Full time

    About the Company : We are looking for a skilled Technical Project Manager to lead and deliver projects in data engineering and analytics. You will manage cross-functional teams to execute data platform, pipeline, and analytics initiatives, ensuring alignment with business goals and timely delivery. About the Role : A short paragraph summarizing the key role...

  • Senior Azure Data Engineer

    3 weeks ago

    Anand, India Delphi Consulting Middle East Full time

    Ready to embark on a journey where your growth is intertwined with our commitment to making a positive impact? Join the Delphi family - where Growth Meets Values.At Delphi Consulting Pvt. Ltd., we foster a thriving environment with a hybrid work model that lets you prioritize what matters most. Interviews and onboarding are conducted virtually, reflecting...