Data Engineer

4 days ago

Amritsar, India Whatjobs IN C2 Full time

Job Title: Data Engineer – Python Expert(Freelance Role) Location: Remote / Hybrid Employment Type: Contract/ Freelance Role Summary We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning. Key Responsibilities Architect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets. Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training. Data Transformation : Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks. Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle. Optimization: Continuously optimize data processing workflows for speed, cost, and reliability. ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs. Required Qualifications 8+ years of professional experience in data engineering, data processing, or backend software engineering. Expert-level proficiency in Python and its data ecosystem (e.G., Pandas, NumPy, Dask, Polars). Proven experience building and maintaining large-scale data pipelines. Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing). Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale. Excellent problem-solving skills and a meticulous attention to detail. Strong communication and collaboration skills, with experience working in a team environment. Preferred Qualifications (Nice-to-Haves) Hands-on experience with the data preprocessing pipeline for an LLM (e.G., LLaMA, BERT, GPT-family). Strong experience with big data frameworks like Apache Spark or Ray. Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers). Familiarity with ML frameworks like PyTorch or TensorFlow. Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services. Why Join Us Opportunity to lead cutting-edge AI and ML projects. Collaborative and innovative team culture. Competitive compensation with continuous learning opportunities. If you are interested, please share your updated CV to along with your expected rate per hour.

  • Data Engineer

    4 days ago

    Amritsar, India Whatjobs IN C2 Full time

    HI Folks Please check the JD and share your updated resume to my email and ping me on whatsapp ( ) along with your resume Data Engineer 100% Remote 1 year contract JOB DESCRIPTION A global law firm with nearly 1,400 lawyers and more than 3,000 employees across 19 offices in the United States, Europe, and Asia is looking to build out a data lake to encompass...

  • Harmony Data Integration Technologies

    2 weeks ago

    Amritsar, Punjab, India Harmony Data Integration Technologies Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Were hiring a Senior QA Engineer who can own test automation across data/ETL pipelines, APIs, and web frontends. Youll design robust QA strategies, build CI/CD-native test suites, drive performance/load testing, and turn system requirements into executable, measurable test plans.What will you do : ETL/Data Quality Automation : -...

  • Manager- Data Governance

    7 days ago

    Amritsar, India Bunge Full time

    Location : Mohali  City : Mohali  State : Punjab (IN-PB)  Country : India (IN)  Requisition Number : 39025  Business Title- Manager - Data Governance and Data Quality Global Job Title-Mgr I Strategy & Trans Global Function-Business Services Global Department-Strategy and Transformation Reporting to Data Governance Lead Role Purpose Statement-The...

  • Data Engineer Python/Scala

    38 minutes ago

    Amritsar, Punjab, India Debut Infotech Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Required Skills Python / Scala, spark/pyspark, Hive, AWS, EMR, S3, SQL, Airflow/ Github- 3-4 years experience developing Data & Analytics solutions- Experience building data lake solutions leveraging one or more ofthe followingAWS, EMR, S3, Hive & Spark- Experience with relational SQL- Experience with scripting languages such as Shell, Python- Experience...

  • AJO Dev

    6 days ago

    Amritsar, Punjab, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID:334481NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a AJO Dev to join our team in Punjab, Punjab (IN-PB), India (IN).Job Title:Senior Adobe Journey Optimizer (AJO)...

  • Test Engineer

    4 days ago

    Amritsar, India Whatjobs IN C2 Full time

    SUMMARY Under limited supervision designs, develops and maintains test procedures, tester hardware and software for electronic circuit board production. TECHNICAL MANAGEMENT RESPONSIBILITIES Review circuit board designs for testability requirements Support manufacturing with failure analysis, tester debugging, reduction of intermittent failures and downtime...

  • Junior Data Scientist

    7 days ago

    Amritsar, India Miratech Full time

    Job DescriptionWe are looking for a specialist to work with Large Language Models (LLMs) to design, develop, and integrate advanced AI solutions within our platform. The role involves automating data processing, generating actionable insights, and enhancing decision-making workflows. You will focus on building scalable, high-impact solutions that improve...

  • Quality Assurance Engineer

    4 weeks ago

    Amritsar, India Whatjobs IN C2 Full time

    Job Title - QA Lead Location - Remote Experience - 7-10 Years About the Role You’ll lead automation, performance, and data integrity initiatives — ensuring our product can handle thousands of customers, hundreds of thousands of users, and enterprise-grade expectations without breaking. This is a hands-on leadership role for someone who can architect...

  • Process Executive

    7 days ago

    Amritsar, India Bunge Full time

    Location : Mohali  City : Mohali  State : Punjab (IN-PB)  Country : India (IN)  Requisition Number : 39660  Job Description Business Title Process Executive - Enterprise Data Management (EDM) Global Function Business Services Global Department Enterprise Data Management (EDM) Reporting to Manager - EDM Role Purpose Statement Responsible for...

  • Trainee Engineers

    3 hours ago

    Amritsar, Punjab, India Naukri Healthcare Jobs Full time ₹ 4,00,000 - ₹ 6,00,000 per year

    Trainee Engineers 1 Pos based in Amritsar.The ideal candidate brings 0-1years and a strong record of GMP/cGMP compliance within regulated pharma, chemicals or biotech environments.Key responsibilities include ownership of day-to-day operations, documentation integrity, SOP creation/review, deviation/OOS handling, CAPA and change control management, audit...