Data Engineer

3 weeks ago

Bikaner, India Aceolution Full time

Job Title: Data Engineer – Python Expert(Freelance Role)Location: Remote / HybridEmployment Type: Contract/ FreelanceRole Summary We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning.Key Responsibilities Architect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets. Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training.Data Transformation: Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks. Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle. Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs.Required Qualifications 8+ years of professional experience in data engineering, data processing, or backend software engineering. Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars). Proven experience building and maintaining large-scale data pipelines. Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing). Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale. Excellent problem-solving skills and a meticulous attention to detail. Strong communication and collaboration skills, with experience working in a team environment.Preferred Qualifications (Nice-to-Haves) Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family). Strong experience with big data frameworks like Apache Spark or Ray. Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers). Familiarity with ML frameworks like PyTorch or TensorFlow. Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services.Why Join UsOpportunity to lead cutting-edge AI and ML projects.Collaborative and innovative team culture.Competitive compensation with continuous learning opportunities.📩 If you are interested, please share your updated CV to sharmila@aceolution.com along with your expected rate per hour.

  • Data Engineer

    3 weeks ago

    Bikaner, India YO HR Consultancy Full time

    Data Engineer/Senior Data EngineerLocation: ChennaiExperience: 2 - 8 YearsRole DescriptionThe Data Engineer will be responsible for designing and developing data engineering solutions, creating and maintaining data models, building Extract Transform Load (ETL) pipelines, implementing data warehousing, and performing data analytics on large...

  • Data Engineer

    4 weeks ago

    Bikaner, India DraconX Full time

    Company Description DraconX specializes in creating intelligent, scalable digital solutions that drive growth and innovation for startups and enterprises. As pioneers in AI business automation and SaaS platforms, we excel in building MVPs, custom software, and automation systems that simplify operations and enhance decision-making. Leveraging expertise in...

  • Data Engineering Specialist

    3 days ago

    bikaner, India beBeeDataEngineer Full time

    Job TitlePlatform Engineer – PythonKey Responsibilities:Design and develop scalable data pipelines and platform services using Python, Apache Spark, and Databricks.Develop and manage notebook-based workflows (Jupyter, Databricks, etc.) for data processing, analysis, and automation.Build and optimize data integration workflows and reusable components for...

  • Data Engineering Visionary

    3 days ago

    bikaner, India beBeeDataEngineering Full time

    Data Engineering ManagerLead and mentor data engineering teams to drive high-quality delivery and best practices.Design, implement, and optimize scalable data pipelines for high-volume data ingestion, transformation, and storage.Develop robust crawling systems using modern frameworks, handling IP rotation, throttling, and dynamic content extraction.Establish...

  • Data Engineering Expert

    20 hours ago

    bikaner, India beBeeData Full time

    About the Role:The Data Specialist evaluates learners' technical proficiency in data engineering, analytics, and cloud data platforms. This role involves comprehensive assessment across SQL, data pipeline development, data modeling, and visualization capabilities, while providing actionable feedback to accelerate learner development. The specialist...

  • Senior Data Engineer

    12 hours ago

    bikaner, India beBeeData Full time

    Job Title: Data AnalystJob Overview:We are seeking a highly skilled professional to join our analytics team. The ideal candidate will design, build, and deploy data-driven solutions leveraging machine learning and cloud-based AI platforms.Main Responsibilities:Develop and optimize machine learning models using Vertex AI tools.Build scalable data pipelines...

  • Senior Data Engineer

    19 hours ago

    bikaner, India beBeeDataScience Full time

    Data Science SpecialistBayone Solutions is seeking a highly experienced Data Scientist to drive innovation within our microservices architecture.We are looking for someone who can lead the development, optimization, and maintenance of data pipelines and back-end services, ensuring seamless integration within a microservices environment.Design, develop, and...

  • Data Engineering Role

    2 weeks ago

    Bikaner, India 100x.inc Full time

    Minimum Requirements: - At least 3 years of professional experience in Data Engineering - Demonstrated end-to-end ownership of ETL pipelines - Deep, hands-on experience with AWS services: EC2, Athena, Lambda, and Step Functions (non-negotiable) - Strong proficiency in MySQL (non-negotiable) - Working knowledge of Docker: setup, deployment, and...

  • Chief Data Engineering Strategist

    3 days ago

    bikaner, India beBeeDataEngineer Full time

    Unlock the Power of Data EngineeringBecoming a leading expert in data engineering is an exceptional opportunity to shape the future of our organization. As a skilled data engineer, you will play a pivotal role in driving innovation within our cutting-edge microservices architecture.Your key responsibilities will include:Designing and Implementing Data...

  • Data Platform Engineer

    3 days ago

    bikaner, India beBeeDatabricks Full time

    Job TitleThe successful candidate will be responsible for implementing and maintaining a cloud-based data platform on AWS, Azure or GCP.Key Responsibilities:Implement and maintain the cloud-based data platform, including workspace setup, user and group management, access control, and security configurations.Provide technical support to data engineering and...