Data Engineer

15 hours ago

guntur, India Aceolution Full time

Job Title: Data Engineer – Python Expert(Freelance Role)Location: Remote / HybridEmployment Type: Contract/ FreelanceRole Summary We are looking for a seasoned Senior Data Engineer to architect, build, and own the data pipelines that power our large language model (LLM) development. As a senior Individual Contributor (IC), you will be the team's expert on data ingestion, processing, and quality for all AI training. Your primary mission is to build scalable, automated systems that transform massive, raw datasets into pristine, model-ready formats. While your focus will be on data engineering, your expertise will be valued in collaborating on model training runs and experiments. You're the perfect fit if you are a Python expert who thrives on solving large-scale data challenges and enjoys working at the intersection of data engineering and machine learning.Key Responsibilities Architect & Build: Design, develop, and own robust, scalable, and automated ETL/ELT pipelines in Python for ingesting and processing terabyte-scale text datasets. Data Quality: Implement rigorous data cleaning, deduplication, filtering, and normalization strategies. Define and enforce data quality standards to ensure the highest integrity for model training.Data Transformation: Efficiently structure and format diverse datasets (JSON, Parquet, etc.) for consumption by LLM training frameworks. Collaboration: Work closely with our team of AI researchers and ML engineers to understand data requirements, define metrics, and support the model training lifecycle. Optimization: Continuously optimize data processing workflows for speed, cost, and reliability.ML Support (Secondary): Occasionally assist in launching, monitoring, and debugging data-related issues during model training runs.Required Qualifications 8+ years of professional experience in data engineering, data processing, or backend software engineering. Expert-level proficiency in Python and its data ecosystem (e.g., Pandas, NumPy, Dask, Polars). Proven experience building and maintaining large-scale data pipelines. Deep understanding of data structures, data modeling, and software engineering best practices (Git, CI/CD, testing). Experience handling and parsing diverse data formats (JSON, CSV, XML, Parquet) at scale. Excellent problem-solving skills and a meticulous attention to detail. Strong communication and collaboration skills, with experience working in a team environment.Preferred Qualifications (Nice-to-Haves) Hands-on experience with the data preprocessing pipeline for an LLM (e.g., LLaMA, BERT, GPT-family). Strong experience with big data frameworks like Apache Spark or Ray. Experience with Hugging Face libraries (Transformers, Datasets, Tokenizers). Familiarity with ML frameworks like PyTorch or TensorFlow. Proficiency with cloud platforms (AWS, GCP, Azure) and their data/storage services.Why Join UsOpportunity to lead cutting-edge AI and ML projects.Collaborative and innovative team culture.Competitive compensation with continuous learning opportunities. If you are interested, please share your updated CV to sharmila@aceolution.com along with your expected rate per hour.

  • Data Engineer

    2 weeks ago

    Guntur, India BayOne Solutions Full time

    The Opportunity:We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engineering and back-end systems. You will be responsible for leading the development, optimization, and maintenance of both data pipelines and back-end...

  • Data Engineer

    2 weeks ago

    Guntur, India BayOne Solutions Full time

    The Opportunity:We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engineering and back-end systems. You will be responsible for leading the development, optimization, and maintenance of both data pipelines and back-end...

  • Data Engineer

    4 days ago

    Guntur, India MRF Full time

    Azure/SQL/Application - Data Engineer Job description: 1. Responsible to maintain the data required for managed application like Advanced Planning System, Dealer Management System, etc., 2. Responsible for user access management on data upload and download 3. Working with functional team to manage the required data in a structured manner 4. Make use of Azure...

  • Cloud Data Engineer

    11 hours ago

    guntur, India beBeeData Full time

    Cloud Data Engineer Job DescriptionWe are seeking an experienced Cloud Data Engineer to manage and maintain our cloud-based data infrastructure.Job Responsibilities:Design, implement, and maintain scalable data pipelines using Azure services such as Data Factory, Storage, Synapse, and Key Vault.Develop and manage Databricks clusters for efficient data...

  • Data Engineer Role

    7 hours ago

    guntur, India beBeeDataEngineer Full time

    Data Engineering OpportunityThe ideal candidate will have a strong background in data engineering, with experience in designing and implementing scalable data pipelines for ingesting, transforming, and activating customer data.We are looking for a professional who can develop and orchestrate workflows using Apache Airflow and Spark.Experience with AWS...

  • Data Engineer(Intern)

    1 week ago

    Guntur, India Tech Phoenix Full time

    Company DescriptionTech Phoenix is a dynamic platform dedicated to all things technology, offering the latest news, trends, and insights from the tech industry. Whether you are a tech enthusiast or a professional, our website features engaging articles, exclusive interviews, and resourceful guides to keep you informed and inspired. We foster an interactive...

  • Data Engineer(Intern)

    1 week ago

    Guntur, India Tech Phoenix Full time

    Company DescriptionTech Phoenix is a dynamic platform dedicated to all things technology, offering the latest news, trends, and insights from the tech industry. Whether you are a tech enthusiast or a professional, our website features engaging articles, exclusive interviews, and resourceful guides to keep you informed and inspired. We foster an interactive...

  • Data Engineer

    3 days ago

    guntur, India beBeeData Full time

    Job Title: Data SpecialistAbout the Role:We are seeking an experienced Data Specialist to implement end-to-end GenAI powered RAG & multi-agent systems.The ideal candidate will provide guidance on system architecture & components, as well as build TTD system components such as LLM Agents, Routing, State management, Vector stores and more generally context...

  • Senior Data Engineer

    2 days ago

    guntur, India beBeeData Full time

    Key Engineering RolesThis position involves multiple facets of data engineering including design, development and maintenance of scalable solutions.Create large-scale ETL/ELT pipelines using a mix of AWS services, Snowflake, Kafka, Airflow and GitHub.Ingest various types of data from different sources such as APIs, databases, files etc.Implement real-time...

  • Data Engineering Specialist

    2 days ago

    guntur, India beBeeDataEngineer Full time

    OverviewWe are seeking a skilled professional to contribute to the design and development of data systems, including pipelines, APIs, analytics, AI, and machine learning at scale.The ideal candidate will have hands-on experience in writing high-quality, scalable, and maintainable code across data pipelines, APIs, and data platform services. They will also...