Big Data Engineer

3 weeks ago

Anywhere in IndiaMultiple Locations, IN D-TechWorks Pvt Ltd Full time

We are hiring for Big Data AI/ML Engineer - Remote , India

Position : Big Data AI/ML Engineer

Experience : 6 to 9 Yrs

Location : Remote

We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology improvements. Specifically, you will be working on projects involving leveraging AI/ML for Data Management Efficiencies & Query Collaborate with cross-functional teams such as Data Scientists, Product Partners and Partner Team Developers to identify oppurtunities for Big Data, Query (Spark, Hive SQL, BigQuery, SQL) tuning opportunities that can be solved using machine learning and generative AI.

- Write clean, high-performance, high-quality, maintainable code.

- Create backend applications using Python, docker, Google Cloud & in-house ML frameworks to orchestrate end to end applications

- Design and develop Big Data Engineering Solutions & generative AI Applications for above ensuring scalability, efficiency, and maintainability of such solutions.

- Implement prompt engineering techniques to fine-tune and enhance LLMs for better performance and application-specific needs.

- Stay abreast of the latest advancements in the field of Generative AI Application Development and actively contribute to the research and development of new Generative AI A Master's or Ph.D. degree in Computer Science or a related field.

- Proven experience working as a Big Data & MLOps Engineer, with a focus on Python, Google Cloud, Spark, Spark SQL, BigQuery and Generative AI Applications.

- Deep understanding and experience in tuning Dataproc, BigQuery, Spark Applications.

- Solid knowledge of software engineering best practices, including version control systems (e.g., Git), code reviews, and testing methodologies.

- Experience with large language models (LLMs) & prompt engineering techniques, vector DB's ( Nice to have )

- Strong communication skills to effectively collaborate and present findings to both technical and non-technical stakeholders.

- Proven ability to adapt and learn new technologies and frameworks quickly.

- A proactive mindset with a passion for continuous learning and research.

(ref:hirist.tech)

  • Lead Big Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Grownex HR Solution Full time

    Role : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Lead Big Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Grownex HR Solution Full time

    Role : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Big Data Architect

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Architect

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Quality Assurance Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Quality Assurance Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data/Databricks Engineer

    2 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN WEN Full time

    Role : Big Data + DatabricksResponsibilities :- Design, develop, and implement scalable Big Data solutions using the Hadoop ecosystem.- Work closely with data scientists and analysts to understand their requirements and translate them into technical solutions.- Develop and optimize Spark and Hive scripts to process, transform, and analyze large datasets.-...

  • GCP Data Architect

    2 months ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Role : GCP Data Architect Description : As a GCP Data Architect, you will be responsible for architecting enterprise data solutions for migration, transformation or modernization on Google Cloud Platform (GCP) integrating native GCP services and other 3rd party data solutions. You will collaborate with cloud partners and our cross-functional teams to enable...

  • GCP Data Architect

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Huquo Full time

    Role : GCP Data Architect Description : As a GCP Data Architect, you will be responsible for architecting enterprise data solutions for migration, transformation or modernization on Google Cloud Platform (GCP) integrating native GCP services and other 3rd party data solutions. You will collaborate with cloud partners and our cross-functional teams to enable...

  • Big Data Architect

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Big Data Architect

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Hadoop Administrator

    2 months ago

    Metros/Anywhere in India/Multiple Locations, IN Dotflick Solutions Full time

    Designation : Hadoop AdministratorOpen for Pan India(Any Location)Job Description :- Development and deployment of data applications- Design Implementation of infrastructure tooling and work on horizontal frameworks and libraries- Creation of data ingestion pipelines between legacy data warehouses and the big data stack- Automation of application back-end...

  • Big Data Architect

    2 months ago

    Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full time

    Job description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...

  • Big Data Architect

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full time

    Job description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...

  • Data Engineer

    2 months ago

    Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full time

    Basic understanding of :- Scheduling and workflow management & working experience in either ADF, Informatica, Airflow, or Similar- Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner, or Similar- Logical/Physical model on Big Data sets or modern data warehouse & working experience in ERwin, ER/Studio,...

  • Data Engineer

    3 weeks ago

    Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full time

    Basic understanding of :- Scheduling and workflow management & working experience in either ADF, Informatica, Airflow, or Similar- Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner, or Similar- Logical/Physical model on Big Data sets or modern data warehouse & working experience in ERwin, ER/Studio,...

  • Aptus Data Labs

    1 month ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...