Big Data Developer

1 month ago

gurugram, India LTIMindtree Full time

PySpark Data Engineer with ML Ops

Experience - 5 to 10 Years

Location - Bangalore (Initial 3 months deputation in Gurgaon) / Rest of India (Initial 3 months deputation in Gurgaon)

Primary - PySpark, Spark, Scala

AWS S3, EMR

Good in troubleshooting on Kubernetes/Docker

Airflow, Jenkins

Model Deployment Experience

What you'll do:

You will work in a cross-functional team of Machine Learning engineers and ML Scientists to design and code large scale batch and real-time data pipelines on the AWS.

Prototype creative solutions quickly by developing minimum viable products and work with seniors and peers in crafting and implementing the technical vision of the team

Communicate and work effectively with geographically distributed cross functional teams

Participate in code reviews to assess overall code quality and flexibility

Resolve problems and roadblocks as they occur with peers and help unblock junior members of the team. Follow through on details and drive issues to closure

Define, develop and maintain artifacts like technical design or partner documentation

Drive for continuous improvement in software and development process within an agile development team

Participate in user story creation in collaboration with the team

Support and troubleshoot data and/or system issues as needed

Who you are:

You have a degree in software engineering, computer science, informatics or a similar field

You are comfortable programming in Python and Scala.

You have hands-on experience with OOAD, design patterns, SQL and NoSQL

You are knowledgeable in big data technologies, in particular Spark, Hive, Hue, Qubole and Databricks

You have experience in crafting real-time streaming applications, preferably in Spark streaming, and Kafka/KStreams

You have a good understanding of machine learning pipelines and ML Lifecycle.

Experience in using cloud services (e.g. AWS)

You have experience with workflow orchestration tools (e.g. Airflow)

You are passionate about learning, especially in the areas of micro-services, system architecture, Data Science and Machine Learning.

You have experience working with Agile/Scrum methodologies

  • Big Data

    1 week ago

    Gurugram, India Virtusa Full time

    Big Data - Informatica Developer - CREQ188335 Description Must have 6-9 year of experience of create, design, and implement data processing jobs to transform the data. Demonstrate proficiency in writing shell script and SQL queries. Hands on Exp with Oracle and DB2 databases. Must have Big Data tech stack experience Hadoop, Hive, Mapr etc. Good...

  • Big Data

    1 week ago

    gurugram, India Virtusa Full time

    Big Data - Informatica Developer - CREQ188335 Description Must have 6-9 year of experience of create, design, and implement data processing jobs to transform the data.Demonstrate proficiency in writing shell script and SQL queries.Hands on Exp with Oracle and DB2 databases.Must have Big Data tech stack experience Hadoop, Hive, Mapr etc.Good Communication...

  • Big Data

    1 week ago

    gurugram, India Virtusa Full time

    Big Data - Informatica Developer - CREQ188336 Description Must have 6-9 year of experience of create, design, and implement data processing jobs to transform the data.Demonstrate proficiency in writing shell script and SQL queries.Hands on Exp with Oracle and DB2 databases.Must have Big Data tech stack experience Hadoop, Hive, Mapr etc.Good Communication...

  • Big Data

    1 week ago

    Gurugram, India Virtusa Full time

    Big Data - Informatica Developer - CREQ188336 Description Must have 6-9 year of experience of create, design, and implement data processing jobs to transform the data. Demonstrate proficiency in writing shell script and SQL queries. Hands on Exp with Oracle and DB2 databases. Must have Big Data tech stack experience Hadoop, Hive, Mapr etc. Good...

  • Big Data Developer

    1 month ago

    Gurugram, India IntraEdge Full time

    Big Data Engineer Position Available in Gurgaon We are looking for a Big Data Engineer to join our team in Gurgaon. The ideal candidate should have a Bachelor's degree in engineering or computer science (or equivalent) OR a Master's in computer applications (or equivalent).The candidate will work closely with the engineering team and product owners, so...

  • Big Data Developer

    1 month ago

    Gurugram, India IntraEdge Full time

    🔴 Big Data Engineer Position Available in Gurgaon 🔴We are looking for a Big Data Engineer to join our team in Gurgaon. The ideal candidate should have a Bachelor's degree in engineering or computer science (or equivalent) OR a Master's in computer applications (or equivalent).The candidate will work closely with the engineering team and product owners,...

  • Big Data Developer

    1 month ago

    gurugram, India IntraEdge Full time

    Big Data Engineer Position Available in Gurgaon We are looking for a Big Data Engineer to join our team in Gurgaon. The ideal candidate should have a Bachelor's degree in engineering or computer science (or equivalent) OR a Master's in computer applications (or equivalent). The candidate will work closely with the engineering team and product owners, so...

  • Java Big Data Developer

    1 week ago

    gurugram, India Virtusa Full time

    Java Big Data Developer - CREQ188170 Description Responsibilities include, but are not limited to - Develops and tests software, including ongoing refactoring of code, and drives continuous improvement in code structure and quality Primary focus is spent writing code, API specs, conducting code reviews and testing in ongoing sprints, or doing proof of...

  • Java Big Data Developer

    2 weeks ago

    Gurugram, India Virtusa Full time

    Java Big Data Developer - CREQ188170 Description Responsibilities include, but are not limited to - Develops and tests software, including ongoing refactoring of code, and drives continuous improvement in code structure and quality Primary focus is spent writing code, API specs, conducting code reviews and testing in ongoing sprints, or doing proof of...

  • Big Data Developer

    1 month ago

    gurugram, India Rocket Consulting Full time

    Big Data EngineerPosition Overview :We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems using cutting-edge technologies. As a Big Data Engineer, you will play a key role in building and optimizing data pipelines, data...

  • Big Data Developer

    2 weeks ago

    Gurugram, India Rocket Consulting Full time

    Big Data EngineerPosition Overview :We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems using cutting-edge technologies. As a Big Data Engineer, you will play a key role in building and optimizing data pipelines, data...

  • Big Data/Scala Developer

    2 weeks ago

    Gurugram, India Waytogo Consultants Full time

    Job Description :In this role, you will play a critical part in designing, developing, and maintaining big data processing pipelines using your expertise in Scala and Apache Spark. You will collaborate closely with data engineers and data scientists to understand their data needs and translate those requirements into robust technical...

  • Big Data/Scala Developer

    1 month ago

    gurugram, India Waytogo Consultants Full time

    Job Description : In this role, you will play a critical part in designing, developing, and maintaining big data processing pipelines using your expertise in Scala and Apache Spark. You will collaborate closely with data engineers and data scientists to understand their data needs and translate those requirements into robust technical...

  • Big Data Developer

    4 weeks ago

    Gurgaon/Gurugram/Bangalore, IN Workassist.in Full time

    Our hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...

  • Big Data Developer

    2 weeks ago

    Gurgaon/Gurugram/Bangalore, India Workassist.in Full time

    Our hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...

  • Big Data Developer

    2 weeks ago

    Gurgaon/Gurugram, India Rocket Consulting Full time

    Big Data EngineerPosition Overview :We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems using cutting-edge technologies. As a Big Data Engineer, you will play a key role in building and optimizing data pipelines, data...

  • Big Data Developer

    4 weeks ago

    Gurgaon/Gurugram, IN Rocket Consulting Full time

    Big Data EngineerPosition Overview :We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems using cutting-edge technologies. As a Big Data Engineer, you will play a key role in building and optimizing data pipelines, data...

  • Big Data/Scala Developer

    4 weeks ago

    Gurgaon/Gurugram, IN Waytogo Consultants Full time

    Job Description :In this role, you will play a critical part in designing, developing, and maintaining big data processing pipelines using your expertise in Scala and Apache Spark. You will collaborate closely with data engineers and data scientists to understand their data needs and translate those requirements into robust technical...

  • Big Data/Scala Developer

    2 weeks ago

    Gurgaon/Gurugram, India Waytogo Consultants Full time

    Job Description : In this role, you will play a critical part in designing, developing, and maintaining big data processing pipelines using your expertise in Scala and Apache Spark. You will collaborate closely with data engineers and data scientists to understand their data needs and translate those requirements into robust technical...

  • Senior Big Data Software Engineer

    1 month ago

    gurugram, India Epam Full time

    Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects...