Big Data Engineer

5 days ago

Kochi, India NOV Full time

This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects.

As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong communication skills and the capability to drive projects forward autonomously. The ideal person for this position loves working with data and strives to optimize large scale data pipelines in support of analytics workloads. The ideal person should excel at solving complex, multi-dimensional data problems and love working as part of a global team.


Key Responsibilities:


  1. Lead and manage assigned projects, providing guidance and support to others on team you are supporting.
  2. Designing and implementing highly performant data ingestion pipelines from multiple sources.
  3. Evaluate and choose appropriate data engineering tools and technologies where necessary.
  4. Optimize data infrastructure and pipelines for maximum efficiency.
  5. Working with event based / streaming technologies to ingest and process data.
  6. Build large scale data migration scripts to migrate big data from native files in OSI PI and Timescale database into Databricks Data Lake.
  7. Develop automated CICD pipelines for optimally deploying data pipelines and data science workloads.
  8. Contribute to development and maintain various streamlit applications.
  9. Collaborate closely and independently with data scientists to provide data for analytics and machine learning.
  10. Implement best practices for data engineering and promote a culture of continuous improvement.


Qualifications and Skills:

  1. Strategic thinking, excellent communication skills and the ability to align data engineering with business goals.
  2. Advanced understanding with spark and/or Databricks(preferred), including spark cluster tuning.
  3. Great understanding of distributed systems and partitioning.
  4. Knowledge in Databricks and AWS data analytics tooling.
  5. Knowledge of Data Science life cycle.
  6. Advanced knowledge in SQL and other languages such as Python, Scala, R or Java.
  7. Experience with developing streamlit in Python applications.
  8. Visualization/reporting tools such as Tableau, Spotfire, Grafana, etc.
  9. Understanding of CI/CD practices and technologies. GitHub and GitHub Actions(preferred), Jenkins, TeamCity, Udeploy, etc.


Optional but Beneficial Skills:

  • Knowledge of Terraform
  • Snowflake
  • Some experience in building ML/AI models (worked with packages such as scikit learn, tensorflow, openai, etc.)
  • Kafka
  • Matillion


Work Environment and Conditions:

  • Flexible and ambition to learn new skills quickly
  • Ability to work independently
  • Ability to mentor others

  • Big Data Engineer

    5 days ago

    kochi, India NOV Full time

    This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects.As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong communication skills and the...

  • Big Data Engineer

    6 days ago

    Kochi, India NOV Full time

    This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects.As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong communication skills and the...

  • Big Data Engineer

    4 days ago

    Kochi, India NOV Full time

    This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects.As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong communication skills and the...

  • Software Engineer-Big Data, Analytics Development

    2 months ago

    Kochi, India Strands Energy Pvt. Ltd Full time

    Software Engineer-Big Data, Analytics Development JOB DESCRIPTION We are on the lookout for talented Software Engineers who have experience in big-data applications. If you are passionate about designing, data analytics, coding and would like to challenge yourself with exciting work in an emerging domain, join us! As a Software developer at Strands...

  • Software Engineer-Big Data, Analytics Development

    2 months ago

    kochi, India Strands Energy Pvt. Ltd Full time

    Software Engineer-Big Data, Analytics Development JOB DESCRIPTIONWe are on the lookout for talented Software Engineers who have experience in big-data applications. If you are passionate about designing, data analytics, coding and would like to challenge yourself with exciting work in an emerging domain, join us!As a Software developer at...

  • Software Engineer-Big Data, Analytics Development

    3 weeks ago

    Kochi, India Strands Energy Pvt. Ltd Full time

    Software Engineer-Big Data, Analytics Development JOB DESCRIPTION We are on the lookout for talented Software Engineers who have experience in big-data applications. If you are passionate about designing, data analytics, coding and would like to challenge yourself with exciting work in an emerging domain, join us! As a Software developer at Strands...

  • Software Engineer-Big Data, Analytics Development

    3 weeks ago

    kochi, India Strands Energy Pvt. Ltd Full time

    Software Engineer-Big Data, Analytics Development JOB DESCRIPTIONWe are on the lookout for talented Software Engineers who have experience in big-data applications. If you are passionate about designing, data analytics, coding and would like to challenge yourself with exciting work in an emerging domain, join us!As a Software developer at...

  • Big Data Engineer

    4 days ago

    kochi, India NOV Full time

    Purpose of the Position:This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects. As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong...

  • Big Data Engineer

    5 days ago

    Kochi, India NOV Full time

    Purpose of the Position:This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects. As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong...

  • ▷ Urgent: Big Data Engineer

    2 days ago

    Kochi, India NOV Full time

    Purpose of the Position:This role is part of the Digital Technology Services team at NOV, a corporate group dedicated to a wide array of data and analytics projects. As a member of this team, you will have the opportunity to work with diverse technologies and collaborate with various departments. This is a highly independent position, demanding strong...

  • Data engineer

    2 weeks ago

    Kochi, India Zaportiv Full time

    Experienced Data Engineer with over 3 years of hands-on experience in developing, testing, and implementing robust data pipelines.Proficient in SQL and Python, adept at leveraging these tools to extract, transform, and load data efficiently.Demonstrated expertise in data analysis and manipulation, utilizing advanced techniques to derive actionable insights...

  • Big Data Engineer

    3 weeks ago

    Kochi, India SPG Consulting Solutions Pvt.Ltd Full time

    Location : Chennai, Pune, Noida, Kochi, Hyderabad, Trivandrum.Job Description :- Proficient in SQL, Spark, Scala, and AWS, with a strong command over these technologies.- Minimum 6-7 years of relevant experience in Spark and SQL, plus 2-3 years of hands-on practice in AWS.- Demonstrated track record of effectively optimizing performance.- Adept at leveraging...

  • Lead Azure Data Engineer

    1 week ago

    Kochi, India Pearlsoft Technologies LLP Full time

    Lead Azure Data EngineerExperience ?10+ years of hands-on work experienceResponsibilities?Design and develop business-critical backend systems using high-quality data pipelines.?Assemble large, complex data sets that meet functional / non-functional business requirements.?Build the infrastructure required for optimal extraction, transformation, and loading...

  • Data Engineer

    4 weeks ago

    Kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data Engineer About Mindcurv We help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer...

  • Data Engineer

    4 weeks ago

    kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data Engineer About MindcurvWe help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer...

  • Data Engineer

    3 weeks ago

    Kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data Engineer About Mindcurv We help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer...

  • Data Engineer

    3 weeks ago

    kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data Engineer About MindcurvWe help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer...

  • Data Engineer

    4 weeks ago

    Kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data EngineerAbout MindcurvWe help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer experiences and...

  • Data Engineer

    3 weeks ago

    Kochi, India Mindcurv Technology Solutions Pvt. Ltd. Full time

    Data EngineerAbout MindcurvWe help our customers rethink their digital business, experiences, and technology to navigate the new digital reality. We do this by designing sustainable and accountable solutions for humans living in a digital world. Mindcurv holistically covers the market’s need to digitalise business processes and customer experiences and...

  • Lead Azure Data Engineer

    1 week ago

    kochi, India Pearlsoft Technologies LLP Full time

    Lead Azure Data Engineer Experience ?10+ years of hands-on work experienceResponsibilities?Design and develop business-critical backend systems using high-quality data pipelines.?Assemble large, complex data sets that meet functional / non-functional business requirements.?Build the infrastructure required for optimal extraction,...