Data engineer

1 week ago

Hyderabad, India ValueLabs Full time
Job Description:
JOB LOCATION - DUBAI UAE
• Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using Py Spark on the Cloudera Data Platform, ensuring data integrity and accuracy.
• Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
• Data Transformation and Processing: Use Py Spark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
• Performance Optimization: Conduct performance tuning of Py Spark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
• Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
• Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
• Monitoring and Maintenance: Monitor pipeline performance, troubleshoot issues, and perform routine maintenance on the Cloudera Data Platform and associated data processes.
• Collaboration: Work closely with other data engineers, analysts, product managers, and other stakeholders to understand data requirements and support various data-driven initiatives.
• Documentation: Maintain thorough documentation of data engineering processes, code, and pipeline configurations.
Qualifications:
Education and Experience:
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 3+ years of experience as a Data Engineer, with a strong focus on Py Spark and the Cloudera Data Platform.
Technical Skills:
• Py Spark: Advanced proficiency in Py Spark, including working with RDDs, Data Frames, and optimization techniques.
• Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
• Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
• Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
• Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
• Scripting and Automation: Strong scripting skills in Linux.
Soft Skills:
• Strong analytical and problem-solving skills.
• Excellent verbal and written communication abilities.
• Ability to work independently and collaboratively in a team environment.
• Attention to detail and commitment to data quality.

  • Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Full time

    Job Description Role Definition:Data engineers build and test scalable Big Data ecosystems for the businesses and help build data systems that are stable and highly optimized. Data engineers also update the existing systems with newer or upgraded versions of the current technologies to improve the efficiency of the databases, data storage layers and process...

  • Data Engineer

    3 months ago

    Hyderabad, India NTT DATA Services Full time

    Role Definition:Data engineers build and test scalable Big Data ecosystems for the businesses and help build data systems that are stable and highly optimized. Data engineers also update the existing systems with newer or upgraded versions of the current technologies to improve the efficiency of the databases, data storage layers and process the data...

  • Innovative Big Data Solutions Engineer

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    About the RoleWe are seeking an experienced Big Data Solutions Engineer to join our team in Hyderabad, India. As a key member of our engineering team, you will be responsible for designing and implementing complex data products components based on requirements with possible technical solutions.You will work closely with Data Scientists, Data Engineers,...

  • Advanced Data Analytics Engineer with Cloud Expertise

    3 weeks ago

    Hyderabad, Telangana, India data Full time

    Unlock the Power of Big DataWe are seeking an experienced Advanced Data Analytics Engineer to join our team in Hyderabad, India. As a key member of our engineering team, you will play a crucial role in designing and implementing innovative data products using agile methodologies and open-source technologies.About the RoleThis is an exciting opportunity to...

  • Data Intelligence Specialist

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    About the RoleWe are seeking a highly skilled Data Intelligence Specialist to join our engineering team in Hyderabad, India.The ideal candidate will have a strong passion for data analysis and quality monitoring, with a deep understanding of complex datasets and statistical measurements.Key ResponsibilitiesTake full ownership of data.ai core intelligence...

  • Senior Data Engineer for Digital Transformation

    3 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description:We are seeking a highly skilled Senior Data Engineer to join our team in Remote, Telangana (IN). This is a full-time position with 40 hours per week.About the Role:The successful candidate will design, develop, and maintain scalable data pipelines in the Python ecosystem. They will also participate in regular team ceremonies and contribute to...

  • SAP Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Full time

    Job Description Req ID: 302158 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a SAP Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Mandatory...

  • Senior Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Business Solutions Full time

    ASenior Data Engineerplays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes :Develop and maintain scalable data pipelines.Create efficient Extract,...

  • Senior Data Engineer

    1 month ago

    hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets. Responsibilities: Designing and Implementing ETL Processes: Develop and maintain scalable data pipelines. Create efficient Extract,...

  • Senior data engineer

    2 months ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets. Responsibilities: Designing and Implementing ETL Processes : Develop and maintain scalable data pipelines. Create efficient...

  • Senior Data Engineer

    2 months ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets. Responsibilities: Designing and Implementing ETL Processes : Develop and maintain scalable data pipelines. Create efficient...

  • Senior Data Engineer

    1 month ago

    hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes:Develop and maintain scalable data pipelines.Create efficient Extract,...

  • Senior Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes:Develop and maintain scalable data pipelines.Create efficient Extract,...

  • Senior Data Engineer

    2 months ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes:Develop and maintain scalable data pipelines.Create efficient Extract,...

  • Senior Data Engineer

    2 months ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes:Develop and maintain scalable data pipelines.Create efficient Extract,...

  • Senior Data Engineer

    2 months ago

    Hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets. Responsibilities: Designing and Implementing ETL Processes : Develop and maintain scalable data pipelines. Create efficient Extract,...

  • Senior Data Engineer

    2 months ago

    hyderabad, India NTT DATA Business Solutions Full time

    A Senior Data Engineer plays a pivotal role in technology-driven organizations, responsible for designing, building, and managing the infrastructure and tools that enable efficient processing and analysis of large data sets.Responsibilities:Designing and Implementing ETL Processes:Develop and maintain scalable data pipelines.Create efficient Extract,...

  • SAP Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 302158 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a SAP Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Mandatory...

  • Data Center Network Infrastructure Engineer

    3 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Job SummaryWe are seeking a highly skilled Data Center Network Infrastructure Engineer to join our team in Hyderabad, India. The successful candidate will be responsible for designing and implementing network infrastructure solutions for our clients.

  • Data Designer

    15 hours ago

    Hyderabad, Telangana, India NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT...