Hadoop Development Lead

1 month ago


Hyderabad, Telangana, India ValueMomentum Full time

About the Role:

We are seeking an experienced Hadoop Data Engineer to join our team at ValueMomentum. The ideal candidate will have extensive experience in Hadoop development, strong knowledge of PySpark programming, and expertise in Hadoop, Hive framework.

Key Responsibilities:

  • Batch Processing Optimization: Analyze and optimize existing batch workflows to ensure efficient data processing.
  • Development & Scripting: Write and maintain batch scripts using Python, Hive, and Shell to automate data workflows.
  • Data Analysis: Utilize Hadoop and related technologies to gather, analyze, and report on data performance and processing metrics.
  • Troubleshooting: Identify and resolve data processing bottlenecks and workflow issues to minimize downtime and ensure smooth operations.
  • Collaboration: Work closely with data engineering, data science, and business intelligence teams to understand requirements and improve the data pipeline.
  • Documentation: Develop and maintain technical documentation for workflows, troubleshooting guides, and best practices.
  • Continuous Improvement: Propose and implement improvements for existing batch processes and pipelines.

Requirements:

  • 5-7 years of experience in batch processing optimization and data workflows, with a strong focus on Hadoop, Python, Hive, and Shell scripting.
  • Proficient in Hadoop ecosystem tools (HDFS, YARN, etc.).
  • Strong knowledge of Python for data processing and automation.
  • Advanced proficiency in Hive query language.
  • Expertise in writing Shell scripts for Linux/Unix environments.
  • Strong troubleshooting and performance analysis skills to optimize workflows.
  • Excellent communication and collaboration skills with a problem-solving mindset.
  • Familiarity with Agile methodologies.
  • Experience working in cross-functional teams.

  • Big Data Engineer

    2 months ago


    Hyderabad, Telangana, India TECHSTAR SOFTWARE DEVELOPMENT INDIA PVT LTD Full time

    Responsibilities of the Candidate :- Be responsible for the design and development of big data solutions. - Partner with domain experts, product managers, analysts, and data scientists to develop Big Data pipelines in Hadoop- Be responsible for moving all legacy workloads to a cloud platform- Work with data scientists to build Client pipelines using...

  • Data Engineer

    1 month ago


    Hyderabad, Telangana, India DBS Bank Full time

    Job SummaryDBS Bank is seeking a highly skilled Hadoop Data Engineer to join our Middle Office Technology team. As a key member of our team, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Hadoop and Spark technologies.Key ResponsibilitiesDesign and develop scalable data processing pipelines using...


  • Hyderabad, Telangana, India SMARTWORK IT SERVICES Full time

    Job SummaryAt SMARTWORK IT SERVICES, we are seeking a skilled Hadoop Developer to join our team. The ideal candidate will have strong expertise in Hadoop, MapReduce, and either Java or Python, along with Data Structures and Algorithms knowledge.Key Responsibilities- Develop and evolve highly scalable, fault-tolerant distributed components using Hadoop...


  • Hyderabad, Telangana, India ValueMomentum Full time

    About ValueMomentum's Engineering CenterAt ValueMomentum, we're a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through a strong engineering foundation and continuously refining our processes, methodologies, tools, agile delivery...


  • Hyderabad, Telangana, India ValueMomentum Full time

    Job Summary:We are seeking a skilled Hadoop Testing Engineer to join our team at ValueMomentum. As a Hadoop Testing Engineer, you will be responsible for testing products or systems to ensure they function properly and meet business needs.Key Responsibilities:Design, develop, and execute test plans and test cases for our Hadoop data ingestion...


  • Hyderabad, Telangana, India ValueMomentum Full time

    Job Summary:We are seeking a skilled Hadoop Tester to join our team at ValueMomentum. As a Hadoop Testing Engineer, you will be responsible for ensuring the quality and reliability of our data pipelines. Your primary focus will be on testing and validating our Hadoop data ingestion processes to guarantee the completeness, correctness, and timeliness of the...


  • Hyderabad, Telangana, India ValueMomentum Full time

    {"Responsibilities": "Key Responsibilities:Design, develop, and execute test plans and test cases for our Hadoop data ingestion pipelines.Ensure the completeness, correctness, and timeliness of data ingested into Hadoop. Validate source to target data mapping and transformations.Evaluate the performance and scalability of our data ingestion processes and...

  • Techstar Software

    2 months ago


    Hyderabad, Telangana, India Techstar Software Development India Pvt. Ltd. Full time

    A Big Data Engineer Lead plays a critical role in managing and guiding the development of big data solutions within an organization. Here's a typical job description for this role.Job Title : Big Data Engineer LeadExp : 5+ YearsLocation : HyderabadNotice Period : Immediate to 30daysJob Summary :The Big Data Engineer Lead is responsible for overseeing the...


  • Hyderabad, Telangana, India Virtusa Full time

    Virtusa seeks a seasoned Senior Big Data Developer to spearhead the development of large-scale data processing systems using Hadoop and Spark. As a key member of our team, you will design, implement, and deploy complex big data solutions that drive business growth and innovation.Job SummaryWe are looking for an experienced Senior Big Data Developer with a...


  • Hyderabad, Telangana, India ValueMomentum Full time

    Hadoop Data Ingestion ExpertiseWe are looking for a skilled Hadoop Testing Engineer to join our team at ValueMomentum. As a key member of our quality engineering team, you will be responsible for testing and validating our Hadoop data ingestion pipelines to ensure they meet the highest standards of quality and reliability.About the RoleDesign, develop, and...


  • Hyderabad, Telangana, India TECHSTAR SOFTWARE DEVELOPMENT INDIA PVT LTD Full time

    Unlock Your Potential as a Senior Data Engineering Project LeadAre you ready to take on a challenging role that combines your technical expertise with leadership skills? Do you have a passion for driving innovation and delivering high-quality results in data engineering projects?We are seeking an experienced Senior Data Engineering Project Lead to join our...


  • Hyderabad, Telangana, India Virtusa Full time

    About the RoleWe are seeking a highly skilled Big Data Engineer to join our team at Virtusa. As a key member of our engineering team, you will design, develop, and maintain large-scale data processing pipelines using big data technologies such as Hadoop, MapReduce, Hive, and Spark.ResponsibilitiesDesign and implement data processing pipelines using Python...

  • Data Engineering Lead

    4 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Data Engineering LeadDescription:As a Data Engineering Lead, you will be responsible for designing, building, and supporting data applications using Spark, Sqoop, and Hive. You will work with large data sets and high-capacity big data processing platforms, and have strong knowledge of Unix and Shell scripting. Advanced knowledge of the Hadoop...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Mempage Full time

    About the Role : We are seeking a highly skilled Big Data Engineer to contribute to the design, development, and maintenance of our data infrastructure. As a key member of our team, you will be responsible for building and optimizing data pipelines to support our business analytics and machine learning initiatives. Your expertise will be instrumental in...


  • Hyderabad, Telangana, India Aricent Full time

    Job DescriptionWe are seeking a skilled Senior Big Data Engineer to design and build solutions for a Fortune 500 client program. The project involves creating a data standardized and curation-based Hadoop cluster.About the RoleThis is a high-visibility, fast-paced key initiative that will integrate data across internal and external sources, provide...

  • Senior Java Developer

    1 month ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Senior Java DeveloperJob Summary:We are seeking a highly skilled Senior Java Developer to join our team at Virtusa. The ideal candidate will have extensive experience in Java, Spark, and Big Data technologies.Key Responsibilities:Design and develop large-scale data processing systems using Java and SparkCollaborate with cross-functional teams to...


  • Hyderabad, Telangana, India Tiger Analytics Full time

    About the RoleWe are seeking a highly skilled Data Governance Architect Lead to join our team at Tiger Analytics. As a key member of our organization, you will be responsible for designing, architecting, and deploying big data-based data governance solutions.Key ResponsibilitiesTechnical management across the full life cycle of big data-based data governance...


  • Hyderabad, Telangana, India Evernorth Health Services Full time

    Job OverviewWe are seeking a Senior Data Engineering Lead to join our team at Evernorth Health Services. In this role, you will be responsible for designing and implementing data pipelines using Spark, Py-Spark, and Hadoop.About UsEvernorth exists to elevate health for all by solving problems others don't, won't or can't. We are passionate about making...


  • Hyderabad, Telangana, India Evoke Technologies Full time

    About the Role:We are seeking a seasoned Data Science Practice Lead to join our team at Evoke Technologies. The ideal candidate will have a strong background in data analytics, machine learning, and business acumen. The Data Science Practice Lead will be responsible for providing consulting, advisory, assessment, and implementation services to customers in...


  • Hyderabad, Telangana, India Tata Consultancy Services Full time

    About Tata Consultancy ServicesTata Consultancy Services is a leading IT services company that empowers global businesses through its cutting-edge technology and innovative solutions.Compensation PackageThe estimated salary for this role is around $120,000 - $180,000 per annum, depending on the location and individual qualifications.About the RoleJob...