HDFS Senior Data Engineer

3 weeks ago


Bengaluru, Karnataka, India Careerconnects Full time
Job Description:

We are seeking an experienced HDFS Sr. Data Engineer to join our team at Careerconnects. In this role, you will be responsible for the entire development process from conception to delivery.

You will collaborate with stakeholders to gather and implement use case details, take ownership of FS and TS documentation, work closely with support engineers for test case documentation, manage multiple projects and deliver on time.

The ideal candidate will have hands-on experience with the Hadoop ecosystem, particularly Spark, Kafka, Hive, and shell scripting. Proficiency in Hadoop, Spark, Hive, Airflow, Shell Scripting, Scala, and Kafka is a must.

Responsibilities include analyzing and writing SQL queries as needed, writing unit tests, ensuring all necessary documentation is in place, developing shell scripts, and scheduling automated jobs. The successful candidate will also focus on writing optimized code with experience in performance testing being a plus.

A strong communicator with excellent stakeholder management skills, you will be able to exhibit a high level of professionalism in your interactions.

Requirements:
  1. Hands-on experience with the Hadoop ecosystem, particularly Spark, Kafka, Hive, and shell scripting.
  2. Familiarity with Airflow, Spark Streaming, cloud platforms, and Agile methodologies is beneficial.
What We Offer:

We offer a competitive salary range of $120,000-$160,000 per year, depending on experience. You will also receive a comprehensive benefits package, including health insurance, retirement plan, and paid time off.



  • Bengaluru, Karnataka, India Tredence Inc. Full time

    Tredence Inc. is a leading provider of big data and analytics solutions.We are currently seeking a highly skilled Senior Data Engineer to join our team in the United States.The successful candidate will have a strong background in data warehousing and processing, with experience in designing and implementing large-scale data pipelines using Hadoop,...


  • Bengaluru, Karnataka, India Mobileum Full time

    We are seeking a highly skilled Senior Big Data Engineer to lead our data analytics initiatives. This role will be responsible for designing, developing, and deploying high-performance data analytics applications.As a key member of our team, you will collaborate with peer engineers and architects to maintain and enhance our data analytics solutions.The ideal...


  • Bengaluru, Karnataka, India IT Source Global Full time

    Job Title: Senior Big Data EngineerJob Description:We are seeking a highly skilled Senior Big Data Engineer to join our team at IT Source Global. As a Senior Big Data Engineer, you will be responsible for designing and implementing scalable, fault-tolerant data pipelines using Hadoop and Spark.Key Responsibilities:Design and implement data pipelines using...


  • Bengaluru, Karnataka, India ATech Full time

    About the JobATech is seeking an experienced Senior Data Engineer to lead our cloud solutions team. This role requires a strong background in designing, building, and optimizing data models and ETL processes using Big Data technologies such as AWS.Key Responsibilities:Design and build scalable data infrastructure on AWSDevelop and deploy data pipelines using...


  • Bengaluru, Karnataka, India Altimetrik Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at Altimetrik in Bangalore. As a key member of our data engineering team, you will play a critical role in designing and developing large-scale data warehouses and ETL jobs using Hadoop/HDFS and AWS services.About the RoleThis is an excellent opportunity for a senior-level...


  • Bengaluru, Karnataka, India GCC client Full time

    Join GCC client as a Senior Cloud Data Architect, leading the design and implementation of scalable data migration strategies to AWS. With a strong focus on cloud technologies, you will develop and optimize Spark jobs for data processing on AWS EMR.We are seeking an expert with 6 to 10 years of experience in big data and cloud technologies, specifically with...


  • Bengaluru, Karnataka, India Publicis Sapient Full time

    Senior Associate Data Engineering Level 2The Senior Associate Data Engineering Level 2 role at Publicis Sapient involves translating client requirements into technical design and implementing components for data engineering solutions. This position requires a deep understanding of data integration and big data design principles to create custom solutions or...


  • Bengaluru, Karnataka, India American Express Full time

    Company OverviewAmerican Express is a global and diverse community of colleagues with an unwavering commitment to back our customers, communities and each other.Job DescriptionWe're seeking a talented Senior Data Engineer Lead to join our team. As part of our tech team, you'll architect, code and ship software that makes us an essential part of our...

  • Senior Data Architect

    1 month ago


    Bengaluru, Karnataka, India Oracle Full time

    Data Engineer Role OverviewOracle is seeking a highly skilled Data Engineer to play a key role in designing, implementing, and optimizing our data pipelines within the Hadoop ecosystem.Key Responsibilities:Lead the integration of data from FLEXCUBE core banking systems into our Hadoop-based data infrastructure.Design, build, and optimize data pipelines...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled Senior Data Engineering Specialist to join our team in Bangalore, India. The successful candidate will have a strong background in ETL development and data engineering using Azure concepts and technologies.Key ResponsibilitiesDesign and implement high and low-level design documents for ETL processes.Develop ETL code in...

  • Senior Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Roku Full time

    Unlock Your Potential as a Roku Data Systems ArchitectRoku is revolutionizing the way the world watches TV, and we're looking for a talented Data Systems Architect to join our team. As a senior engineer, you'll play a critical role in designing and implementing scalable, highly available, and fault-tolerant distributed data processing systems that process...


  • Bengaluru, Karnataka, India Publicis Sapient Full time

    Data Engineering Senior AssociateAs a Senior Associate Data Engineer at Publicis Sapient, you will be responsible for translating client requirements into technical design and implementing components for data engineering solutions. You will utilize your deep understanding of data integration and big data design principles to create custom solutions or...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Photon Full time

    At Photon, we are pushing the boundaries of data engineering to deliver personalized customer experiences.We are seeking a skilled Data Engineer to join our team and help design and develop our next generation Cloud Data Lake and Analytics Platform for Workplace Solutions.The purpose of this role is to participate in end-to-end development, including design,...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Data Engineering Senior Specialist RoleDiverse and dynamic Data Engineering activities on Azure Data platform. The role involves building and delivering Data pipelines connecting various enterprise data sources, both RDBMS and NoSQL. Develop data mappings to existing data sources and design big data processing notebooks using Azure Databricks. Scripting and...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    We are currently seeking a highly skilled Data Solutions Architect to join our team in Bangalore, India.About the Role:As a Data Solutions Architect, you will design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack.Main...


  • Bengaluru, Karnataka, India gethyr Full time

    Data Engineering ExpertiseWe are seeking a seasoned Data Engineering Specialist to join our team at Gethyr. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines using PySpark to process and transform large datasets efficiently.Key Responsibilities:Design and implement data pipelines using PySpark to...


  • Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full time

    About the RoleWe are seeking a Senior Data Engineering Manager to lead our engineering team in successfully building and deploying cutting-edge analytics products for consumption by various stakeholders.As a key member of our team, you will be responsible for leading the architecture, design, and delivery of enterprise-grade microservices. You will also...

  • Bosch Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Bosch Full time

    Job OverviewWe are seeking an experienced Senior Data Engineer to join our team at Bosch. This is a challenging opportunity for a highly skilled professional with a strong background in large-scale Java or Scala software development and distributed systems.Key Responsibilities:Design and build data pipelines on a global scale, handling datasets ranging from...


  • Bengaluru, Karnataka, India Saaki Argus & Averil Consulting Full time

    As a Snowflake Data Engineer at Saaki Argus & Averil Consulting, you will be responsible for managing and manipulating data within Snowflake or Netezza. This role involves developing and implementing UNIX shell scripts for data processing and automation, as well as working with Hadoop, HDFS, Spark, and NoSQL databases to support data workflows.Additionally,...


  • Bengaluru, Karnataka, India Oracle Full time

    We are seeking a highly skilled Senior Data Integration Specialist to join our team at Oracle.About the RoleThis is an exciting opportunity to work on cutting-edge data projects and contribute to data-driven decision-making in the financial sector.The ideal candidate will have extensive experience in designing and implementing data solutions within the...