Big Data Engineer

4 weeks ago


Anywhere in IndiaMultiple LocationsRemote Hum Technologies Full time

Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients

Company Name : Hum Technologies

Client : One of the Fortune 500 Companies

Location : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting


Roles & Responsibilities :


- Responsible for designing and implementing data pipelines, troubleshooting and mentoring.


- Devise technical solution options aligned to the strategic architecture.


- Design data models / structures to support capital modelling and reporting processes.


- Influence the design and implementation of batched processes to create model inputs and execute models


- Liaise with other projects to achieve synergies through mutually beneficial design / delivery decisions.


- Act as a technical mentor/coach to developers to help them deliver solutions that meet customer requirements and architectural standards.


- Be available to discuss and resolve technical issues with individuals within and without the team.


- Help to embed the use of new technologies to deliver project goals.


- Help to ensure work carried out is in line with quality Skills :


- Strong knowledge of Hadoop/Big Data concepts


- Experience in working with Cloudera clusters is a plus


- Experience in Analysing data sets and creating data models using SQL


- Good working knowledge of Apache Spark with Java/Scala


- Knowledge of Spark Streaming with Kafka


- Must have hands on experience in building ETL pipelines using Apache NiFi


- Must have hands on experience in Impala with Kudu


- Experience in writing transformations using SQL


- Experience in monitoring, debugging and troubleshooting Spark Jobs/Impala Queries in the cluster


- Basic knowledge of Unix shell scripting


- Good to have knowledge of Spring boot, Spring MVC, REST APIs and Microservices.


- Good to have knowledge of AWS Cloud (EC2, IAM, S3, Lambda, CloudWatch etc.)

(ref:hirist.tech)
  • Big Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    AWS Big Data Engineer Job DescriptionWe are seeking a highly skilled AWS Big Data Engineer to join our team at D-TechWorks Pvt Ltd. As a key member of our delivery team, you will be responsible for designing, developing, and supporting big data solutions using AWS technologies.Key Responsibilities:Design and develop big data architectures using AWS services...


  • Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    About the Role:At D-TechWorks Pvt Ltd, we are seeking a skilled Data Engineer for Big Data to join our team. As a key member of our delivery team, you will be responsible for ensuring effective design, development, validation, and support activities in line with client needs and architectural requirements.About the Position:This is a Remote Big Data Position...


  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    Job Title: Senior Big Data EngineerJob Summary:US Technology International Pvt Ltd is seeking a Senior Big Data Engineer to lead the development and implementation of big data solutions. The ideal candidate will have expertise in Hadoop, Spark, and other big data technologies, as well as strong leadership and communication skills.Key Responsibilities:Design,...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full time

    We are looking for Aws big data engineer In Remote , India Position : Aws big data engineerExperience : 7 to 9 YearLocation : Remote , India Mandate skills : BigData Python Pyspark AWS Glue Airflow Redshift Responsibilities :- Ensure effective Design, Development, Validation and Support activities in line with client needs and architectural requirements.,-...


  • Anywhere in India/Multiple Locations US Technology International Pvt Ltd Full time

    Job Title: Senior Big Data EngineerWe are seeking a highly skilled Senior Big Data Engineer to join our team at US Technology International Pvt Ltd.Key Responsibilities:Design and develop large-scale data processing systems using Hadoop/Spark.Collaborate with cross-functional teams to integrate data processing systems with existing infrastructure.Develop and...


  • Anywhere in India/Multiple Locations/Remote, IN NucleusTeq Consulting Pvt. Ltd. Full time

    Role : Big Data Engineer Responsibilities :- Big Data Engineer with 5-6 years of experience- Experience in leading a team of engineers and a good attitude toward learning the domain and implementation- Strong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.- Finalizing the scope of the system and...

  • Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations Innotatz IT Solutions Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Innotatz IT Solutions. As a Data Engineer, you will play a crucial role in designing and implementing robust data pipelines to collect, connect, and curate data from various sources.Responsibilities:Design and develop scalable data pipeline architectures to meet...

  • Big Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations IT Source Global Full time

    Job SummaryWe are seeking a skilled Hadoop Developer to join our team at IT Source Global. As a Hadoop Developer, you will be responsible for designing and developing complex data pipelines for ingestion, transformation, and processing of structured and unstructured data.Key ResponsibilitiesDevelop, implement, and maintain large-scale data processing...

  • Big Data Lead

    3 weeks ago


    Anywhere in India/Multiple Locations Persistent Systems Full time

    About PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above.Our TeamWe work with many industry-leading organizations across the world. Our growth trajectory continues, as we reported...

  • Big Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations Stratosphere IT Services PVT Ltd Full time

    Job Description : - Azure data Engineer - Azure Data Factory - azure databricks- python - sql- Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database- Optional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB and Purview.- Strong programming, unit testing & debugging skills in SQL, Python or Scala/Java.- Some experience...

  • Lead Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations, IN Talentoj Full time

    Requirements : - 6+ years' experience with Big Data Technologies such as Apache Spark, Cloud Native, and Data Lakes, and Databricks. - Delivered ETL/ELT solutions including data extraction, transformation, cleansing, data integration, and data management. - Implemented batch ingestion pipelines. - Strong Python skills - automate tasks, write unit tests...


  • Anywhere in India/Multiple Locations/Remote NucleusTeq Consulting Pvt. Ltd. Full time

    Role: As a Senior Big Data Engineer at NucleusTeq Consulting Pvt. Ltd., you will be responsible for leading a team of engineers in designing and implementing scalable big data solutions using Spark, Scala, and AWS. Your expertise in Airflow orchestration and strong exposure to Hadoop and similar frameworks will be valuable assets to our team. Key...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations/Remote, IN Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scriptingRoles & Responsibilities :- Responsible for designing and implementing data...

  • Big Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations, IN TalentXO Full time

    Required Qualifications :- 7-8 years of experience as a Data Engineer or in a similar role- Proficient in Python programming and PySpark for data processing and transformation- Extensive experience in designing and implementing data pipelines using AWS Glue and AWS MWAA (Airflow)- Strong SQL skills, including experience with AWS Redshift- Familiarity with...


  • Anywhere in India/Multiple Locations Aexonic Technologies Private Limited Full time

    Job SummaryWe are seeking a highly skilled Senior Big Data Engineer to join our team at Aexonic Technologies Private Limited. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.Key ResponsibilitiesDesign and develop...


  • Metros/Anywhere in India/Multiple Locations Dotflick Solutions Full time

    Job SummaryWe are seeking a highly skilled Data Engineering Expert to join our team at Dotflick Solutions. The ideal candidate will have a strong background in big data infrastructure and experience with Hadoop clusters.Key ResponsibilitiesDesign and implement data ingestion pipelines between legacy data warehouses and big data stacksDevelop and deploy data...

  • Big Data Expert

    4 weeks ago


    Metros/Anywhere in India/Multiple Locations Dotflick Solutions Full time

    Job Title: Hadoop AdministratorAbout the Role:This is a key position for our company, Dotflick Solutions, in which you will serve as a development and support expert with responsibility for the design, development, automation, testing, support and administration of our Enterprise Infrastructure Big Data Service.Key Responsibilities:Design, development,...

  • Big Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations, IN Changeleaders Full time

    Key Responsibilities :Data Pipeline Development :- Design, develop, and maintain scalable data pipelines using Big Data technologies (e.g., Apache Hadoop, Apache Spark, Kafka).- Implement data ingestion strategies to efficiently load and transform large datasets from various sources.Data Architecture and Modeling :- Collaborate with data architects to design...

  • Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations Info Services Full time

    Data Engineer RoleWe are seeking a skilled Data Engineer to join our team at Info Services. As a Data Engineer, you will be responsible for designing and implementing efficient data pipelines using big data frameworks (e.g, Spark) for ingesting, transforming, and processing large datasets.Key Responsibilities- Design and implement data pipelines using big...

  • Persistent Systems

    4 weeks ago


    Anywhere in India/Multiple Locations Persistent Systems Full time

    About Persistent : We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with...