Senior Big Data Engineer

1 day ago

Gurugram, India FxConsulting Full time

Position : Big Data Engineer

Experience : 59 Years

Job Type : Full-time

Job Summary :

We are seeking a highly skilled Big Data Engineer with 59 years of experience to develop, maintain, and optimize robust, scalable big data solutions. This role requires expertise in designing and implementing solutions using Hadoop-based technologies and AWS. The engineer will be crucial in building various complex Big Data platform components, focusing on optimizing data access, handling high data volumes, and ensuring real-time performance using frameworks like Spark, Kafka, and other cloud technologies.

Key Responsibilities :

Big Data Platform Architecture and Design :

- Lead the design of comprehensive data solutions leveraging Hadoop-based technologies in conjunction with AWS cloud infrastructure.

- Implement and design various critical Big Data platform components, including Batch Processing, Live Stream Processing, In-Memory Cache, Query Layer (SQL), Rule Engine, and Action Framework.

- Design and implement a robust Data Access Layer capable of connecting to diverse data sources.

Performance and Scalability :

- Implement scalable technical solutions to reliably handle ever-increasing data volumes using modern big data and cloud technologies like Spark, Kafka, and Cloud computing services.

- Utilize advanced caching techniques within the Data Access Layer to ensure fast responses to real-time SQL queries against Big Data Technologies.

Data Pipeline and Infrastructure :

- Set up and operate end-to-end data pipelines and the overall infrastructure required for executing ETL jobs.

- Apply proficiency with distributed file systems and computing concepts to optimize data :

- Experience: 59 years of hands-on Data engineering experience.

- Programming Languages: Good hands-on knowledge and proficiency in one of the primary languages: Java, Scala, or Python.

- Data Handling Frameworks: Strong hands-on experience with at least one of the major data handling frameworks: Apache Spark, Apache Beam, or Apache Flink.

- Data Store: Good technical understanding of both SQL and NoSQL databases (e.g., MySQL, Mongo).

- Data Infrastructure: Familiarity with cloud solutions for data infrastructure, with AWS preferably.

- Distributed Concepts: Proficient with distributed file systems and computing concepts (e.g., HDFS, YARN).

Preferred Skills :

- Familiarity with different types of efficient data formats, including Apache Parquet, Avro, and ORC.

- Direct experience integrating streaming technologies like Kafka with processing frameworks like Spark or Flink.

- Experience in designing a Rule Engine or Action Framework for real-time data processing.

- Certifications in AWS Data Analytics or related Big Data technologies.


(ref:hirist.tech)

  • Big Data Engineer

    2 weeks ago

    Gurugram, India Insight Global Full time

    Pay Range:LPA (INR) Required Qualifications: 3–5 years of hands-on experience in Big Data engineering. Proficiency in Python (Java is a plus). Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus. Solid problem-solving skills and a structured thought process. Experience with data migration and...

  • Big Data Developer

    2 weeks ago

    Gurugram, India Kezan Inc Full time

    Position: Big Data Engineer ( Immediate Joiner) Experience: 5+ Years Location: ( Gurugram / Bangalore ) Joining: Immediate Joiner Budget LPA Job Summary: We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal...

  • Big Data Developer

    2 weeks ago

    Gurugram, India Kezan Inc Full time

    Position: Big Data Engineer Experience: 5+ Years Location: ( Gurugram / Bangalore ) Joining: Immediate Joiner Budget=16.5 LPA Job Summary: We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal candidate will have...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India DEUTSCHE TELEKOM DIGITAL LABS PRIVATE LIMITED Full time

    Responsibilities :- You will be involved in the design of data solutions using Hadoop based technologies along with Hadoop, AWS.- Responsibilities includes design and implementation of various Big Data platform components like (Batch Processing, Live Stream Processing, In- Memory Cache, Query Layer (SQL), Rule Engine and Action Framework ).- Design and...

  • Big Data Tester

    2 weeks ago

    Gurugram, India Kezan Inc Full time

    Job Title:Big Data Tester (SQL) Experience:3+Years Location:(Gurugram/Bangalore) Employment Type:Full-Time Job Summary: We are seeking a highly skilledBig Data Tester with strong SQL expertiseto join our team. The ideal candidate will be responsible for validating, testing, and ensuring the quality of large-scale data pipelines, data lakes, and big data...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India Fabric Full time

    Company Name - Aays Analytics Job Summary - You will act as a key member of the Data consulting team, working directly with partners and senior stakeholders. You will design and implement big data and analytics solutions. Communication and organisational skills are key for this position. Key Responsibilities Develop data solutions within a Big Data Azure...

  • Big Data Engineer

    2 weeks ago

    Gurugram, India Qcentrio Full time

    Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support. Solve complex...

  • Big Data Tester

    2 days ago

    Bengaluru, Gurugram, India Kezan Consulting Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    **ONLY FOR IMMEDIATE JOINERS**Position: Big Data TesterExperience: 4+ YearsLocation: ( Gurugram / Bangalore )Joining: Immediate JoinerJob Summary:We are seeking an experienced Big Data Tester with a strong background in HDFS, Hive, ETL testing, and SQL. The ideal candidate will have a experience in Big Data ecosystems, ETL/SQL testing, and in the...

  • Apps Associates

    2 weeks ago

    Gurugram, India Apps Associates (I) Pvt. Ltd Full time

    Job Description :Key Responsibilities : - Data Architecture & Strategy : Lead the design and implementation of scalable, reliable, and high performance data architectures, data lakes, and data warehouses.- ETL/ELT Pipeline Development : Architect, build, and optimize complex ETL/ELT pipelines to ingest, transform, and load data from diverse sources into...

  • Senior Big Data Engineer

    2 weeks ago

    Gurugram, India dunnhumby Full time

    dunnhumbyis the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First. Our mission:to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of the...