Big Data Engineer

6 days ago

Gurgaon, Haryana, India Idyllic Services Pvt Ltd Full time

Job Title : Big Data Engineer - Scala

Experience : 7 to 10Years (Minimum 3+ years in Scala)

Notice Period : Immediate to 30 Days

Role Overview :

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities :

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.

- Collaborate with cross-functional teams to design scalable cloud-based data architectures.

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

- Build monitoring and alerting systems leveraging Splunk or equivalent tools.

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills :

- Scala - Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python.

- Hands-on experience with Apache Kafka.

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years).

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture.

Preferred Skills :

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions.

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques

(ref:hirist.tech)

  • Big Data Engineer

    4 days ago

    Gurgaon, Haryana, India Infogain Full time

    Posted on August 17 2024ROLES RESPONSIBILITIESBig Data Engineer Senior As a Big Data Engineer Senior you will be responsible for designing and implementing complex big data solutions You will work closely with clients and team members to understand their requirements and develop architectures that meet their needs You will also be responsible for...

  • Big Data Engineer

    6 days ago

    Gurgaon, Haryana, India Insight Global Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Pay Range: LPA (INR)Required Qualifications:3–5 years of hands-on experience in Big Data engineering.Proficiency in Python (Java is a plus).Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus.Solid problem-solving skills and a structured thought process.Experience with data migration and...

  • Big Data Engineer

    7 days ago

    Gurgaon, Haryana, India beBeeData Full time ₹ 2,50,00,000 - ₹ 3,00,00,000

    Cloud Engineering Lead">We are looking for a Cloud Engineering Lead to drive the design and implementation of scalable, secure, and high-performance cloud-based big data solutions. As a key member of our team, you will be responsible for leading the development and deployment of cloud-native technologies and architectures.">Develop and implement cloud...

  • Data Scientist/Big Data Engineer

    24 hours ago

    Gurgaon, Haryana, India Techmora Full time

    Job Description : Data Science / Big Data EngineerLocation : Gurgaon / PuneEmployment Type : Full-timeNotice Period : 30 DaysExperience : 4-6 YearsAbout the Role :We are seeking a skilled Data Science / Big Data Engineer with strong expertise in data wrangling, large-scale data processing, and advanced analytics. The ideal candidate will have hands-on...

  • Senior Big Data Engineer

    3 days ago

    Gurgaon, Haryana, India dunnhumby Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    We're looking for a Senior Big Data Engineer who expects more from their career. It's chance to extend and improve dunnhumby's Data Engineering Team. It's an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers. What we expect from you • 5 years of experience in Information Technology...

  • Big Data Modernization Engineer

    1 week ago

    Gurgaon, Haryana, India Infosys Limited Full time

    Job Description- Engineer has to help Director in Modernization- Convert Talend jobs into PySpark or Data proc jobs- Big data skills- Good in written and verbal Communication- Good problem-solving skills and programming skills

  • Big Data Engineer

    4 days ago

    Gurgaon, Haryana, India NITS Solutions Full time

    We have an urgent opening for Big Data Engineer role.Job Location - GurgaonExperience Required - 2 -6 YearsSkills :- Understand business needs & apply analytical concepts to generate actionable business insights/optimization models- Write Java coding for Spark- Write Python/SQL code- Write Scala coding for Spark- Converting hard and complex techniques as...

  • Big Data Tester

    6 days ago

    Gurgaon, Haryana, India Kezan Inc Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Title:Big Data Tester (SQL)Experience:3+YearsLocation:[Gurugram/Bangalore]Employment Type:Full-TimeJob Summary:We are seeking a highly skilledBig Data Tester with strong SQL expertiseto join our team. The ideal candidate will be responsible for validating, testing, and ensuring the quality of large-scale data pipelines, data lakes, and big data...

  • Big Data Architect

    6 days ago

    Gurgaon, Haryana, India beBeeData Full time ₹ 2,50,00,000 - ₹ 3,50,00,000

    Senior Data Engineer PositionWe're seeking an experienced Senior Big Data Engineer to spearhead the design, development, and implementation of large-scale data systems using Redshift, AWS, Spark, and Scala.Data Architecture- Design and develop large-scale data warehouses using Amazon Redshift to maximize efficiency and scalability.- Optimize Redshift cluster...

  • Big Data Developer

    6 days ago

    Gurgaon, Haryana, India Kezan Inc Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Position: Big Data Engineer ( Immediate Joiner)Experience: 5+ YearsLocation: ( Gurugram / Bangalore )Joining: Immediate JoinerBudget LPAJob Summary:We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal candidate...