Big Data Engineer

14 hours ago

Gurgaon, Haryana, India MyCareernet Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Company:Indian / Global Digital Organization

Key Skills:
Pyspark, AWS, Python, SCALA, ETL

Roles and Responsibilities:

  • Develop and deploy ETL and data warehousing solutions using Python libraries and Linux bash scripts on AWS EC2, with data stored in Redshift.
  • Collaborate with product and analytics teams to scope business needs, design metrics, and build reports/dashboards.
  • Automate and optimize existing data sets and ETL pipelines for efficiency and reliability.
  • Work with multi-terabyte data sets and write complex SQL queries to support analytics.
  • Design and implement ETL solutions integrating multiple data sources using Pentaho.
  • Utilize Linux/Unix scripting for data processing tasks.
  • Leverage AWS services (Redshift, S3, EC2) for storage, processing, and pipeline automation.
  • Follow software engineering best practices for coding standards, code reviews, source control, testing, and operations.

Skills Required:

Must-Have:

  • Hands-on experience with PySpark for big data processing
  • Strong knowledge of AWS services (Redshift, S3, EC2)
  • Proficiency in Python for data processing and automation
  • Strong SQL skills for working with RDBMS and multi-terabyte data sets

Nice-to-Have:

  • Experience with SCALA for distributed data processing
  • Knowledge of ETL tools such as Pentaho
  • Familiarity with Linux/Unix scripting for data operations
  • Exposure to data modeling, pipelines, and visualization

Education:
Bachelor's degree in Computer Science, Information Technology, or a related field

  • Big Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India Infogain Full time

    Posted on August 17 2024ROLES RESPONSIBILITIESBig Data Engineer Senior As a Big Data Engineer Senior you will be responsible for designing and implementing complex big data solutions You will work closely with clients and team members to understand their requirements and develop architectures that meet their needs You will also be responsible for...

  • Big Data Engineer

    16 hours ago

    Gurgaon, Haryana, India Insight Global Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Pay Range: LPA (INR)Required Qualifications:3–5 years of hands-on experience in Big Data engineering.Proficiency in Python (Java is a plus).Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus.Solid problem-solving skills and a structured thought process.Experience with data migration and...

  • Big Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India Idyllic Services Pvt Ltd Full time

    Job Title : Big Data Engineer - ScalaExperience : 7 to 10Years (Minimum 3+ years in Scala)Notice Period : Immediate to 30 DaysRole Overview : We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track...

  • Data Scientist/Big Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India Techmora Full time

    Job Description : Data Science / Big Data EngineerLocation : Gurgaon / PuneEmployment Type : Full-timeNotice Period : 30 DaysExperience : 4-6 YearsAbout the Role :We are seeking a skilled Data Science / Big Data Engineer with strong expertise in data wrangling, large-scale data processing, and advanced analytics. The ideal candidate will have hands-on...

  • Big Data Developer

    15 hours ago

    Gurgaon, Haryana, India Viraaj HR Solutions Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About The OpportunityA staffing and HR services firm focused on placing talent into Enterprise IT, Analytics, and Big Data engineering projects across Banking, Retail, Telecom, and SaaS sectors. We partner with product and services organisations to deliver scalable data platforms, real-time analytics, and high-throughput ETL solutions.We are hiring an...

  • Big Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India NITS Solutions Full time

    We have an urgent opening for Big Data Engineer role.Job Location - GurgaonExperience Required - 2 -6 YearsSkills :- Understand business needs & apply analytical concepts to generate actionable business insights/optimization models- Write Java coding for Spark- Write Python/SQL code- Write Scala coding for Spark- Converting hard and complex techniques as...

  • Big Data Modernization Engineer

    4 weeks ago

    Gurgaon, Haryana, India Infosys Limited Full time

    Job Description- Engineer has to help Director in Modernization- Convert Talend jobs into PySpark or Data proc jobs- Big data skills- Good in written and verbal Communication- Good problem-solving skills and programming skills

  • Big Data Tester

    16 hours ago

    Gurgaon, Haryana, India Kezan Inc Full time ₹ 6,00,000 - ₹ 15,00,000 per year

    Job Title:Big Data Tester (SQL)Experience:3+YearsLocation:[Gurugram/Bangalore]Employment Type:Full-TimeJob Summary:We are seeking a highly skilledBig Data Tester with strong SQL expertiseto join our team. The ideal candidate will be responsible for validating, testing, and ensuring the quality of large-scale data pipelines, data lakes, and big data...

  • Big Data Developer

    16 hours ago

    Gurgaon, Haryana, India Kezan Inc Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Position: Big Data Engineer ( Immediate Joiner)Experience: 5+ YearsLocation: ( Gurugram / Bangalore )Joining: Immediate JoinerBudget LPAJob Summary:We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal candidate...

  • Big Data Tester

    11 hours ago

    Gurgaon, Haryana, India Kezan Inc Full time ₹ 9,60,000 - ₹ 13,20,000 per year

    Big Data Tester (AWS + SQL) (Immediate joiner)Location:Gurugram / Bangalore (Hybrid)Experience:4–7 YearsBudget:Up to ₹11 LPAAbout the Role:We are looking for a skilled and detail-orientedBig Data Testerwith expertise inAWSandSQLto join our growing team. This role requires hands-on experience in data validation, test automation for big data pipelines, and...