Big Data Engineer

3 weeks ago


Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

Big Data Engineer (PySpark)

Location: Pune/Nagpur (WFO)

Experience: 8 - 12 Years

Employment Type: Full-time

Job Overview

We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities

  • Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
  • Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
  • Write efficient HiveQL and SQL queries to handle large-scale datasets.
  • Perform performance tuning and optimization of distributed data systems.
  • Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
  • Manage and schedule workflows using Apache Airflow or Oozie.
  • Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills

  • Proven experience in Big Data Engineering with a focus on PySpark.
  • Strong knowledge of HDFS, Hive, Sqoop, and related tools.
  • Proficiency in SQL/HiveQL for large datasets.
  • Expertise in performance tuning and optimization of distributed systems.
  • Familiarity with Agile methodology and collaborative team practices.
  • Experience with workflow orchestration tools (Airflow/Oozie).
  • Strong problem-solving, analytical, and communication skills.

Good to Have

  • Knowledge of data modeling and data warehousing concepts.
  • Exposure to DevOps practices and CI/CD pipelines for data engineering.
  • Experience with other Big Data frameworks such as Spark Streaming or Kafka.

  • Big Data Engineer

    3 weeks ago


    Pune, Maharashtra, India Merito Full time

    Job Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...

  • Big Data Engineer

    11 hours ago


    Pune, Maharashtra, India RiskInsight Consulting Pvt Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databasesBuild and maintain scalable data pipelines for effective data ingestion, transformation, and analysisCollaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them...

  • Big Data Engineer

    2 days ago


    Pune, Maharashtra, India Information Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Big Data EngineerKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal solutions. - Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks. -...

  • Big Data Engineer

    3 weeks ago


    Pune, Maharashtra, India WorkiFicient Full time

    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...


  • Pune, Maharashtra, India beBeeData Full time

    Job DescriptionWe are seeking a highly skilled Big Data Developer to join our team. This individual will be responsible for designing, developing, and maintaining large-scale big data pipelines, real-time systems, and data warehouses.Required Skills and QualificationsCoding Experience: 2-5 years of experience in Java programming with a solid understanding of...

  • Big Data Engineer

    2 days ago


    Pune, Maharashtra, India Information Technology Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks.Develop distributed data processing systems...


  • Pune, Maharashtra, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Lead Big Data EngineerCome work at a place where innovation and collaboration come together to support the most exciting missions in the world.Job Description:We are seeking a talented Big Data Engineering Specialist to deliver roadmap features of our Enterprise TruRisk Platform which would help customers to measure, communicate and eliminate cyber risks...

  • Big Data Engineer

    2 weeks ago


    Pune, Maharashtra, India Tata Consultancy Services Full time

    Greetings from TCS TCS is hiring for Big Data Location: - Chennai/Mumbai/Pune Desired Experience Range: 6 to 12 years Must-Have • PySpark • Hive Good-to-Have • Spark • HBase • DQ tool • Agile Scrum experience • Exposure in data ingestion from disparate sources onto Big Data platform Thanks Anshika


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 35,00,000

    Senior Big Data EngineerWe are seeking a highly skilled and experienced Senior Big Data Engineer to fill this role.Design, develop, and maintain scalable and high-performance data processing systems for maximum efficiency.Build robust ETL/ELT workflows for structured and unstructured data using big data technologies like Hadoop, Spark, Hive, HBase, Kafka,...

  • Big Data Developer

    2 weeks ago


    Pune, Maharashtra, India MRO Full time

    Analyst - Big Data DeveloperWe are seeking a highly skilled Analyst -Big Data Developer to join our dynamic team.The ideal candidate will have extensive experience with big data technologies and a strong background in developing and optimizing data integration frameworks and applications.You will be responsible for designing, implementing, and maintaining...