Big Data Engineer

1 day ago


Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

Big Data Engineer (PySpark)

Location: Pune/Nagpur (WFO)

Experience: 8 - 12 Years

Employment Type: Full-time

Job Overview

We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities

  • Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
  • Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
  • Write efficient HiveQL and SQL queries to handle large-scale datasets.
  • Perform performance tuning and optimization of distributed data systems.
  • Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
  • Manage and schedule workflows using Apache Airflow or Oozie.
  • Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills

  • Proven experience in Big Data Engineering with a focus on PySpark.
  • Strong knowledge of HDFS, Hive, Sqoop, and related tools.
  • Proficiency in SQL/HiveQL for large datasets.
  • Expertise in performance tuning and optimization of distributed systems.
  • Familiarity with Agile methodology and collaborative team practices.
  • Experience with workflow orchestration tools (Airflow/Oozie).
  • Strong problem-solving, analytical, and communication skills.

Good to Have

  • Knowledge of data modeling and data warehousing concepts.
  • Exposure to DevOps practices and CI/CD pipelines for data engineering.
  • Experience with other Big Data frameworks such as Spark Streaming or Kafka.

  • Big Data Engineer

    2 weeks ago


    Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full time

    Job Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...

  • Big Data Engineer

    3 days ago


    Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Engineer">We are seeking a highly skilled Big Data Engineer to drive the design and implementation of scalable data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. This is an excellent opportunity for experienced professionals to join our organization and leverage their expertise to drive business outcomes through...

  • Big Data Engineer

    3 weeks ago


    Pune, Maharashtra, India WorkiFicient Full time

    Roles and Responsibility :- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.- Design and implement systems to cleanse,...


  • Pune, Maharashtra, India Devlats Pvt Ltd Full time

    About the Role : We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in designing, building, and optimizing data pipelines and architectures, as well as a solid understanding of big data tools and Responsibilities : - Design, develop, and maintain scalable and high-performance...


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Our company is seeking a highly skilled Data Architect to design, develop and implement scalable data solutions using cutting-edge technologies.Key Responsibilities:Data Ingestion and Processing: Develop and maintain high-quality code, adhering to industry standards and best practices, for efficient data ingestion from disparate sources onto our Big Data...


  • Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking ambitious professionals for the role of Lead Apache Hadoop Engineer in our Technology, Data, and Innovation (TDI) team.Key ResponsibilitiesDevelop robust architectures and designs for big data platforms and applications within the Apache Hadoop ecosystem.Implement and deploy big data platforms and solutions on-premises and in...

  • Big Data Engineer

    3 days ago


    Pune, Maharashtra, India beBeeData Full time ₹ 13,70,400 - ₹ 25,10,800

    Job Title: Data ArchitectWe are seeking an experienced Data Architect to lead the development of our big data strategy.About the RoleThis is a highly visible and rewarding role for a seasoned professional with expertise in designing, implementing, and maintaining large-scale data systems.Key ResponsibilitiesLeverage deep technical expertise to drive...


  • Pune, Maharashtra, India Citigroup Full time

    The Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team The overall objective of this role is to contribute to applications systems analysis and programming...


  • Pune, Maharashtra, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Job Title:Big Data EngineerWe are seeking a skilled Big Data Engineer to design, develop, and maintain large-scale data systems. This role requires expertise in big data technologies, data integration, and data architecture.Key Responsibilities:Design and develop scalable big data architectures and pipelines for efficient data processing and...


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Big Data Engineer PositionOur organization is looking for a highly skilled Big Data Engineer to join our team.Required Skills:Must have 3-8 years of strong experience in Java J2EE technologies with expertise in big data technology stacks such as Hadoop, Spark, Kafka, HBase, and Hive.Strong fundamentals including data structures, algorithms, object-oriented...