Big data specialist

4 weeks ago

Pune, India Atomic North Full time

Big Data EngineerLocation: Pune/Nagpur (WFO)Experience Required: 8+ Years**Long term onsite to Abu Dhabi**Job Summary:- We are looking for an experienced Big Data Engineer with strong expertise in Py Spark and Big Data ecosystems.- The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.Key Responsibilities:• Design, develop, and maintain data pipelines using Py Spark and related Big Data technologies.• Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.• Write efficient Hive QL and SQL queries to handle large-scale datasets.• Perform performance tuning and optimization of distributed data systems.• Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions. • Manage and schedule workflows using Apache Airflow or Oozie.• Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.Required Skills:• Proven experience in Big Data Engineering with a focus on Py Spark.• Strong knowledge of HDFS, Hive, Sqoop, and related tools.• Proficiency in SQL/Hive QL for large datasets.• Expertise in performance tuning and optimization of distributed systems.• Familiarity with Agile methodology and collaborative team practices.• Experience with workflow orchestration tools (Airflow/Oozie).• Strong problem-solving, analytical, and communication skills.Good to Have:• Knowledge of data modeling and data warehousing concepts.• Exposure to Dev Ops practices and CI/CD pipelines for data engineering.• Experience with other Big Data frameworks such as Spark Streaming or Kafka.

  • Big Data Specialist

    3 weeks ago

    Pune, India Emergys Full time

    Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 DaysExperience: 8+ YearsMust Have Skills:- Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.- Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple...

  • Big data specialist

    21 hours ago

    Pune, India Emergys Full time

    Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 DaysExperience: 8+ YearsMust Have Skills:- Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.- Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple...

  • Big Data Specialist

    3 weeks ago

    Pune, India Emergys Full time

    Work Location: Pune (Work-from-Office) ⏳ Notice Period: Immediate to 30 Days Experience: 8+ Years Must Have Skills: Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems. Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple...

  • Big Data Specialist

    3 weeks ago

    Pune, India Emergys Full time

    📍 Work Location: Pune (Work-from-Office) ⏳ Notice Period: Immediate to 30 Days 💼 Experience: 8+ Years Must Have Skills: - Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems. - Data Modeling & Integration – Strong ability to build efficient data models and integrate data...

  • Big Data Specialist

    3 days ago

    Pune, Maharashtra, India Emergys Full time

    Work Location:Pune(Work-from-Office)Notice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple sources.Big...

  • Big Data Specialist

    3 weeks ago

    Pune, India Emergys Full time

    Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 DaysExperience: 8+ YearsMust Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple...

  • Big Data Specialist

    3 weeks ago

    Pune, India Emergys Full time

    📍 Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 Days💼 Experience: 8+ Years Must Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from...

  • Big Data Specialist

    7 hours ago

    pune, India Emergys Full time

    📍 Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 Days💼 Experience: 8+ Years Must Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from...

  • Advanced Big Data Specialist

    1 week ago

    pune, India beBeeData Full time

    Big Data Tester RoleAs a big data tester, you will play a key role in ensuring the quality and integrity of our large-scale data processing systems.Key Responsibilities:Design and execute detailed test plans to validate ETL processes and data pipelinesDevelop and maintain strong SQL, HiveQL, and Spark SQL skills to analyze and validate large...

  • Big Data Specialist

    3 weeks ago

    Pune, Maharashtra, India, Maharashtra Emergys Full time

    Work Location: Pune (Work-from-Office)⏳ Notice Period: Immediate to 30 Days Experience: 8+ Years Must Have Skills:Data Architecture & Design – Expertise in designing scalable and resilient data architectures for batch and streaming systems.Data Modeling & Integration – Strong ability to build efficient data models and integrate data from multiple...