Big Data Specialist

3 days ago

Pune, Maharashtra, India Atomic North Full time ₹ 12,00,000 - ₹ 24,00,000 per year

Big Data Engineer

Location: Pune/Nagpur (WFO)

Experience Required: 8+ Years

**Long term onsite to Abu Dhabi**

Job Summary
:

  • We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.
  • The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities:


• Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.


• Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.


• Write efficient HiveQL and SQL queries to handle large-scale datasets.


• Perform performance tuning and optimization of distributed data systems.


• Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
• Manage and schedule workflows using Apache Airflow or Oozie.


• Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills
:


• Proven experience in Big Data Engineering with a focus on PySpark.


• Strong knowledge of HDFS, Hive, Sqoop, and related tools.


• Proficiency in SQL/HiveQL for large datasets.


• Expertise in performance tuning and optimization of distributed systems.


• Familiarity with Agile methodology and collaborative team practices.


• Experience with workflow orchestration tools (Airflow/Oozie).


• Strong problem-solving, analytical, and communication skills.

Good to Have:


• Knowledge of data modeling and data warehousing concepts.


• Exposure to DevOps practices and CI/CD pipelines for data engineering.


• Experience with other Big Data frameworks such as Spark Streaming or Kafka.

  • Big Data Lead

    1 week ago

    Pune, Maharashtra, India Populace World Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Designation: Big Data LeadLocation: PuneExperience: 8-10 yearsReports To – Product Engineering HeadJob OverviewWe are looking to hire a talented big data engineer to develop and manage our company's Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with...

  • Big Data Tester

    5 days ago

    Pune, Maharashtra, India Shavish Hr And Digital Marketing Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Big data Tester have a good Big Data testing, focusing on both automated & manual testing for data validation.Strong understanding of Hadoop ecosystem.Skilled in writing & executing SQL queries. Hands-on exp. with JIRA or Bugzilla.

  • Big Data Developer

    2 weeks ago

    Pune, Maharashtra, India Populace World Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Description : Big Data DeveloperLocation : Pune, IndiaExperience : 7 YearsAbout the Role : We are seeking a highly experienced and technically proficient Big Data Developer to join our team in Pune. In this role, you will be responsible for designing, developing, and optimizing large-scale data processing pipelines and platforms. The ideal candidate...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India RiskInsight Consulting Pvt Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    ResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databasesBuild and maintain scalable data pipelines for effective data ingestion, transformation, and analysisCollaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them...

  • Big Data Engineer

    1 day ago

    Pune, Maharashtra, India Talent Sketchers Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Designation: Big Data EngineerExperience: 4+ YearsWork Mode: Remote OpportunityNotice Period: Immediate Joiners/ Serving Notice PeriodJob Description:Role: Big Data EngineerThis Data Engineer will be engaged in data science-related research and software application development and engineering duties related to our enterprise-grade Wi-Fi technology to provide...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Coforge Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala.Experience Required: 3 to 6 YearsJob Locations: Pune, Hyderabad, Greater Noida Only.Send your CV to: For queries, contact via WhatsApp: Key Responsibilities:-• Design, develop, and optimize Big Data architectures leveraging AWS services for large-scale,...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India Information Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Big Data EngineerKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal solutions. - Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks. -...

  • Big Data Lead

    5 days ago

    Pune, Maharashtra, India Persistent Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many...

  • Big Data Architect

    1 week ago

    Pune, Maharashtra, India Datametica Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Big Data Architect GCPExperience: 10–12 YearsLocation: PuneRoleWe are looking for a Big Data Architect with deep GCP expertise to define, design, and lead cloud-native data platforms for enterprise clients. You will be responsible for architecting large-scale data lakes, warehouses, and pipelines on GCP, modernizing on-premise workloads, and...

  • Big Data Lead

    1 week ago

    Pune, Maharashtra, India DemandMatrix Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company Description At DemandMatrix, our vision is to disrupt the $100 billion sales and marketing intelligence industry by using domain knowledge, machine learning and AI. Fortune 100 companies like Microsoft, Google, Adobe, Amazon, IBM trust us to identify their next customer. Job Description What will you do?To help us go to the next level we are looking...