Big Data Specialist

2 weeks ago

Pune, Maharashtra, India Atomic North Full time

Big Data Engineer

Location: Pune/Nagpur (WFO)

Experience Required: 8+ Years

**Long term onsite to Abu Dhabi**

Job Summary
:

  • We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.
  • The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.

Key Responsibilities:


• Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.


• Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.


• Write efficient HiveQL and SQL queries to handle large-scale datasets.


• Perform performance tuning and optimization of distributed data systems.


• Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
• Manage and schedule workflows using Apache Airflow or Oozie.


• Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.

Required Skills
:


• Proven experience in Big Data Engineering with a focus on PySpark.


• Strong knowledge of HDFS, Hive, Sqoop, and related tools.


• Proficiency in SQL/HiveQL for large datasets.


• Expertise in performance tuning and optimization of distributed systems.


• Familiarity with Agile methodology and collaborative team practices.


• Experience with workflow orchestration tools (Airflow/Oozie).


• Strong problem-solving, analytical, and communication skills.

Good to Have:


• Knowledge of data modeling and data warehousing concepts.


• Exposure to DevOps practices and CI/CD pipelines for data engineering.


• Experience with other Big Data frameworks such as Spark Streaming or Kafka.

  • Big Data Lead

    2 weeks ago

    Pune, Maharashtra, India Populace World Solutions Full time

    Designation: Big Data LeadLocation: PuneExperience: 8-10 yearsReports To – Product Engineering HeadJob OverviewWe are looking to hire a talented big data engineer to develop and manage our company's Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with...

  • Big Data Developer

    37 minutes ago

    Pune, Maharashtra, India Talent Worx Full time

    We're hiring for one of the world's leading professional services firms, renowned for its commitment to innovation, excellence, and global impact. With a presence in over 150 countries, this organization provides services across consulting, audit, tax, risk advisory, and financial advisory — helping Fortune 500 companies and governments navigate complex...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Real Jobs Consulting Services Full time

    Company DescriptionReal Jobs Consulting Services, based in Gurgaon, is a trusted provider of comprehensive placement services. With expertise in HR consultancy, placement consultancy, manpower recruitment, and career guidance, the company is dedicated to enhancing business potential through strategic and tailored solutions. Our team of professionals excels...

  • Big Data Engineer

    5 days ago

    Pune, Maharashtra, India Vodafone Full time

    Design, develop, and maintain scalable data pipelines for processing structured and unstructured data. Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses. Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows. Ensure high performance and...

  • Big Data Architect

    2 days ago

    Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full time

    Experience: 10-12 yearsKey Responsibilities:Data Architecture Design: Designing and implementing scalable and resilient dataarchitectures for both batch and streaming data processing.Data Modeling: Developing data models and database structures to ensure efficientdata storage and retrieval.Data Security and Governance: Ensuring data security, integrity, and...

  • Big Data Lead

    2 weeks ago

    Pune, Maharashtra, India Persistent Full time

    About PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Aexonic Technologies Full time

    Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Net2source Llp Full time

    Interested candidates, please share your updated profile at: Were Hiring | Engineer (Big Data) | Pune, MHWe are hiring an Engineer (Big Data) for an exciting opportunity with one of our reputed clients. If you have strong experience in Apache Spark and modern data engineering platforms, this role could be a great fit for you.Job Title: EngineerWork Location:...

  • Big Data Solution Architect

    7 days ago

    Pune, Maharashtra, India Emergys Full time

    Work Location:PuneNotice Period:Immediate to 30 DaysExperience:8+ YearsMust Have Skills:Hadoop Ecosystem Expertise – Strong hands-on experience with core Hadoop components and related big data technologies.DevOps & Cloud (Preferably Azure) – Experience using DevOps tools and CI/CD processes on cloud platforms, with a preference for Microsoft...

  • Big Data Platform Engineer

    2 weeks ago

    Pune, Maharashtra, India Cognyte Full time

    Big Data Platform Engineer / BD Site Reliability Engineer (SRE) - Job RequirementsPosition OverviewWe are seeking a Big Data Platform Engineer to support our enterprise customers' big data infrastructure deployments and operations. This role requires a hands-on technical professional who can work both remotely and on-site at customer locations, including...