iKrux - Big Data Engineer - Python/Scala

2 weeks ago


Pune, India Ikrux Engineeering Full time

We are seeking a highly skilled and motivated Big Data Engineer with strong experience in building, optimizing, and managing large-scale data processing pipelines. The ideal candidate will have hands-on expertise in Scala, Python, Apache Spark, Kafka, and/or Apache NiFi, and will work closely with cross-functional teams to design and implement scalable data solutions that drive business intelligence and analytics.Key Responsibilities : - Design, build, and maintain large-scale distributed data processing systems using Spark, Kafka, and/or NiFi.- Write efficient and reusable code in Scala and Python for data transformation, ingestion, and enrichment.- Collaborate with data scientists, analysts, and engineers to ensure data quality, consistency, and availability.- Optimize data pipelines for performance, reliability, and scalability.- Integrate diverse data sources, ensuring data governance and security compliance.- Monitor and troubleshoot data workflows, resolving bottlenecks and issues.- Participate in code reviews, architecture discussions, and best practice development.- Stay updated on emerging big data technologies and frameworks to continuously improve system efficiency.Required Skills and Experience : - Strong programming experience in Scala and Python for data engineering.- Extensive experience with Apache Spark, including optimization, tuning, and resource management.- Hands-on experience with Kafka for real-time data streaming and messaging systems.- Experience with Apache NiFi for data flow automation and integration.- Working knowledge of Hadoop ecosystem components is a plus.- Understanding of cloud-based data solutions (AWS, Azure, or GCP) is desirable.- Familiarity with relational and NoSQL databases (e.g., PostgreSQL, MongoDB, Cassandra).- Experience in working with data warehousing solutions and ETL pipelines.- Good analytical, problem-solving, and debugging skills.- Excellent communication and teamwork abilities. (ref:hirist.tech)



  • Pune, India Ikrux Engineeering Full time

    Job Description :We are seeking a highly skilled Big Data Tester with strong expertise in ETL Testing, Hadoop, and Spark/PySpark. The candidate will be responsible for ensuring data quality, validating large-scale data pipelines, and working closely with data engineers and analysts to deliver reliable big data solutions.Key Responsibilities :- Perform...

  • Big Data Engineer

    5 days ago


    Pune, India Leinex Consulting Full time

    Role & responsibilities : - Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders- Design, develop, and maintain highly scalable data processing applications- Write efficient, reusable and well documented code- Deliver big data projects using Spark, Scala , Python, SQL- Maintain and tune existing Spark...


  • Pune, India LIFTU TECHNOLOGY PRIVATE LIMITED Full time

    Job Summary :We are seeking an experienced Big Data Architect to lead the design and implementation of scalable, secure, and high-performing big data solutions. The ideal candidate will possess a deep understanding of big data technologies, data modeling, and cloud-based data services. You will collaborate with cross-functional teams including data...


  • Pune, India Consulting Pandits Full time

    Exciting Job Opportunity in Pune!Are you an experienced Senior Big Data Engineer with over 5 years of expertise? We have a fantastic opportunity waiting for you!Key Requirements :- Minimum of 5+ years of experience- Proficient in Spark development for batch and streaming applications- Skilled in Pyspark and Data Engineering- Expert in ETL implementation and...

  • Big Data Engineer

    4 weeks ago


    Pune, India Funic Tech Full time

    Description :Job Opening : Big Data Engineer (7-10 yrs)Location : Aundh, Pune (Work From Office)Job Type : Full-Time | General Shift | Monday FridayKey Responsibilities :- Build and manage highly scalable Big Data pipelines for large structured datasets.- Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.-...

  • Big Data Architect

    1 week ago


    Pune, India Talent Velocity Full time

    Job Title : Big Data ArchitectExperience : 8+ YearsLocation : Nagpur / Pune (Onsite 5 Days Working)Notice Period : 0 to 30 days maxAbout the Role :We are looking for a highly skilled Big Data Architect to design and deliver scalable, production-grade big data solutions integrating Hadoop ecosystems with cloud platforms. The ideal candidate will have hands-on...

  • Big Data Engineer

    2 weeks ago


    Pune, India HyrEzy Talent Solutions Full time

    Role : Big Data EngineerAbout Organization :Our Client is an IT consulting organization focused on providing AI Led Digital Transformation Consulting services to enterprise customers. They have adopted AI First strategy to drive value in form of Agility, Quality, Efficiency, Competitive advantage, Customer experience as core principles. Our focus is to drive...


  • Pune, India Coforge Full time

    We are looking for a passionate and experienced AWS-Scala Data Engineer to join our growing data engineering team. This role requires a strong foundation in cloud technologies, data processing, and programming, with a focus on building scalable and efficient data solutions.Key Responsibilities:- Design, develop, and maintain scalable data pipelines on AWS.-...


  • Pune, Maharashtra, India LTIMindtree Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job description:Mandate Skills: Scala, Spark, SQLResponsibility:Overall, more than 5 Yrs of experience in Big Data projects.Experience in Scala programming languagesExperience in Big Data technologies including Spark Scala and KafkaWho have a good understanding of data structures and algorithmsHas worked in Implementation projects building data pipelines...

  • DHI Solutions

    3 weeks ago


    Pune, India DHI Solutions Full time

    Job Description : We are seeking a highly skilled Data Engineer (Scala) with strong expertise in Big Data, Cloud, and Data Pipeline development. The ideal candidate will have hands-on experience in designing, building, and optimizing large-scale data solutions and should be comfortable working in a collaborative, client-facing environment.Key...