iKrux - Big Data Engineer - Python/Scala
2 weeks ago
We are seeking a highly skilled and motivated Big Data Engineer with strong experience in building, optimizing, and managing large-scale data processing pipelines. The ideal candidate will have hands-on expertise in Scala, Python, Apache Spark, Kafka, and/or Apache NiFi, and will work closely with cross-functional teams to design and implement scalable data solutions that drive business intelligence and analytics.Key Responsibilities : - Design, build, and maintain large-scale distributed data processing systems using Spark, Kafka, and/or NiFi.- Write efficient and reusable code in Scala and Python for data transformation, ingestion, and enrichment.- Collaborate with data scientists, analysts, and engineers to ensure data quality, consistency, and availability.- Optimize data pipelines for performance, reliability, and scalability.- Integrate diverse data sources, ensuring data governance and security compliance.- Monitor and troubleshoot data workflows, resolving bottlenecks and issues.- Participate in code reviews, architecture discussions, and best practice development.- Stay updated on emerging big data technologies and frameworks to continuously improve system efficiency.Required Skills and Experience : - Strong programming experience in Scala and Python for data engineering.- Extensive experience with Apache Spark, including optimization, tuning, and resource management.- Hands-on experience with Kafka for real-time data streaming and messaging systems.- Experience with Apache NiFi for data flow automation and integration.- Working knowledge of Hadoop ecosystem components is a plus.- Understanding of cloud-based data solutions (AWS, Azure, or GCP) is desirable.- Familiarity with relational and NoSQL databases (e.g., PostgreSQL, MongoDB, Cassandra).- Experience in working with data warehousing solutions and ETL pipelines.- Good analytical, problem-solving, and debugging skills.- Excellent communication and teamwork abilities. (ref:hirist.tech)
-
iKrux - Big Data Tester
5 days ago
Pune, India Ikrux Engineeering Full timeJob Description :We are seeking a highly skilled Big Data Tester with strong expertise in ETL Testing, Hadoop, and Spark/PySpark. The candidate will be responsible for ensuring data quality, validating large-scale data pipelines, and working closely with data engineers and analysts to deliver reliable big data solutions.Key Responsibilities :- Perform...
-
Big Data Engineer
5 days ago
Pune, India Leinex Consulting Full timeRole & responsibilities : - Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders- Design, develop, and maintain highly scalable data processing applications- Write efficient, reusable and well documented code- Deliver big data projects using Spark, Scala , Python, SQL- Maintain and tune existing Spark...
-
Senior Big Data Architect
3 weeks ago
Pune, India LIFTU TECHNOLOGY PRIVATE LIMITED Full timeJob Summary :We are seeking an experienced Big Data Architect to lead the design and implementation of scalable, secure, and high-performing big data solutions. The ideal candidate will possess a deep understanding of big data technologies, data modeling, and cloud-based data services. You will collaborate with cross-functional teams including data...
-
Senior Big Data Engineer
2 weeks ago
Pune, India Consulting Pandits Full timeExciting Job Opportunity in Pune!Are you an experienced Senior Big Data Engineer with over 5 years of expertise? We have a fantastic opportunity waiting for you!Key Requirements :- Minimum of 5+ years of experience- Proficient in Spark development for batch and streaming applications- Skilled in Pyspark and Data Engineering- Expert in ETL implementation and...
-
Big Data Engineer
4 weeks ago
Pune, India Funic Tech Full timeDescription :Job Opening : Big Data Engineer (7-10 yrs)Location : Aundh, Pune (Work From Office)Job Type : Full-Time | General Shift | Monday FridayKey Responsibilities :- Build and manage highly scalable Big Data pipelines for large structured datasets.- Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.-...
-
Big Data Architect
1 week ago
Pune, India Talent Velocity Full timeJob Title : Big Data ArchitectExperience : 8+ YearsLocation : Nagpur / Pune (Onsite 5 Days Working)Notice Period : 0 to 30 days maxAbout the Role :We are looking for a highly skilled Big Data Architect to design and deliver scalable, production-grade big data solutions integrating Hadoop ecosystems with cloud platforms. The ideal candidate will have hands-on...
-
Big Data Engineer
2 weeks ago
Pune, India HyrEzy Talent Solutions Full timeRole : Big Data EngineerAbout Organization :Our Client is an IT consulting organization focused on providing AI Led Digital Transformation Consulting services to enterprise customers. They have adopted AI First strategy to drive value in form of Agility, Quality, Efficiency, Competitive advantage, Customer experience as core principles. Our focus is to drive...
-
AWS-Scala Data Engineer
4 weeks ago
Pune, India Coforge Full timeWe are looking for a passionate and experienced AWS-Scala Data Engineer to join our growing data engineering team. This role requires a strong foundation in cloud technologies, data processing, and programming, with a focus on building scalable and efficient data solutions.Key Responsibilities:- Design, develop, and maintain scalable data pipelines on AWS.-...
-
Senior Big Data Engineer
1 week ago
Pune, Maharashtra, India LTIMindtree Full time ₹ 6,00,000 - ₹ 18,00,000 per yearJob description:Mandate Skills: Scala, Spark, SQLResponsibility:Overall, more than 5 Yrs of experience in Big Data projects.Experience in Scala programming languagesExperience in Big Data technologies including Spark Scala and KafkaWho have a good understanding of data structures and algorithmsHas worked in Implementation projects building data pipelines...
-
DHI Solutions
3 weeks ago
Pune, India DHI Solutions Full timeJob Description : We are seeking a highly skilled Data Engineer (Scala) with strong expertise in Big Data, Cloud, and Data Pipeline development. The ideal candidate will have hands-on experience in designing, building, and optimizing large-scale data solutions and should be comfortable working in a collaborative, client-facing environment.Key...