
Big Data Engineer
1 day ago
Big Data Engineer (PySpark)
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
- Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
- Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
- Write efficient HiveQL and SQL queries to handle large-scale datasets.
- Perform performance tuning and optimization of distributed data systems.
- Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
- Manage and schedule workflows using Apache Airflow or Oozie.
- Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
- Proven experience in Big Data Engineering with a focus on PySpark.
- Strong knowledge of HDFS, Hive, Sqoop, and related tools.
- Proficiency in SQL/HiveQL for large datasets.
- Expertise in performance tuning and optimization of distributed systems.
- Familiarity with Agile methodology and collaborative team practices.
- Experience with workflow orchestration tools (Airflow/Oozie).
- Strong problem-solving, analytical, and communication skills.
Good to Have
- Knowledge of data modeling and data warehousing concepts.
- Exposure to DevOps practices and CI/CD pipelines for data engineering.
- Experience with other Big Data frameworks such as Spark Streaming or Kafka.
-
Big Data Engineer
2 weeks ago
Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full timeJob Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...
-
Big Data Engineer
3 days ago
Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Big Data Engineer">We are seeking a highly skilled Big Data Engineer to drive the design and implementation of scalable data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. This is an excellent opportunity for experienced professionals to join our organization and leverage their expertise to drive business outcomes through...
-
Big Data Engineer
3 weeks ago
Pune, Maharashtra, India WorkiFicient Full timeRoles and Responsibility :- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.- Design and implement systems to cleanse,...
-
Senior Big Data Engineer
4 weeks ago
Pune, Maharashtra, India Devlats Pvt Ltd Full timeAbout the Role : We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in designing, building, and optimizing data pipelines and architectures, as well as a solid understanding of big data tools and Responsibilities : - Design, develop, and maintain scalable and high-performance...
-
Senior Big Data Engineer
3 hours ago
Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000Our company is seeking a highly skilled Data Architect to design, develop and implement scalable data solutions using cutting-edge technologies.Key Responsibilities:Data Ingestion and Processing: Develop and maintain high-quality code, adhering to industry standards and best practices, for efficient data ingestion from disparate sources onto our Big Data...
-
Big Data Engineering Leadership
11 hours ago
Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job OverviewWe are seeking ambitious professionals for the role of Lead Apache Hadoop Engineer in our Technology, Data, and Innovation (TDI) team.Key ResponsibilitiesDevelop robust architectures and designs for big data platforms and applications within the Apache Hadoop ecosystem.Implement and deploy big data platforms and solutions on-premises and in...
-
Big Data Engineer
3 days ago
Pune, Maharashtra, India beBeeData Full time ₹ 13,70,400 - ₹ 25,10,800Job Title: Data ArchitectWe are seeking an experienced Data Architect to lead the development of our big data strategy.About the RoleThis is a highly visible and rewarding role for a seasoned professional with expertise in designing, implementing, and maintaining large-scale data systems.Key ResponsibilitiesLeverage deep technical expertise to drive...
-
Senior Big Data Engineer
4 weeks ago
Pune, Maharashtra, India Citigroup Full timeThe Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team The overall objective of this role is to contribute to applications systems analysis and programming...
-
Highly Skilled Big Data Engineer Required
3 days ago
Pune, Maharashtra, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000Job Title:Big Data EngineerWe are seeking a skilled Big Data Engineer to design, develop, and maintain large-scale data systems. This role requires expertise in big data technologies, data integration, and data architecture.Key Responsibilities:Design and develop scalable big data architectures and pipelines for efficient data processing and...
-
Big Data Technology Specialist
23 hours ago
Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878Big Data Engineer PositionOur organization is looking for a highly skilled Big Data Engineer to join our team.Required Skills:Must have 3-8 years of strong experience in Java J2EE technologies with expertise in big data technology stacks such as Hadoop, Spark, Kafka, HBase, and Hive.Strong fundamentals including data structures, algorithms, object-oriented...