
Big Data Engineer
24 hours ago
Location: Pune/Nagpur (WFO)
Experience: 8 - 12 Years
Employment Type: Full-time
Job Overview
We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability.
Key Responsibilities
- Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
- Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
- Write efficient HiveQL and SQL queries to handle large-scale datasets.
- Perform performance tuning and optimization of distributed data systems.
- Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions.
- Manage and schedule workflows using Apache Airflow or Oozie.
- Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
Required Skills
- Proven experience in Big Data Engineering with a focus on PySpark.
- Strong knowledge of HDFS, Hive, Sqoop, and related tools.
- Proficiency in SQL/HiveQL for large datasets.
- Expertise in performance tuning and optimization of distributed systems.
- Familiarity with Agile methodology and collaborative team practices.
- Experience with workflow orchestration tools (Airflow/Oozie).
- Strong problem-solving, analytical, and communication skills.
Good to Have
- Knowledge of data modeling and data warehousing concepts.
- Exposure to DevOps practices and CI/CD pipelines for data engineering.
- Experience with other Big Data frameworks such as Spark Streaming or Kafka.
-
Big Data Engineer
5 days ago
Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per yearBig Data Engineer (PySpark)Location:Pune/Nagpur (WFO)Experience:8 - 12 YearsEmployment Type:Full-timeJob OverviewWe are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance...
-
Big Data Engineer
4 days ago
Pune, Maharashtra, India Merito Full timeJob Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...
-
Big Data Engineer
3 weeks ago
Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full timeJob Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...
-
Big Data Engineer
1 week ago
Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000Big Data Engineer">We are seeking a highly skilled Big Data Engineer to drive the design and implementation of scalable data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. This is an excellent opportunity for experienced professionals to join our organization and leverage their expertise to drive business outcomes through...
-
Big Data Engineer
5 days ago
Pune, Maharashtra, India WorkiFicient Full timeTo be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...
-
Big Data Engineer
5 days ago
Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000Job Title:Big Data EngineerAbout the RoleWe're seeking an experienced data engineer to lead the development of large-scale data processing pipelines using Apache Spark and related technologies.The ideal candidate will have a strong background in data warehousing, ETL concepts, and SQL/NoSQL operations.Familiarity with UNIX shell scripting for scheduling and...
-
Senior Big Data Engineer
5 days ago
Pune, Maharashtra, India Devlats Pvt Ltd Full time US$ 1,50,000 - US$ 2,00,000 per yearAbout The RoleWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in designing, building, and optimizing data pipelines and architectures, as well as a solid understanding of big data tools and Responsibilities :Design, develop, and maintain scalable and high-performance data...
-
Senior Big Data Engineer
7 days ago
Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000Our company is seeking a highly skilled Data Architect to design, develop and implement scalable data solutions using cutting-edge technologies.Key Responsibilities:Data Ingestion and Processing: Develop and maintain high-quality code, adhering to industry standards and best practices, for efficient data ingestion from disparate sources onto our Big Data...
-
Big Data Engineering Leadership
7 days ago
Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job OverviewWe are seeking ambitious professionals for the role of Lead Apache Hadoop Engineer in our Technology, Data, and Innovation (TDI) team.Key ResponsibilitiesDevelop robust architectures and designs for big data platforms and applications within the Apache Hadoop ecosystem.Implement and deploy big data platforms and solutions on-premises and in...
-
Big Data Engineering Specialist
6 days ago
Pune, Maharashtra, India beBeeBigData Full time ₹ 1,04,000 - ₹ 1,30,878We are seeking a seasoned professional to fill the role of Big Data Engineering Specialist. This is an exceptional opportunity for individuals with expertise in designing and developing large-scale data processing systems.The ideal candidate will have extensive knowledge of big data technologies and a strong track record of delivering projects on time and...