
Big Data Engineer
6 days ago
Role & responsibilities :
- Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders
- Design, develop, and maintain highly scalable data processing applications
- Write efficient, reusable and well documented code
- Deliver big data projects using Spark, Scala , Python, SQL
- Maintain and tune existing Spark applications to the fullest.
- Find opportunities for optimizing existing spark applications.
- Work closely with QA, Operations and various teams to deliver error free software on time
- Actively lead / participate daily agile / scrum meetings
- Take responsibility for Apache Spark development and implementation
- Translate complex technical and functional requirements into detailed designs
- Investigate alternatives for data storing and processing to ensure implementation of the most streamlined solutions
- Serve as a mentor for junior staff members by conducting technical training sessions and reviewing project outputs
Qualifications :
- Engineering graduates in computer science backgrounds preferred with 8 years of software development experience with Hadoop framework components(HDFS, Spark, Spark, Scala, PySpark)
- Excellent at verbal, written and presentation skills
- Ability to present and defend a solution with technical facts & business proficiency
- Understanding of data-warehousing and data-modeling techniques
- Strong data engineering skills
- Knowledge of Core Java, Linux, SQL, and any scripting language
- At least 6 years of experience using Python / Scala, Spark, SQL
- Knowledge of shell scripting is a plus
- Knowledge of Core and Advance Java is a plus.
- Experience in developing and tuning spark applications
- Excellent understanding of spark architecture, data frames and tuning spark
- Strong knowledge of database concepts, systems architecture, and data structures is a must Process oriented with strong analytical and problem solving skills
- Experience in writing Python standalone applications dealing with PySpark API
- Knowledge of DELTA.
- IO package is a plus
-
Big Data Engineer
3 weeks ago
Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full timeBig Data Engineer (PySpark)Location: Pune/Nagpur (WFO)Experience: 8 - 12 YearsEmployment Type: Full-timeJob OverviewWe are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...
-
Big Data Engineer
4 weeks ago
Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full timeBig Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...
-
Big Data Engineer
2 days ago
Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time ₹ 20,00,000 - ₹ 25,00,000 per yearBig Data Engineer (PySpark)Location:Pune/Nagpur (WFO)Experience:8 - 12 YearsEmployment Type:Full-timeJob OverviewWe are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance...
-
Big Data Engineer
3 weeks ago
Pune, Maharashtra, India Merito Full timeJob Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...
-
Big Data Engineer
5 days ago
Pune, Maharashtra, India RiskInsight Consulting Pvt Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per yearResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databasesBuild and maintain scalable data pipelines for effective data ingestion, transformation, and analysisCollaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them...
-
Big Data Engineer
7 days ago
Pune, Maharashtra, India Information Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per yearBig Data EngineerKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal solutions. - Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks. -...
-
Big Data Engineer
3 weeks ago
Pune, Maharashtra, India WorkiFicient Full timeTo be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...
-
Big Data Engineer
7 days ago
Pune, Maharashtra, India Information Technology Full time ₹ 15,00,000 - ₹ 25,00,000 per yearKey ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design discussions and contribute to selecting, integrating, and maintaining Big Data frameworks.Develop distributed data processing systems...
-
Big Data Developer
2 days ago
Pune, Maharashtra, India Wroots Global Full time ₹ 15,00,000 - ₹ 25,00,000 per yearJob Title: Big Data Engineer / Software EngineerLocation: Pune Kharadi (Client Location) | WFO 3 days a weekExperience: 6 to 10 yearsJob Description:We are hiring an experienced Software Engineer with strong expertise in big data and cloud technologies.Key Skills:Hands-on experience in Scala / Java / PythonStrong knowledge of Big Data technologies: Spark,...
-
Big Data Engineer
2 days ago
Pune, Maharashtra, India Capco Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJob Title:Big Data Engineer - ScalaAbout Us"Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 byAvtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across...