Big Data Engineer

3 weeks ago

Pune, India Leinex Consulting Full time

Role & responsibilities :

- Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders

- Design, develop, and maintain highly scalable data processing applications

- Write efficient, reusable and well documented code

- Deliver big data projects using Spark, Scala , Python, SQL

- Maintain and tune existing Spark applications to the fullest.

- Find opportunities for optimizing existing spark applications.

- Work closely with QA, Operations and various teams to deliver error free software on time

- Actively lead / participate daily agile / scrum meetings

- Take responsibility for Apache Spark development and implementation

- Translate complex technical and functional requirements into detailed designs

- Investigate alternatives for data storing and processing to ensure implementation of the most streamlined solutions

- Serve as a mentor for junior staff members by conducting technical training sessions and reviewing project outputs

Qualifications :

- Engineering graduates in computer science backgrounds preferred with 8+ years of software development experience with Hadoop framework components(HDFS, Spark, Spark, Scala, PySpark)

- Excellent at verbal, written and presentation skills

- Ability to present and defend a solution with technical facts & business proficiency

- Understanding of data-warehousing and data-modeling techniques

- Strong data engineering skills

- Knowledge of Core Java, Linux, SQL, and any scripting language

- At least 6+ years of experience using Python / Scala, Spark, SQL

- Knowledge of shell scripting is a plus

- Knowledge of Core and Advance Java is a plus.

- Experience in developing and tuning spark applications

- Excellent understanding of spark architecture, data frames and tuning spark

- Strong knowledge of database concepts, systems architecture, and data structures is a must Process oriented with strong analytical and problem solving skills

- Experience in writing Python standalone applications dealing with PySpark API

- Knowledge of DELTA.

- IO package is a plus


(ref:hirist.tech)

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Talent Sketchers Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Designation: Big Data EngineerExperience: 4+ YearsWork Mode: Remote OpportunityNotice Period: Immediate Joiners/ Serving Notice PeriodJob Description:Role: Big Data EngineerThis Data Engineer will be engaged in data science-related research and software application development and engineering duties related to our enterprise-grade Wi-Fi technology to provide...

  • Big Data Engineer

    4 weeks ago

    Pune, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location:Pune/Nagpur (WFO) Experience:8 - 12 Years Employment Type:Full-time Job Overview We are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Funic Tech Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Big Data EngineerLocation: Aundh, Pune (Work From Office)Job Type: Full-Time | General Shift | Monday – FridayKey ResponsibilitiesBuild and manage highly scalable Big Data pipelines for large structured datasets.Collaborate with Product Management & Engineering leadership to design and implement optimal solutions.Participate in design...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India FUNIC TECH PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description :Job Opening : Big Data Engineer (7-10 yrs)Location : Aundh, Pune (Work From Office)Job Type : Full-Time | General Shift | Monday FridayKey Responsibilities : - Build and manage highly scalable Big Data pipelines for large structured datasets. - Collaborate with Product Management & Engineering leadership to design and implement optimal...

  • Big Data Engineer

    2 days ago

    Pune, India WorkiFicient Full time

    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...

  • Big Data Engineer

    23 hours ago

    Pune, India Coforge Full time

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala.Experience Required: 3 to 6 YearsJob Locations: Pune, Hyderabad, Greater Noida Only.Send your CV to: queries, contact via WhatsApp: Key Responsibilities:-• Design, develop, and optimize Big Data architectures leveraging AWS services for large-scale,...

  • Big Data Engineer

    17 hours ago

    Pune, India Coforge Full time

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala. Experience Required: 3 to 6 Years Job Locations: Pune, Hyderabad, Greater Noida Only. Send your CV to: Gaurav.2.Kumar@coforge.com For queries, contact via WhatsApp: 9667427662 Key Responsibilities:-• Design, develop, and optimize Big Data architectures...

  • Big Data Engineer

    5 hours ago

    Pune, India Coforge Full time

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala. Experience Required: 3 to 6 Years Job Locations: Pune, Hyderabad, Greater Noida Only. Send your CV to: Gaurav.2.Kumar@coforge.com For queries, contact via WhatsApp: 9667427662 Key Responsibilities:-• Design, develop, and optimize Big Data architectures...

  • Big Data Engineer

    6 hours ago

    pune, India Coforge Full time

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala. Experience Required: 3 to 6 Years Job Locations: Pune, Hyderabad, Greater Noida Only. Send your CV to: Gaurav.2.Kumar@coforge.com For queries, contact via WhatsApp: 9667427662 Key Responsibilities:-• Design, develop, and optimize Big Data architectures...

  • Big Data Engineer

    18 hours ago

    Pune, India Coforge Full time

    Coforge Ltd is Hiring for Big Data Engineer – AWS, Spark & Scala.Must Have Skills:- AWS, Spark & Scala. Experience Required: 3 to 6 Years Job Locations: Pune, Hyderabad, Greater Noida Only. Send your CV to: Gaurav.2.Kumar@coforge.com For queries, contact via WhatsApp: 9667427662 Key Responsibilities:-• Design, develop, and optimize Big Data architectures...