Deloitte - Data Engineer - PySpark/Python

2 days ago

pune, India Deloitte India Full time

Job Description :

- Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.


- Implement the tools and processes, handling performance, scale, availability, accuracy, and monitoring.


- Liaison with BAs to ensure that requirements are correctly interpreted and implemented.

- Participation in regular planning and status meetings.


- Input to the development process through the involvement in Sprint reviews and retrospectives.


- Input into system architecture and design.

- Peer code reviews.

- Scala development and design using Scala 2.10+ or Java development and design using Java 1.8+.

- Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, PySpark, Spark streaming, YARN, Kafka, Hive, Python, ETL frameworks, Map Reduce, SQL, RESTFUL API services).

- Sound knowledge on working Unix/Linux Platform

- Hands-on experience building data pipelines using Hadoop components - Hive, Spark, Spark SQL, PySpark.

- Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible & Jenkins) and requirement management in JIRA.

- Understanding of big data modelling techniques using relational and non-relational techniques

- Experience on Debugging the Code issues and then publishing the highlighted differences to the development team/Architects.

Good to have skills :

- Experience with time-series/analytics dB's such as ElasticSearch.

- Experience with scheduling tools such as Airflow, Control-M.

- Understanding or experience of Cloud design patterns

- Exposure to DevOps & Agile Project methodology such as Scrum and Kanban.

- Experience with developing Hive QL, UDF's for analyzing semi structured/structured datasets.

(ref:hirist.tech)

  • Deloitte - Data Engineer - PySpark/Python

    2 weeks ago

    Pune, Maharashtra, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    3 weeks ago

    Pune, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. - Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    1 month ago

    Pune, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.- Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    1 month ago

    Pune, Maharashtra, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.- Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    1 month ago

    Pune, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting. - Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    3 weeks ago

    Pune, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.- Implement the tools and processes, handling performance, scale,...

  • Deloitte - Data Engineer - PySpark/Python

    4 weeks ago

    Pune, Maharashtra, India Deloitte India Full time

    Job Description : - Software design, Scala & Spark, PySpark & Python development, automated testing of new and existing components in an Agile, DevOps and dynamic environment Promoting development standards, code reviews, mentoring, knowledge sharing Production support & troubleshooting.- Implement the tools and processes, handling performance, scale,...

  • Data Engineer

    3 weeks ago

    Pune, India Overture Rede Private Limited. Full time

    Description: - 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    4 weeks ago

    Pune, India Overture Rede Private Limited. Full time

    Description:- 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    4 weeks ago

    pune, India Overture Rede Private Limited. Full time

    Description: - 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Overture Rede Private Limited. Full time

    Description:- 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    4 weeks ago

    Pune, India Overture Rede Private Limited. Full time

    Description: - 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    2 weeks ago

    pune, India Overture Rede Private Limited. Full time

    Description: - 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    3 weeks ago

    Pune, India Overture Rede Private Limited. Full time

    Description:- 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Overture Rede Private Limited. Full time

    Description:- 3 to 6 years years experience in Big Data tech stack- Strong development background- Java or scala 3-6 years- Spark 1-4 years. Certifications are a plus- Azure - 1-3 years. Certifications are a plus- Databricks 1-2 years. Certifications are a plus- Strong SQL background- UI Development experience is plusJob Requirements: - Participate in the...

  • Hiring For Pyspark Data Engineer

    3 days ago

    Pune, India timesjobs Full time

    Hiring For Pyspark Data EngineerDesired ProfileSkills: Pyspark, Scala, Hive, Airflow, Control M, CICD, Python, Hadoop, Data Modelling, Experience with Jira, Confluence, GitHub or other similar Agile Scrum technologiesNotice Period: immediate-45 daysEducation: full-time graduationLocation: Pune & HyderabadEmail: career@krazymantra.com

  • Azure Databricks Engineer

    6 days ago

    pune, India RapidBraiins Full time

    No. of openings - 10+Primary Skill : SQL, Azure DatabricksSecondary Skills : PySpark, Azure Data FactoryResponsibilities : As an Azure Data Engineer, your day-to-day work activities will be as follows : - Design & develop the ETL- Good experience in writing SQL, Python and PySpark programming- Create the Pipelines (simple and complex) using ADF.- Work with...

  • AWS Data Engineer

    3 weeks ago

    Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India Change Leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • AWS Data Engineer

    2 months ago

    Pune/Chennai/Mumbai/Bangalore/Delhi NCR/Hyderabad, India Change Leaders Full time

    Job Description : AWS Glue + Python PysparkJob Overview :Candidate should be responsible for the configuration, administration, and optimization of the AWS Glue + Python Pyspark platform. This role involves ensuring the smooth operation of AWS Glue + Python Pyspark for data integration, transformation, and loading tasks in alignment with the...

  • Hiring For Pyspark Data Engineer

    3 days ago

    Pune, India timesjobs Full time

    Job Title: Pyspark Data EngineerDesired Profile:Skills required:HadoopSpark/PysparkHiveYARNScala development and design using Scala 2.10 or PythonData ModellingOperating systems such as Windows and Linux/UnixExperience with Jira, Confluence, GitHub or other similar Agile Scrum technologiesTest methodologiesNotice Period: ImmediateEducation: Full-time...