PySpark Developer

1 week ago

Bengaluru, India Deloitte Full time
Deloitte India is hiring for PySpark Developer

Level:

Deputy ManagerLocation:

Bangalore/Delhi NCR/MumbaiJob Type:

Full-timeExperience Required:

4-6 years

Job Summary:We are seeking a talented and experienced Deputy Manager - PySpark Developer with 4 to 6 years of experience to join our dynamic team. The ideal candidate will possess extensive expertise in PySpark, big data technologies, and data engineering practices. As a Deputy Manager, you will play a critical role in developing and optimizing data processing pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver high-impact data solutions.

Key Responsibilities:PySpark Development:Design, develop, and maintain scalable and efficient PySpark applications to process large datasets.Write clean, maintainable, and efficient code following best practices and coding standards.Optimize and tune PySpark jobs to improve performance and reduce execution times.Big Data Technologies:Work with big data tools and frameworks such as Hadoop, Hive, and Spark to manage and process large-scale data.Implement and maintain data pipelines using Apache Airflow or similar orchestration tools.Ensure data is ingested, processed, and stored in an optimal manner for further analysis and reporting.Data Engineering:Design and implement ETL processes to extract, transform, and load data from various sources.Ensure data quality and consistency through robust validation and testing procedures.Collaborate with data scientists and analysts to understand data requirements and deliver solutions that meet their needs.Cloud Integration:Deploy and manage big data solutions on cloud platforms such as AWS, Azure, or Google Cloud.Leverage cloud services for storage, computing, and data processing to enhance scalability and performance.Optimize cloud resource utilization to reduce costs and improve efficiency.Collaboration and Communication:Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand requirements and deliver data solutions.Communicate effectively with team members and stakeholders, providing regular updates on project status.Document technical designs, processes, and procedures to ensure knowledge sharing and continuity.Continuous Improvement:Stay up-to-date with the latest industry trends and technologies in big data and data engineering.Identify opportunities for process improvement and implement best practices to enhance productivity and efficiency.Participate in code reviews and provide constructive feedback to ensure code quality and maintainability.

Qualifications:Bachelor's degree in Computer Science, Data Science, or a related field.4-6 years of professional experience in PySpark development and big data technologies.Strong understanding of distributed computing and parallel processing concepts.Proficiency in SQL and experience with data warehousing solutions.Familiarity with data modeling, ETL processes, and data pipeline orchestration.Experience with cloud platforms (e.g., AWS, Azure, GCP) and related services.Solid understanding of data structures, algorithms, and software engineering principles.Excellent problem-solving skills and attention to detail.Strong communication and interpersonal skills.

Preferred Qualifications:Relevant certifications (e.g., Databricks Certified Associate Developer for Apache Spark).Experience with real-time data processing and streaming technologies (e.g., Kafka, Flink).Knowledge of machine learning frameworks and libraries.Familiarity with containerization technologies (e.g., Docker, Kubernetes).Experience with version control systems (e.g., Git).

How you’ll grow

Connect for impactOur exceptional team of professionals across the globe are solving some of the world’s most complex business problems, as well as directly supporting our communities, the planet, and each other. Know more in our

Global Impact Report

and our

India Impact Report

.

Empower to leadYou can be a leader irrespective of your career level. Our colleagues are characterised by their ability to inspire, support, and provide opportunities for people to deliver their best and grow both as professionals and human beings. Know more about

Deloitte and our One Young World partnership.

Inclusion for allAt Deloitte, people are valued and respected for who they are and are trusted to add value to their clients, teams and communities in a way that reflects their own unique capabilities.

Know more about everyday steps that you can take to be more inclusive.

At Deloitte, we believe in the unique skills, attitude and potential each and every one of us brings to the table to make an impact that matters.

Drive your careerAt Deloitte, you are encouraged to take ownership of your career. We recognise there is no one size fits all career path, and global, cross-business mobility and up / re-skilling are all within the range of possibilities to shape a unique and fulfilling career. Know more about

Life at Deloitte.

Everyone’s welcome… entrust your happiness to usOur workspaces and initiatives are geared towards your 360-degree happiness. This includes specific needs you may have in terms of accessibility, flexibility, safety and security, and caregiving. Here’s a glimpse of things that are in store for you.

Interview tipsWe want job seekers exploring opportunities at Deloitte to feel prepared, confident and comfortable. To help you with your interview, we suggest that you do your research, know some background about the organisation and the business area you’re applying to.

Check out recruiting tips from Deloitte professionals.

  • Lead PySpark Developer

    1 week ago

    Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    1 week ago

    Bengaluru, Karnataka, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    1 week ago

    Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    1 week ago

    Bengaluru, India LTIMindtree Full time

    Job Description:Primary Skill Set: Lead PySpark DeveloperSeconday Skill Set: SAS toolsLead a team of data engineers in the design and implementation of big data solutions using PySpark.Collaborate with cross-functional teams to understand business needs and develop data-driven solutions.Design and implement scalable and robust data pipelines using...

  • Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Manvision Technologies Full time

    SNRequired InformationDetails1RolePySpark Developer2Required Technical Skill SetPySpark, Python, HDFS, Hadoop, SQL3No of Requirements024Desired Experience Range5+ yearsLocation of RequirementIndiaDesired Competencies (Technical/Behavioral Competency)Must-HaveSound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured &...

  • Pyspark

    1 week ago

    Bengaluru, Karnataka, India Riverforest Connections Full time

    Position Purpose For the projects which are implemented on Data SSC platform, we need senior developer for python- pyspark development to migrate project from spark/scala to python-pyspark._ResponsibilitiesDirect ResponsibilitiesDesign high quality deliverables adhering to business requirement with defined standards and design principles, patternsDevelop and...

  • Pyspark Developer

    2 months ago

    Bengaluru, India Corporate Resources Full time

    Job DescriptionDesigning and developing big data applications using the PySpark framework to meet the needs of the businessWriting and optimizing Spark SQL statements to extract and manipulate large datasetsDeveloping and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processingDebugging and...

  • Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Corporate Resources Full time

    Job DescriptionDesigning and developing big data applications using the PySpark framework to meet the needs of the businessWriting and optimizing Spark SQL statements to extract and manipulate large datasetsDeveloping and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processingDebugging and...

  • Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Manvision Technologies Full time

    SN Required Information Details 1 Role PySpark Developer 2 Required Technical Skill Set PySpark, Python, HDFS, Hadoop, SQL 3 No of Requirements 02 4 Desired Experience Range 5+ years Location of Requirement India Desired Competencies (Technical/Behavioral Competency) Must-Have Sound...

  • Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Corporate Resources Full time

    Job Description Designing and developing big data applications using the PySpark framework to meet the needs of the business Writing and optimizing Spark SQL statements to extract and manipulate large datasets Developing and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processing ...

  • PySpark Developer

    17 hours ago

    Bengaluru, India Deloitte Full time

    Deloitte India is hiring for PySpark DeveloperLevel: Deputy ManagerLocation: Bangalore/Delhi NCR/MumbaiJob Type: Full-timeExperience Required: 4-6 yearsJob Summary:We are seeking a talented and experienced Deputy Manager - PySpark Developer with 4 to 6 years of experience to join our dynamic team. The ideal candidate will possess extensive expertise in...

  • AM-Pyspark Developer

    3 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world's biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We're harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    1 month ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    3 weeks ago

    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • Bigdata + Pyspark

    1 week ago

    Bengaluru, Karnataka, India Impetus Technologies Full time

    BIgdata + Pyspark :Big Data; Hadoop / HDFS,Pyspark,PythonExperience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL) as appropriateHands-on expertise with application design, software developmentProficient in Big Data technologies Designs, codes, tests, corrects and documents large and/or complex programs and...

  • Bigdata + Pyspark

    1 week ago

    Bengaluru, Karnataka, India Impetus Technologies Full time

    BIgdata + Pyspark : Big Data; Hadoop / HDFS,Pyspark,Python Experience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL) as appropriateHands-on expertise with application design, software development Proficient in Big Data technologies Designs, codes, tests, corrects and documents large and/or complex programs and...

  • AM-Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world's biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We're harnessing the power of technology and humanity to create...

  • Python+Pyspark

    3 weeks ago

    Bengaluru, India Wissen Full time

    Wissen Technology is now hiringPython+PysparkWe are seeking a skilled Python Developer with a strong background in PySpark to develop and optimize data processing applications. The ideal candidate will be responsible for building robust and scalable data processing solutions using Python and...

  • Python, Pyspark

    1 week ago

    Bengaluru, India Ziniosedge Full time

    Python, Pyspark 5 Technology Lead (Data Engineer) BE - 8+ years of industry experience as Lead Developer - Experience in implementing ETL and ELT data pipelines with PySpark - Spark Structured API, SPARK SQL & Spark Performance tuning are highly preferred. - Experience in building data pipelines on data lake or Lakehouse(AWS Databricks) & handling...