Pyspark Developer

4 weeks ago


Bengaluru, Karnataka, India Tata Consultancy Services Full time
Pyspark Developer Job Description

We are seeking a highly skilled Pyspark Developer to join our team at Tata Consultancy Services. As a Pyspark Developer, you will be responsible for designing and implementing Big Data pipelines using PySpark, database migration, transformation, and integration solutions for Data warehousing projects.

Key Responsibilities:
  • Design and implement Big Data pipelines using PySpark, database migration, transformation, and integration solutions for Data warehousing projects.
  • Develop data processing tasks using PySpark, including reading data from external sources, merging data, aggregating data, performing data enrichment, and loading into target data destinations.
  • Work with Jupyter Notebook/ PyCharm etc. IDE to develop and deploy data processing tasks.
  • Fine-tune processes and troubleshoot performance issues.
  • Experience in source code control systems like git, GitHub, SVN etc.
  • Deployment and operationalizing the code, knowledge of scheduling tools like Control-M etc. is preferred.
  • Experience in CI/CD tools like Jenkins, Nexus, Ansible etc.
  • Excellent communication and interpersonal skills.
  • Understanding of Unix/Linux Shell Scripting.
  • Experience in Agile delivery methodologies.
  • Hands-on experience in data validation, writing unit test cases.
  • Working experience of ETL Tools like Data Stage, Talend etc. is an added advantage.
  • Working experience on Databricks / Delta Lake is an added advantage.

Requirements:

  • 4 to 12 Years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for Data warehousing projects.
  • Excellent knowledge in Apache Spark and Python programming experience.

  • PySpark Developer

    3 weeks ago


    Bengaluru, Karnataka, India Capgemini Engineering Full time

    Job Title: Pyspark EngineerAt Capgemini Engineering, we are seeking a highly skilled Pyspark Engineer to join our team.Responsibilities:Design and develop large-scale data processing pipelines using PySpark.Collaborate with cross-functional teams to integrate data engineering solutions.Develop and maintain complex data processing applications using...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Title:PySpark Hadoop EngineerExperience: 5 to 8 yearsLocation: Pune, Bangalore, Hyderabad, DelhiKey Skills/Knowledge:We are seeking a highly skilled professional with a strong background in data and experience in Hadoop and PySpark. The ideal candidate will have a minimum of 4 years of experience, with at least 2 years of experience in PySpark....


  • Bengaluru, Karnataka, India Hiring House Technology Full time

    About the RoleHiring House Technology is seeking a highly skilled PYSPARK DEVELOPER to join our team and contribute to the design and development of robust PySpark applications for large-scale data processing.Key Responsibilities• Design and implement efficient data ingestion, transformation, and storage processes.• Develop and optimize PySpark...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Title: Senior PySpark DeveloperAt Tata Consultancy Services, we are seeking a highly skilled Senior PySpark Developer to join our team.Key Responsibilities:Design and implement Big Data pipelines using PySpark, database migration, transformation, and integration solutions for Data warehousing projects.Develop data processing tasks using PySpark,...


  • Bengaluru, Karnataka, India Hiring House Technology Full time

    Hiring House Technology is seeking a skilled Sr. PySpark Developer to join our team.Job SummaryWe are looking for a highly motivated and experienced Big Data Engineer to design, develop, and maintain large-scale data processing systems using PySpark.Key ResponsibilitiesDesign and develop robust PySpark applications for data processing and analysis.Build and...

  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Data Engineer (Pyspark Developer) Job DescriptionTata Consultancy Services is seeking a highly skilled Data Engineer with expertise in Pyspark to join our team.Key Responsibilities:Design and implement Big Data pipelines using PySparkDevelop data processing tasks using PySpark, including reading data from external sources, merging data, aggregating data,...

  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Capgemini Engineering Full time

    Job Title: Pyspark EngineerAt Capgemini Engineering, we are seeking a highly skilled Pyspark Engineer to join our team. As a Pyspark Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using the PySpark framework.Responsibilities:Design and develop ETL pipelines using PySpark to extract, transform,...


  • Bengaluru, Karnataka, India Capgemini Full time

    About the Role:We are looking for a skilled PySpark developer to join our team at Capgemini.Key Responsibilities:Manage large teams and deliver complex data processing projectsDevelop and maintain large-scale data processing systems using PySparkDesign and implement big data processing tools and techniquesCollaborate with cross-functional teams to drive...


  • Bengaluru, Karnataka, India Wipro Full time

    Job Title: Senior Java Developer with PySpark ExperienceJob Summary:We are seeking a highly skilled Senior Java Developer with expertise in PySpark to join our team at Wipro. As a Senior Java Developer, you will be responsible for designing, developing, and maintaining large-scale Java applications using PySpark.Key Responsibilities:Design and develop...


  • Bengaluru, Karnataka, India Minutes to Seconds Pty Ltd Full time

    About the JobAt Minutes to Seconds, we're on a mission to match talented individuals with the perfect job opportunities. Our goal is to provide a seamless fit between skills and roles, driving success for both our candidates and clients.We're currently seeking a skilled Senior PySpark Developer to join our dynamic team. As a key member of our team, you'll be...

  • Big Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    About the Job:We are looking for a skilled Hadoop developer with experience in PySpark to join our team at Tata Consultancy Services. The ideal candidate will have a strong background in data transformation and a minimum of 2 years of experience with PySpark.Responsibilities:Design and implement data transformation workflows using PySparkWork with large...

  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Title: Data Engineer - PySparkWe are seeking a highly skilled Data Engineer with expertise in PySpark to join our team at Tata Consultancy Services.Key Responsibilities:Design and develop scalable data processing pipelines using PySparkWork with large datasets and ensure data quality and integrityCollaborate with cross-functional teams to integrate data...

  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Title: Data Engineer - PySparkWe are seeking a highly skilled Data Engineer with expertise in PySpark to join our team at Tata Consultancy Services.Key Responsibilities:Design and develop scalable data processing pipelines using PySparkWork with large datasets and ensure data quality and integrityCollaborate with cross-functional teams to integrate data...

  • Senior Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Retail Kloud9 Technologies India Private Limited Full time

    Job Title: Senior Data Engineer - PySparkCompany: Retail Kloud9 Technologies India Private LimitedJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Retail Kloud9 Technologies India Private Limited. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems...

  • Senior Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    We are seeking a highly skilled Senior Data Engineer with expertise in PySpark to join our team at Tata Consultancy Services.Key Responsibilities:Design and develop large-scale data processing systems using PySparkCollaborate with cross-functional teams to integrate data pipelines and ensure data qualityDevelop and maintain data transformation scripts using...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Description: We are currently hiring for a Data Transformation Specialist with expertise in Hadoop and PySpark. The ideal candidate will have a good data background and minimum 2 years of experience in PySpark. The role involves working with the Hadoop platform and developing data transformation expertise.Requirements:4 years of experience in big...

  • Senior Cloud Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Epsilon Full time

    About EpsilonEpsilon is a leading company in the digital marketing space, and we're looking for talented individuals to join our team. As a Senior Cloud Engineer (Pyspark), you will play a key role in developing and building products that help us stay ahead of the competition.Job DescriptionWe're seeking a highly skilled Cloud Engineer with experience in...

  • Senior Cloud Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Epsilon Full time

    About the RoleWe are seeking a highly skilled Senior Cloud Engineer (Pyspark) to join our team at Epsilon. As a key member of our Prospect Product Development team, you will be responsible for establishing configuration management, automating our infrastructure, and implementing continuous integration frameworks along with data engineering platform...


  • Bengaluru, Karnataka, India Epsilon Full time

    About EpsilonEpsilon is a leading provider of data-driven marketing solutions. Our team of innovative thinkers develops and builds products that help us connect with millions of customers. We're looking for talented individuals who can help us solve complex problems and exceed expectations.Job DescriptionWe're seeking a Senior or Lead/Principal Engineer to...


  • Bengaluru, Karnataka, India KPI Partners Full time

    About the Role:We are seeking a highly skilled PySpark Engineer to join our dynamic team at KPI Partners. As a key member of our team, you will play a crucial role in designing and developing scalable data pipelines and analytics solutions.Key Responsibilities:Design, develop, and maintain efficient data pipelines using PySpark.Implement data workflows using...