Senior PySpark Developer

4 days ago

Hyderabad Secunderabad Telangana, India beBeeBigData Full time
PySpark Big Data Engineer

Join our team of expert data engineers and take on the challenge of working with large-scale data processing and analytics using PySpark.

As a PySpark Big Data Engineer, you will be responsible for designing, implementing, and optimizing efficient PySpark solutions for complex data processing tasks. You will work closely with cross-functional teams to understand business requirements and develop appropriate data solutions.

Some key responsibilities of this role include:
  • Designing and implementing high-performance PySpark data pipelines to handle large-scale data processing and analysis.
  • Developing and optimizing Spark SQL queries for big data analytics and reporting.
  • Working with cloud platforms like AWS, Azure, or Google Cloud to deploy Spark-based applications and leverage cloud-native services for big data processing.
  • Maintaining clear and comprehensive documentation of data pipelines, architectures, and processes to ensure knowledge sharing and continuity within the team.


Required qualifications for this role include:
  • 2-4 years of professional experience working with PySpark and big data technologies.
  • Strong expertise in Python programming with a focus on data processing and manipulation.
  • Familiarity with cloud platforms like AWS, Azure, or Google Cloud, and experience with cloud-native big data tools.
  • Knowledge of ETL processes and tools, as well as data storage technologies like HDFS, S3, or Google Cloud Storage.
  • Experience with distributed computing and cluster management, including proficiency in Spark SQL for data querying and transformation.


We are looking for a skilled and experienced PySpark Big Data Engineer who is passionate about working with large-scale data processing and analytics. If you have a strong background in PySpark and big data technologies, and enjoy collaborating with cross-functional teams to drive business outcomes, we encourage you to apply for this exciting opportunity.

  • Data Engineer

    4 hours ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    PySpark Data EngineerJob Description:Lead a team of data engineers in designing, developing and maintaining large-scale PySpark data pipelines. 5+ years of hands-on experience with PySpark, including expertise in Spark SQL, DataFrames, Datasets, and MLlib. Experience with Agile development methodologies, such as Scrum or Kanban, and familiar with tools...

  • PySpark Developer

    4 days ago

    Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full time

    Job DescriptionROLE RESPONSIBILITIESData Engineering and Processing:Develop and manage data pipelines using PySpark on Databricks.Implement ETL/ELT processes to process structured and unstructured data at scale.Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks.Databricks Platform Expertise:Experience in Perform Design,...

  • Python,pyspark,apachespark

    3 days ago

    Hyderabad, Telangana, India NTT DATA Full time

    **Req ID**: 327063 We are currently seeking a python,pySpark,ApacheSpark to join our team in Hyderabad, Telangana (IN-TG), India (IN). "At NTT DATA, we know that with the right people on board, anything is possible. The quality, integrity, and commitment of our employees are key factors in our company's growth, market presence and our ability to help our...

  • Python,pyspark,apachespark

    15 hours ago

    Hyderabad, Telangana, India NTT DATA Full time

    **Req ID**: 327061 We are currently seeking a python,pySpark,ApacheSpark to join our team in Hyderabad, Telangana (IN-TG), India (IN). "At NTT DATA, we know that with the right people on board, anything is possible. The quality, integrity, and commitment of our employees are key factors in our company's growth, market presence and our ability to help our...

  • Senior Data Engineer

    4 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeData Full time

    Job Title: Senior Data EngineerKey Responsibilities:We are seeking a highly skilled and experienced Senior Data Engineer to lead the design, development, and implementation of large-scale data pipelines using Python, Pyspark, and SQL. The ideal candidate will have strong technical expertise in data engineering concepts, with the ability to handle multiple...

  • Pyspark Developer

    3 days ago

    Hyderabad, Bengaluru, Pune, India Infosys Limited Full time

    Job Description- Bachelors degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education- At least 5 years of experience in Pyspark, Spark with Hadoop distributed frameworks while handling large amount of big data using Spark and Hadoop...

  • Pyspark, Aws

    15 hours ago

    Hyderabad, Telangana, India Cognizant Full time

    **Exp: 4 to 13 years** **Skill: Pyspark, AWS** **Location : Bangalore/Hyderabad/Kolkata/Pune/Chennai** **Technical Skills**: Python,AWS Glue Studio,AWS Glue ETL,AWS Glue Catalog,Amazon S3,Apache Spark **Responsibilities : -** Develop and maintain data pipelines using AWS Glue Studio and AWS Glue ETL. - Ensure data quality and integrity by implementing...

  • Senior Data Engineer Position

    4 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Engineer (PySpark)We are seeking a skilled data engineer with experience in PySpark and data processing. The ideal candidate will possess strong knowledge of data pipelines, HDFS, Hive, Spark, PySpark, and Python. You will be instrumental in designing, developing, and optimizing data processing solutions for large datasets.Key...

  • AWS Cloud Development Expert Wanted

    4 hours ago

    Hyderabad / Secunderabad, Telangana, India beBeeAws Full time ₹ 1,04,000 - ₹ 1,30,878

    Senior AWS Developer Position">Total years of experience: 8 to 10 yearsRelevant years of experience: 4 to 6 yearsShould have expertise in AWSStrong understanding and hands-on experience with AWS pipelinesGood communication skillsSolid experience in AWS development, preferably with cloud-based architecturesMandatory Skills: AWSDesired Skills: SQL, PySpark

  • Data Engineer Architect- Pyspark

    5 days ago

    Hyderabad, Telangana, India Mount Talent Consulting Pvt Ltd. Full time

    Professional & Technical Skills:- Must To Have Skills: Proficiency in PySpark, Amazon Web Services (AWS), Python (Programming Language).- Strong understanding of data processing and analysis which includes relational database- Experience in designing and implementing scalable applications.- Proficient in troubleshooting and problem-solving in application...