Pyspark Developer

3 days ago

Bengaluru Hyderabad, India Sonata Software Full time US$ 90,000 - US$ 1,20,000 per year

We are seeking a skilled and experienced PySpark Developer to join our dynamic team. The ideal candidate will have a strong background in software development, with a focus on data transformation, processing, and performance optimization using PySpark. Expertise in SQL and its functions is required. Developer will be responsible for designing, developing, and maintaining scalable data transformation solutions, as well as developing efficient Spark jobs and managing job scheduling.

  • Strong expertise in Python programming:

    Deep knowledge of Python language features, libraries, and best practices.
  • Extensive experience with Apache Spark and PySpark:

    Writing optimized Spark jobs, transformations, actions, and working with RDDs and DataFrames.
  • Solid understanding of big data processing concepts:

    Distributed computing, fault tolerance, partitioning, and data shuffling.
  • Experience with ETL pipeline development:

    Designing, implementing, and maintaining scalable data pipelines.
  • Proficient in working with large datasets:

    Handling data ingestion, cleansing, and transformation efficiently.
  • Good understanding of SQL:

    Writing complex SQL queries and integrating Spark SQL where needed.
  • Hands-on experience with FastAPI:

    Developing high-performance, scalable RESTful APIs and microservices using FastAPI.
  • Experience with cloud platforms and services:

    Working knowledge of AWS (EMR, S3), Azure, or GCP cloud environments.
  • Familiarity with Hadoop ecosystem components:

    HDFS, Hive, HBase, or similar tools.
  • Knowledge of data serialization formats:

    JSON, Parquet, Avro, ORC.
  • Strong debugging and troubleshooting skills:

    Ability to profile, optimize, and debug Spark and API applications.
  • Experience with version control systems:

    Git or similar tools.
  • Understanding of software development lifecycle and Agile methodologies:

    Participating in code reviews, CI/CD pipelines, and sprint planning.
  • Excellent problem-solving and communication skills:

    Ability to collaborate with data engineers, analysts, and stakeholders.

  • Pyspark Developer

    3 days ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title : PySpark SMEJob Location – HyderabadExperience: 4-8 Years of experience in IT industry with exposure to Python, Python , PySpark , Delta Lake , DataStage , Terada ( Bteq ,)Strong in Pyspark and SQL Requirement:Strong knowledge of Pyspark and SQL2 + years of experience in PySpark.4+Years of experience in DataStage, Terada.6+ Years of experience...

  • PySpark Developer

    34 minutes ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job Title: PySpark DeveloperExperience: 6 to 8 YearsLocation: Hyderabad ( Onsite )Job DescriptionMust-HaveStrong knowledge of Pyspark and SQL2 + years of experience in PySpark.4+Years of experience in DataStage, Terada.6+ Years of experience in Python.Experience in GCP cloud ServicesDelta LakeGood-to-HavePysparkGCPGoogle Cloud / Google...

  • Pyspark Developer

    3 days ago

    Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...

  • PySpark Developer

    1 hour ago

    Hyderabad, Telangana, India Algoleap Technologies Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    SUMMARY Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • PySpark Developer

    1 week ago

    Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full time

    Job DescriptionROLE RESPONSIBILITIESData Engineering and Processing:Develop and manage data pipelines using PySpark on Databricks.Implement ETL/ELT processes to process structured and unstructured data at scale.Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks.Databricks Platform Expertise:Experience in Perform Design,...

  • Pyspark

    9 hours ago

    Bengaluru, India Riverforest Connections Full time

    Position Purpose - For the projects which are implemented on Data SSC platform, we need senior developer for python - pyspark development to migrate project from spark/scala to python-pyspark._ **Responsibilities** **Direct Responsibilities** Design high quality deliverables adhering to business requirement with defined standards and design principles,...

  • Pyspark Developer

    3 days ago

    Bengaluru, Karnataka, India Sigma Allied Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Position - Pyspark DeveloperLocation - BangaloreExperience - 6 to 9 yrsRequired Skills -Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact.Must have skills : PySparkMinimum 5 year(s) of experience is requiredEducational Qualification : 15 years full time educationSummary:As an...

  • eSoftLabs - PySpark Developer

    4 days ago

    Hyderabad, Telangana, India ENTERPRISE SOFTLABS PRIVATE LINITED Full time

    Job Title : Pyspark DeveloperLocation : HyderabadExperience Required : 4- 8Keywords : AWS, Pyspark, DatabricksSkills and experiences required :- 3- 6 years of hands-on development in PySpark.- Experience with Databricks and performance tuning using Spark UI.- Strong understanding of AWS services, Kafka, and distributed data processing.- Proficient in...

  • Pyspark Developer

    3 days ago

    Bengaluru, Chennai, Pune, India Tekskills Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    JOB SUMMARY:Experience 6+years (relevant minimum 5+ years)Detailed job description - Skill Set:Candidate should have good hands on experience in Pyspark preferrable more than 7 years.Should have very good understanding of Agile development methodologies and excellent team handling experience.Mandatory Skills - Pyspark, Python, Agile

  • Pyspark Developer

    3 days ago

    Bengaluru, Karnataka, India Unified Excellence Full time US$ 90,000 - US$ 1,20,000 per year

    Responsibilities:* Collaborate with cross-functional teams on project delivery.* Ensure code quality through testing and documentation.* Design, develop, and maintain PySpark applications using Python and Spark.