Current jobs related to Pyspark Engineer - Bengaluru, Karnataka - Pan Asia HR Solutions

  • Senior PySpark Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeePySpark Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior PySpark EngineerJob Overview:Greetings from our company. We are seeking a highly skilled Senior PySpark Engineer to join our team.Key Responsibilities:Implement data ingestion pipelines from various data sources, including databases, S3, and files.Design and develop ETL/ Data Warehouse transformation processes using PySpark.Work with structured and...

  • Pyspark Data Engineer

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time

    We are seeking an experienced Data Engineer to join our team in Chennai, Pune, Bangalore, Hyderabad, Trivandrum or Kochi. The ideal candidate will have a strong background in Python and proficiency in Pyspark, SparkSQL, SQL and relational databases, ETL and Data Warehousing.Required Skills and QualificationsPyspark: Experience with Pyspark development using...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time

    Job Title: Senior Data Engineer - PySparkWe are seeking an experienced Senior Data Engineer to join our team and drive the development of scalable, high-performance data pipelines using PySpark.Data Engineering: Design, develop, and maintain complex data pipelines using PySpark, ensuring data integrity and accuracy.Data Ingestion: Implement and manage data...

  • Chief Pyspark Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeDataScientist Full time

    Job Title: Pyspark DeveloperWe are seeking a skilled Pyspark Developer to join our team. The ideal candidate will have extensive experience in developing and fine-tuning programs and applications using Python, PySpark, and Scala on Big Data/Hadoop platforms.The successful candidate will work closely with our Risk Management team to implement risk models in...

  • PySpark Developer

    3 weeks ago

    Bengaluru, Karnataka, India ValueLabs Full time

    Job Title- PySpark Data EngineerWe're growing our Data Engineering team at ValueLabs and looking for a talented individual to build scalable data pipelines on Cloudera Data PlatformExperience- 5years to 9years.Pyspark Job Description:• Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the...

  • PySpark Developer

    2 weeks ago

    Bengaluru, Karnataka, India ValueLabs Full time

    Job Title- PySpark Data EngineerWe're growing our Data Engineering team at ValueLabs and looking for a talented individual to build scalable data pipelines on Cloudera Data PlatformExperience- 5years to 9years.Pyspark Job Description:• Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the...

  • Pyspark Developer

    3 weeks ago

    Bengaluru, Karnataka, India ValueLabs Full time

    Pyspark Developer Location - Bangalore (5 Days WFO)Experience Level - 5+ yrsNotice Period - Immediate to 15 daysJob Description:We are seeking a highly skilled Python & PySpark Developer to join our dynamic team. This position will be responsible for developing and maintaining complex data processing systems using Python and PySpark, ensuring high...

  • Pyspark Developer

    3 weeks ago

    Bengaluru, Karnataka, India ValueLabs Full time

    Pyspark DeveloperLocation - Bangalore (5 Days WFO)Experience Level - 5+ yrsNotice Period - Immediate to 15 daysJob Description:We are seeking a highly skilled Python & PySpark Developer to join our dynamic team. This position will be responsible for developing and maintaining complex data processing systems using Python and PySpark, ensuring high performance...

  • Senior Pyspark Developer

    1 week ago

    Bengaluru, Karnataka, India beBeePyspark Full time

    Pyspark Engineer OpportunityOverview:This exciting role involves working with a cutting-edge technology, PySpark. As a Pyspark Engineer, you will have the opportunity to work on complex data processing tasks and contribute to the growth of our organization.Responsibilities:Design and implement efficient PySpark solutions for large-scale data processing...

  • Pyspark Developer

    3 weeks ago

    Bengaluru, Karnataka, India ValueLabs Full time

    Pyspark DeveloperLocation - Bangalore (5 Days WFO)Experience Level - 5+ yrs Notice Period - Immediate to 15 daysJob Description:We are seeking a highly skilled Python & PySpark Developer to join our dynamic team. This position will be responsible for developing and maintaining complex data processing systems using Python and PySpark, ensuring high...

Pyspark Engineer

1 month ago

Bengaluru, Karnataka, India Pan Asia HR Solutions Full time

Job Description :

We are looking for a skilled PySpark Developer with expertise in ETL processes, Python, and SQL to design, develop, and optimize large-scale data processing pipelines. The ideal candidate will have hands-on experience in PySpark for data transformation, aggregation, and analysis in a Big Data environment.

Key Responsibilities :

- Design, develop, and maintain efficient ETL pipelines using PySpark.

- Write optimized Spark SQL queries and perform data transformations.

- Work with structured and semi-structured data (Parquet, Avro, JSON, etc.).

- Optimize Spark jobs for performance and scalability.

- Collaborate with Data Engineers, Analysts, and Business Teams to understand data requirements.

- Troubleshoot and debug data processing issues in Spark applications.

- Ensure data quality, integrity, and consistency across pipelines.

- Integrate data from multiple sources (Databases, APIs, Streaming platforms).

Required Skills & Qualifications :

- Strong experience in PySpark (Spark with Python) for Big Data processing.

- Proficiency in Python programming (Pandas, NumPy is a plus).

- Strong SQL skills (query optimization, joins, aggregations).

- Hands-on experience in ETL pipeline development and data warehousing concepts.

- Familiarity with Big Data tools (Hadoop, Hive, HDFS) is a plus.

- Knowledge of AWS/GCP/Azure data services (Glue, Databricks, EMR, BigQuery) is a bonus.

- Experience with version control (Git) and CI/CD pipelines.

- Good understanding of distributed computing principles.

Good to Have :

- Experience with Kafka/Spark Streaming for real-time data processing.

- Knowledge of Airflow/Luigi for workflow orchestration.

- Understanding of Data Lake & Delta Lake architectures.

(ref:hirist.tech)