Experienced PySpark Engineer

3 days ago

Bengaluru, Karnataka, India beBeePySparkEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

Job Title: Experienced PySpark Engineer

Job Summary:

Our organization is seeking an experienced and highly skilled PySpark engineer to join our team. As a key member of the data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems using PySpark.

Key Responsibilities:

  • Design and implement complex data pipelines using PySpark to process large datasets and provide insights into business operations.
  • Develop and optimize Spark SQL queries to improve query performance and reduce latency.
  • Work with cross-functional teams to understand business requirements and develop data solutions that meet those needs.
  • Collaborate with data scientists and analysts to design and implement machine learning models and deploy them into production.
  • Troubleshoot and resolve issues related to data processing, application failures, and cluster resource utilization.

Required Skills and Qualifications:

  • 4+ years of experience working with PySpark and big data technologies.
  • Strong expertise in Python programming with a focus on data processing and manipulation.
  • Hands-on experience with Apache Spark, particularly with PySpark for distributed computing.
  • Proficiency in Spark SQL for data querying and transformation.
  • Familiarity with cloud platforms like AWS, Azure, or Google Cloud, and experience with cloud-native big data tools.
  • Knowledge of ETL processes and tools.
  • Experience with data storage technologies like HDFS, S3, or Google Cloud Storage.
  • Knowledge of data formats such as Parquet, ORC, Avro, or JSON.
  • Experience with distributed computing and cluster management.
  • Familiarity with Linux/Unix and command-line operations.
  • Strong problem-solving skills and ability to troubleshoot data processing issues.

Benefits:

This role offers a competitive salary and benefits package, including a comprehensive health insurance plan, 401(k) matching program, and generous paid time off.

Others:

We are committed to providing a diverse and inclusive work environment and encourage applications from candidates from all backgrounds.

  • Pyspark Architect

    2 days ago

    Bengaluru, Karnataka, India Talent Worx Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Description: PySpark ArchitectLocation: Bangalore,Pune,Kolkata Experience Required: 7 to 13 YearsRole Overview:We are seeking a highly skilled and motivated PySpark Manager to lead our dataengineering initiatives. The ideal candidate will have deep expertise in PySpark,Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible...

  • Data Engineer, PySpark

    19 hours ago

    Bengaluru, Karnataka, India RBS Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Join us as a Data Engineer, PySparkThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank by developing innovative data driven solutions, using insight to be commercially successful, and keeping our customers' and the bank's data...

  • Python/pyspark

    3 weeks ago

    Bengaluru, Karnataka, India CIEL HR Full time

    HiGreetings of the Day We have a critical opening with our client for the SAP ABAP Development for Python Pyspark for Bangalore Location Experience 8-12 years - Work Form Office Payroll Company Name CIEL Mode of Employment ContractNotice Period Immediate 15 days 30 days MaxJob descriptionBig data technologies Hadoop data lakes SQL scripting...

  • Software Engineer, PySpark, AVP

    3 days ago

    Bengaluru, Karnataka, India RBS Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Join us as a Software Engineer, PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate vice president levelWhat...

  • SDE-Python/Pyspark

    3 weeks ago

    Bengaluru, Karnataka, India VidPro Consultancy Services Full time

    Job DescriptionExperience: 4 - 7 YearsLocation: Whitefield - BangaloreWork Mode: HybridMandatory Skills: Python, PySpark, AWS, SQL, DSA, Databricks,Hadoop,spark,mapreduce,HBase.Responsibilities: Good development practices- Hands on coder with good experience in programming languages likePython or Pyspark .- Hands-on experience on the Big Data stack like...

  • 15h Left Application Lead – PySpark

    3 weeks ago

    Bengaluru, Karnataka, India Sigma Allied Services Private Limited Full time

    Job DescriptionTitle: Application Lead PySparkLocation: Bengaluru, Karnataka, IndiaExperience: Minimum 5 Years in PySparkEducation: 15 Years Full-Time EducationJob Description:We are seeking an experienced Application Lead with strong expertise in PySpark to lead the design, development, and configuration of applications. You will act as the primary point of...

  • Experienced Data Engineer Wanted

    24 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Job Opportunity: Experienced Data Engineer NeededWe are seeking a highly skilled professional to join our team as a Data Engineer. The ideal candidate will have a strong background in data integration and management, with expertise in cloud-based services.The successful candidate will be responsible for designing, implementing, and maintaining large-scale...

  • Principal Data Engineering Specialist

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Lead Data Engineer PositionWe are looking for an experienced Lead Data Engineer to oversee our data engineering projects. This role requires expertise in building and maintaining scalable data pipelines, along with leadership capabilities to guide a team of data engineers.The ideal candidate will have proficiency in Python, SQL, Pyspark, Airflow, AWS,...

  • Senior Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time

    Big Data Developer JobWe're seeking an experienced Big Data Developer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing large-scale data processing pipelines using Databricks and PySpark.This role is ideal for someone with strong hands-on experience in big data processing and analytics, as...

  • Azure Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India ALIQAN Technologies Full time

    Job Title : Senior Azure Data Engineer (Contract)Company : ALIQAN Technologies (Client: MNC)Location : Bangalore, IndiaExperience : 8-13 yearsNotice Period : Immediate joiners onlyJob Type : ContractAbout the Role :We're looking for an experienced Senior Azure Data Engineer to join our client's team in Bangalore. As a Senior Azure Data Engineer, you will...