Pyspark Lead

3 days ago

Bengaluru, Karnataka, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.

Roles and Responsibility

  • Design and develop scalable data pipelines using PySpark and other relevant technologies.
  • Collaborate with cross-functional teams to identify and prioritize project requirements.
  • Develop and maintain large-scale data systems, ensuring high performance and reliability.
  • Troubleshoot and resolve complex technical issues related to data processing and analysis.
  • Implement automated testing and validation frameworks to ensure data quality and integrity.
  • Work closely with stakeholders to understand business needs and provide technical solutions.

Job Requirements

  • Strong proficiency in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.
  • Experience with big data processing and analysis is highly desirable.
  • Strong problem-solving skills and attention to detail are essential.
  • Excellent communication and collaboration skills are required.
  • Ability to work in a fast-paced environment and meet deadlines.
  • A Bachelor's or Master's degree in Computer Science or a related field (BE, B.Tech, ME, M.Tech) is preferred.
  • Notice period: 0 to 30 days.
  • Specific domain / industry: N/A.

  • Pyspark developer

    5 days ago

    Bengaluru, Karnataka, India Infosys Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Description:In the role of Data Engineer you will interface with key stakeholders and apply your technical proficiency across different stages of the Software Development Life Cycle including Requirements Elicitation Application Architecture definition and DesignYou will play an important role in creating the high level design artifactsYou will also...

  • PySpark Developer

    2 weeks ago

    Bengaluru, Karnataka, India INFOBELL IT SOLUTIONS PVT LTD Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title : PySpark DeveloperLocation : Bangalore (Work from Office)Experience : 5+ YearsNotice Period : Immediate Joiners PreferredAbout Us : Infobell IT is a technology-driven organization specializing in product engineering, AI, cloud modernization, and data analytics. We collaborate with leading global enterprises to deliver scalable and...

  • Software Engineer, PySpark

    2 weeks ago

    Bengaluru, Karnataka, India NatWest Group Full time ₹ 2,00,00,000 - ₹ 2,50,00,000 per year

    Software Engineer, PySpark, VP Join us as a Software Engineer, PySparkThis is an opportunity for a technically minded individual to join us as a Software Engineer You'll be designing, producing, testing and implementing working software, working across the lifecycle of the system Hone your existing software engineering skills and advance your career...

  • Sr. Snowflake Data Engineer Pyspark

    2 weeks ago

    Bengaluru, Karnataka, India EduRun Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Senior Data Engineer | 10-12 Years Experience | Must Have: Proven expertise in building scalable batch and streaming data pipelines using Databricks (PySpark) and Snowflake.Lead the design, implementation, and optimization of application data stores using PostgreSQL, DynamoDB, and advanced SQL.Strong programming skills in SQL, Python, and PySpark for...

  • Pyspark Developer

    2 weeks ago

    Bengaluru, Karnataka, India Infosys Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Pyspark Developer• Primary skills:Domain->Finacle-Core-Functional->Finacle-Core-WMS->Grand Master • Knowledge of more than one technology • Basics of Architecture and Design fundamentals • Knowledge of Testing tools • Knowledge of agile methodologies • Understanding of Project life cycle activities on development and maintenance projects •...

  • Technical Lead with ETL/SQL/Pyspark

    3 days ago

    Bengaluru, Karnataka, India EdgeVerve Systems Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Pipeline/ETL (Extract, Transform, Load) processes, API integration, scripting languages (Python), and big data technologies (Trio, Iceberg, DuckDB/Parquet).Database design, data modeling, and data warehousing. SQL and at least one cloud platform.Analytical tools (Superset, Power BI), statistical analysis, and SQL. To derive insights from data.Data governance...

  • PySpark Python Consultant

    3 days ago

    Bengaluru, Karnataka, India IQVIA Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Support ongoing recurring deliverables in PL/SQL, PySpark, SAS, Python and BDF technologiesContribute to adhoc projects and POCs planned as part of US AA support TeamEnhance and share APLD and healthcare domain expertise from project learningsGood communication (both verbal and written) and analytical skillsLogical thinking and reasoning capabilitiesAbility...

  • PySpark Python Consultant

    3 days ago

    Bengaluru, Karnataka, India IQVIA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    • Support ongoing recurring deliverables in PL/SQL, PySpark, SAS, Python and BDF technologies• Contribute to adhoc projects and POCs planned as part of US AA support Team• Enhance and share APLD and healthcare domain expertise from project learnings• Good communication (both verbal and written) and analytical skills• Logical thinking and reasoning...

  • Data Engineer – Databricks

    5 days ago

    Bengaluru, Karnataka, India, Karnataka Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...

  • Data Engineer with SQL/ETL/Pyspark

    6 days ago

    Bengaluru, Karnataka, India Infosys Finacle Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Mandate Skills - SQL, ETL ,hadoop, pyspark, Apache, KafkaRequired Skills5+ years of relevant Data engineering experience using Python or Java as tech be well versed into HadoopDevelopment expertise with 2+ years of experience, in SQL, ETL processes and other similar technologiesSkilled in Identifying and optimizing database queriesStrong understanding of...