Senior Pyspark Developer

19 hours ago

Bengaluru Karnataka India, Karnataka Luxoft Full time

Project description

We need a Senior Python and Pyspark Developer to work for a leading investment bank client.


Responsibilities

  • Develop software applications based on business requirements
  • Maintain software applications and make enhancements according to project specifications
  • Participate in requirement analysis, design, development, testing, and implementation activities
  • Propose new techniques and technologies for software development.
  • Perform unit testing and user acceptance testing to evaluate application functionality
  • Ensure to complete the assigned development tasks within the deadlines
  • Work in compliance with coding standards and best practices
  • Provide assistance to Junior Developers when needed.
  • Perform code reviews and recommend improvements.
  • Review business requirements and recommend changes to develop reliable applications.
  • Develop coding documentation and other technical specifications for assigned projects.
  • Act as primary contact for development queries and concerns.
  • Analyze and resolve development issues accurately.


Mandatory Skills

  • Experience in data intensive Pyspark development.
  • Experience as a core Python developer.
  • Experience developing Classes, OOPS, exception handling, parallel processing .
  • Strong knowledge of DB connectivity, data loading , transformation, calculation.
  • Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations.
  • Lambda Functions, Decorators.
  • Vector operations on Pandas dataframes /series.
  • Application of applymap, apply, map functions.
  • Concurrency and error handling data pipeline batch of size [1-10 gb].
  • Ability to understand business requirements and translate them into technical requirements.
  • Ability to design architecture of data pipeline for concurrent data processing.
  • Familiar with creating/designing RESTful services and APIs.
  • Familiar with application unit tests.
  • Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services.
  • Debugging application.


Nice-to-Have Skills

  • Knowledge of web backend technology
  • Django, Python, PostgreSQL.
  • Apache Airflow
  • Atlassian Jira
  • Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
  • Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus

  • Pyspark Developer

    2 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Pyspark DeveloperRequired Skill Set: Pyspark, Python, SQL and relational databases, SparkSQL, Spark Scripting, UNIX Shell Scripting, ETL, Data Warehousing, CI/CDDesired Experience Range: 4 to 10 YearsJob Location: Hyderabad, Bangalore, Chennai, Kolkata, PuneMust Have:Data Engineer, Python developer with specialty in...

  • Pyspark developer

    4 days ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!TCS is hiring for Pyspark DeveloperVirtual Interview Date-11th October 2025Desired Experience: 4-11 yearsJob Location: Bangalore / Chennai / HyderabadRequire skillset: Big Data, Hadoop, Spark, Hive ,PysparkJob Description:-Must Have Technical Qualifications:SparkHadoopHiveNice to Have Qualifications:Data Engineer CertificationExperience...

  • Pyspark designer

    2 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    We are looking for an experienced PySpark AWS Cloud Designer/Developer with a strong background in data integration and ETL. The ideal candidate will have: - Over 5 years’ experience designing and developing ETL integration patterns using Python and Spark. - 3+ years working with PySpark for data processing (Databricks or Apache Spark experience is a...

  • Pyspark developer

    4 days ago

    Bengaluru, Karnataka, India Infosys Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Description:In the role of Data Engineer you will interface with key stakeholders and apply your technical proficiency across different stages of the Software Development Life Cycle including Requirements Elicitation Application Architecture definition and DesignYou will play an important role in creating the high level design artifactsYou will also...

  • PySpark Developer

    2 weeks ago

    Bengaluru, Karnataka, India INFOBELL IT SOLUTIONS PVT LTD Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title : PySpark DeveloperLocation : Bangalore (Work from Office)Experience : 5+ YearsNotice Period : Immediate Joiners PreferredAbout Us : Infobell IT is a technology-driven organization specializing in product engineering, AI, cloud modernization, and data analytics. We collaborate with leading global enterprises to deliver scalable and...

  • TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Bangalore

    2 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Job Title: AWS Senior Data Engineer with Pyspark, AWS, GlueLocation: BangaloreExperience: 6 to 10 YearsNotice Period: 30-45 daysJob Description:Must: PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC[Cloud Formation / Terraform], GitKind Regards,Priyankha M

  • Developer (Python | PySpark)

    3 days ago

    Bengaluru, Karnataka, India ONOC Full time ₹ 1,00,00,000 - ₹ 1,40,00,000 per year

    We're Hiring – Developer (Python | PySpark)Location: Chennai / BangaloreCTC: ₹13 – 14 LPAExperience: 8 – 10 YearsNotice Period: Max 20 DaysKey Responsibilities:Design, develop & maintain scalable data pipelines using PySpark on cloud (AWS / Azure / GCP)Write complex, optimized SQL queries for data extraction, transformation & analysisBuild automation...

  • Data Engineer – Databricks

    4 days ago

    Bengaluru, Karnataka, India, Karnataka Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...

  • Pyspark Lead

    2 days ago

    Bengaluru, Karnataka, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.Roles and ResponsibilityDesign and develop scalable data pipelines using PySpark...

  • Pyspark & Databricks Developer

    3 days ago

    Bengaluru, Karnataka, India Growel Softech Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    1.Total 3 to 6 years of experience as a Pyspark and Databricks Developer with a strong portfolio demonstrating your expertise 2. Good hands on knowledge with database technologies such as SQL and NoSQL 3. Familiarity with Snowflake is added advantage 4. Good in Troubleshooting debugging maintaining and improving existing software 5. Excellent problem-solving...