Pyspark

23 hours ago

Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Summary

As a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life integration.

Responsibilities

  • Optimize Pyspark applications to improve performance and resource utilization ensuring efficient data processing and analysis.
  • Develop and maintain scalable solutions using AWS EC2 and AWS EMR to support business needs and data-driven decision-making.
  • Implement PySpark frameworks to process large datasets enhancing data accessibility and insights for stakeholders.
  • Collaborate with cross-functional teams to design and deploy robust data pipelines ensuring seamless integration with existing systems.
  • Utilize Python programming to automate tasks and streamline workflows contributing to increased productivity and reduced operational costs.
  • Analyze and interpret data within the Cards & Payments domain providing actionable insights to drive strategic initiatives.
  • Ensure data security and compliance with industry standards safeguarding sensitive information and maintaining trust with clients.
  • Troubleshoot and resolve technical issues related to Pyspark and AWS environments minimizing downtime and optimizing system performance.
  • Participate in code reviews and provide constructive feedback to peers fostering a culture of continuous improvement and innovation.
  • Document technical processes and solutions creating comprehensive guides for future reference and knowledge sharing.
  • Stay updated with the latest advancements in PSpark and AWS technologies applying new techniques to enhance project outcomes.
  • Engage in collaborative problem-solving sessions contributing to the development of creative solutions for complex challenges.
  • Support the team in achieving project milestones and deliverables ensuring alignment with organizational goals and objectives.

Qualifications

  • Demonstrate proficiency in Spark Optimization techniques showcasing the ability to enhance data processing efficiency.
  • Exhibit strong experience with AWS EC2 and AWS EMR highlighting expertise in cloud-based solutions and infrastructure management.
  • Possess advanced skills in Apache Spark demonstrating capability in handling large-scale data processing tasks.
  • Showcase expertise in Python programming emphasizing ability to automate processes and develop efficient solutions.
  • Have domain knowledge in Cards & Payments providing insights and understanding of industry-specific challenges and opportunities.
  • Display a minimum of 5 years of relevant experience ensuring a solid foundation in technical and domain skills.
  • Present a maximum of 8 years of experience indicating a balance of expertise and adaptability in evolving technologies.

  • Pyspark Developer

    3 weeks ago

    Chennai, Tamil Nadu, India MP DOMINIC AND CO Full time

    Job Summary - Design develop and implement scalable data pipelines and streaming use cases using PySpark and Spark on a distributed computing platform - Possess strong programming skills in Spark streaming - Have familiarity with cloud platforms like GCP - Gain experience in big data technologies such as Hadoop Hive and HDFS - Perform ETL operations...

  • Data Engineer- PySpark

    1 week ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Build up data pipelines for consumption by the data science team.Clear understanding and experience with Python and PySpark.Experience in writing Python programs and SQL queries.Experience in SQL Query tuning.Build and maintain data pipelines in Pyspark with SQL and Python.Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best...

  • Pyspark Developer

    1 week ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Discover your future at CitiWorking at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact.Job OverviewAt Citi we're not just building technology, we're building the...

  • databricks/pyspark

    5 days ago

    Chennai, Tamil Nadu, India NRM Analytix Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Responsibilities:* Design, develop & maintain PySpark solutions using Databricks platform* Optimize performance through efficient data processing techniques* Collaborate with cross-functional teams on project deliveryAccessible workspace

  • Pyspark, AWS

    21 hours ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are seeking a skilled Developer with 4 to 6 years of experience to join our team. The ideal candidate will have expertise in Amazon S3 and PySpark and a strong background in the Cards & Payments domain. This hybrid role offers the opportunity to work on cutting-edge projects in a dynamic environment contributing to the companys growth and...

  • Pyspark Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilledData Engineerwith hands-on experience inPySparkandDatabricksto design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.Key Responsibilities:Develop...

  • Databrick Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Key Responsibilities:Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Software Engineer, PySpark and AWS, AVP

    7 days ago

    Chennai, Tamil Nadu, India Natwest Digitalx Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Join us as a Software Engineer, PySpark and AWSThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIts a chance to hone your existing technical skills and advance your careerWe're offering this role at associate vice president...

  • Python/Pyspark Development

    7 days ago

    Chennai, Tamil Nadu, India Standard Chartered Bank Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job ID: 39129Location: Chennai, INArea of interest: TechnologyJob type: Regular EmployeeWork style: Office WorkingOpening date: 10 Sept 2025Job SummaryStrategy - Deliver on the functional requirements for the Regulatory exercisesBusiness - Group Functions - FinanceProcesses - Regulatory Stress Testing.Risk Management - Group standard requirementsKey...

  • TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Chennai

    2 days ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Job Title: AWS Senior Data Engineer with Pyspark, AWS, GlueLocation: ChennaiExperience: 6 to 10 YearsNotice Period: 30-45 daysJob Description:Must: PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC[Cloud Formation / Terraform], GitKind Regards,Priyankha M