Pyspark

3 days ago

Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Summary

As a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life integration.

Responsibilities

  • Optimize Pyspark applications to improve performance and resource utilization ensuring efficient data processing and analysis.
  • Develop and maintain scalable solutions using AWS EC2 and AWS EMR to support business needs and data-driven decision-making.
  • Implement PySpark frameworks to process large datasets enhancing data accessibility and insights for stakeholders.
  • Collaborate with cross-functional teams to design and deploy robust data pipelines ensuring seamless integration with existing systems.
  • Utilize Python programming to automate tasks and streamline workflows contributing to increased productivity and reduced operational costs.
  • Analyze and interpret data within the Cards & Payments domain providing actionable insights to drive strategic initiatives.
  • Ensure data security and compliance with industry standards safeguarding sensitive information and maintaining trust with clients.
  • Troubleshoot and resolve technical issues related to Pyspark and AWS environments minimizing downtime and optimizing system performance.
  • Participate in code reviews and provide constructive feedback to peers fostering a culture of continuous improvement and innovation.
  • Document technical processes and solutions creating comprehensive guides for future reference and knowledge sharing.
  • Stay updated with the latest advancements in PSpark and AWS technologies applying new techniques to enhance project outcomes.
  • Engage in collaborative problem-solving sessions contributing to the development of creative solutions for complex challenges.
  • Support the team in achieving project milestones and deliverables ensuring alignment with organizational goals and objectives.

Qualifications

  • Demonstrate proficiency in Spark Optimization techniques showcasing the ability to enhance data processing efficiency.
  • Exhibit strong experience with AWS EC2 and AWS EMR highlighting expertise in cloud-based solutions and infrastructure management.
  • Possess advanced skills in Apache Spark demonstrating capability in handling large-scale data processing tasks.
  • Showcase expertise in Python programming emphasizing ability to automate processes and develop efficient solutions.
  • Have domain knowledge in Cards & Payments providing insights and understanding of industry-specific challenges and opportunities.
  • Display a minimum of 5 years of relevant experience ensuring a solid foundation in technical and domain skills.
  • Present a maximum of 8 years of experience indicating a balance of expertise and adaptability in evolving technologies.

  • AWS + Pyspark

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Skills- AWS +PysparkExperience: 4 to 9 yearsLocation: AIA-PuneAs an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and...

  • Data Engineer- PySpark

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Build up data pipelines for consumption by the data science team.Clear understanding and experience with Python and PySpark.Experience in writing Python programs and SQL queries.Experience in SQL Query tuning.Build and maintain data pipelines in Pyspark with SQL and Python.Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best...

  • Pyspark Developer

    2 weeks ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Discover your future at CitiWorking at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact.Job OverviewAt Citi we're not just building technology, we're building the...

  • databricks/pyspark

    1 week ago

    Chennai, Tamil Nadu, India NRM Analytix Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Responsibilities:* Design, develop & maintain PySpark solutions using Databricks platform* Optimize performance through efficient data processing techniques* Collaborate with cross-functional teams on project deliveryAccessible workspace

  • Pyspark, AWS

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are seeking a skilled Developer with 4 to 6 years of experience to join our team. The ideal candidate will have expertise in Amazon S3 and PySpark and a strong background in the Cards & Payments domain. This hybrid role offers the opportunity to work on cutting-edge projects in a dynamic environment contributing to the companys growth and...

  • Pyspark Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilledData Engineerwith hands-on experience inPySparkandDatabricksto design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.Key Responsibilities:Develop...

  • Databrick Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Key Responsibilities:Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Pyspark Developer(Chennai,Bangalore,Hyderabad,Kolkata,Pune)

    1 day ago

    Chennai, Tamil Nadu, India Tata Consultancy Servicess Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Skill : PySpark DeveloperExperience Range: 4 to 8Notice : (Immediate to 60 Days)Job Description:Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scriptingob Description:4+ Years of experience with Developing/Fine tuning and implementing programs/applicationsUsing Python/PySpark/Scala on Big Data/Hadoop Platform.Roles and...

  • Big Data Developer(Hive, Pyspark)

    1 week ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    TCS is conductingWeekend Virtual DriveforBig Data Developer(Hive, Pyspark)Years of Experience: 4 to 10 yrsLocation: ChennaiNotice Period: 0- 90 daysDate of Interview: 25th Oct, 2025Mode of Interview: Virtual Drive( Cisco or WebEx)Roles & Responsibilities:Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools.Should be able to...

  • Software Engineer, Tableau And PySpark, AVP

    1 day ago

    Chennai, Tamil Nadu, India NatWest Group Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Join us as a Software Engineer, Tableau And PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate vice...