Pyspark

3 days ago

Chennai Tamil Nadu, India Cognizant Full time

**Job Summary**

As a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life integration.

**Responsibilities**
- Develop and maintain scalable solutions using AWS EC2 and AWS EMR to support business needs and data-driven decision-making.
- Implement PySpark frameworks to process large datasets enhancing data accessibility and insights for stakeholders.
- Collaborate with cross-functional teams to design and deploy robust data pipelines ensuring seamless integration with existing systems.
- Utilize Python programming to automate tasks and streamline workflows contributing to increased productivity and reduced operational costs.
- Analyze and interpret data within the Cards & Payments domain providing actionable insights to drive strategic initiatives.
- Ensure data security and compliance with industry standards safeguarding sensitive information and maintaining trust with clients.
- Troubleshoot and resolve technical issues related to Pyspark and AWS environments minimizing downtime and optimizing system performance.
- Participate in code reviews and provide constructive feedback to peers fostering a culture of continuous improvement and innovation.
- Document technical processes and solutions creating comprehensive guides for future reference and knowledge sharing.
- Engage in collaborative problem-solving sessions contributing to the development of creative solutions for complex challenges.
- Support the team in achieving project milestones and deliverables ensuring alignment with organizational goals and objectives.

**Qualifications**
- Demonstrate proficiency in Spark Optimization techniques showcasing the ability to enhance data processing efficiency.
- Exhibit strong experience with AWS EC2 and AWS EMR highlighting expertise in cloud-based solutions and infrastructure management.
- Possess advanced skills in Apache Spark demonstrating capability in handling large-scale data processing tasks.
- Showcase expertise in Python programming emphasizing ability to automate processes and develop efficient solutions.
- Have domain knowledge in Cards & Payments providing insights and understanding of industry-specific challenges and opportunities.
- Display a minimum of 5 years of relevant experience ensuring a solid foundation in technical and domain skills.
- Present a maximum of 8 years of experience indicating a balance of expertise and adaptability in evolving technologies.

  • Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain data solutions using Databricks SQL Databricks Delta Lake and Databricks Workflows. - Optimize and enhance existing data workflows to improve performance and efficiency. - Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs. -...

  • Aws + Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skills - AWS +Pyspark** **Experience: 4 to 9 years** **Location: AIA-Pune** **As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights...

  • Big Data + Pyspark

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    AIA-Pune **Job Summary** Data Engineer 4 to 9 Years **Responsibilities** Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation...

  • Pyspark, Aws

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain scalable data processing solutions using PySpark to support business objectives in the Cards & Payments domain. - Collaborate with cross-functional teams to design and implement data storage solutions on Amazon S3 ensuring data integrity and security. - Analyze and optimize existing data workflows...

  • Ai Ml

    1 day ago

    Chennai, Tamil Nadu, India Prodapt Full time

    **Overview**: AI ML (GCP/BQ/spark/pyspark ) **Responsibilities**: AI ML (GCP/BQ/spark/pyspark ) **Requirements**: AI ML (GCP/BQ/spark/pyspark )

  • Python Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Data engineer 1) Overall 7+ in IT Experience with relevant 5+ years of experience in Data Engineering. Required technical skills: Python, PySpark, SQL, AWS 2) Required Domain: Healthcare Insurance 4) Extensive data analysis skills and Good communication skills 5) Experience working in Agile **About Virtusa** Teamwork, quality of life, professional and...

  • Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryAs a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life...

  • Pyspark + Databricks

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skill: Pyspark + Databricks** ** Experience**: 6 to 9 Years ** Location : AIA - Gurgaon** **Job Summary** **Responsibilities** - Lead the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. - Oversee the integration of Databricks Unity Catalog to ensure seamless data governance and...

  • Pyspark Developer

    3 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    15 hours ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore