Pyspark

1 week ago

Chennai Tamil Nadu, India Cognizant Full time

**Job Summary**

**Responsibilities**
- Develop and maintain data solutions using Databricks SQL Databricks Delta Lake and Databricks Workflows.
- Optimize and enhance existing data workflows to improve performance and efficiency.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
- Implement data transformation and processing using PySpark to ensure data quality and integrity.
- Provide technical guidance and support to junior developers and team members.
- Conduct code reviews and ensure adherence to best practices and coding standards.
- Troubleshoot and resolve issues related to data workflows and processing.
- Monitor and maintain data pipelines to ensure smooth and reliable operation.
- Participate in the design and architecture of data solutions to support business objectives.
- Stay updated with the latest industry trends and technologies to continuously improve data solutions.
- Document technical specifications and maintain comprehensive documentation for data solutions.
- Ensure data security and compliance with relevant regulations and standards.
- Collaborate with stakeholders to gather requirements and provide regular updates on project progress.

**Qualifications**
- Possess strong expertise in Databricks SQL Databricks Delta Lake and Databricks Workflows.
- Have extensive experience with PySpark for data transformation and processing.
- Demonstrate excellent problem-solving skills and attention to detail.
- Exhibit strong communication and collaboration skills to work effectively with cross-functional teams.
- Show proficiency in optimizing and enhancing data workflows for performance and efficiency.
- Have a solid understanding of data security and compliance requirements.
- Display the ability to provide technical guidance and support to junior team members.
- Maintain a proactive approach to staying updated with industry trends and technologies.
- Possess strong documentation skills to maintain comprehensive technical specifications.
- Demonstrate the ability to troubleshoot and resolve data workflow issues effectively.
- Show experience in conducting code reviews and ensuring adherence to best practices.
- Exhibit the ability to gather requirements and provide regular project updates to stakeholders.
- Have a proven track record of developing and maintaining data solutions that drive business success.

**Certifications Required**

Databricks Certified Associate Developer for Apache Spark Databricks Certified Professional Data Engineer

  • Pyspark

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** As a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life...

  • Aws + Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skills - AWS +Pyspark** **Experience: 4 to 9 years** **Location: AIA-Pune** **As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights...

  • Big Data + Pyspark

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    AIA-Pune **Job Summary** Data Engineer 4 to 9 Years **Responsibilities** Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation...

  • Pyspark, Aws

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain scalable data processing solutions using PySpark to support business objectives in the Cards & Payments domain. - Collaborate with cross-functional teams to design and implement data storage solutions on Amazon S3 ensuring data integrity and security. - Analyze and optimize existing data workflows...

  • Ai Ml

    21 hours ago

    Chennai, Tamil Nadu, India Prodapt Full time

    **Overview**: AI ML (GCP/BQ/spark/pyspark ) **Responsibilities**: AI ML (GCP/BQ/spark/pyspark ) **Requirements**: AI ML (GCP/BQ/spark/pyspark )

  • Python Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Data engineer 1) Overall 7+ in IT Experience with relevant 5+ years of experience in Data Engineering. Required technical skills: Python, PySpark, SQL, AWS 2) Required Domain: Healthcare Insurance 4) Extensive data analysis skills and Good communication skills 5) Experience working in Agile **About Virtusa** Teamwork, quality of life, professional and...

  • Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryAs a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life...

  • Pyspark + Databricks

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skill: Pyspark + Databricks** ** Experience**: 6 to 9 Years ** Location : AIA - Gurgaon** **Job Summary** **Responsibilities** - Lead the design and implementation of scalable data solutions using Databricks Workflows to enhance data processing capabilities. - Oversee the integration of Databricks Unity Catalog to ensure seamless data governance and...

  • Pyspark Developer

    3 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    10 hours ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore