PySpark Developer

Description :Key Responsibilities :- As part of the Infosys delivery team your primary role would be to interface with the client for quality assurance issue resolution and ensuring high customer satisfaction- You will understand requirements create and review designs validate the architecture and ensure high levels of service offerings to clients in the technology domain- You will participate in project estimation provide inputs for solution delivery conduct technical risk planning perform code reviews and unit test plan reviews- You will lead and guide your teams towards developing optimized high quality code deliverables continual knowledge management and adherence to the organizational guidelines and processes- Develop and maintain scalable data pipelines using Python and PySpark.- Collaborate with data engineers and data scientists to understand and fulfil data processing needs.- Optimize and troubleshoot existing PySpark applications for performance improvements.- Write clean, efficient, and well-documented code following best practices.- Participate in design and code reviews.- Develop and implement ETL processes to extract, transform, and load data.- Ensure data integrity and quality throughout the data lifecycle.- Stay current with the latest industry trends and technologies in big data and cloud computing.- You would be a key contributor to building efficient programs systems and if you think you fit right in to help our clients navigate their next in their digital transformation journey this is the place for you- If you think you fit right in to help our clients navigate their next in their digital transformation journey this is the place for youTechnical Requirements :- Must have excellent knowledge in Apache Spark and Python programming experience.- Experience in developing data processing tasks using PySpark such as reading data from external sources, merging data, aggregating data, performing data enrichment, and loading into target data destinations.- Hands-on project experience on Jupyter Notebook/ PyCharm etc. IDE- Should have experience in fine-tuning process and troubleshooting performance issue. (ref:hirist.tech)