Python Bigdata Developer

Job Title Data Engineer Location TechM Blr ITPL Years of Experience 7 10 Years Job Summary We are seeking a highly skilled Data Engineer with a strong background in PySpark to join our dynamic team The ideal candidate will have 7 10 years of experience in data engineering with a proven track record of designing building and maintaining scalable data pipelines You will play a crucial role in transforming raw data into actionable insights enabling data driven decision making across the organization Skills Python Pyspark Hadoop HiveQL Devops Responsibilities Design develop and maintain robust data pipelines using PySpark to process large datasets efficiently Collaborate with data scientists and analysts to understand data requirements and deliver high quality data solutions Implement data integration processes from various sources ensuring data quality and consistency Optimize existing data workflows and troubleshoot performance issues to enhance data processing efficiency Work with cloud based data platforms and tools to manage and store data securely Document data engineering processes and maintain clear communication with stakeholders regarding project status and challenges Stay updated with the latest industry trends and technologies in data engineering and big data Mandatory Skills Strong expertise in PySpark for data processing and transformation Proficient in SQL and experience with relational databases Experience with data warehousing solutions and ETL processes Familiarity with cloud platforms such as AWS Azure or Google Cloud Solid understanding of data modeling and data architecture principles Ability to work in a fast paced environment and manage multiple priorities effectively Preferred Skills Experience with Apache Kafka or similar streaming technologies Knowledge of machine learning concepts and frameworks Familiarity with containerization technologies like Docker Experience with version control systems such as Git Strong analytical and problem solving skills Qualifications Bachelor s or Master s degree in Computer Science Information Technology or a related field 7 10 years of experience in data engineering or a related role Proven experience in building and optimizing data pipelines using PySpark Excellent communication skills and the ability to work collaboratively in a team environment If you are passionate about data engineering and possess the required skills we invite you to apply and join our innovative team at TechM Blr ITPL Tech Mahindra represents the connected world offering innovative and customer-centric information technology experiences enabling Enterprises Associates and the Society to Rise We are a USD 4 9 billion company with 121 840 professionals across 90 countries helping over 935 global customers including Fortune 500 companies Our convergent digital design experiences innovation platforms and reusable assets connect across a number of technologies to deliver tangible business value and experiences to our stakeholders Tech Mahindra is the highest ranked Non-U S company in the Forbes Global Digital 100 list 2018 and in the Forbes Fab 50 companies in Asia 2018