Principal Consultant- Databricks Developer With Experience In Unity Catalog + Python

Ready to shape the future of work At Genpact we don t just adapt to change we drive it AI and digital innovation are redefining industries and we re leading the charge Genpact s AI Gigafactory our industry-first accelerator is an example of how we re scaling advanced technology solutions to help global enterprises work smarter grow faster and transform at scale From large-scale models to agentic AI our breakthrough solutions tackle companies most complex challenges If you thrive in a fast-moving tech-driven environment love solving real-world problems and want to be part of a team that s shaping the future this is your moment Genpact NYSE G is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally Through our deep business knowledge operational excellence and cutting-edge solutions - we help companies across industries get ahead and stay ahead Powered by curiosity courage and innovation our teams implement data technology and AI to create tomorrow today Get to know us at genpact com and on LinkedIn X YouTube and Facebook Inviting applications for the role of Principal Consultant- Databricks Developer with experience in Unity Catalog Python Spark Kafka for ETL In this role the Databricks Developer is responsible for solving the real world cutting edge problem to meet both functional and non-functional requirements Responsibilities Develop and maintain scalable ETL pipelines using Databricks with a focus on Unity Catalog for data asset management Implement data processing frameworks using Apache Spark for large-scale data transformation and aggregation Integrate real-time data streams using Apache Kafka and Databricks to enable near real-time data processing Develop data workflows and orchestrate data pipelines using Databricks Workflows or other orchestration tools Design and enforce data governance policies access controls and security protocols within Unity Catalog Monitor data pipeline performance troubleshoot issues and implement optimizations for scalability and efficiency Write efficient Python scripts for data extraction transformation and loading Collaborate with data scientists and analysts to deliver data solutions that meet business requirements Maintain data documentation including data dictionaries data lineage and data governance frameworks Qualifications we seek in you Minimum qualifications Bachelor s degree in Computer Science Data Engineering or a related field experience in data engineering with a focus on Databricks development Proven expertise in Databricks Unity Catalog and data lake management Strong programming skills in Python for data processing and automation Experience with Apache Spark for distributed data processing and optimization Hands-on experience with Apache Kafka for data streaming and event processing Proficiency in SQL for data querying and transformation Strong understanding of data governance data security and data quality frameworks Excellent communication skills and the ability to work in a cross-functional environ Must have experience in Data Engineering domain Must have implemented at least 2 project end-to-end in Databricks Must have at least experience on databricks which consists of various components as below o Delta lake o dbConnect o db API 2 0 o Databricks workflows orchestration Must be well versed with Databricks Lakehouse concept and its implementation in enterprise environments Must have good understanding to create complex data pipeline Must have good knowledge of Data structure algorithms Must be strong in SQL and sprak-sql Must have strong performance optimization skills to improve efficiency and reduce cost Must have worked on both Batch and streaming data pipeline Must have extensive knowledge of Spark and Hive data processing framework Must have worked on any cloud Azure AWS GCP and most common services like ADLS S3 ADF Lambda CosmosDB DynamoDB ASB SQS Cloud databases Must be strong in writing unit test case and integration test Must have strong communication skills and have worked on the team of size 5 plus Must have great attitude towards learning new skills and upskilling the existing skills Preferred Qualifications Good to have Unity catalog and basic governance knowledge Good to have Databricks SQL Endpoint understanding Good To have CI CD experience to build the pipeline for Databricks jobs Good to have if worked on migration project to build Unified data platform Good to have knowledge of DBT Good to have knowledge of docker and Kubernetes Why join Genpact Be a transformation leader - Work at the cutting edge of AI automation and digital innovation Make an impact - Drive change for global enterprises and solve business challenges that matter Accelerate your career - Get hands-on experience mentorship and continuous learning opportunities Work with the best - Join 140 000 bold thinkers and problem-solvers who push boundaries every day Thrive in a values-driven culture - Our courage curiosity and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress Come join the tech shapers and growth makers at Genpact and take your career in the only direction that matters Up Let s build tomorrow together Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race color religion or belief sex age national origin citizenship status marital status military veteran status genetic information sexual orientation gender identity physical or mental disability or any other characteristic protected by applicable laws Genpact is committed to creating a dynamic work environment that values respect and integrity customer focus and innovation Furthermore please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way Examples of such scams include purchasing a starter kit paying to apply or purchasing equipment or training