Sr. Data Engineer

About Position: We are currently seeking a highly proficient Sr Data Engineer to be part of a global team and function in a lead developer role. You will perform analysis, design and development of data processing ETL job streams to support the data warehouses, data marts and other data stores in support of the Optum business. You will do hands-on development and testing work as part of a scrum team that is global in nature. The primary technology stack will be Azure Databricks environment, with Delta Lake, Airflow, Python components and a Snowflake data store. You will work closely with external and internal stakeholders to design solutions to solve problems and resolve issues.Role: Sr. Data EngineerLocation: All Persistent LocationsExperience: 7 to 10 YearsJob Type: Full Time EmploymentWhat You'll Do: As Data Engineer, you will evaluate, design, develop, maintain and test Big Data solutions. You will be involved in data engineering activities like creating pipelines/ workflows for source to target mappingsResponsible for data ingestion from files, streams and databases. Process the data using Azure Databricks, Python, SQL.Design and development of ETL data flows using Azure Databricks ecosystem components. Lead the development of large-scale, high-speed, and low-latency data solutions in the areas of large-scale data manipulation, long-term data storage, data warehousing, low-latency retrieval systems, real-time reporting and analytics data applications, ad-hoc analytics. Performance tune long running jobs.Develop programs in Python and SQL as part of data cleaning and processing.Develop efficient code for multiple use cases leveraging Python and Big Data technologies on the Azure platform in an optimal mannerMaintain high operational excellence guaranteeing high availability and platform stabilityDocument architecture and design components. Be able to communicate these to other engineers on the team. Guide and mentor junior engineersFollow DevOps practices to automate the data pipelinesExpertise You'll Bring: 5+ years of overall experience in Big Data with prior experience in ETLFamiliarity with the application of AI/ML tools and models to data processing and ETL workloadsStrong, advanced expertise in Azure Databricks, Delta Lake and PythonStrong knowledge of SQL3+ years of DevOps and CI/CD pipeline implementation and managementWorking experience with Git hubExperience designing and developing components for a cloud application, preferably AzureGood oral and written communication skills.Benefits:Competitive salary and benefits packageCulture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certificationsOpportunity to work with cutting-edge technologiesEmployee engagement initiatives such as project parties, flexible work hours, and Long Service awardsAnnual health check-upsInsurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parentsValues-Driven, People-Centric & Inclusive Work Environment:Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.We support hybrid work and flexible hours to fit diverse lifestyles.Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employmentLet’s unleash your full potential at Persistent - persistent.com/careers“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”