Data Engineer Lead

About the RoleWe are seeking an experiencedData Engineer Leadto design, develop, and maintain scalable data solutions onAzure and Databricksas part of our enterprise data modernization initiatives. The ideal candidate will have a strong background indata pipeline development, data integration frameworks , andcloud-based data engineering , with deep expertise in tools such asDatabricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica .This role will lead the design and delivery ofhigh-performance, governed data architecturesfor large-scale enterprise clients, driving data reliability, compliance, and analytics readiness.Key ResponsibilitiesDesign and implementscalable and reusabledata pipelinesusingDatabricks (PySpark, Delta Lake)andAzure Data Factory , ensuring high throughput and low latency.Develop and maintainCommon Data Platforms (CDP)andMedallion Architecture-based data lakesto consolidate legacy systems and improve data availability.Create and managebackend data servicesusingPython, Spark, or C# , ensuring performance, reliability, and scalability.Develop data ingestion frameworksusing metadata-driven and schema-drift–handling designs to support multi-source data integration.Leaddata integration and migration projectsusingTalend, Ab Initio, Alteryx, and Informatica , modernizing traditional ETL workflows into cloud-native pipelines.Implement data quality, lineage, and governance frameworks(Unity Catalog, Data Stewardship, Data Catalog, MDM).Design and maintainGDPR/CCPA-compliant data processingand ensure adherence to global data privacy standards.Build and optimizeFeature Store Data Martsto support AI/ML analytics teams and downstream BI consumers.Collaborate with cross-functional teams (Data Scientists, BI Developers, Product Owners) to define and deliver data-driven business outcomes.Mentor junior engineers inbest practices of data engineering , CI/CD, and infrastructure-as-code deployments.UtilizeRedisand caching mechanisms to optimize performance and reduce load latency in analytic workloads.Take end-to-end ownership of assigned modules ensuring timely delivery, documentation, and performance optimization.Technical EnvironmentCloud Platforms:Microsoft Azure (Data Factory, Synapse, Data Lake, Databricks), AWS (S3, EC2, Glue, Redshift)Data Engineering Tools:Databricks (DLT, Auto Loader, Unity Catalog), Talend, Ab Initio, Informatica, AlteryxProgramming Languages:Python, PySpark, SQL, Unix Shell ScriptingData Modeling:Dimensional, Relational, and Medallion architectureDevOps / CI-CD:GitHub Actions, Azure DevOps, Databricks Asset Bundles (DABs)Data Governance & Security:RBAC/ABAC, Data Stewardship, MDM, GDPR/CCPA complianceMonitoring & Scheduling:Airflow, Logic Apps, Azure MonitorRequired Qualifications12+ years of experience inData Engineering, Data Architecture, or ETL Development .Proven experience leadingAzure + Databricks –based modernization or migration programs.Deep understanding ofdata integration tools(Talend, Informatica, Ab Initio, Alteryx) andmetadata-driven pipeline design .Hands-on experience withPySpark, SQL, and Azure Data Services .Strong understanding ofdata modeling, data quality, and governance frameworks .Experience withreal-time and batch data ingestion , including Kafka, Kinesis, or Event Hub.Excellent problem-solving, communication, and stakeholder management skills.Preferred QualificationsBachelor’s degree in computer science, or a related fieldStrong analytical and problem-solving skillsAzure Data Engineer Certification or equivalentFamiliarity withAI/ML data preparation and Feature Store designExperience implementingcost optimization and cluster governanceon DatabricksWhy Join UsBe part of adata modernization journeywith a global enterprise brand.Work with cutting-edgecloud and data engineering tools .Collaborate with ahighly skilled, cross-functional teamof architects, analysts, and data scientists.Opportunity to lead and innovate within a large-scale, multi-domaindata transformation initiative .