Data Engineer

Key Skills and Responsibilities:We are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation.This role will play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.Kindly click on the below link to apply: Responsibilities:● Design and implement scalable, robust, and secure data pipelines using:● Azure Data Factory, Databricks, Synapse Analytics● Azure Data Lake Gen2, Event Hub, Azure Functions● Develop and maintain ETL/ELT processes for structured and unstructured data● Implement Change Data Capture (CDC), streaming pipelines, and batch ingestion● Work with AWS Glue, S3, Redshift or GCP BigQuery, Dataflow as needed● Optimize performance and cost across data workloads and cloud environments● Establish and maintain data quality frameworks, observability tools, and monitoring dashboards● Define and enforce data rules, validations, anomaly detection and reconciliation● Implement data lineage and metadata tracking using Azure Purview● Drive adoption of data cataloguing, profiling, and classification tools● Collaborate with data stewards and compliance teams to ensure governance alignment● Implement role-based access control (RBAC), data masking, encryption, and tokenization● Responsible for technical design, coding, unit testing, test case documentation, and walkthroughsfor all assigned Azure related projects to support company business and operational needs.● Ensures software developed follows the defined programming standards and follows the code /design review processes.● Critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details, abstract up from low level information to a general understanding, and distinguish user requests from the underlying true needs.● Collaborate with developers and subject matter experts to establish the technical vision and analyse trade-offs between usability and performance needs.● Mentor junior data engineers and contribute to technical best practicesQualification & Experience Technical Skills:● 4+ years in data engineering with at least 2+ years on Azure cloud platform● Hands-on with Azure Data Factory, Azure Data Lake Gen2, Databricks, Synapse, Purview● Proficiency in SQL, PySpark, Python, and data orchestration tools● Strong understanding of data architecture patterns – Lakehouse, medallion, delta architecture● Familiarity with Snowflake, BigQuery, Redshift, or AWS Glue● Experience with data versioning and GitOps for data● Working knowledge of data observability, lineage, cataloguing, data quality● Exposure to privacy-enhancing techniques, access control, security auditing● Exposure to machine learning use cases in data engineering pipelines, data quality, anomaly detection, and schema change detection● Exposure to GenAI or agentic AI to Automate data cataloguing and metadata enrichment etc.● Experience with NLP or LLMs in metadata extraction or data classification is a plus Soft Skills: -● Strong problem-solving, communication, and stakeholder collaboration skills● Ability to lead data initiatives and mentor team members● Proactive in learning and adopting emerging technologies in data & AI Qualification: - ·Candidate must be BE/BTech/MCA with 4 to 8 years’ experience in Data Engineering.Personal Attributes/Traits• Consultative • Socially confident • Achievement oriented • Decisive and action oriented • Creative • Eager to learn • Resilient Competencies •Business Foresight • Influencing Others • Fostering Partnerships With Customers • Managing Transformation • Driving Excellence • Leading Teams• Working Across Boundaries