Data engineer

Key Skills and Responsibilities:We are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation.This role will play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.Kindly click on the below link to apply: Uh P5s PEJKey Responsibilities:● Design and implement scalable, robust, and secure data pipelines using:● Azure Data Factory, Databricks, Synapse Analytics● Azure Data Lake Gen2, Event Hub, Azure Functions● Develop and maintain ETL/ELT processes for structured and unstructured data● Implement Change Data Capture (CDC), streaming pipelines, and batch ingestion● Work with AWS Glue, S3, Redshift or GCP Big Query, Dataflow as needed● Optimize performance and cost across data workloads and cloud environments● Establish and maintain data quality frameworks, observability tools, and monitoring dashboards● Define and enforce data rules, validations, anomaly detection and reconciliation● Implement data lineage and metadata tracking using Azure Purview● Drive adoption of data cataloguing, profiling, and classification tools● Collaborate with data stewards and compliance teams to ensure governance alignment● Implement role-based access control (RBAC), data masking, encryption, and tokenization● Responsible for technical design, coding, unit testing, test case documentation, and walkthroughsfor all assigned Azure related projects to support company business and operational needs.● Ensures software developed follows the defined programming standards and follows the code /design review processes. ● Critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details, abstract up from low level information to a general understanding, and distinguish user requests from the underlying true needs. ● Collaborate with developers and subject matter experts to establish the technical vision and analyse trade-offs between usability and performance needs. ● Mentor junior data engineers and contribute to technical best practicesQualification & Experience Technical Skills: ● 4+ years in data engineering with at least 2+ years on Azure cloud platform ● Hands-on with Azure Data Factory, Azure Data Lake Gen2, Databricks, Synapse, Purview ● Proficiency in SQL, Py Spark, Python, and data orchestration tools ● Strong understanding of data architecture patterns – Lakehouse, medallion, delta architecture ● Familiarity with Snowflake, Big Query, Redshift, or AWS Glue ● Experience with data versioning and Git Ops for data ● Working knowledge of data observability, lineage, cataloguing, data quality ● Exposure to privacy-enhancing techniques, access control, security auditing ● Exposure to machine learning use cases in data engineering pipelines, data quality, anomaly detection, and schema change detection ● Exposure to Gen AI or agentic AI to Automate data cataloguing and metadata enrichment etc. ● Experience with NLP or LLMs in metadata extraction or data classification is a plus Soft Skills: - ● Strong problem-solving, communication, and stakeholder collaboration skills ● Ability to lead data initiatives and mentor team members ● Proactive in learning and adopting emerging technologies in data & AI Qualification: - ·Candidate must be BE/BTech/MCA with 4 to 8 years' experience in Data Engineering.Personal Attributes/Traits
• Consultative
• Socially confident
• Achievement oriented
• Decisive and action oriented
• Creative
• Eager to learn
• Resilient Competencies
•Business Foresight
• Influencing Others
• Fostering Partnerships With Customers
• Managing Transformation
• Driving Excellence
• Leading Teams
• Working Across Boundaries