Data Engineer

Key Skills and Responsibilities:

We are seeking a Senior Data Engineer with a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. The ideal candidate will have deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation.

This role will play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.

Kindly click on the below link to apply:

https://forms.office.com/r/tfUhP5sPEJ

Key Responsibilities:

● Design and implement scalable, robust, and secure data pipelines using:

● Azure Data Factory, Databricks, Synapse Analytics

● Azure Data Lake Gen2, Event Hub, Azure Functions

● Develop and maintain ETL/ELT processes for structured and unstructured data

● Implement Change Data Capture (CDC), streaming pipelines, and batch ingestion

● Work with AWS Glue, S3, Redshift or GCP BigQuery, Dataflow as needed

● Optimize performance and cost across data workloads and cloud environments

● Establish and maintain data quality frameworks, observability tools, and monitoring dashboards

● Define and enforce data rules, validations, anomaly detection and reconciliation

● Implement data lineage and metadata tracking using Azure Purview

● Drive adoption of data cataloguing, profiling, and classification tools

● Collaborate with data stewards and compliance teams to ensure governance alignment

● Implement role-based access control (RBAC), data masking, encryption, and tokenization

● Responsible for technical design, coding, unit testing, test case documentation, and walkthroughs

for all assigned Azure related projects to support company business and operational needs.

● Ensures software developed follows the defined programming standards and follows the code /design review processes.

● Critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details, abstract up from low level information to a general understanding, and distinguish user requests from the underlying true needs.

● Collaborate with developers and subject matter experts to establish the technical vision and analyse trade-offs between usability and performance needs.

● Mentor junior data engineers and contribute to technical best practices

Qualification & Experience Technical Skills:

● 4+ years in data engineering with at least 2+ years on Azure cloud platform

● Hands-on with Azure Data Factory, Azure Data Lake Gen2, Databricks, Synapse, Purview

● Proficiency in SQL, PySpark, Python, and data orchestration tools

● Strong understanding of data architecture patterns – Lakehouse, medallion, delta architecture

● Familiarity with Snowflake, BigQuery, Redshift, or AWS Glue

● Experience with data versioning and GitOps for data

● Working knowledge of data observability, lineage, cataloguing, data quality

● Exposure to privacy-enhancing techniques, access control, security auditing

● Exposure to machine learning use cases in data engineering pipelines, data quality, anomaly detection, and schema change detection

● Exposure to GenAI or agentic AI to Automate data cataloguing and metadata enrichment etc.

● Experience with NLP or LLMs in metadata extraction or data classification is a plus Soft Skills: -

● Strong problem-solving, communication, and stakeholder collaboration skills

● Ability to lead data initiatives and mentor team members

● Proactive in learning and adopting emerging technologies in data & AI Qualification: - ·

Candidate must be BE/BTech/MCA with 4 to 8 years' experience in Data Engineering.

Personal Attributes/Traits

• Consultative
• Socially confident
• Achievement oriented
• Decisive and action oriented
• Creative
• Eager to learn
• Resilient Competencies
•Business Foresight
• Influencing Others
• Fostering Partnerships With Customers
• Managing Transformation
• Driving Excellence
• Leading Teams

• Working Across Boundaries