Lead II

Job Title: Lead II - Data Engineering - Data Analysis, Data Structures

Domain:Information technology (IT)

Location:India - Bangalore

Salary: INR

Experience:7 - 9 Years

3 days in office, Hybrid model.

Must-Have Skill : Data Analysis, Data Structures,Data Warehousing

Job Description

Job Summary

A senior-level Data Engineer role focused on independently designing, developing, and optimizing large-scale, cloud-based data pipelines and architectures. The position requires end-to-end project ownership, from design to production, and includes responsibilities for team leadership, technical guidance, and stakeholder engagement.

Key Responsibilities & Outcomes

Data Engineering & Development

*   Design, code, and test data pipelines for ingesting, wrangling, transforming, and joining data from various sources.

*   Develop and manage data storage solutions including data warehouses, data lakes, Delta Lakes, and relational/NoSQL databases.

*   Create optimal data architectures and designs (HLD, LLD) and effective data schemas and models.

*   Tune performance, optimize costs, and resolve performance issues in data pipelines and warehouses.

*   Implement data governance, including metadata management, data quality rules, and lineage.

Project & Team Management:

*   Support the Project Manager in project execution, planning, and resource estimation.

*   Manage the delivery of project modules and oversee the developmental activities of the team.

*   Set team goals, provide feedback, guide career development, and proactively manage team attrition.

*   Mentor team members and guide their upskilling and certification efforts.

Technical Leadership & Process

*   Define and govern coding standards, configuration management, and engineering processes.

*   Review code, design documents, and test plans created by the team and peers.

*   Conduct defect root cause analysis and implement mitigation strategies.

*   Interface with customers to clarify requirements, present design options, and conduct product demos.

*   Manage the release process and ensure adherence to SLAs and timelines.

Required Qualifications & Skills

Technical Skills:

*   Programming Languages:** Expert proficiency in **Python** and **SQL**; strong skills in **PySpark**.

*   ETL/ELT Tools:Hands-on experience with **AWS Glue**, **Databricks**, **DataProc**, and Informatica.

*   Cloud Platforms & Data Warehousing:Expert-level experience with **AWS**; intermediate with Azure. Expertise in **Snowflake**, **BigQuery**, **Redshift**, and data lake/lakehouse architectures (e.g., **Delta Lake**).

*   Data Modeling & Databases:Strong understanding of data schemas, models (e.g., Dimensional modeling), RDBMS (PostgreSQL, SQL Server), and NoSQL (MongoDB, Elasticsearch).

*  DevOps & Infrastructure:Experience with CI/CD (Jenkins, CircleCI), containerization (Docker, Kubernetes), and Infrastructure as Code (Terraform). Ability to calculate infrastructure costs.

Professional Skills

*   Independence and proficiency across various data domains.

*   Ability to creatively solve problems, optimize solutions, and reuse proven patterns.

*   Strong communication skills to explain technical details to customers and stakeholders.

*   Experience in project estimation, RFP responses, and solutioning.

*   Proven ability to lead, mentor, and upskill a team of data engineers.

Measures of Success

*   Adherence to project timelines, engineering standards, and SLAs.

*   Low number of defects and non-compliance issues post-delivery.

*   Quick turnaround of production bugs.

*   Efficiency improvements in pipelines (reduced resource use, faster run times).

*   Completion of relevant technical and domain certifications.

*   High levels of customer satisfaction and team engagement.