Lead Data Engineer
1 month ago
Lead Data Engineer-MS Fabric.
Exp : 10+ years.
Type : Contract(remote).
Shift Time : 3PM to 12AM IST.
Description :
We are looking for a Lead Data Engineer with extensive experience in developing ETL processes using PySpark Notebooks and Microsoft Fabric and supporting existing legacy SQL Server environments.
The ideal candidate will possess a strong background in Spark-based development, demonstrate a high proficiency in SQL, and be comfortable working independently, collaboratively within a team, or leading other developers when required.
- Design, develop, and maintain ETL pipelines using PySpark Notebooks and Microsoft Fabric.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver efficient data solutions.
- Migrate and integrate data from legacy SQL Server environments into modern data platforms.
- Optimize data pipelines and workflows for scalability, efficiency, and reliability.
- Provide technical leadership and mentorship to junior developers and other team members.
- Troubleshoot and resolve complex data engineering issues related to performance, data quality, and system scalability.
- Develop, maintain, and enforce data engineering best practices, coding standards, and documentation.
- Conduct code reviews and provide constructive feedback to improve team productivity and code quality.
- Support data-driven decision-making processes by ensuring data integrity, availability, and consistency across different platforms.
Requirements :
- Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field.
- Experience with Microsoft Fabric or similar cloud-based data integration platforms is a must.
- 10+ years of experience in data engineering, with a strong focus on ETL development using PySpark or other Spark-based tools.
- Proficiency in SQL with extensive experience in complex queries, performance tuning, and data modeling.
- Strong knowledge of data warehousing concepts, ETL frameworks, and big data processing.
- Familiarity with other data processing technologies (e.g, Hadoop, Hive, Kafka) is an advantage.
- Experience working with both structured and unstructured data sources.
- Excellent problem-solving skills and the ability to troubleshoot complex data engineering issues.
- Proven ability to work independently, as part of a team, and in leadership roles.
- Strong communication skills with the ability to translate complex technical concepts into business terms.
Mandate skills :
- Experience with Data lake, Data warehouse, Delta lake.
- Experience with Azure Data Services, including Azure Data Factory, Azure Synapse, or similar tools.
- Knowledge of scripting languages (e.g, Python, Scala) for data manipulation and automation.
- Familiarity with DevOps practices, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.
-
Data Engineering Manager Lead
2 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeScale Data Pipeline Services with HevoWe are seeking an experienced Data Engineering Manager to lead the development of scalable platform services supporting thousands of customers' data pipelines.This role offers a unique opportunity to build and own critical components of our cloud warehouse solution, ensuring reliable data loading and streamlining...
-
Chief Data Engineering Lead
2 weeks ago
Bengaluru, Karnataka, India Data Inc Full timeAbout the RoleWe are seeking a highly skilled Chief Data Engineering Lead to join our team at Data Inc.This is an exciting opportunity to lead the development and implementation of architecture strategy and standards pertinent to the domain, and establish Standards and Best practices.
-
Data Engineering Lead with Scalable Platforms
2 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeUnlock Career Growth as a Data Engineering LeadWe're seeking an exceptional Data Engineering Lead to join our team at Hevo Data. In this role, you'll have the opportunity to own and build scalable platform services that support thousands of customers' data pipelines and ensure load data reliability to cloud warehouses.The Hevo platform core services will...
-
Data engineer
1 month ago
Bengaluru, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...
-
Lead Data Engineering Manager
3 weeks ago
Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full timeAbout UsData Labs - LANDMARK GROUP is a leading data engineering firm committed to delivering cutting-edge analytics solutions.Job DescriptionWe are seeking an experienced Lead Data Engineering Manager to oversee the development and deployment of our analytics products. This role involves leading a team of engineers in designing, building, and deploying...
-
Data Engineer
3 months ago
Bengaluru, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...
-
Data Engineer
3 months ago
Bengaluru, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...
-
Data Engineer
2 months ago
Bengaluru, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...
-
Data Science
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeJob OverviewWe are seeking an experienced Data Science & AI Engineer to lead our machine learning initiatives and drive innovation in artificial intelligence.About the RoleThe successful candidate will have a strong background in developing and implementing traditional machine learning algorithms, with experience deploying models in production environments....
-
Data Engineering Director
2 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeEngineering Leadership OpportunityThis is a unique chance to own and build scalable platform services that support thousands of customers' data pipelines, ensuring load data reliability to cloud warehouses.The Hevo platform core services will provide no-code-low pipeline building capabilities, making a significant impact on the company's business.About the...
-
(15h Left) Data Engineer
2 months ago
Bengaluru, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.- Lead the architecture, design, and delivery of enterprise grade microservices.- Deliver assigned projects within budgeted time, cost and agreed...
-
Lead Data Architect
3 weeks ago
Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data ArchitectLocation: Yemalur, BangaloreAt Data Labs - LANDMARK GROUP, we are seeking a skilled Lead Data Architect to spearhead our engineering team in successfully designing and deploying cutting-edge analytics products.Main Responsibilities:Lead the architecture, design, and delivery of enterprise-grade microservices.Partner with Product...
-
Data-Driven Customer Experience Lead
4 weeks ago
Bengaluru, Karnataka, India Hevo Data Full timeAbout Hevo Data:We are a leading provider of fully managed Automated Pipelines, used by thousands of data-driven companies worldwide. Our mission is to enable every company to be data-driven, and we're looking for an experienced Customer Experience Lead to join our team.As a Customer Experience Lead at Hevo Data, you will be responsible for ensuring...
-
Senior SAP Data Migration Lead
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the Role:-Sr Syniti ADM Consultant:As a highly experienced SAP Data Migration Consultant, you will lead the execution of data migration projects from SAP ECC to SAP S4/HANA, utilizing your expertise in data cleansing, profiling, and harmonization. Your proficiency in project planning, cutover planning, and gathering business requirements is essential...
-
AI Data Science and Engineering Leader
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeJob Title: AI Data Science and Engineering LeaderAbout the Role:We are seeking a highly skilled Ai Data Science and Engineering Leader to join our team in Bangalore, India. This is an excellent opportunity for a talented individual to lead our data science and engineering efforts.Responsibilities:Develop and implement traditional machine learning algorithms...
-
GCP Lead Engineer
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a skilled GCP Lead Engineer to join our team at NTT DATA. This role is ideal for someone with expertise in designing and implementing Google Cloud Platform (GCP) solutions.Key ResponsibilitiesAssess, design, and implement GCP solutions, including compute, network, storage, identity, security, and disaster recovery.Develop and...
-
Data Engineering Specialist
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeJob Title: Data Engineering Specialist - BangaloreAbout the Role:We are seeking a highly skilled Data Engineering Specialist to join our team in Bangalore, India. As a key member of our engineering team, you will be responsible for designing and developing high-quality data pipelines using ETL code in Databricks.Key Responsibilities:Develop high-level design...
-
AI Data Engineer
1 month ago
Bengaluru, India NTT DATA Full timeJob Description Job Title: Data Science & AIML, GenAI Lead/EngineerKey Responsibilities:• Develop and implement traditional machine learning algorithms.• Deploy at least one model in a production environment.• Write and maintain Python code for data science and machine learning projects.Preferred Qualifications:• Knowledge of Deep Learning (DL)...
-
Data Engineering Expert
1 month ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the RoleWe are seeking a skilled Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will play a crucial role in designing and implementing automated data pipelines.
-
Cloud Data Engineer for Big Data Solutions
4 weeks ago
Bengaluru, Karnataka, India NTT DATA Full timeAbout the Role:As a Cloud Data Engineer, you will be responsible for designing and implementing large-scale data processing systems using GCP's BigQuery, Dataflow, and Dataproc technologies. Your expertise in ETL/ELT pipelines, data warehousing, and lakehouse solutions will enable us to efficiently process structured and unstructured data.Key...