Lead data engineer

4 days ago

Delhi, India SourceBae Full time
Lead Data Engineer-MS Fabric.
Exp: 10+ years
contract(remote)
shift Time-3 PM to 12 AM IST
Description
We are looking for a Lead Data Engineer with extensive experience in developing ETL processes using Py Spark Notebooks and Microsoft Fabric and supporting existing legacy SQL Server environments. The ideal candidate will possess a strong background in Spark-based development, demonstrate a high proficiency in SQL, and be comfortable working independently, collaboratively within a team, or leading other developers when required.
Design, develop, and maintain ETL pipelines using Py Spark Notebooks and Microsoft Fabric.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver efficient data solutions.
Migrate and integrate data from legacy SQL Server environments into modern data platforms.
Optimize data pipelines and workflows for scalability, efficiency, and reliability.
Provide technical leadership and mentorship to junior developers and other team members.
Troubleshoot and resolve complex data engineering issues related to performance, data quality, and system scalability.
Develop, maintain, and enforce data engineering best practices, coding standards, and documentation.
Conduct code reviews and provide constructive feedback to improve team productivity and code quality.
Support data-driven decision-making processes by ensuring data integrity, availability, and consistency across different platforms.
Requirements
Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field.
Experience with Microsoft Fabric or similar cloud-based data integration platforms is a must.
10+ years of experience in data engineering, with a strong focus on ETL development using Py Spark or other Spark-based tools.
Proficiency in SQL with extensive experience in complex queries, performance tuning, and data modeling.
Strong knowledge of data warehousing concepts, ETL frameworks, and big data processing.
Familiarity with other data processing technologies (e.g., Hadoop, Hive, Kafka) is an advantage.
Experience working with both structured and unstructured data sources.
Excellent problem-solving skills and the ability to troubleshoot complex data engineering issues.
Proven ability to work independently, as part of a team, and in leadership roles.
Strong communication skills with the ability to translate complex technical concepts into business terms.
Mandate skills
Experience with Data lake, Data warehouse, Delta lake
Experience with Azure Data Services, including Azure Data Factory, Azure Synapse, or similar tools.
Knowledge of scripting languages (e.g., Python, Scala) for data manipulation and automation.
Familiarity with Dev Ops practices, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.
If Intrested. Please submit your CV to or share it via Whats App at 8827565832
Stay updated with our latest job opportunities and company news by following us on Linked In: :

  • Lead data engineer

    5 days ago

    Delhi, India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...

  • Java software engineer

    4 days ago

    Delhi, India Lead Masters AI Full time

    Company DescriptionLead Masters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. Lead...

  • Data Engineering Lead

    16 hours ago

    Delhi, India Welspun Transformation Services Limited Full time

    As a Data Engineering Lead, you will be responsible for managing and leading the data engineering team in our IT department. You will play a crucial role in designing, building, and maintaining our data infrastructure, data processing systems, and data pipelines. You will also be responsible for implementing data visualization tools and techniques to help...

  • Java Software Engineer

    2 months ago

    delhi, India Lead Masters AI Full time

    Company Description LeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Java Software Engineer

    2 months ago

    Delhi, India Lead Masters AI Full time

    Company Description LeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Java Software Engineer

    2 months ago

    delhi, India Lead Masters AI Full time

    Company DescriptionLeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Java Software Engineer

    2 months ago

    Delhi, India Lead Masters AI Full time

    Company DescriptionLeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Java Software Engineer

    6 days ago

    delhi, India Lead Masters AI Full time

    Company DescriptionLeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Lead Data Engineer

    6 days ago

    delhi, India SourceBae Full time

    Lead Data Engineer-MS Fabric.Exp: 10+ yearscontract(remote)shift Time-3PM to 12AM ISTDescriptionWe are looking for a Lead Data Engineer with extensive experience in developing ETL processes using PySpark Notebooks and Microsoft Fabric and supporting existing legacy SQL Server environments. The ideal candidate will possess a strong background in Spark-based...

  • Lead Data Engineer

    2 weeks ago

    Delhi, India OnFocus Soft Full time

    Experience8+ work experienceCompany DescriptionWe are hiring for one of the top notch MNC based out of the United States of America. This is an exciting opportunity to be a core team member of their first ever deployment in India! Exciting days ahead!!Role DescriptionThis is a full-time hybrid role for a Data Engineer (Automation team) specializing in...

  • AWS Data Engineer

    1 week ago

    delhi, India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    2 weeks ago

    delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • AWS Data Engineer

    3 months ago

    Delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • Aws data engineer

    1 week ago

    Delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • Data Engineering Lead

    3 weeks ago

    Delhi/Bangalore, India Hero Fincorp Ltd Full time

    Data Engineering LeadWe are seeking an experienced Data Engineering Lead to join our team at Hero Fincorp Ltd. The ideal candidate will have a strong background in data engineering, with expertise in Databricks and AWS. Key responsibilities include designing and developing data pipelines, managing data quality, and leading a team of engineers. A bachelor's...

  • Lead Data Engineer

    2 weeks ago

    delhi, India Intuitive.Cloud Full time

    About us: Intuitive.Cloud is one of the fastest-growing (INC 5000, CRN) Cloud & SDx solution and services companies supporting enterprise customers on a global scale. Intuitive is an "Engineering Company" delivering measurable value and key business outcomes. Intuitive Superpowers: - DataOps & AI/ML - Cloud Native, AppSecOps, DevSecOps - Cloud Migration...

  • Data Engineering Lead

    3 weeks ago

    Delhi, Delhi, India Info Services Full time

    We are seeking an experienced Data Engineering Lead to join Info Services, responsible for designing, building, and maintaining large-scale data systems. Key skills include strong coding skills in Scala, Python, or Java, as well as proficiency in Spark, big data frameworks, and cloud platforms such as AWS, GCP, or Azure.Skill Requirements:Strong coding...

  • ▷ Only 24h Left: Java Software Engineer

    4 days ago

    Delhi, India Lead Masters AI Full time

    Company DescriptionLeadMasters.ai is an innovative AI-driven platform for lead generation and ad optimization, utilizing advanced algorithms to analyze top-performing ads across social media. The platform empowers businesses of all sizes to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. LeadMasters.ai...

  • Lead Data Engineer

    1 week ago

    delhi, India Intuitive.Cloud Full time

    About us:Intuitive.Cloud is one of the fastest-growing (INC 5000, CRN) Cloud & SDx solution and services companies supporting enterprise customers on a global scale. Intuitive is an "Engineering Company" delivering measurable value and key business outcomes.Intuitive Superpowers:- DataOps & AI/ML- Cloud Native, AppSecOps, DevSecOps- Cloud Migration &...

  • Lead Data Engineer

    2 weeks ago

    delhi, India Intuitive.Cloud Full time

    About us:Intuitive.Cloud is one of the fastest-growing (INC 5000, CRN) Cloud & SDx solution and services companies supporting enterprise customers on a global scale. Intuitive is an "Engineering Company" delivering measurable value and key business outcomes.Intuitive Superpowers:- DataOps & AI/ML- Cloud Native, AppSecOps, DevSecOps- Cloud Migration &...