Lead Python Data Engineer

2 days ago

Noida, Uttar Pradesh, India Algoworks Full time ₹ 12,00,000 - ₹ 24,00,000 per year

Lead Python Data Engineer - Databricks (Unity Catalog, Delta, CI/CD) 

Job Summary

We are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL, and Power BI. This role requires a proactive individual who can work independently and collaboratively in a fast-paced environment.

Required Technical Skills:

  • 7+ years of hands-on experience in software development, with a strong emphasis on Python programming.
  • Deep expertise in PySpark (Spark SQL, DataFrames, and Datasets) for building and optimizing scalable data pipelines. (PySpark)
  • Solid experience with Azure Databricks (Unity Catalog), including working with Delta tables, workflows, and scripting in Python.
  • Understanding of data governance, data management, security, and compliance in cloud environments.
  • Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory,
  • Experience with Power BI, including DAX and Power Query.
  • Modeling data for Data Warehousing and Visualization projects through Semantic Layers.
  • Skilled in writing complex SQL queries and building views involving joins across multiple tables.
  • Proficient in tools like DevOps (project management), and version control and CI/CD pipelines.

Nice to have Skills:

  • Experience Azure AI, Azure Copilot Studio and Azure Fabric.
  • Experience building workflows with Power Automate.
  • EventHub or Kafka for streaming data ingestion.
  • Proficient in designing and implementing real-time data streaming solutions.
  • Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies.

Soft Skills & Collaboration:

  • Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination.
  • Excellent problem-solving and code review skills.
  • Strong communication skills with the ability to understand business requirements and translate them into technical solutions.
  • Proactive and responsive during business hours for any ad-hoc calls or meetings.
  • Demonstrated ability to prioritize effectively, think creatively, and adapt quickly to new technologies and feedback.

  • Python developer

    4 days ago

    Noida, Uttar Pradesh, India Gauge data solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Essential SkillsDevelop, enhance and maintain Python related projects, data services, platforms and processes.Apply and maintain data quality checks to ensure data integrity and completeness.Able to integrate multiple data sources and databases.Collaborate with cross-functional teams across, Decision Sciences, Search, Database Management. To design...

  • Lead Data Engineer

    1 week ago

    Noida, Uttar Pradesh, India Uplers Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Lead Data Engineer - Azure CloudExperience: Years Exp.Salary: CompetitivePreferred Notice Period: 30 DaysShift: 10:00 AM to 7:00 PM ISTOpportunity Type: Noida (Remote for 6 months, Later Hybrid)Placement Type: Permanent(*Note: This is a requirement for one of Uplers' Clients)Must have skills required :Engineering management, Data Engineering, Azure Data...

  • Python Data Engineer

    6 days ago

    Noida, Uttar Pradesh, India ShyftLabs Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    POSITION OVERVIEW We are seeking a highly skilled Python Data Engineer with a strong background in designing, developing, and maintaining scalable data solutions. The ideal candidate will have 5+ years of professional experience in Python and SQL, with proven expertise in building ETL/ELT pipelines, managing data streaming solutions. This role requires a...

  • Lead Data Engineer

    2 days ago

    Noida, Uttar Pradesh, India LUMIQ Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Who we are:LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance (FSI) industry. We are trusted by the world's largest FSIs, including insurers, banks, AMCs, and NBFCs, to address their data challenges. Our clients include 40+ enterprises with over $10B in deposits/AUM, collectively representing about 1B customers...

  • Lead Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Infinium Associates Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Role : We are seeking a talented Lead Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes.Responsibilities : - Data...

  • Lead Data Engineer

    2 days ago

    Noida, Uttar Pradesh, India 3Pillar Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Embark on an exciting journey into the realm of software product development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Lead, Data Engineering, you will hold a...

  • Lead Data Engineer

    4 days ago

    Noida, Uttar Pradesh, India 3Pillar Full time US$ 40,000 - US$ 80,000 per year

    Embark on an exciting journey into the realm of software product development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Lead, Data Engineering, you will hold a...

  • Data Engineer Lead

    14 hours ago

    Noida, Uttar Pradesh, India Vserv Careers Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Profile - Data Engineer LeadExp- 7+ YearsLocation - NoidaWork Mode - HybridSkills - Data Engineering , Team Lead , Snowflake , AWS/Azure , Databricks , DBT , Data Warehousingfor applying you can share your CV on Key ResponsibilitiesData Engineering & Pipeline Development:·      Develop and maintain robust, scalable ETL pipelines to ingest, clean, and...

  • Tech Lead Python

    3 days ago

    Noida, Uttar Pradesh, India Dailoqa Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Role name : Technical lead (Python)Location: NoidaExp : yearsWe're looking at people who :·       Are proactive, curious adaptable, and patient·       Shape the company's vision and will have a direct impact on its success.·       Have the opportunity for fast career growth.·       Have the opportunity to participate in the...

  • Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India EXL SERVICES Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Lead Assistant Manager Job Description - We are seeking a skilled and motivated Data Engineer with hands-on experience in Google Cloud Platform (GCP) and Python, and a strong understanding of the Aetna Healthcare domain. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and solutions that support...