Lead Data Engineer

21 hours ago

Noida, Uttar Pradesh, India CGvak Software Exports Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Title: Data Engineer (Scala, Spark, Pyspark, Databricks)

Remark - final round will be the F2F

Experience: 10+ Years (Lead Role)

Responsibilities:

  • Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.
  • Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency.
  • Optimize Spark applications for performance and cost-efficiency within the Databricks environment.
  • Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning.
  • Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions.
  • Implement data governance and security best practices within Databricks.
  • Troubleshoot and resolve data-related issues, ensuring data availability and reliability.
  • Stay updated with the latest advancements in Spark, Scala, Databricks, and related big data technologies.

Required Skills and Experience:

  • Proven experience as a Data Engineer with a strong focus on big data technologies.
  • Expertise in Scala programming language for data processing and Spark application development.
  • In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark Core.
  • Proficiency in using Databricks platform features, including notebooks, jobs, workflows, and Unity Catalog.
  • Experience with Delta Lake and its capabilities for building data lakes.
  • Strong understanding of data warehousing concepts, data modeling, and relational databases.
  • Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
  • Experience with version control systems like Git.
  • Excellent problem-solving and analytical skills.
  • Ability to work independently and as part of a team.

Preferred Qualifications (Optional):

  • Experience with other big data technologies like Kafka, Flink, or Hadoop ecosystem components.
  • Knowledge of data visualization tools.
  • Understanding of DevOps principles and CI/CD pipelines for data engineering.
  • Relevant certifications in Spark or Databrick

  • Lead Data Engineer

    5 days ago

    Noida, Uttar Pradesh, India 3Pillar Full time US$ 40,000 - US$ 80,000 per year

    Embark on an exciting journey into the realm of software product development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Lead, Data Engineering, you will hold a...

  • Lead Data Engineer

    3 days ago

    Noida, Uttar Pradesh, India 3Pillar Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Embark on an exciting journey into the realm of software product development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Lead, Data Engineering, you will hold a...

  • Lead Data Engineer

    3 days ago

    Noida, Uttar Pradesh, India LUMIQ Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Who we are:LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance (FSI) industry. We are trusted by the world's largest FSIs, including insurers, banks, AMCs, and NBFCs, to address their data challenges. Our clients include 40+ enterprises with over $10B in deposits/AUM, collectively representing about 1B customers...

  • Lead Data Engineer

    1 week ago

    Noida, Uttar Pradesh, India Uplers Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Lead Data Engineer - Azure CloudExperience: Years Exp.Salary: CompetitivePreferred Notice Period: 30 DaysShift: 10:00 AM to 7:00 PM ISTOpportunity Type: Noida (Remote for 6 months, Later Hybrid)Placement Type: Permanent(*Note: This is a requirement for one of Uplers' Clients)Must have skills required :Engineering management, Data Engineering, Azure Data...

  • Data Engineer Lead

    23 hours ago

    Noida, Uttar Pradesh, India Vserv Careers Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Profile - Data Engineer LeadExp- 7+ YearsLocation - NoidaWork Mode - HybridSkills - Data Engineering , Team Lead , Snowflake , AWS/Azure , Databricks , DBT , Data Warehousingfor applying you can share your CV on Key ResponsibilitiesData Engineering & Pipeline Development:·      Develop and maintain robust, scalable ETL pipelines to ingest, clean, and...

  • Lead Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Infinium Associates Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Role : We are seeking a talented Lead Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes.Responsibilities : - Data...

  • Lead Data Engineer

    1 week ago

    Noida, Uttar Pradesh, India Conduent Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job OverviewWe are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition,...

  • Data Lead

    1 week ago

    Noida, Uttar Pradesh, India ShyftLabs Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position Overview: We're looking for a hands-on Data Lead to architect and deliver end-to-end data features that power our booking and content management systems. You'll lead a team, own the data pipeline lifecycle, and play a critical role in shaping and scaling our data infrastructure — spanning batch, real-time, and NoSQL environments. ShyftLabs is a...

  • Lead Python Data Engineer

    3 days ago

    Noida, Uttar Pradesh, India Algoworks Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Lead Python Data Engineer - Databricks (Unity Catalog, Delta, CI/CD) Job SummaryWe are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL, and...

  • Systems Engineering Specialist Advisor

    1 day ago

    Noida, Uttar Pradesh, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    SCCM Administration: Lead the configuration, deployment, and maintenance of SCCM infrastructure. Manage software distribution, patch management, OS deployment, and compliance settings. Monitor and troubleshoot SCCM client health and performance. Design and implement mobile device management (MDM) and mobile application management (MAM) strategies using...