Lead Data Engineer

2 days ago

Pune, Maharashtra, India CG-VAK Software & Exports Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Title:
Data Engineer (Scala, Spark, Pyspark, Databricks)
Remark - final round will be the F2F

Experience: 10+ Years (Lead Role)
Responsibilities

  • Design, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.
  • Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data quality, integrity, and efficiency.
  • Optimize Spark applications for performance and cost-efficiency within the Databricks environment.
  • Work with Delta Lake for building reliable data lakes and data warehouses, ensuring ACID transactions and data versioning.
  • Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions.
  • Implement data governance and security best practices within Databricks.
  • Troubleshoot and resolve data-related issues, ensuring data availability and reliability.
  • Stay updated with the latest advancements in Spark, Scala, Databricks, and related big data technologies.

Required Skills And Experience

  • Proven experience as a Data Engineer with a strong focus on big data technologies.
  • Expertise in Scala programming language for data processing and Spark application development.
  • In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark Core.
  • Proficiency in using Databricks platform features, including notebooks, jobs, workflows, and Unity Catalog.
  • Experience with Delta Lake and its capabilities for building data lakes.
  • Strong understanding of data warehousing concepts, data modeling, and relational databases.
  • Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
  • Experience with version control systems like Git.
  • Excellent problem-solving and analytical skills.
  • Ability to work independently and as part of a team.

Preferred Qualifications (Optional)

  • Experience with other big data technologies like Kafka, Flink, or Hadoop ecosystem components.
  • Knowledge of data visualization tools.
  • Understanding of DevOps principles and CI/CD pipelines for data engineering.
  • Relevant certifications in Spark or Databrick

Skills: scala,big data technologies,data,apache spark,spark,big data

  • Tech Lead Data Engineer

    4 days ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    About Data Axle:Data Axle Inc.  has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now as an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...

  • Data Engineer

    5 days ago

    Pune, Maharashtra, India Jash Data Sciences Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...

  • Lead Data Engineer

    2 weeks ago

    Pune, Maharashtra, India EXL Talent Acquisition Team Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Lead Data Engineer Job DescriptionRole Overview:We are looking for a skilled Lead Data Engineer to design, build, and manage scalable data pipelines and architectures. You will lead a team of data engineers, collaborate with cross-functional stakeholders, and drive best practices in data engineering to support data-driven decision-making.Key...

  • Lead Data Engineer

    1 week ago

    Pune, Maharashtra, India EXL Service Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description: Lead Data Engineer Job DescriptionRole Overview:We are looking for a skilled Lead Data Engineer to design, build, and manage scalable data pipelines and architectures. You will lead a team of data engineers, collaborate with cross-functional stakeholders, and drive best practices in data engineering to support data-driven decision-making.Key...

  • Lead Data Engineer

    3 days ago

    Pune, Maharashtra, India Principal Global Services Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    ResponsibilitiesWhat You'll do:Seeking a Lead Data Engineer with demonstrated ability guiding a team, developing platforms, and aligning strategy with goals.You'll have Opportunit to: - Lead the technical direction and strategy for data engineering projects   - Lead a team of data engineers, providing technical mentorship and guidance   - Establish and...

  • Lead Software Engineer

    1 week ago

    Pune, Maharashtra, India Data Direct Networks Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    OverviewThis is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to...

  • Lead Software Engineer

    2 weeks ago

    Pune, Maharashtra, India Data Direct Networks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    OverviewThis is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to...

  • Lead Software Engineer

    1 week ago

    Pune, Maharashtra, India Data Direct Networks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    OverviewThis is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to...

  • Lead Data Engineer

    5 days ago

    Pune, Maharashtra, India Primate Consulting Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Company DescriptionPrimate Consulting is a prominent IT service provider specializing in Mobile and Web Application Development, Full-Stack Development, and Custom Product Development. The company also offers Consulting Services, Staff Augmentation, and Sales & Marketing solutions. Primate Consulting delivers cost-effective services across industries,...

  • Lead Software Engineer

    1 day ago

    Pune, Maharashtra, India Data Direct Networks Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    OverviewThis is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to...