Spark & Delta Lake developer

2 weeks ago


Hyderabad, Telangana, India Hsbc Full time ₹ 15,00,000 - ₹ 28,00,000 per year

We are currently seeking an experienced professional to join our team in the role of Senior Software Engineer.

In this role you will:

  • Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL and Spark Streaming.
  • Experience with Spark optimization techniques.
  • Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning.
  • Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer.
  • Proficiency in Python/Scala/Java for Spark development and integrate with ETL process.
  • Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database)
  • Understanding of data quality best practices and data validation techniques.

Qualifications - External

To be successful in this role, you should meet the following requirements:

  • Understanding of data warehouse concepts, data modelling techniques.
  • Expertise in Git for code management.
  • Familiarity with CI/CD pipelines and containerization technologies.
  • Nice to have: Experience using data integration tools like DataStage/Prophecy/Informatica/Ab Initio.

  • Databricks+spark

    7 days ago


    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. Experience in Sales & Marketing domains is a plus. The role involves developing and optimizing data solutions to...

  • Databricks Engineer

    8 hours ago


    Hyderabad, Telangana, India NTT DATA Business Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Roles and ResponsibilitiesDesign, develop, test, deploy, and maintain large-scale data pipelines using Apache Spark, Delta Lake, ETL processes.Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.Develop complex SQL queries to optimize database performance and troubleshoot issues.Ensure scalability,...

  • Databricks

    2 weeks ago


    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team. The ideal candidate will have extensive experience in Spark in Scala Delta Sharing Databricks Unity Catalog Admin Databricks CLI Delta Live Pipelines Structured Streaming Risk Management Apache Airflow Amazon S3 Amazon Redshift Python Databricks SQL...


  • Hyderabad, Telangana, India Backbase Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    At Backbase, we're reimagining the future of banking through data and intelligence. Our Engagement Banking Platform enables financial institutions to deliver seamless, personalized customer experiences—and data is at the core of that transformation.As a Senior Data Engineer – Consultant, you'll play a key role in building scalable, secure, and...

  • Big Data Engineer

    2 weeks ago


    Hyderabad, Telangana, India Techno Facts Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Experience in developing and delivering scalable big data pipelines using Apache Spark and Databricks on AWS.Position Requirements :Must Have : Build and maintain scalable data pipelines using Databricks and Apache Spark. Develop and optimize ETL/ELT processes for structured and unstructured data. Knowledge of Lakehouse architecture for efficient data...


  • Hyderabad, Telangana, India Bilvantis Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description: DataBricks-BigData Development – Lead Programmer Analyst (Exp: 5 Years to 7 Years)We are looking for a highly skilled and experienced DataBricks-BigData Development skilled as a Lead Programmer Analyst.Experience should have 5 Years to 7 Years into BigData, DataBricks, Python & PySpark development.Experience in data engineering with strong...


  • Hyderabad, Telangana, India Bilvantis Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description: AI-ML Engineering – Senior Programmer Analyst (Exp: 3 Years to 5 Years)We are looking for a highly skilled and experienced AI-ML Engineering skilled as a Senior Programmer Analyst.Technical Expertise:Experience should have 3 Years to 5 Years in AI-ML Engineering.Strong expertise in Python, PySpark, MLflow.Predictive Modelling...

  • AWS Data Engineer

    2 weeks ago


    Hyderabad, Telangana, India Talent Worx Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Senior AWS Data EngineerOverview:We are seeking experienced AWS Data Engineers to design, implement, and maintain robust data pipelines and analytics solutions using AWS services. The ideal candidate will have a strong background in AWS data services, big data technologies, and programming languages.Exp- 3 to 7 years Location- Bangalore, Pune, Hyderabad,...


  • Hyderabad, Telangana, India Cropboy solutions Full time ₹ 1,80,000 - ₹ 18,00,000 per year

    Position OverviewWe are looking for an experienced Data Engineer to design and implement a scalable data platform using Databricks. The ideal candidate will have strong expertise in building modern data architectures, integrating diverse data sources (SQL Server, MongoDB, InfluxDB), and enabling analytics and reporting use cases.Key Responsibilities●...

  • Database Engineer

    2 weeks ago


    Hyderabad, Telangana, India NationsBenefits Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    About NationsBenefits:At NationsBenefits, we are leading the transformation of the insurance industry by developing innovative benefits management solutions. We focus on modernizing complex back-office systems to create scalable, secure, and high-performing platforms that streamline operations for our clients. As part of our strategic growth, we are focused...