Spark & Delta Lake

1 day ago

Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 5,00,000 - ₹ 15,00,000 per year

Spark & Delta Lake

Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL and Spark Streaming.

Experience with Spark optimization techniques.

Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning.

Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer.

Proficiency in Python/Scala/Java for Spark development and integrate with ETL process.

Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database)

Understanding of data quality best practices and data validation techniques.

Other Skills:

Understanding of data warehouse concepts, data modelling techniques.

Expertise in Git for code management.

Familiarity with CI/CD pipelines and containerization technologies.

Nice to have experience using data integration tools like DataStage/Prophecy/Informatica/Ab Initio"

  • Senior Data Consultant

    3 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCSTCS is Hiring for Senior Data Consultant Interview Mode: Virtual Required Experience: 10-16 years Work location: PAN INDIAMust-Have Databricks (including Py Spark, Delta Lake, MLflow)Snowflake (data warehousing, performance tuning, data sharing)Data Lake architecture Cloud platforms: Azure, AWS, GCPCI/CD and Data Ops practices Legacy system...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Backbase Full time ₹ 7,00,000 - ₹ 12,00,000 per year

    At Backbase, we're reimagining the future of banking through data and intelligence. Our Engagement Banking Platform enables financial institutions to deliver seamless, personalized customer experiences—and data is at the core of that transformation.As a Senior Data Engineer – Consultant, you'll play a key role in building scalable, secure, and...

  • Databricjks- Hyderabad

    2 weeks ago

    Hyderabad, Telangana, India Diverse Lynx Full time

    Hiring for Databricks- Hyderabad Design and implement ETL ELT pipelines in Databricks using PySpark SQL and Delta Lake Collaborate with data analysts and data scientists to support advanced analytics and machine learning workflows Optimize and troubleshoot Spark jobs for performance scalability and cost efficiency Integrate Databricks with various...

  • Senior Data Consultant

    4 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCSTCS is Hiring for Senior Data ConsultantInterview Mode: VirtualRequired Experience: 10-16 yearsWork location: PAN INDIAMust-Have- Databricks (including PySpark, Delta Lake, MLflow)- Snowflake (data warehousing, performance tuning, data sharing)- Data Lake architecture- Cloud platforms: Azure, AWS, GCP- CI/CD and DataOps practices- Legacy...

  • Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCSTCS is hiring Data Engineer (Databricks)Location – PAN INDIAYears of Experience – 10-16 yearsJob DescriptionRole - Senior Data Engineer – DatabricksRequired Technical Skill SetDatabricks, PySpark, Delta Lake, MLflow, Azure/AWS/GCP, Data Lakehouse architecture, CI/CD, DataOpsDesired CompetenciesMust-Have- Expertise in building scalable...

  • Senior Data Consultant

    3 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCS TCS is Hiring for Senior Data Consultant Interview Mode: Virtual Required Experience: 10-16 years Work location: PAN INDIA Must-Have Databricks (including PySpark, Delta Lake, MLflow) Snowflake (data warehousing, performance tuning, data sharing) Data Lake architecture Cloud platforms: Azure, AWS, GCP CI/CD and DataOps practices Legacy...

  • Senior Data Consultant

    3 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCSTCS is Hiring for Senior Data ConsultantInterview Mode: VirtualRequired Experience: 10-16 yearsWork location: PAN INDIAMust-HaveDatabricks (including PySpark, Delta Lake, MLflow)Snowflake (data warehousing, performance tuning, data sharing)Data Lake architectureCloud platforms: Azure, AWS, GCPCI/CD and DataOps practicesLegacy system...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCS TCS is hiring Data Engineer (Databricks) Location – PAN INDIA Years of Experience – 10-16 years Job Description Role - Senior Data Engineer – Databricks Required Technical Skill Set Databricks, PySpark, Delta Lake, MLflow, Azure/AWS/GCP, Data Lakehouse architecture, CI/CD, DataOps Desired Competencies Must-Have ...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCS TCS is hiring Data Engineer (Databricks) Location – PAN INDIA Years of Experience – 10-16 yearsJob DescriptionRole - Senior Data Engineer – DatabricksRequired Technical Skill SetDatabricks, PySpark, Delta Lake, MLflow, Azure/AWS/GCP, Data Lakehouse architecture, CI/CD, DataOpsDesired Competencies Must-HaveExpertise in building...

  • FactSet - Software Engineer III - Python/Machine Learning

    23 hours ago

    Hyderabad, Telangana, India Factset Research Systems Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Were seeking a passionate and experienced Senior Python & Machine Learning Engineer to join our Data domain team. Youll work on unique, one-of-a-kind problem statements using advanced GenAI, large language models (LLMs), and modern data engineering frameworks. You will help conceptualize and deliver impactful solutions that push the boundaries of data...