
Spark & Delta Lake
3 days ago
Spark & Delta Lake
Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL and Spark Streaming.
Experience with Spark optimization techniques.
Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning.
Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer.
Proficiency in Python/Scala/Java for Spark development and integrate with ETL process.
Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database)
Understanding of data quality best practices and data validation techniques.
Other Skills:
Understanding of data warehouse concepts, data modelling techniques.
Expertise in Git for code management.
Familiarity with CI/CD pipelines and containerization technologies.
Nice to have experience using data integration tools like DataStage/Prophecy/Informatica/Ab Initio"
-
Databricks+spark
2 weeks ago
Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJob SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. Experience in Sales & Marketing domains is a plus. The role involves developing and optimizing data solutions to...
-
Senior Data Engineer
1 day ago
Hyderabad, Telangana, India Careernet Full time ₹ 15,00,000 - ₹ 20,00,000 per yearKey Skills: Spark, Azure Databricks, Apache Spark, Python, Delta Lake, Kafka, Snowflake, Qlik, CI/CD, AzureRoles & Responsibilities:Design, develop, and maintain scalable data pipelines using Azure Databricks and Spark.Implement data processing solutions with Delta Lakes and manage multiple file formats.Collaborate with cross-functional teams to ensure data...
-
Databricks Engineer
7 days ago
Hyderabad, Telangana, India NTT DATA Business Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRoles and ResponsibilitiesDesign, develop, test, deploy, and maintain large-scale data pipelines using Apache Spark, Delta Lake, ETL processes.Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.Develop complex SQL queries to optimize database performance and troubleshoot issues.Ensure scalability,...
-
Senior Data Engineer
2 weeks ago
Hyderabad, Telangana, India Backbase Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAt Backbase, we're reimagining the future of banking through data and intelligence. Our Engagement Banking Platform enables financial institutions to deliver seamless, personalized customer experiences—and data is at the core of that transformation.As a Senior Data Engineer – Consultant, you'll play a key role in building scalable, secure, and...
-
Databricks Developer
1 day ago
Hyderabad, Telangana, India Zetamicron Full time ₹ 8,00,000 - ₹ 24,00,000 per yearJob Title:Databricks DeveloperLocation:Bangalore, Chennai, Hyderabad, PuneExperience:5–7 YearsAbout the RoleWe are seeking an experiencedDatabricks Developerwith 5–7 years of expertise in Big Data engineering and analytics platforms. The ideal candidate will have strong hands-on experience inApache Spark, Databricks, PySpark, and cloud platforms such...
-
AWS Data Engineer
7 days ago
Hyderabad, Telangana, India Talent Worx Full time ₹ 8,00,000 - ₹ 24,00,000 per yearSenior AWS Data EngineerOverview:We are seeking experienced AWS Data Engineers to design, implement, and maintain robust data pipelines and analytics solutions using AWS services. The ideal candidate will have a strong background in AWS data services, big data technologies, and programming languages.Exp- 3 to 7 years Location- Bangalore, Pune, Hyderabad,...
-
Data Engineer
1 day ago
Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 20,00,000 per yearGreetings from TCSTCS is hiring Data Engineer (Databricks)Location – PAN INDIAYears of Experience – 10-16 yearsJob DescriptionRole- Senior Data Engineer – DatabricksRequired Technical Skill SetDatabricks, PySpark, Delta Lake, MLflow, Azure/AWS/GCP, Data Lakehouse architecture, CI/CD, DataOpsDesired CompetenciesMust-HaveExpertise in building scalable...
-
Databricks Engineer
1 day ago
Hyderabad, Telangana, India Oblytech Full time US$ 90,000 - US$ 1,20,000 per yearJob Title:Databricks Engineer (PySpark Developer)Employment Type:Full-Time | Permanent | Remote (Work From Home)Industry:IT Services & Consulting | Software Development | Data Engineering | Cloud ServicesFunctional Area:Data Engineering | Big Data | Cloud Platforms | AnalyticsAbout Oblytech:Oblytech is a fast-growing IT consulting and software services firm,...
-
Hyderabad, Telangana, India Factset Research Systems Full time ₹ 20,00,000 - ₹ 25,00,000 per yearWere seeking a passionate and experienced Senior Python & Machine Learning Engineer to join our Data domain team. Youll work on unique, one-of-a-kind problem statements using advanced GenAI, large language models (LLMs), and modern data engineering frameworks. You will help conceptualize and deliver impactful solutions that push the boundaries of data...
-
Big Data Engineer
1 day ago
Hyderabad, Telangana, India Grid Dynamics Full time ₹ 20,00,000 - ₹ 25,00,000 per yearNotice Period : Immediate to 30 daysMandatoryScala and PythonApache Spark (batch & streaming) mustDeep knowledge of HDFS internals and migration strategies.Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.Running Spark and/or Flink jobs on Kubernetes (e.g.,...