Current jobs related to Databricks Engineer - Trivandrum - RapidBrains

  • High Salary: Data Engineer

    4 weeks ago

    Thiruvananthapuram / Trivandrum, India TalentBasket Full time

    Job Description We are seeking a skilled Data Engineer with expertise in PySpark, Databricks, and SQL to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate will have experience in handling large datasets, optimizing performance, and working with cloud-based data platforms. Key Responsibilities: - Develop and...

Databricks Engineer

4 weeks ago

Trivandrum, India RapidBrains Full time
Databricks Engineer
Experience : 4+ Years
Location : Trivandrum (Hybrid)
Contract Duration : 6+ Months
Work Time : IST Shift
Job Description
We are seeking a skilled Data Engineer to design and develop data warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, and Power BI. The role involves building and optimizing data pipelines, working with large datasets, and implementing automation using DevOps/CI/CD frameworks. The ideal candidate should have expertise in Azure, AWS, Terraform, ETL, Python, and data lifecycle management while collaborating on architecture frameworks and best practices.
Roles and Responsibilities
Design and develop data warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services.
Develop and optimize SQL queries and work with SSIS for ETL processes.
Work in an onshore-offshore model, handling challenging scenarios effectively.
Build and optimize data pipelines for ETL workloads and extract insights using Data Science and Analytics.
Work with large datasets (structured and unstructured) and implement data transformation processes.
Utilize DevOps/CI/CD frameworks for automation, leveraging Infrastructure as Code (IaC) with Terraform and configuration management tools.
Participate in architecture framework discussions, best practices, and data warehouse design.
Implement and maintain Azure Data Factory pipelines for ETL projects.
Ensure effective data ingestion, transformation, loading, validation, and performance tuning.
Work independently with minimal supervision and communicate effectively across teams.
Skills & Requirements
Expertise in Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services .
Strong experience in SQL, SSIS, and Query Optimization .
Hands-on experience with ETL pipelines, Data Warehousing, and Analytics .
Proficiency in Azure, AWS, and DevOps/CI/CD frameworks .
Experience with Terraform, Jenkins, and Infrastructure as Code (IaC) .
Strong knowledge of Python for data processing and transformation.
Experience handling data ingestion, transformation, validation, and performance tuning .
Ability to work in an ambiguous environment and translate vague requirements into concrete deliverables.