Databricks Senior Data Engineer

3 weeks ago

Thiruvananthapuram, India Luxoft Full time

Project description

We are hiring for projects in the Middle East. The team comprises front-end and back-end developers, data analysts and data scientists, architects, analysts, and project managers. You will work with modern technologies and be an integral part of continuing to evolve our high-performing platforms. If you are serious about being a part of this fast-paced and exciting journey, this project should be top of the list

This role will be primarily responsible for closely collaborating with a business SME to implement data validation rules.


Responsibilities

Collaborating with a business SME to implement data validation rules.

Understand business requirements and actively provide input from a data perspective.

Understand the underlying data and flow of data.

Understand overall platform architecture and capabilities and how to load data effectively.

Recognize and adapt to the changes in processes.


Skills

Must have

7 years + of relevant experience.

Proficient hands-on experience with Azure Databricks (3y+).

Proficiency in Python and PySpark (5y+).

Advanced knowledge in SQL (5y+).

Should be able to analyse and understand complex data.

Knowledge of Azure data/delta lake, Azure DevOps, CI/CD, and Git is essential.

Should clearly understand the data platform lifecycle and contribute to preparing Design documents, Unit Test plans, and Code review reports.

Knowledge of data Lake house and business intelligence architecture.

Experience working in an Agile environment (Scrum, Agile) is a plus.

Interpersonal/communication skills (both oral/written) with the ability to communicate at various levels with clarity & precision.

Bachelor's degree in IT/Computer Science or related field


Nice to have

Knowledge of the financial markets, portfolio theory, and risk management is a plus


Other

Languages

English: C1 Advanced


Seniority

Senior

  • Senior Data Engineer

    4 weeks ago

    Thiruvananthapuram, Kerala, India Techvantage Analytics Full time

    Job DescriptionAt Techvantage Analytics, we are seeking a highly skilled Senior Data Engineer to join our team. The ideal candidate will have expertise in designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server.The Senior Data Engineer will be responsible for collaborating with various stakeholders,...

  • Senior data engineer

    1 month ago

    Thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced...

  • Senior Data Engineer

    1 month ago

    Thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced...

  • Senior Data Engineer

    3 weeks ago

    thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced analytics...

  • Senior Data Engineer

    1 month ago

    thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced analytics...

  • Senior Data Engineer

    1 month ago

    Thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced analytics...

  • Senior Data Engineer

    1 month ago

    Thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced analytics...

  • Senior Data Engineer

    1 month ago

    Thiruvananthapuram, India NeoITO Full time

    We are seeking an experienced Data Engineer with 3+ years of hands-on experience in designing, building, and maintaining scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced...

  • Senior Data Architect

    1 month ago

    Thiruvananthapuram, Kerala, India NeoITO Full time

    We are seeking an experienced Data Engineer to design, build, and maintain scalable data infrastructure. The ideal candidate will have deep expertise in the Python stack and be skilled in building robust data pipelines, working with large datasets, and optimizing data architectures to support advanced analytics and business needs.Experience with Databricks...

  • Technical Lead

    3 weeks ago

    Thiruvananthapuram, Kerala, India Mig Staffing Full time

    Data Engineering ManagerAs a seasoned Data Engineering Manager at Mig Staffing, you will lead the design, implementation, and maintenance of data solutions on the Microsoft Azure Data platform and SQL Server. Your expertise in designing scalable and reliable data pipelines will drive the efficient processing, storage, and retrieval of large volumes of...

  • Data Engineering Lead for Edge Computing Infrastructure

    5 hours ago

    Thiruvananthapuram, Kerala, India Armada Systems Full time

    Armada Systems is an innovative edge computing startup that bridges the digital divide with advanced technology infrastructure.About the OpportunityWe are seeking a highly motivated Data Engineering Lead to join our Data Platform team and help shape the future of data ingestion, processing, and analysis for our Edge Computing AI Platform.Job...

  • Technical Lead

    3 months ago

    Thiruvananthapuram, India Mig Staffing Full time

    Please apply if you are interested in C2HWe are looking for a qualified Data Engineer to join our team in TrivandrumRoles and responsibilities :- Lead Data Engineer will be responsible for designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server (SSIS, SSAS, UC4 Atomic)- Leading a team of data engineers,...

  • Techvantage Analytics

    1 month ago

    Trivandrum/Thiruvananthapuram, India Techvantage Analytics Full time

    What roles and responsibilities will be performed by the selected candidate ? Senior Data Engineer will be responsible for designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server (SSIS, SSAS, UC4 Atomic). Collaborate with various stakeholders, and ensuring the efficient processing, storage, and retrieval of...

  • Data Engineer

    5 days ago

    Thiruvananthapuram, India Alp Consulting Ltd. Full time

    Job Summary: As a Senior DBT/Azure Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data transformation pipelines within the Azure cloud environment or GCP Cloud environment using DBT (Data Build Tool). Your extensive experience in Azure services and data warehousing will enable you to deliver robust and scalable data...

  • Data Engineer

    5 days ago

    Thiruvananthapuram, India Alp Consulting Ltd. Full time

    Job Summary: As a Senior DBT/Azure Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data transformation pipelines within the Azure cloud environment or GCP Cloud environment using DBT (Data Build Tool). Your extensive experience in Azure services and data warehousing will enable you to deliver robust and scalable data...

  • Data Engineer

    5 days ago

    Thiruvananthapuram, India Alp Consulting Ltd. Full time

    Job Summary: As a Senior DBT/Azure Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data transformation pipelines within the Azure cloud environment or GCP Cloud environment using DBT (Data Build Tool). Your extensive experience in Azure services and data warehousing will enable you to deliver robust and scalable data...

  • Data Engineer

    4 days ago

    Thiruvananthapuram, India Alp Consulting Ltd. Full time

    Job Summary: As a Senior DBT/Azure Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data transformation pipelines within the Azure cloud environment or GCP Cloud environment using DBT (Data Build Tool). Your extensive experience in Azure services and data warehousing will enable you to deliver robust and scalable data...

  • Data Engineer

    1 month ago

    Trivandrum/Thiruvananthapuram, India Ospyn Technologies Pvt Ltd Full time

    Data EngineerJob Summary:We are seeking a highly skilled Data Engineer to join our team at Ospyn Technologies Pvt Ltd. The successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines to support data-driven initiatives.Key Responsibilities:Design and develop data pipelines using ETL/ELT processes and tools such...

  • Cloud Infrastructure Engineer

    2 weeks ago

    thiruvananthapuram, India Qubiqon Full time

    Ready to build a rock-solid cloud foundation for the future of data and AI? At Qubiqon Consulting, we need aCloud Infrastructure Engineer with 3-6 years of experiencewho’s got the chops to create, maintain, and secure an Azure environment that’s as dependable as a favorite pair of sneakers.This role is all about keeping the infrastructure running like a...

  • Senior Data Lead

    4 weeks ago

    Thiruvananthapuram, Kerala, India Scipy Technologies Full time

    We are seeking a Senior Data Lead to spearhead our data analytics team at Scipy Technologies. As a key member of our team, you will leverage your expertise in data science to drive business growth through data-driven insights. Your role will involve leading the development of predictive models and machine learning algorithms, designing and implementing data...