Data Engineering Azure databricks

19 hours ago

New Delhi, India EXL Full time

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes. - Create the data integration and data diagram documentation. - Lead the data validation, UAT and regression test for new data asset creation. - Create and maintain data models, including schema design and optimization. - Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.Qualifications and Skills:- Strong knowledge on Python and Pyspark - Expectation is to have ability to write Pyspark scripts for developing data workflows. - Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum - Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. - Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. - Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. - Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. - Expectation is to have strong problem-solving and troubleshooting skills. - Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. - Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. - 4-7 years of experience in Data Engineer. - Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. - Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks. - Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Senior Azure Databricks Engineer

    19 hours ago

    New Delhi, India MLAI Digital Private Ltd. Full time

    Company DescriptionMLAI_Digital is a GenAI_Agentic Specialised Organisation(8 Years old) providing Various Cognitive Products well suited for major BFSI Institutions.Offices in US(ohio),Singapore(PTE), Malaysia , Indiaand Australia(MEL)Domain expertise:- FINANCIAL INSTITUTIONS(Banks & Insurance) and Fraud Analytics for Financial TransactionsRole...

  • Data Engineering Azure databricks

    7 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.- Create the data integration and data diagram documentation.-...

  • Associate Architect – Data

    18 hours ago

    New Delhi, India Quantiphi Full time

    Job Title : Associate Architect - Data EngineeringExperience : 6 to 9 YearsLocation : Bangalore / Mumbai (any) (Hybrid WFO)About the Role:We are looking for a skilled Data Engineers with hands-on experience in the Microsoft Azuredata ecosystem. The ideal candidate will play a critical role in designing, developing, andmanaging data pipelines and...

  • Azure Databricks Engineer

    3 days ago

    Delhi, India Tata Consultancy Services Full time

    Role** Databricks Engineer (Individual Contributor)Required Technical Skill Set** Azure Databricks, PysparkExperience: 4 to 10 YearsLocation: Hyd,Kolkata, Noida, Chennai, BangaloreDesired Competencies (Technical/Behavioral Competency)Must-Have**1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem-solving skillGood-to-HaveAzure DevopsAzure Data...

  • Data Engineering Azure databricks

    1 week ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.- Create the data integration and data diagram documentation.-...

  • Data Engineering Azure databricks

    1 week ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead...

  • Data Engineering Azure databricks

    1 week ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead...

  • Azure Data Engineer

    19 hours ago

    New Delhi, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy Services Job Openings at TCS SkillAzure Data Engineer (ADF / Databrick) & ETL Developer Exp range 8yrs -12yrs location : Chennai / Bangalore Pls find the Job Description below. Azure Data Engineers: Experience working with Azure cloud services, such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics...

  • Azure Databricks Engineer

    3 days ago

    Delhi, India Tata Consultancy Services Full time

    Role : Azure Databricks EngineerRequired Technical Skill Set : Azure Databricks,PysparkExperience : 4 to 10 yearsLocation : Hyderabad, Kolkata, Noida, Chennai & BengaluruJob Description :Desired Competencies (Technical/Behavioral Competency)Must-Have :-1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem solving skill.Good-to-Have :-Azure...

  • Azure Databricks Engineer

    2 days ago

    Delhi, India Tata Consultancy Services Full time

    Role : Azure Databricks EngineerRequired Technical Skill Set : Azure Databricks,PysparkExperience : 4 to 10 yearsLocation : Hyderabad, Kolkata, Noida, Chennai & BengaluruJob Description :Desired Competencies (Technical/Behavioral Competency)Must-Have :-1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem solving skill.Good-to-Have :-Azure...