Data Engineering Azure databricks

2 days ago

Delhi, India EXL Full time

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation, UAT and regression test for new data asset creation.Create and maintain data models, including schema design and optimization.Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.Qualifications and Skills:Strong knowledge on Python and PysparkExpectation is to have ability to write Pyspark scripts for developing data workflows.Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and GreenplumExpectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.Expectation is to have strong problem-solving and troubleshooting skills.Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.4-7 years of experience in Data Engineer.Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Data Engineering Azure databricks

    2 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.- Create the data integration and data diagram documentation.-...

  • Data Engineering Azure databricks

    3 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.- Create the data integration and data diagram documentation.-...

  • Data Engineering Azure databricks

    4 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead...

  • Azure Databricks Engineer

    3 weeks ago

    Delhi, Delhi, India Techmynd Consulting Full time

    Azure Databricks EngineerLocation : PAN India (Hybrid)Experience : 6-9 YearsMust-Have :- Experience with Azure Databricks , PySpark Notebooks , Spark SQL and Unity Catalog.- Data Modelling experience with strong SQL knowledge.Key Responsibilities :- Develop, maintain, and optimize PySpark notebooks on Azure Databricks for data processing and analytics...

  • Azure Databricks Engineer

    5 days ago

    Delhi, NCR, India Proso Ai Technologies Full time ₹ 8,33,333 - ₹ 25,00,000 per year

    We are looking for an experienced Azure Databricks Engineer for a short-term role (3-4 months).Experience Required: 7+ yearsKey Skills & Responsibilities:Create/modify Azure Data Factory pipelinesDevelop and optimize Databricks notebooks using SQL & PythonWork with file formats: Parquet, Delta, CSVExperience with Azure DevOps & version control (GIT),...

  • Engineering Manager – Big Data, Azure

    2 weeks ago

    Delhi, India DataE2E Technologies Full time

    Job Title: Engineering Manager – Big Data, Azure & DatabricksLocation: RemoteEmployment Type: Full-timeDepartment: Data Engineering / TechnologyExperience: 8+ years (3+ years in a managerial role)Salary - 30 LPAAbout the Role:We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms, Microsoft Azure,...

  • Azure Databricks

    1 week ago

    Delhi, India Tata Consultancy Services Full time

    TCS present an excellent opportunity for Azure DatabricksJob Location: KochiExperience required : 7-10 YrsMust-HaveHands-on experience of developing and operating ETL/ELT pipelines with Azure Data Factory, Azure Databricks (Spark), Azure Data Lake, Synapse, Azure SQL Database.Should have a good understanding of SQL and Data Warehousing...

  • Azure Data Engineer

    2 weeks ago

    Delhi, India RapidBrains Full time

    Position – Senior Azure Data EngineerLocation: PAN IndiaEmployment Type: Full-timeNotice Period - Immediate Joiners OnlyAbout the RoleWe are seeking an experienced Senior Data Engineer to join our IT division and play a pivotal role in supporting the company’s global operations across diverse sectors. The ideal candidate will have strong expertise in...

  • Azure Data Engineer

    1 week ago

    Delhi, India RapidBrains Full time

    Position – Senior Azure Data EngineerLocation: PAN IndiaEmployment Type: Full-timeNotice Period - Immediate Joiners OnlyAbout the RoleWe are seeking an experienced Senior Data Engineer to join our IT division and play a pivotal role in supporting the company’s global operations across diverse sectors. The ideal candidate will have strong expertise in...

  • Lead Azure Data Engineer

    2 weeks ago

    Delhi, India RandomTrees Full time

    Immediate #HIRING for a highly motivated and experienced Lead Azure Data Engineer to join our growing team.We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologiesIf you're passionate about building robust applications and thrive in a dynamic environment,...