Data Engineering Azure databricks

3 days ago

Delhi, India EXL Full time
Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.
Responsibilities:
- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
- Create the data integration and data diagram documentation.
- Lead the data validation, UAT and regression test for new data asset creation.
- Create and maintain data models, including schema design and optimization.
- Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.
Qualifications and Skills:
- Strong knowledge on Python and Pyspark
- Expectation is to have ability to write Pyspark scripts for developing data workflows.
- Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
- Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
- Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
- Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
- Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
- Expectation is to have strong problem-solving and troubleshooting skills.
- Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
- Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
- 4-7 years of experience in Data Engineer.
- Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
- Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
- Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Data Engineering Azure databricks

    2 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:- Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.- Create the data integration and data diagram documentation.-...

  • Data Engineering Azure databricks

    4 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead...

  • Data Engineering Azure databricks

    2 days ago

    Delhi, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead...

  • Azure Databricks Engineer

    3 weeks ago

    Delhi, Delhi, India Techmynd Consulting Full time

    Azure Databricks EngineerLocation : PAN India (Hybrid)Experience : 6-9 YearsMust-Have :- Experience with Azure Databricks , PySpark Notebooks , Spark SQL and Unity Catalog.- Data Modelling experience with strong SQL knowledge.Key Responsibilities :- Develop, maintain, and optimize PySpark notebooks on Azure Databricks for data processing and analytics...

  • Azure Databricks Engineer

    5 days ago

    Delhi, NCR, India Proso Ai Technologies Full time ₹ 8,33,333 - ₹ 25,00,000 per year

    We are looking for an experienced Azure Databricks Engineer for a short-term role (3-4 months).Experience Required: 7+ yearsKey Skills & Responsibilities:Create/modify Azure Data Factory pipelinesDevelop and optimize Databricks notebooks using SQL & PythonWork with file formats: Parquet, Delta, CSVExperience with Azure DevOps & version control (GIT),...

  • Engineering Manager – Big Data, Azure

    2 weeks ago

    Delhi, India DataE2E Technologies Full time

    Job Title: Engineering Manager – Big Data, Azure & DatabricksLocation: RemoteEmployment Type: Full-timeDepartment: Data Engineering / TechnologyExperience: 8+ years (3+ years in a managerial role)Salary - 30 LPAAbout the Role:We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms, Microsoft Azure,...

  • Azure Databricks

    1 week ago

    Delhi, India Tata Consultancy Services Full time

    TCS present an excellent opportunity for Azure DatabricksJob Location: KochiExperience required : 7-10 YrsMust-HaveHands-on experience of developing and operating ETL/ELT pipelines with Azure Data Factory, Azure Databricks (Spark), Azure Data Lake, Synapse, Azure SQL Database.Should have a good understanding of SQL and Data Warehousing...

  • Azure Data Engineer

    2 weeks ago

    Delhi, India RapidBrains Full time

    Position – Senior Azure Data EngineerLocation: PAN IndiaEmployment Type: Full-timeNotice Period - Immediate Joiners OnlyAbout the RoleWe are seeking an experienced Senior Data Engineer to join our IT division and play a pivotal role in supporting the company’s global operations across diverse sectors. The ideal candidate will have strong expertise in...

  • Azure Data Engineer

    1 week ago

    Delhi, India RapidBrains Full time

    Position – Senior Azure Data EngineerLocation: PAN IndiaEmployment Type: Full-timeNotice Period - Immediate Joiners OnlyAbout the RoleWe are seeking an experienced Senior Data Engineer to join our IT division and play a pivotal role in supporting the company’s global operations across diverse sectors. The ideal candidate will have strong expertise in...

  • Lead Azure Data Engineer

    2 weeks ago

    Delhi, India RandomTrees Full time

    Immediate #HIRING for a highly motivated and experienced Lead Azure Data Engineer to join our growing team.We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologiesIf you're passionate about building robust applications and thrive in a dynamic environment,...