Data Engineering Azure databricks

2 days ago

bangalore, India EXL Full time

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.

Responsibilities:

  • Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
  • Create the data integration and data diagram documentation.
  • Lead the data validation, UAT and regression test for new data asset creation.
  • Create and maintain data models, including schema design and optimization.
  • Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

Qualifications and Skills:

  • Strong knowledge on Python and Pyspark
  • Expectation is to have ability to write Pyspark scripts for developing data workflows.
  • Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
  • Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
  • Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
  • Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
  • Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
  • Expectation is to have strong problem-solving and troubleshooting skills.
  • Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
  • Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
  • 4-7 years of experience in Data Engineer.
  • Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
  • Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
  • Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Data engineer | azure + databricks

    7 days ago

    Bangalore, India Xebia Full time

    ???? Job Opportunity – Data Engineer | Azure + Databricks ???? Location: Bangalore (Whitefield) | Hybrid (3 days office/week) Experience: 6+ years Notice Period: Immediate to 2 weeks – only apply if you can join early About the Role: We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and data...

  • Data Engineer | Azure + Databricks

    3 days ago

    Bangalore, India Xebia Full time

    Job Opportunity – Data Engineer | Azure + Databricks Bangalore (Whitefield) | Hybrid (3 days office/week) Experience: 6+ years Immediate to 2 weeks – only apply if you can join early We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and data science use cases . The ideal candidate will have...

  • Data Engineer | Azure + Databricks

    1 week ago

    Bangalore, India Xebia Full time

    Job Opportunity – Data Engineer | Azure + Databricks Location: Bangalore (Whitefield) | Hybrid (3 days office/week) Experience: 6+ years Notice Period: Immediate to 2 weeks – only apply if you can join early About the Role: We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and...

  • Data Engineering Azure databricks

    13 hours ago

    bangalore, India EXL Full time

    Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Responsibilities: Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes. Create the data integration and data diagram...

  • Data Engineer | Azure + Databricks

    1 week ago

    bangalore, India Xebia Full time

    🔹 Job Opportunity – Data Engineer | Azure + Databricks 🔹Location: Bangalore (Whitefield) | Hybrid (3 days office/week)Experience: 6+ yearsNotice Period: Immediate to 2 weeks – only apply if you can join earlyAbout the Role:We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and data science use cases....

  • Azure Databricks Engineer

    2 days ago

    Bangalore Rural, Bengaluru, India BLJ Tech Geeks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description: Azure Databricks EngineerPosition: Azure Databricks EngineerLocation: [Specify Location / Remote / Hybrid]Experience Required: 4 10 years (depending on role level)Role OverviewWe are seeking an experienced Azure Databricks Engineer to design, develop, and optimize data pipelines and analytics solutions on Azure. The ideal candidate will have...

  • Azure Data Engineer

    2 weeks ago

    bangalore, India iO Associates Full time

    Azure Data Engineer - Databricks ExpertREMOTEFull-timeShift Timings: 2 pm - 11 pm IST Our client is a leading AI-driven tech company transforming global enterprises with innovative solutions in data intelligence, automation, and responsible AI. With a mission to help enterprises modernize and scale, they focus on delivering intelligent, future-ready...

  • Data Engineer | Azure + Databricks

    6 days ago

    bangalore district, India Xebia Full time

    🔹 Job Opportunity – Data Engineer | Azure + Databricks 🔹 Location: Bangalore (Whitefield) | Hybrid (3 days office/week) Experience: 6+ years Notice Period: Immediate to 2 weeks – only apply if you can join early About the Role: We are looking for a skilled Data Engineer to join our team, working on end-to-end data engineering and data...

  • Azure databricks, PySpark

    1 week ago

    bangalore, India Tata Consultancy Services Full time

    Role: Azure Data Engineer(ADF, ADB)Required Technical Skill Set: Azure Data Engineer, ADF, Azure Databricks Spark (PySpark or Scala), Python, PL/SQLExp : 4 to 6 YearLocation : Kolkata, Pune, Mumbai, Bangalore, BhubaneshwarMust-Have · Strong experience in Azure Data Factory , ADB( Azure Databricks) Synapse; establishing the cloud connectivity between...

  • Associate architect

    1 week ago

    Bangalore, India Quantiphi Full time

    Job Title : Associate Architect - Data Engineering Experience : 6 to 9 Years Location : Bangalore / Mumbai (any) (Hybrid WFO) About the Role: We are looking for a skilled Data Engineers with hands-on experience in the Microsoft Azure data ecosystem. The ideal candidate will play a critical role in designing, developing, and managing data pipelines...