Data Engineering Azure databricks

3 weeks ago

Jodhpur, India EXL Full time

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.

Responsibilities:

  • Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
  • Create the data integration and data diagram documentation.
  • Lead the data validation, UAT and regression test for new data asset creation.
  • Create and maintain data models, including schema design and optimization.
  • Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

Qualifications and Skills:

  • Strong knowledge on Python and Pyspark
  • Expectation is to have ability to write Pyspark scripts for developing data workflows.
  • Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
  • Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
  • Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
  • Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
  • Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
  • Expectation is to have strong problem-solving and troubleshooting skills.
  • Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
  • Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
  • 4-7 years of experience in Data Engineer.
  • Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
  • Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
  • Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Databricks Engineer

    3 weeks ago

    Jodhpur, India RapidBrains Full time

    Job Title: Databricks EngineerExperience: 7.5+ YearsLocation: Remote (IST Shift)Contract Type: Long-TermOverviewWe are seeking an experienced Databricks Engineer with 7.5+ years of expertise in database development, ETL, and cloud-based data solutions. The ideal candidate should be highly skilled in SQL, Databricks, PySpark, and cloud technologies, with...

  • IT Data Integration Engineer

    7 days ago

    Jodhpur, India Classified Jobs Full time

    Key Responsibilities:Design and develop integration solutions using Azure services such as Azure Data Factory, Azure Logic Apps, Azure Functions for building scalable integration workflowsDevelop and manage APIs through Azure API Management to facilitate seamless integration between various systems.Implement best practices for Enterprise Application...

  • Senior Data Engineer

    3 weeks ago

    Jodhpur, India CareerXperts Consulting Full time

    We are looking for a highly skilled Senior Data Engineer to design, build, and optimize scalable data pipelines and infrastructure. The ideal candidate will have strong expertise in modern data engineering tools and frameworks, along with the ability to collaborate closely with data scientists, analysts, and product teams to deliver reliable and efficient...

  • Data modeler

    2 weeks ago

    Jodhpur, India Manuh Technologies Full time

    5+ years of professional experience in data engineering, data science engineering, or related fields.Strong proficiency in Python programming (Pandas, Num Py, Py Spark, or similar).Hands-on experience with Dask for large-scale distributed data processing.Proven expertise as a Data Modeler (conceptual, logical, and physical modeling).Solid understanding of...

  • Senior DevOps Engineer

    2 weeks ago

    Jodhpur, India thinkbridge Full time

    We are seeking a highly skilled and experienced Senior DevOps Engineer (Azure) to join our team. In this role, you will be responsible for designing, implementing, and managing our infrastructure and deployment processes. You will collaborate closely with software development and operations teams to automate workflows, streamline processes, and ensure high...

  • .net azure architect

    4 days ago

    Jodhpur, India Nous Infosystems Full time

    NET/CLOUD—ARCHITECTABOUT NOUSNous Infosystems is a CMMi Level 5 SVC + SSD v1.3, ISO 9001:2008, ISO/IEC 27001:2013 certified global information technology firm providing software solutions and services across a broad spectrum of industries and domains. Nous Infosystems has been delivering quality technology outsourcing solutions to customers across varied...

  • Engineering Leader in Oil

    3 weeks ago

    Jodhpur, India Capgemini Engineering Full time

    Engineering Leader – Oil & Gas Field Services (Digital Solutions)Location: Mumbai / Thane / Navi MumbaiExperience: 15+ yearsPosition OverviewAs an Engineering Leader for Oil & Gas Field Services, you will spearhead digital transformation initiatives within the oil field services domain. This role demands strategic thinking, technical expertise, and...

  • Senior Data Engineer

    7 days ago

    Jodhpur, India NOQ Full time

    Senior Data & Reporting Developer (Remote | India | Contract Role)Company: NOQ GroupLocation: RemoteContract Duration: 6 months (possibility of extension based on performance)Start Date: Immediate⸻About NOQ GroupNOQ Group is a technology-driven organization dedicated to delivering innovative data solutions for businesses in the food and beverage industry...

  • Software engineer in test

    2 days ago

    Jodhpur, India Bright Matrix Global Full time

    Must have Backbase platform testing (mandatory) on Resume Job Description:We are seeking a Senior QE / SDET Engineer with strong experience in Backbase platform testing, Selenium automation, and API validation for enterprise-level banking applications. The ideal candidate will have a solid understanding of the banking domain, hands-on experience with C#...

  • 3 Days Left: Senior Data

    6 days ago

    Jodhpur, India Kooner Transport Group - KTG Full time

    Location: Remote/ Uttarakhand - Ramnagar Office Type: 3-Month Contract (extendable based on performance) Work Hours: Must overlap 3–4 hours/day with Eastern Time (Canada) Compensation: ₹100,000 – ₹150,000 INR per month (depending on skill & experience) About the Role Kooner Transport Group (Canada) operates a large cross-border trucking fleet with...