Data Engineering Azure databricks

2 days ago

Jodhpur, India EXL Full time

Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.

Responsibilities:

  • Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
  • Create the data integration and data diagram documentation.
  • Lead the data validation, UAT and regression test for new data asset creation.
  • Create and maintain data models, including schema design and optimization.
  • Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

Qualifications and Skills:

  • Strong knowledge on Python and Pyspark
  • Expectation is to have ability to write Pyspark scripts for developing data workflows.
  • Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
  • Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
  • Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
  • Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
  • Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
  • Expectation is to have strong problem-solving and troubleshooting skills.
  • Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
  • Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
  • 4-7 years of experience in Data Engineer.
  • Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
  • Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
  • Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

  • Azure Data Engineer

    4 weeks ago

    Jodhpur, Rajasthan, India techolution Full time

    We are seeking a skilled Azure Data Engineer with hands-on experience in modern data engineering tools and platforms within the Azure ecosystem. The ideal candidate will have a strong foundation in data integration, transformation, and migration, along with a passion for working on complex data migration projects.Job Title: Sr. Azure Data EngineerLocation:...

  • Senior Data Engineer

    4 weeks ago

    Jodhpur, Rajasthan, India EXL Full time

    About the RoleWe are seeking a highly skilled Data Engineer with deep expertise in Microsoft Azure to design, build, and optimize scalable data pipelines and Lakehouse architectures. You will work closely with cross-functional teams to provide thought leadership deliver robust, secure, and high-performance data solutions that empower analytics and data...

  • Data Engineer

    3 weeks ago

    Jodhpur, Rajasthan, India Infilon Technologies Pvt ltd Full time

    Infilon Technologies is hiring an experienced Data Engineer (Immediate Joiner) for our US-based client, CalcuQuote - www.calcuquote.com. You'll play a key role in building and optimizing data pipelines, ensuring data is clean, secure, and accessible for analytics and operations. This role offers the chance to work closely with talented teams, solve...

  • AI Engineer – Azure AI

    3 weeks ago

    Jodhpur, Rajasthan, India Apidel Technologies Full time

    We are seeking an experienced and highly skilled AI Engineer to design, configure, and optimize enterprise-grade AI solutions using Microsoft Azure AI services. The ideal candidate will have hands-on experience building Retrieval-Augmented Generation (RAG) applications and intelligent agents, integrating services like Azure OpenAI, Cognitive Search, Speech,...

  • Technical Delivery Lead(10+ Years of Experience)

    3 weeks ago

    Jodhpur, Rajasthan, India OptiLnX Full time

    Technical Delivery Lead10+ yearsRemote but on need basis candidate should be ready to go to client location.11am to 8:30 pmBackground Verification will be mandatoryPosition Overview: We are seeking a highly skilled and experienced Technical Delivery Lead to join our team for a Cloud Data Modernization project. The successful candidate willbe responsible...

  • Data Engineer

    2 weeks ago

    Jodhpur, India Canopus Infosystems - A CMMI Level 3 Company Full time

    Position: Data EngineerExperience: 2+ YearsLocation: RemoteJoining: Immediate Joiners PreferredAbout the Role:We are looking for a skilled Data Engineer with strong Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization. The ideal candidate should be capable of building data pipelines, performing...

  • Senior Backend Engineer

    3 weeks ago

    Jodhpur, Rajasthan, India RapidBrains Full time

    Senior Backend EngineerExperience: 8–10 YearsLocation: NoidaContract Type: Long TermWork Time: IST ShiftOverviewWe are seeking a highly experienced Senior Backend Engineer with 8–10 years of expertise in designing and building scalable backend systems. The ideal candidate will be proficient in Node.js (NestJS framework preferred) and TypeScript, with...

  • Senior Data Analyst

    3 weeks ago

    Jodhpur, Rajasthan, India Nityo Infotech Full time

    Job ResponsibilitiesDevelop a strong understanding of internal stakeholders' data and analytical needs, with a focus on customer-facing leadership models.Advocate for the integration of local customer insights into strategic planning and resource allocation decisions.Partner with US Sales Force Operations to drive key business processes and collaborate on...

  • Lead Data Engineer

    3 weeks ago

    Jodhpur, Rajasthan, India The Conqueror Challenges Full time

    ???? About the RoleWe're looking for a relentlessly curious Data Engineer to join our team as a Marketing Platform Specialist.This role is for someone who will dedicate themselves to extracting every possible byte of data from marketing platforms - the kind of person who gets excited about discovering hidden API endpoints and undocumented features..You'll...

  • Engineering Leader in Oil

    1 day ago

    Jodhpur, India Capgemini Engineering Full time

    Engineering Leader – Oil & Gas Field Services (Digital Solutions)Location: Mumbai / Thane / Navi MumbaiExperience: 15+ yearsPosition OverviewAs an Engineering Leader for Oil & Gas Field Services, you will spearhead digital transformation initiatives within the oil field services domain. This role demands strategic thinking, technical expertise, and...