Data Engineer

4 weeks ago

Coimbatore, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

Job Description :


- Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.

- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness

- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.

- Diagnose system performance issue related to data processing and implement solution to address them.

- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.

- Maintain integrity and quality across all pipelines and environments.

- Understand and follow secure coding practice to make sure code is not vulnerable

Skills :

- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.

- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.

- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.

- Proficient in SQl and database Design concepts.

- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.

- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.

- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.

- Knowledge on GitHub and build management.

- Passion for data and experience working within a data driven organization

- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management

(ref:hirist.tech)

  • Wavicle Data Solutions

    2 weeks ago

    Chennai/Coimbatore, India Wavicle data solutions Full time

    Job Description :- We are seeking a highly experienced Senior and Lead Data Engineer with over 6 years of expertise in data engineering.- As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.- Your proficiency in Python, PySpark, AWS/Azure, Databricks/Snowflake, SQL, and leadership skills will be crucial for...

  • Data Engineer

    23 hours ago

    Coimbatore, India Numentica LLC Full time

    This is a remoteposition.As a Salesforce Data Engineer you willbe responsible for designing and implementing data pipelines toextract transform and load (ETL) Salesforce data. You willcollaborate closely with clients to understand their data needs andrequirements and develop customized solutions to normalize cleanseand finetune their Salesforce data. You...

  • Data Engineer

    2 weeks ago

    Coimbatore, India Numentica LLC Full time

    This is a remote position.As a Salesforce Data Engineer , you will be responsible for designing and implementing data pipelines to extract, transform, and load (ETL) Salesforce data. You will collaborate closely with clients to understand their data needs and requirements, and develop customized solutions to normalize, cleanse, and fine-tune their Salesforce...

  • Data Engineer

    5 days ago

    coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview: TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities: Designing andimplementing robust ETL...

  • Data Engineer

    4 weeks ago

    Coimbatore, India CGVAK Software AND Exports Limited Full time

    Overview:TheDatabricks Spark Developer plays a crucial role in harnessing thepower of data using Databricks and Spark to develop and maintainefficient data pipelines. They are responsible for implementingscalable and reliable solutions that enable datadrivendecisionmaking within the organization.KeyResponsibilities:Designing andimplementing robust ETL...

  • Lead Data Engineer

    4 weeks ago

    Coimbatore, India Presidio Full time

    SEIZE THE OPPORTUNITY TO BE A PART OF SOMETHING GREAT! Presidio is on the leading edge of a technology-driven movement to transform the way business is done, for our customers and our customers' customers. Joining Presidio means immersing yourself in a culture of self-starters, collaborators and innovators who make real, lasting change in the marketplace...

  • Azure Data Engineer

    3 weeks ago

    coimbatore, India Numentica LLC Full time

    This is a remoteposition. JobDescription:We are seeking a talentedand experienced Azure Data Engineer to join our team. The idealcandidate will have a strong background in data engineering with afocus on Microsoft Azure services. The candidate should have aminimum of 510 years of handson experience with Azure Data Factory(ADF) Azure Synapse Analytics Synapse...

  • Senior Data Engineer

    1 month ago

    coimbatore, India Presidio Full time

    SEIZE THE OPPORTUNITY TO BE A PART OF SOMETHING GREAT! Presidio is on the leading edge of a technology-driven movement to transform the way business is done, for our customers and our customers’ customers. Joining Presidio means immersing yourself in a culture of self-starters, collaborators and innovators who make real, lasting change in the...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India Salvo Software Full time

    We are looking for an experienced Automotive Aftermarket Scan Tool Data Engineer to bring onto our team. This position would oversee the management and modification of OEM diagnostic data for the engineering team and advise them on best practices. The person who holds this position will also be responsible for understanding how to integrate this data from...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India Salvo Software Full time

    We are looking for an experienced Automotive Aftermarket Scan Tool Data Engineer to bring onto our team. This position would oversee the management and modification of OEM diagnostic data for the engineering team and advise them on best practices. The person who holds this position will also be responsible for understanding how to integrate this data from...

  • Senior Data Engineer

    2 weeks ago

    coimbatore, India Salvo Software Full time

    We are looking for an experienced Automotive Aftermarket Scan Tool Data Engineer to bring onto our team. This position would oversee the management and modification of OEM diagnostic data for the engineering team and advise them on best practices. The person who holds this position will also be responsible for understanding how to integrate this data from...

  • Senior Data Engineer

    4 weeks ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Sr. DATA EngineerLocation : CoimbatorePosition Type : Full-Time / 6 months contract to hireKey Responsibilities :Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Mid-Level Data Engineer to...

  • Senior Data Engineer

    1 month ago

    Coimbatore, India BlueByte Technologies Full time

    Job Title : Sr. DATA EngineerLocation : CoimbatorePosition Type : Full-Time / 6 months contract to hireKey Responsibilities :Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Mid-Level Data Engineer to...

  • Senior Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India BlueByte Technologies Full time

    Job Title : Sr. DATA EngineerLocation : CoimbatorePosition Type : Full-Time / 6 months contract to hireKey Responsibilities :Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Mid-Level Data Engineer to...

  • GCP Data Engineer

    1 month ago

    Coimbatore, India LTIMindtree Full time

    • 5+ years of applicable engineering experience• Experience in GCP, Big Query• Strong proficiency in Python with an emphasis in building data pipelines• Ability to write complex SQL to perform common types of analysis and aggregations• Experience with Apache Airflow or Google Composer• Detail-oriented and document all the work• Ability to work...

  • GCP Data Engineer

    1 month ago

    Coimbatore, India LTIMindtree Full time

    • 5+ years of applicable engineering experience • Experience in GCP, Big Query • Strong proficiency in Python with an emphasis in building data pipelines • Ability to write complex SQL to perform common types of analysis and aggregations • Experience with Apache Airflow or Google Composer • Detail-oriented and document all the work • Ability to...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    2 weeks ago

    Coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    1 month ago

    coimbatore, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...

  • Senior Data Engineer

    3 weeks ago

    Coimbatore, Tamil Nadu, India PureSearch Full time

    Job Description : We are seeking a talented Senior Data Engineer with 3 to 5 years of experience to join the client's team. The ideal candidate will have strong expertise in SQL and ETL processes using tools such as Pentaho, Talend, Azure Data Factory (ADF), Databricks, or AWS Glue. Familiarity with event-driven architecture patterns such as Lambda,...