Data Engineer

3 weeks ago

Anywhere in IndiaMultiple Locations Excelra Knowledge Solutions Full time

Job Description :


At Excelra ,We are seeking a highly skilled Data Engineer with expertise in Databricks, SQL, Python, and data flow pipelines.

The ideal candidate will have experience in creating Azure Data Factory (ADF) pipelines, deploying code in Azure DevOps using Git Repos and YAML files, and exposure to Snowflake. This role involves designing, implementing, and maintaining robust data solutions to support various business needs.

Key Responsibilities :

Databricks Expertise :

- Develop and maintain data processing workflows using Databricks.

- Optimize Databricks jobs for performance and cost efficiency.

- Implement data transformations and aggregations within the Databricks environment.

- Collaborate with data scientists and analysts to integrate and process large datasets.

SQL Proficiency :

- Write complex SQL queries to retrieve and manipulate data from various databases.

- Ensure data accuracy and consistency through effective querying and data validation techniques.

- Optimize SQL queries for performance and scalability.

Python Programming :

- Develop data pipelines and ETL processes using Python.

- Create reusable code modules and libraries for data processing tasks.

- Perform data cleaning, transformation, and analysis using Python-based tools and frameworks.

Data Flow Pipelines :

- Design and implement data flow pipelines to move and transform data across different systems.

- Ensure data integrity and reliability in data pipelines.

- Monitor and troubleshoot data pipelines to resolve issues promptly.

Azure Data Factory (ADF) Pipelines :

- Create and manage ADF pipelines to orchestrate data movement and transformation activities.

- Develop custom ADF activities using Azure Functions or other services.

- Implement logging, monitoring, and alerting mechanisms for ADF pipelines.

Azure DevOps Deployment Pipelines :

- Set up and maintain deployment pipelines in Azure DevOps.

- Use Git Repos for version control of data flow related code and Databricks notebooks.

- Create and manage YAML files for continuous integration and deployment (CI/CD) processes.

- Ensure seamless deployment of Databricks and other data-related code through Azure DevOps.

Snowflake Exposure :

- Work with Snowflake data warehouse to store and manage large volumes of data.

- Write and optimize SQL queries for Snowflake.

- Integrate Snowflake with other data processing tools and platforms.

Qualifications :

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. A master's degree is a plus.

Experience :

- Proven experience as a Data Engineer or similar role.

- Hands-on experience with Databricks and Spark.

- Strong SQL skills with experience in query optimization.

- Proficiency in Python for data processing and automation.

- Experience in building and managing ADF pipelines.

- Familiarity with Azure DevOps, Git, and YAML for CI/CD processes.

- Exposure to Snowflake and understanding of its architecture and capabilities.

Skills :

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration abilities.

- Ability to work in a fast-paced, dynamic environment.

- Attention to detail and commitment to data quality.

Additional Desirable Skills :

- Experience with other cloud platforms (AWS, GCP).

- Knowledge of data governance and security best practices.

- Familiarity with data visualization tools (e.g., Power BI, Tableau).

(ref:hirist.tech)

  • Data Architect

    6 days ago

    Anywhere In India/multiple Locations Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ Years Location - Remote Skills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS Cloud We are seeking a highly skilled and experienced Data Architect to join our team.The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS...

  • Sigmoid Analytics

    3 weeks ago

    Anywhere in India/Multiple Locations ASTAR DATA LLP Full time

    Job Description : This position will be part of a growing team working towards building world-class large scale Big Data architectures.Responsibilities : Good Development Practices :- Hands-on coder with good experience in programming languages like Python or Scala.- Good Understanding of programming principles and development practices like checkin policy,...

  • Sigmoid Analytics

    2 months ago

    Anywhere in India,Multiple Locations ASTAR DATA LLP Full time

    Job Description : This position will be part of a growing team working towards building world-class large scale Big Data architectures.Responsibilities : Good Development Practices :- Hands-on coder with good experience in programming languages like Python or Scala.- Good Understanding of programming principles and development practices like checkin policy,...

  • Aptus Data Labs

    3 weeks ago

    Anywhere in India/Multiple Locations Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    1 month ago

    Anywhere in India/Multiple Locations Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    1 month ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Aptus Data Labs

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Aptus Data Labs Full time

    Role - Data Architect/Data Engineer Lead- AWSExp - 8+ YearsLocation - RemoteSkills - Data architecture, Data modelling & design, Data Governance, MDM, Python, SQL, ETL, AWS CloudWe are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have 8+ years of experience in data architecture, with a focus on AWS cloud...

  • Sigmoid Analytics

    3 weeks ago

    Anywhere in India/Multiple Locations ASTAR DATA LLP Full time

    This position will be part of a growing team working towards building world-class large scale Big Data Good Development Practices:- Hands-on coder with good experience in programming languages like Python or Scala.- Good Understanding of programming principles and development practices like checkin policy, unit testing, code deployment- Self-starter to be...

  • Sigmoid Analytics

    2 months ago

    Anywhere in India,Multiple Locations ASTAR DATA LLP Full time

    This position will be part of a growing team working towards building world-class large scale Big Data Good Development Practices:- Hands-on coder with good experience in programming languages like Python or Scala.- Good Understanding of programming principles and development practices like checkin policy, unit testing, code deployment- Self-starter to be...

  • MLOps Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • MLOps Engineer

    1 month ago

    Anywhere in India/Multiple Locations GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • MLOps Engineer

    1 month ago

    Anywhere in India/Multiple Locations, IN GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • MLOps Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • Data Engineer

    6 days ago

    Anywhere in India/Multiple Locations BlueByte Technologies Full time

    Position : Data engineerLocation : Remote/ HybridMust have - AKS, Python and SQLGood to have - Erwin toolJob Description :seeking a skilled and motivated AKS (Azure Kubernetes Service) Data Engineer to join our dynamic team (can be open for GKS exp in place of AKS too). The ideal candidate will have a strong background in data engineering, experience with...

  • Data Engineer

    5 days ago

    Anywhere in India/Multiple Locations Arting Digital Full time

    Position Title : Data EngineerExp : 5+ yearsLocation : RemotePrimary Skills : Data warehousing, ETL leveraging Snowflake, ADF ,Python Qualification : AnyRoles and responsibilities : An offshore Data Engineer to work with the Investment Analytics team. This person needs to have a strong exp of Data warehousing and ETL leveraging Snowflake, ADF and Python. A...

  • Data Engineer

    5 days ago

    Anywhere in India/Multiple Locations CliqHR Full time

    Work Mode : Remote (but need to do WFO after joining of 1st week)Preferred candidate profile : Minimum 5+ years of work experience in Data Analysis, BI or Data Engineering roles with a quantitative degree preferably in Computer Science, Statistics or Mathematics. Excellent Data handling and strong analytical skills to collect, transform and discover...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Data Engineer (Integration) Experience Required : 10+yrs Salary : As per the Market Standard or Else whatever budget you have Notice period : Immediate joiner Job Type : Full time Location : Remote Job Description :- Understand all forms of data integration including APIs - Event-based, Streaming, REST and GraphQL API, CDC/Replication,...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Data Engineer (Integration)Experience Required : 10+yrsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :- Understand all forms of data integration including APIs - Event-based, Streaming, REST and GraphQL API, CDC/Replication, Batch,...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Data Engineer (Integration)Experience Required : 10+yrsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :- Understand all forms of data integration including APIs - Event-based, Streaming, REST and GraphQL API, CDC/Replication, Batch,...

  • Data Engineer

    5 days ago

    Anywhere in India/Multiple Locations ARAHAS TECHNOLOGIES PRIVATE LIMITED Full time

    We are currently hiring a Pyspark Data Engineer with AWS expertise with7 years of experience Job Description :The candidate should have strong skills in Pyspark coding and have a background in AWS technologies such as Airflow Experience working with complex SQL queries Good with PySpark, Databricks, and Airflow Bachelor's or Master's degree in computer...