ETL Data Engineer

4 weeks ago


Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.

- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness

- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.

- Diagnose system performance issue related to data processing and implement solution to address them.

- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.

- Maintain integrity and quality across all pipelines and environments.

- Understand and follow secure coding practice to make sure code is not vulnerable

Skills :

- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.

- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.

- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.

- Proficient in SQl and database Design concepts.

- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.

- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.

- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.

- Knowledge on GitHub and build management.

- Passion for data and experience working within a data driven organization

- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management

(ref:hirist.tech)
  • Data Engineer

    3 weeks ago


    Bangalore/Hyderabad, Karnataka, India Indium Software Full time

    Job Description:We are seeking a talented and motivated Data Engineer to join our team. The ideal candidate should have a strong background in SQL, Python, ETL, and data visualization, with experience working closely with stakeholders and providing data support to Data Analysts. As a Data Engineer, you will play a key role in developing data pipelines,...

  • Data Engineer

    1 month ago


    Bangalore, Karnataka, India D Square Consulting Service Full time

    Role : Sr. Data Engineer DOMOExperience : 5+ YearsLocation : Bangalore- Candidates who can join within 30 days, may only applyWorking Mode : Work From OfficeJob Highlights :Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital marketing projects.- Strong knowledge of digital marketing toolsPreferred...

  • Data Engineer

    1 week ago


    Bangalore, Karnataka, India WIZSTAFFING PRIVATE LIMITED Full time

    Job Description :We at Captain Fresh, are building smart supply chain to deliver the highest quality seafood and meat for the Indian consumer.Our innovations in process management and workforce orchestration along with strong industry credentials are enabling us to deliver the fastest harvest-to-fork in the industry. Our endeavor is to leverage experience...


  • Bangalore, Karnataka, India Innover Digital Inc. Full time

    Job Description :- The primary purpose of the Senior Extract Transform Load (ETL) Developer is to help develop and deploy data warehouse BI solutions, data analytics and data integrations that support business.- Design, develop, and maintain SSIS packages to load data in a variety of formats and from a variety of source systems using built-in and third-party...

  • Azure Synapse Analyst

    1 month ago


    Bangalore/Hyderabad, Karnataka, India Swift Strategic Staff Solutions INC Full time

    Notice : Immediate Joiner.Skill : Azure Data Factory, Azure cloud, Data Lake, Azure Synapse and ETL.Job Description:Roles and Responsibilities :- The person in this position will work in a team environment to support applications using Microsoft.- Data Access Technologies.- The role of a Support Engineer within the DS group focuses on delivering high quality...


  • Bangalore, Karnataka, India Pluralsight Full time

    Job Description : Working at Pluralsight Founded in 2004 and trusted by Fortune 500 companies, Pluralsight is the technology skills platform organizations and individuals in 150+ countries count on to create progress for the world. Our platform helps technologists master their craft and take control of their careers. We empower businesses everywhere to build...


  • Bangalore, Karnataka, India BOOTLABS TECHNOLOGIES PRIVATE LIMITED Full time

    Responsibility :- At least 4 years of experience in AWS technologiesThe documented ability of delivering innovative solutions in an applied environment related to at least one of the following areas :- ETL, Data pipeline, Workflow- Strong knowledge of programming in Python, Java, or Scala.- High level of competence in global and cross-skilled collaborative...


  • Bangalore, Karnataka, India BOOTLABS TECHNOLOGIES PRIVATE LIMITED Full time

    Job Description :- At least 3 years of experience in AWS technologies- The documented ability of delivering innovative solutions in an applied environment related to at least one of the following areas : ETL, Data pipeline, Workflow- Strong knowledge of programming in Python, Java, or Scala.- High level of competence in global and cross-skilled collaborative...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Varite India Full time

    Duties :- Experience working in Azure/Databricks.- Expert in Python to create application and ETL to extract data from SharePoint, Salesforce, SQL, etc.- Experience working with APIs and integrating data from various sources into Python-based applications.- Experience in SQL along with writing complex queries.- Exposure to C# and Azure function along with...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Recruise Full time

    Job Description :- AWS technologies such as S3, Redshift, Lambda, Athena,Glue, etc, Python, SQL- Bioinformatics or Life science experience Mandate- Programming experience with SQL and Python.- Strong Hands-on experience in built/deploy of end-to-end project.- Experience in ETL processing in AWS cloud, experience in works on orchestration tools preferably...

  • SAS Consultant

    1 month ago


    Bangalore, Karnataka, India Talent Corner HR Full time

    We are hiring for one of our client based from Ahmedabad.Position Name : SAS Developer. (Data Integration, Visual Analytics, Enterprise Guide). Job Responsibilities/description:- Experience with data management and analysis using SAS. SAS DI Studio, SAS 9. 1/9. 2/9. 3, SAS Enterprise Guide.- Exposure to ETL process with experience of data structures, design,...

  • Talend Lead

    3 weeks ago


    Bangalore, Karnataka, India Selsoft Full time

    Job role: TALEND LEADJob Location: BangaloreProject Timing: 11 AM - 8 PM Job Description:- Having 8 -12 Years experience in Talend Complex SQL:- Must have strong experience on ETL - Talend and Complex SQL - Should have strong working experience on Oracle, MySQL and NoSQL databases. Example: RDBMS-Oracle/MySql, NoSql-Couchbase/MongoDB- Good to have the cloud...

  • Python Developer

    1 month ago


    Bangalore, Karnataka, India IT Full time

    Job Overview :We are looking for a seasoned Python Developer with over 10 years of experience in IT, with a focus on data engineering. The ideal candidate will have extensive knowledge of Python, Pyspark, Azure Databricks, ADF, SQL, and data warehouse concepts. This role requires strong technical skills, project management capabilities, and excellent...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India GLAXOSMITHKLINE ASIA PVT. LTD Full time

    A new world-leading consumer health company. Shaped by all who join us. Together, we're improving everyday health for billions of people. By growing and innovating our global portfolio of category-leading brands including Sensodyne, Panadol, Advil, Voltaren, Theraflu, Otrivin, and Centrum through a unique combination of deep human understanding and...

  • Juego Studios

    3 weeks ago


    Bangalore, Karnataka, India Juego Studios Full time

    As a Python Data Engineer for Test at Client you will be at the forefront of ensuring the quality and reliability of our data systems. You will play a critical role in analyzing raw data, building test frameworks, and validating data products using Python. Collaborating closely with data analytics experts and stakeholders, you will contribute to the...


  • Bangalore/Hyderabad, Karnataka, India FINDERNEST SOFTWARE SERVICES Full time

    About the job :As a Microsoft Power BI developer, you will play a crucial role in transforming data into actionable insights by designing and developing Power BI reports and dashboards. This role is integral to the organization's data-driven decision-making processes and will involve collaborating with various teams to understand the business needs and...


  • Bangalore, Karnataka, India Digihelic Solutions Pvt. Ltd. Full time

    Experience- 4+ years. Must Have: - Over 4 years of relevant experience in SAP Analytics Cloud (SAC).- Good experience in creating both analytic and planning data modelling (Data Modelling).- Story building experience with Smart Insights and Smart Discovery.- Familiarity with Data Sphere management.- ETL (Extraction, Transfer, Loading) knowledge is an added...

  • Engineering Manager

    1 month ago


    Bangalore, Karnataka, India Freelancer Recruiter Full time

    Mandatory requirements :- Apache Kafka- Elastic search- Spark Spring- NoSql- Swift- Distributed systems and Java (preferred)- Only from product organisationsResponsibilities : - Manage software development projects by setting requirements, goals and timelines. - Create clear career paths for team members and help them grow with regular & deep mentoring....

  • Principal Engineer

    1 month ago


    Bangalore, Karnataka, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

    What Success Will Look Like :- Design, build and maintain the stream processing, time-series analysis system which is at the heart of Data Platform's products- Responsible for architecture of the platform- Build features, enhancements, new services and bug fixing in Scala and Rust on a Jenkins based pipeline to be deployed as Docker containers on...

  • Big Data Engineer

    1 month ago


    Hyderabad/Bangalore, Karnataka, India HRwing Full time

    Role & Responsibilities:Good development practices:Hands on coder with good experience in programming languages like Java or Python.Hands-on experience on the Big Data stack like PySpark, Hbase, Hadoop, Mapreduce and ElasticSearch.Good understanding of programming principles and development practices like checkin policy, unit testing, code deploymentSelf...