ETL Data Engineer

4 weeks ago


Bengaluru, India Codersbrain technology pvt ltd Full time

Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.

- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness

- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.

- Diagnose system performance issue related to data processing and implement solution to address them.

- Collaborate with other teams to ensure successful integration of data pipelines into larger system architecture requirement.

- Maintain integrity and quality across all pipelines and environments.

- Understand and follow secure coding practice to make sure code is not vulnerable

Skills :

- Should have Strong background in Software development with experience in ingest, transform and store data from large datasets using Pyspark in Azure Databricks with strong knowledge on distributed computing concepts.

- Must have hands on experience in designing and developing ETL Pipelines in Pyspark in Azure Databricks with strong python scripting exposure like list comprehensions, Dictionary variables etc.

- Must have minimum 5 years of exposure and good proficiency in data warehousing concepts.

- Proficient in SQl and database Design concepts.

- Hands on experience with good proficiency in Delta table and delta file operations like merge, Insert override, Partition overrides etc.

- Hands on Experience in CICD in Azure DevOps/Harness and ADF/Stone branch for orchestration.

- Knowledge of Azure cloud computing platform with Azure Synapse, ADLS.

- Knowledge on GitHub and build management.

- Passion for data and experience working within a data driven organization

- Excellent presentation, communication (oral & written), and relationship building skills, across all levels of management

(ref:hirist.tech)
  • Data Engineer

    1 month ago


    Bengaluru, India Mobile Programming LLC Full time

    Job Description :Position : Data EngineerExperience : 6+ yearsWork Mode : Work from OfficeLocation : BangaloreExperience in Nifi or Tibco is mandatory.Mandatory Skills : ETL, DevOps platform, Nifi or TibcoPlease note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data...

  • Azure Data Engineer

    4 weeks ago


    Bengaluru, India Connectio IT Pvt Ltd Full time

    Role Description :This is a full-time Azure Data Engineer role at Connect io IT Pvt Ltd located in Bengaluru. As an Azure Data Engineer, you will be responsible for tasks such as data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. You will work on-site in Bengaluru.Qualifications :- Data Engineering, Data...

  • Data Engineer

    3 weeks ago


    Bengaluru, India Sterlite Technologies Limited Full time

    Role as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...

  • Data Engineer

    4 weeks ago


    Bengaluru, India CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

    Company Description : CST Cyber Sapient is a company dedicated to helping clients achieve advocacy through exceptional product and service experiences. We work closely with our clients to drive change within their organization and provide support in areas such as strategic vision, product innovation, user experience, talent support, and technical delivery....

  • Data Engineer

    4 days ago


    Bengaluru, India Axtria - Ingenious Insights Full time

    Job Title: Data Engineer / ETL DeveloperJob Description:We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/DatabricksResponsibilities:Design, develop, and implement ETL processes using Azure Data Factory (ADF), Matillion, Snowflake, Or...

  • Data Engineer

    5 days ago


    Bengaluru, India Axtria - Ingenious Insights Full time

    Job Title: Data Engineer / ETL Developer Job Description:We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/DatabricksResponsibilities:Design, develop, and implement ETL processes using Azure Data Factory (ADF), Matillion, Snowflake, Or...

  • Data Engineer

    5 days ago


    Bengaluru, India Axtria - Ingenious Insights Full time

    Job Title: Data Engineer / ETL Developer Job Description:We are seeking an experienced ETL Developer / Data Engineer to join our team. The ideal candidate will have a strong background in Azure Data Factory (ADF), SQL, Pyspark/DatabricksResponsibilities:Design, develop, and implement ETL processes using Azure Data Factory (ADF), Matillion, Snowflake, Or...

  • Data Engineer

    2 weeks ago


    Bengaluru, India Careerzgraph Full time

    Total experience required : 15-20 years.Notice Period - Not More then 15 Days / Only Immediate Joiners For Location - Bengaluru (KA)/Electronic City Phase 1. - Work from Office Job Description :- Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines.- Develop advanced scripts using Python for...

  • Data Engineer

    1 month ago


    Bengaluru, India Excellence and Eminence LLP Full time

    About the Role:We are seeking a talented and motivated Data Engineer to join our team and play a crucial role in building and maintaining our data infrastructure. This role is ideal for a candidate with a strong foundation in data engineering principles and experience working with data pipelines and Design, develop, and implement data pipelines for...

  • Senior Data Engineer

    1 month ago


    Bengaluru, India LOGIC PLANET IT SERVICES Full time

    About the job :Project Role : Data EngineerMust have skills : Google BigQueryMinimum 5 year(s) of experience is requiredAs a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and...

  • Etl Testing

    1 month ago


    Bengaluru, India NTT DATA Full time

    **Description**: "To develop & execute automation test cases both manually and using Bamboo. Analyzing business requirement in order to drive testing efficiently Participate in regression testing and verification/ validation activities. Exposure to understand the requirement and to prepare test data." Primary Skill Set Manula Testing/ ETL Testing...


  • Bengaluru, India PhysicsWallah Full time

    Working Experience :- Minimum 1 year of work experience.- Startup experience preferred, Streaming Platform, Edtech work experience bonus. Roles & Responsibilities :- Develop ETL (Extract, Transform, Load) processes to move and transform data between systems.- Integrate data from various sources, ensuring consistency and accuracy.- Monitor system performance...

  • Senior Data Engineer

    4 weeks ago


    Bengaluru, India Amicon Hub Full time

    Data Engineer role.Location : BangaloreNeed only Immediate Joiners and candidates from Product based/Start up Companies.Responsibilities :- Analyze and organize raw data - Build data systems and pipelines- Evaluate business needs and objectives- Interpret trends and patterns- Conduct complex data analysis and report on results - Prepare data for prescriptive...

  • ETL Test Engineer

    1 month ago


    Bengaluru, India Corporate Resources Full time

    Job DescriptionDeveloping ETL test cases and test plans that ensure data quality, accuracy, and completeness.Conducting functional and non-functional testing of ETL processes to validate the integrity of the data being transferred.Identifying and documenting defects, issues, and potential improvements in the ETL process and sharing them with the development...

  • Etl, Data Migration

    4 weeks ago


    Bengaluru, India Zensar Technologies Full time

    Data/ETL Tester: - 3+ years of Years of experience in DB/ ETL testing, Good knowledge in ETL process, Expertise in writing complex SQL queries. - Experience creating test plans and test cases for assigned modules - Good knowledge in any ETL tools (SAP BODS/Informatica / DataStage / Talend etc.) - Good knowledge in GenRocket tool or any other Test data...


  • Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Data Engineer

    4 weeks ago


    Bengaluru, India CareerNet Consulting Full time

    WHAT YOU'LL DO : - Help build robust and scalable data pipelines and ETL services for processing data from internal and external Data sources.- Help automating business processes requiring data manipulation, aggregation, and sharing- Work in collaboration with other members of the group located in California.- Optimize and monitor proactively existing...

  • Data Engineer

    3 days ago


    Bengaluru, India TD Newton Full time

    As Data Engineer, you will :- Support in designing and rolling out the data architecture and infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources- Identify data source, design and implement data schema/models and integrate data that meet the requirements of the business stakeholders- Play an...


  • Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...


  • Bengaluru, India Techno Wise Full time

    Responsibilities :1. Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and PySpark.2. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement data solutions that align with business needs.3. Optimize and tune existing data pipelines for performance and...