ETL Data Engineer

3 weeks ago

New Delhi, India The Techgalore Full time

Pls rate the candidate (from 1 to 5, 1 lowest, 5 highest ) in these areas

  1. Big Data
  2. PySpark
  3. AWS
  4. Redshift

Position Summary

Experienced ETL Developers and Data Engineers to ingest and analyze data from multiple enterprise sources into Adobe Experience Platform

Requirements

  • About 4-6 years of professional technology experience mostly focused on the following:
  • 4+ year of experience on developing data ingestion pipelines using Pyspark(batch and streaming).
  • 4+ years experience on multiple Data engineering related services on AWS, e.g. Glue, Athena, DynamoDb, Kinesis, Kafka, Lambda, Redshift etc.
  • 1+ years of experience of working with Redshift esp the following.

o Experience and knowledge of loading data from various sources, e.g. s3 bucket and on-prem data sources into Redshift.

o Experience of optimizing data ingestion into Redshift.

o Experience of designing, developing and optimizing queries on Redshift using SQL or PySparkSQL

o Experience of designing tables in Redshift(distribution key, compression etc., vacuuming,etc. )

Experience of developing applications that consume the services exposed as ReST APIs. Experience and ability to write and analyze complex and performant SQLs

Special Consideration given for

  • 2 years of Developing and supporting ETL pipelines using enterprise-grade ETL tools like Pentaho, Informatica, Talend
  • Good knowledge on Data Modellin g(design patterns and best practices)
  • Experience with Reporting Technologies (i.e. Tableau, PowerBI)

What youll do

Analyze and understand customers use case and data sources and extract, transform and load data from multitude of customers enterprise sources and ingest into Adobe Experience Platform

Design and build data ingestion pipelines into the platform using PySpark

Ensure ingestion is designed and implemented in a performant manner to support the throughout and latency needed.

Develop and test complex SQLs to extract\analyze and report the data ingested into the Adobe Experience platform.

Ensure the SQLs are implemented in compliance with the best practice to they are performant.

Migrate platform configurations, including the data ingestion pipelines and SQL, across various sandboxes.

Debug any issues reported on data ingestion, SQL or any other functionalities of the platform and resolve the issues.

Support Data Architects in implementing data model in the platform.

Contribute to the innovation charter and develop intellectual property for the organization.

Present on advanced features and complex use case implementations at multiple forums.

Attend regular scrum events or equivalent and provide update on the deliverables.

Work independently across multiple engagements with none or minimum supervision.



  • PySpark & ETL Data Engineer

    2 days ago

    New Delhi, India CirrusLabs Full time

    We are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...

  • PySpark & ETL Data Engineer

    9 hours ago

    New Delhi, India CirrusLabs Full time

    We areCirrusLabs . Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...

  • ETL Data Engineer

    3 weeks ago

    New Delhi, India The Techgalore Full time

    Pls rate the candidate (from 1 to 5, 1 lowest, 5 highest ) in these areas Big Data PySpark AWS Redshift Position Summary Experienced ETL Developers and Data Engineers to ingest and analyze data from multiple enterprise sources into Adobe Experience Platform Requirements About 4-6 years of professional technology experience...

  • ETL QA Engineer

    2 weeks ago

    New Delhi, India Saransh Inc Full time

    We're Hiring: Senior ETL QA EngineerExperience: 6–8 Years ExperienceLocation: Hyderabad / ChennaiJoin: Immediate Joiners PreferredEmployment Type: Full-Time | HybridAre you passionate about ensuring data integrity and accuracy in high-performance systems? Do you excel in ETL testing and SQL validation? We’re looking for an experienced Senior ETL QA...

  • ETL Automation Tester

    1 week ago

    New Delhi, India NTT DATA, Inc. Full time

    Mandatory Skills SQL,ELT/ETL Testing, Python, Data Validation, Pyspark/Pytest/Junit/TestNG and Azure Data services, Databricks QA, Data Lake QAGood to have Skills Devops, Azure cloud Devops, Data governance, API testing

  • Lead - Data Governance & ETL

    1 week ago

    New Delhi, India ICICIDirect Full time

    Position Summary The Lead - Data Governance & ETL is responsible for defining and implementing the organization’s enterprise-wide data governance strategy, ensuring data quality, security, compliance, and consistency across business functions. The role also oversees the design, development, and operations of robust ETL (Extract, Transform, Load) processes...

  • ETL QA Engineer

    6 days ago

    New Delhi, India Saransh Inc Full time

    We're Hiring: Senior ETL QA Engineer Experience:6–8 Years Experience Location : Hyderabad / Chennai Join:Immediate Joiners Preferred Employment Type : Full-Time | HybridAre you passionate about ensuringdata integrity and accuracyin high-performance systems? Do you excel inETL testing and SQL validation ? We’re looking for an experiencedSenior ETL QA...

  • ETL Developer

    1 week ago

    New Delhi, India Tag Full time

    Position Summary:We are seeking a highly skilled ETL Developer with 5–8 years of experience in data integration, transformation, and pipeline optimization. This role is a key part of our Data Engineering function within the Business Intelligence team, responsible for enabling robust data flows that power enterprise dashboards, analytics, and machine...

  • ETL Data Testing/ Data Quality Engineer

    1 week ago

    New Delhi, India Tredence Inc. Full time

    About the company: Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI. Headquartered in San Jose, California, with offices in San Jose, Foster City, Chicago, London, and Toronto, the company embraces a vertical-first approach and an...

  • ETL Data Testing/ Data Quality Engineer

    6 days ago

    New Delhi, India Tredence Inc. Full time

    About the company:Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI. Headquartered in San Jose, California, with offices in San Jose, Foster City, Chicago, London, and Toronto, the company embraces a vertical-first approach and an...