ETL Lead

3 weeks ago

Hyderabad, India Milestone Technologies, Inc. Full time

Position Overview:

We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for extensive data processing, analytics, and business intelligence functions. Demonstrating profound knowledge in cloud-native tools and platforms including AWS, Azure, Oracle, Snowflake, and Databricks, the Senior Data Engineer will also excel in deploying advanced data integration techniques utilizing cloud-native ETL tools (e.g., AWS Glue, Azure Data Factory) and Oracle Data Integrator (ODI).

Key Responsibilities:

  • Lead the architecture and development of scalable data solutions across both batch and streaming platforms, optimizing the use of cloud technologies (AWS or similar) to meet business and technical demands, including real-time data processing.
  • Analyze complex data elements and systems, engaging in the development of conceptual, logical, and physical data models, along with the validation and execution of ETL/ELT mappings and transformation logic.
  • Craft and peer review data models and schemas, enhancing new and current data sources for the data warehouse, ensuring efficiency and scalability.
  • Conduct thorough unit, integration, and system testing of data sources to ensure data integrity against source systems, aiming for continuous performance optimization.
  • Innovate with Snowflake’s Snowpark and Snowpipe to architect scalable data management solutions, facilitating real-time data ingestion.
  • Leverage Apache Spark for comprehensive data processing and analytics, enhancing data workflow efficiency and scalability.
  • Implement Databricks Delta Live Tables to automate and ensure data ETL processes, maintaining data quality and consistency for real-time analytics.
  • Maintain high-performance data pipelines, making extensive use of AWS, Azure, and Databricks, along with seamless integration with Oracle Data Integrator (ODI).
  • Utilize cloud native ETL tools to streamline data integration and transformation across diverse data platforms.
  • Develop sophisticated data transformations and machine learning models within Snowflake using Snowpark to boost analytics capabilities.
  • Ensure the availability of real-time data and analytics through efficient configuration and optimization of Snowpipe.
  • Collaborate with business stakeholders and IT departments to transform business needs into technical specifications and practical data models.
  • Uphold data governance, security, and compliance standards across all data platforms and tools.
  • Remain abreast of emerging trends and technologies in data engineering, promoting innovative solutions that propel the data strategy.

Qualifications:

  • Bachelor’s or master’s degree in computer science, Information Technology, Engineering, or a related field.
  • Proven expertise in data processing and analytics using Apache Spark, with hands-on experience in Databricks Delta Live Tables for managing real-time data.
  • Proficiency in Snowflake, including mastery of Snowpark and Snowpipe, paired with experience in cloud native ETL tools (AWS Glue, Azure Data Factory, Google Cloud Dataflow).
  • Comprehensive background in Oracle, Snowflake, Databricks, and Oracle Data Integrator (ODI) for in-depth data integration and processing.
  • Strong programming skills in SQL, Python, Java, or Scala, with the capability to develop intricate data models and algorithms.
  • In-depth knowledge of data warehousing, data modeling, and data lake concepts within a cloud environment.
  • Exceptional problem-solving skills, complemented by the ability to work both independently and collaboratively.
  • Outstanding communication skills, with proficiency in engaging with both technical and non-technical team members.
  • Experience in managing and leading team

  • ETL Lead

    2 weeks ago

    hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview:We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    2 weeks ago

    hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview: We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    2 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview: We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • Etl lead

    3 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview:We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    3 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview: We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    3 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview:We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    3 weeks ago

    hyderabad, India Milestone Technologies, Inc. Full time

    Position Overview:We are looking for Lead ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for...

  • ETL Lead

    2 months ago

    Noida/Hyderabad, India Qualihires Private Limited Full time

    Salary INR 20 - 30 - The Integration Lead is responsible for designing and developing the integration Architecture using ETL. - This includes creating data model, mappings, workflows, deriving ingestion framework and publication mechanism etc.Must have tool experience : - Informatica CDI, Informatica CDQ, Strong SQL Knowledge- Good to have tool...

  • ETL Lead

    3 weeks ago

    Hyderabad, India Flipped.ai Full time

    ETL Lead-Contract(6 Months)Awign Expert is an Enterprise focused platform that helps businesses Hire, Assess and Manage highly skilled resources for Gig Based Projects. We provide our Experts a gateway to work for and build a freelance/consulting career with large scale Enterprises. We are a newly launched business division of Awign, which is one of the...

  • Senior ETL Tester

    4 weeks ago

    Hyderabad, India ILink Digital Full time

    About the RoleILink Digital is seeking an experienced Senior ETL Tester to lead the creation of comprehensive test plans and strategies for ETL processes. The ideal candidate will have a strong background in data quality assurance, data pipelines, and test automation.Key Responsibilities:Develop automated test scripts using Python to validate data...

  • ETL Testing Specialist

    4 weeks ago

    Hyderabad, Telangana, India SPG Consulting Full time

    Data Services and Cloud-Based Applications ExpertiseWe are seeking a skilled ETL testing specialist to join our team at SPG Consulting.As an ETL testing specialist, you will be responsible for testing data services and cloud-based applications, with a focus on ETL applications and tools such as Python, Java, TestNG, TestRail, Jenkins, and Amazon S3. You will...

  • ETL Lead

    2 months ago

    Hyderabad, India Evnek Technologies Pvt Ltd Full time

    About the RoleDuration:  6 MonthLocation:  HyderabadMode - Work From Office (5 Days) Time - India Time and U.S overlap.Timings: Full Time (As per company timings)Notice Period: within 15 days or immediate joinerExperience: 10+ Years  Job Description:Proven expert at writing SQL code with at least 10  years of experience. Must have 5+ years of...

  • Senior ETL Developer

    4 weeks ago

    Hyderabad, Telangana, India ZenithCad Full time

    About the RoleZenithCad is seeking a talented Module Lead to join our team in Hyderabad on a contract-to-hire basis. This mid-level position requires a strong background in ETL, Data Modeling, and Performance Tuning.Key ResponsibilitiesDesign, develop, and implement ETL processes to extract, transform, and load data.Create and maintain comprehensive data...

  • Senior ETL Tester

    2 months ago

    Hyderabad, India ILink Digital Full time

    Requirement :- Should have 5 Plus years of experience in Data/BI/ETL testing.- Candidate should be able work from client office in Hyderabad on need basis.- Extensive experience in Database/ETL data validations (DQ Rules and transformations).-Experience in writing strong complex SQL queries.- Extensive knowledge in Data warehousing concepts (e.g,FACT, DIM...

  • Azure Data Lead

    4 weeks ago

    Hyderabad, India CGT SOLUTIONS (IND) PRIVATE LIMITED Full time

    Role: Azure Data LeadExperience : 8+ yearsLocation : Hyderabad (on site)Position : PermanentClient : Acuvate.Job Responsibilities :- To collaborate with various teams/regions in driving facilitating data design, identifying architectural risks and key areas of improvement in data landscape, and developing and refining data models and architecture frameworks-...

  • Informatica IICS Lead

    1 day ago

    Hyderabad, Telangana, India Virtusa Full time

    Job OverviewVirtusa is seeking an experienced ETL developer to join our team as an Informatica IICS Lead. This role involves developing and implementing data integration solutions using Informatica cloud, IICS, and Power Center.About the RoleThis position requires a strong understanding of RDBMS, DWH-BI concepts, and experience in developing application...

  • India - Designer - Etl Iv

    6 months ago

    Hyderabad, India VARITE INDIA PRIVATE LIMITED Full time

    Description: - 9 + years minimum of Tableau experience in developing, maintaining, and managing Tableau-driven dashboards & analytics and working knowledge of Tableau administration/architecture. - Working knowledge of Tableau desktop and creating interactive dashboards - Hands-on experience in fine-tuning the reports and understanding the business needs to...

  • Senior ETL Developer

    2 weeks ago

    hyderabad, India Milestone Technologies, Inc. Full time

    We are looking for a Senior ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for extensive data...

  • Senior ETL Developer

    3 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    We are looking for a Senior ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for extensive data...

  • Senior etl developer

    3 weeks ago

    Hyderabad, India Milestone Technologies, Inc. Full time

    We are looking for a Senior ETL Developer. This pivotal role is dedicated to harnessing state-of-the-art data processing technologies such as Snowpark, Snow pipe, Apache Spark, and Databricks Delta Live Tables. The successful candidate will play a key role in evolving, deploying, and overseeing our advanced data infrastructure, pivotal for extensive data...