Lead Data Pipeline Architect

5 days ago


India NeenOpal Inc. Full time

NeenOpal Inc. is seeking a skilled Senior Data Engineer to build and maintain robust data pipelines and integrations from 3rd party data sources.

The ideal candidate will have experience in designing database import/export processes, generating client data warehouse deliverables, and working with various data integration tools.

We offer a competitive salary range of $140,000 - $200,000 per annum, depending on experience, as well as opportunities for career growth and professional development.

Requirements:

  • 2+ years experience as an ETL developer with strong data architecture knowledge around data warehousing concepts, SQL development and optimization, and operational support models.
  • Experience using Python to automate ETL/Data Processes jobs.
  • Design and develop ETL and data processing solutions using data integration tools, python scripts, and cloud environments.
  • Experience / Willingness to learn AWS Glue / AWS Data Pipeline / Azure Data Factory for Data Integration.
  • Develop and create transformation queries, views, and stored procedures for ETL processes, and process automation.
  • Document data mappings, data dictionaries, processes, programs, and solutions as per established standards for data governance.
  • Work with the data analytics team to assess and troubleshoot potential data quality issues at key intake points such as validating control totals at intake and then upon transformation, and transparently build lessons learned into future data quality assessments
  • Solid experience with data modeling, business logic, and RESTful APIs.
  • Solid experience in the Linux environment.
  • Experience with NoSQL / PostgreSQL preferred
  • Experience working with databases such as MySQL, NoSQL, and Postgres, and enterprise-level connectivity experience (such as connecting over TLS and through proxies).
  • Experience with NGINX and SSL.
  • Performance tune data processes and SQL queries, and recommend and implement data process optimization and query tuning techniques.


  • India FLYONIT Full time

    Job DescriptionFlyonit is seeking an experienced AWS Data Pipeline Architect to lead the design and implementation of secure, scalable, and efficient data pipelines on AWS. As a key member of our team, you will be responsible for creating reliable and maintainable data integration solutions using services like Glue, Lambda, Step Functions, and...


  • India Wavicle Data Solutions Full time

    Wavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...

  • Lead data engineer

    3 weeks ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...

  • Lead Data Engineer

    4 weeks ago


    India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...

  • Lead Data Engineer

    4 weeks ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...


  • India TIU Consulting Full time

    TIU Consulting is seeking a highly skilled Data Engineer to join our team in Pune.We offer an estimated salary of ₹45,00,000 - ₹75,00,000 per annum, based on experience and qualifications.Job DescriptionAs a Data Engineer at TIU Consulting, you will be part of a cross-functional Scrum team that develops and supports our products. You will work closely...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...


  • india Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...


  • India Aidetic Full time

    We are seeking an experienced Lead Data Architect to join our team at Aidetic. The successful candidate will have a proven track record of designing and implementing scalable data engineering solutions using Databricks or Snowflake.This role will focus on leading a team of data engineers who are building, maintaining, and optimizing large-scale data...

  • Data Architect Lead

    3 days ago


    India Quantiphi Full time

    At Quantiphi, we are seeking an exceptional Data Architect Lead to spearhead our data engineering efforts. As a key member of our team, you will be responsible for leading the design and implementation of cloud-native solutions using cutting-edge technologies.About the Role:We offer a competitive salary range of $150,000 - $200,000 per annum, commensurate...


  • India MacroHire Full time

    Exciting Cloud Data Architect Lead Opportunity at MacroHireWe are seeking an experienced Cloud Data Architect Lead to join our team at MacroHire. This is a unique opportunity for a skilled professional to lead the design and implementation of Enterprise Data platforms using Snowflake.About the Role:As a Cloud Technical Lead, you will be responsible for...


  • india Crayon Data Full time

    Role:AI ArchitectExp: 10 to 15 yrsLocation: Chennai(Hybrid)In non-cloud Large Language Model (LLM) deployments, the AI Architect serves as a strategic leader, driving the alignment of AI solutions with business objectives while addressing critical challenges like scalability, governance, and security. Their role ensures sustainable and impactful AI adoption...


  • india Crayon Data Full time

    Role:AI Architect Exp: 10 to 15 yrs Location: Chennai(Hybrid) In non-cloud Large Language Model (LLM) deployments, the AI Architect serves as a strategic leader, driving the alignment of AI solutions with business objectives while addressing critical challenges like scalability, governance, and security. Their role ensures sustainable and impactful AI...


  • India Tekskills Inc. Full time

    Job TitleSr. Data Architect - AI/ML ExpertAbout UsTekskills Inc., a leading provider of data-driven solutions, is seeking an experienced Sr. Data Architect to lead our team in designing and implementing cutting-edge data pipelines that drive business growth.Job SummaryThe successful candidate will be responsible for architecting and implementing complex data...

  • Lead Data Architect

    7 days ago


    India VXI Global Solutions Full time

    We are seeking a Lead Data Architect to spearhead the development and implementation of our data infrastructure. As a critical member of our team, you will play a pivotal role in designing, implementing, and maintaining scalable data solutions to support our customer experience and management initiatives.This is an exciting opportunity to work with...


  • India Gravity Infosolutions Full time

    Job Title: Data Architect Lead - MumbaiEstimated Salary: ₹2,400,000 - ₹3,600,000 per annumCompany Overview:We are Gravity Infosolutions, a leading provider of innovative data solutions. Our team is passionate about delivering cutting-edge technology to drive business success.About the Role:We are seeking an experienced Senior Data Architect to lead our...


  • India ITI Data Full time

    ITI Data is seeking a highly skilled Cloud Data Engineering Lead to spearhead the design and development of scalable data pipelines on AWS. With a strong background in PySpark, Python, and AWS services, you will be responsible for building robust data integration solutions that enable data-driven insights across various internal and external sources.This is...


  • India Innova ESI Full time

    Innova ESI is seeking a highly skilled Data Solutions Architect to lead the design and implementation of data pipelines, models, and transformations. As a key member of our team, you will be responsible for ensuring the timely delivery of end-to-end data solutions that meet the needs of our business.The ideal candidate will have a strong background in data...


  • India LTIMindtree Full time

    Exciting Opportunity at LTIMindtreeWe are seeking a highly skilled Lead Data AI Architect to join our team in Hyderabad.A minimum of 10 years of overall experience with at least 4-5 years of experience in Machine Learning Deep Learning is required.Extensive experience working on NLP and conversant with Python programming is essential.The ideal candidate...