Data Pipeline Architect

2 weeks ago


Bengaluru, Karnataka, India Neerinfo Solutions Full time
Job Requirements

We are seeking a highly skilled Cloud Data Infrastructure Specialist to join our team at Neerinfo Solutions. The successful candidate will be responsible for designing and implementing scalable cloud data infrastructure using AWS services.

Key Responsibilities:

  • Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.
  • Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage solutions.
  • Develop data transformation scripts using Python and PySpark to process large datasets in the cloud.
  • Work with Apache Spark and the Hadoop ecosystem to manage large-scale data processing workloads.
  • Optimize Apache Spark and Hadoop jobs for performance and scalability, ensuring data pipelines run efficiently at scale.
  • Write and optimize complex SQL queries for data manipulation and aggregation in cloud data warehouses.

Qualifications:

  • 4-8 years of experience in data engineering, with significant hands-on experience in AWS services and data pipeline development.
  • Strong experience with AWS services including Redshift, Glue, EMR, Lambda, and S3.
  • In-depth experience with Apache Spark and Hadoop ecosystem for distributed data processing and analysis.
  • Proficiency in Python and PySpark for data engineering tasks, data transformation, and automation.

Salary Range: $120,000 - $180,000 per year



  • Bengaluru, Karnataka, India Aditi Consulting Full time

    Aditi Consulting is a leading provider of data-driven solutions. We are looking for a skilled Data Pipeline Architect to join our team.Location: Bangalore/HybridEstimated Salary: ₹12,00,000 - ₹20,00,000 per annumCompany OverviewAt Aditi Consulting, we believe in harnessing the power of data to drive business decisions. Our team of experts works...


  • Bengaluru, Karnataka, India LTIMindtree Full time

    At LTIMindtree, we are seeking a skilled Data Pipeline Architect to join our team. With a strong background in data engineering and expertise in Snowflake, DBT, and Terraform, you will design and build end-to-end data pipelines that drive business insights.We offer a competitive salary of approximately $120,000 per year, commensurate with experience. Our...

  • Pipeline Architect

    2 weeks ago


    Bengaluru, Karnataka, India ITQube LTD Full time

    Becoming Part of Our TeamWe are looking for a talented Pipeline Architect to collaborate with our team in designing, building, and optimizing data transformation, cleansing, and mapping pipelines using SnapLogic. The ideal candidate should have a solid understanding of cloud platforms, particularly AWS.


  • Bengaluru, Karnataka, India Pro5 (formerly Mangtas) Full time

    Job Description">As a Senior Data Pipeline Architect at Pro5 (formerly Mangtas), you will play a critical role in designing and delivering high-quality data pipelines that meet business requirements.About the Role">You will work closely with cross-functional teams to design, implement, and deploy scalable data solutions using modern data stacks, including...


  • Bengaluru, Karnataka, India Synechron Full time

    We are looking for a highly skilled Senior Data Engineer to join our team at Synechron. The estimated salary for this role is $140,000 - $200,000 per year.About the RoleThis is a unique opportunity to work on complex data engineering projects and contribute to the growth and success of our company. As a Senior Data Engineer, you will be responsible for...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. This role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS.The ideal candidate will have experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development....


  • Bengaluru, Karnataka, India Sigmoid Full time

    Sigmoid is seeking a highly skilled Senior Data Pipeline Architect to lead our DevOps Engineering team.Job OverviewThis role offers the opportunity to design and implement scalable data pipelines that will serve as the foundation for collecting, storing, modeling, and analyzing massive datasets from multiple channels.The ideal candidate will have a strong...


  • Bengaluru, Karnataka, India TMRW Full time

    Job DescriptionWe are seeking a Big Data Infrastructure Specialist to design, develop, and maintain robust and scalable data pipelines for business intelligence and machine learning. The successful candidate will collaborate with data analysts, data scientists, and other data consumers within the business to manage the data warehouse table structure and...


  • Bengaluru, Karnataka, India Coders Brain Technology Private Limited Full time

    Job Overview: We are looking for a talented individual to lead our data modeling and pipeline optimization efforts. As a Senior Data Architect - DBT, you will be responsible for designing, building, and maintaining data models using DBT, implementing transformations to ensure data quality and consistency, and collaborating with data engineering teams to...


  • Bengaluru, Karnataka, India Synechron Full time

    Synechron Job OpeningWe are looking for a highly skilled Data Pipeline Developer to join our team. As a Senior Data Engineer, you will be responsible for building and maintaining complex data pipelines using cloud-native technologies like AWS CDK and Apache Beam.Key Responsibilities:Data Streaming Pipeline Development: Develop and implement data streaming...


  • Bengaluru, Karnataka, India Hevo Data Full time

    Job Overview:As a Sales Development Representative at Hevo Data, you will be responsible for managing your account book and engaging with prospects to introduce and position Hevo as a one-stop solution for their data integration and pipeline needs.Key Accountabilities:Develop and execute territory plans to identify and pursue new business opportunitiesBuild...

  • Eximietas Design

    1 week ago


    Bengaluru, Karnataka, India Eximietas Design Full time

    We are seeking a highly skilled Senior Data Pipeline Architect to join our team at Eximietas Design. This is a fantastic opportunity for an experienced professional to lead the design and implementation of large-scale data pipelines using Java, Kafka, S3, CDC, Spark, and SQL.The ideal candidate will have a strong background in big data engineering with a...


  • Bengaluru, Karnataka, India Infometry Full time

    We are seeking a skilled Cloud Data Pipeline Developer to join our team at Infometry. As a key member of our infrastructure group, you will design, develop, and maintain robust data pipelines using Google Cloud Platform (GCP) services. In this role, you will collaborate closely with data scientists, analysts, and engineers to ensure seamless data flow and...


  • Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Key ResponsibilitiesData Pipeline Development : Develop and manage ETL/ELT workflows using AWS Glue and PySpark to process large datasets.Data Storage and Management : Architect and manage data storage in Amazon S3 , ensuring performance, cost-efficiency, and security.Compute and Serverless : Build and deploy serverless solutions using AWS Lambda for...


  • Bengaluru, Karnataka, India Gloify Full time

    About GloifyWe are seeking an experienced Data Pipeline Architect to design, develop, and maintain our data pipeline architecture.Our team works with cutting-edge technology, focusing on scalability and efficiency. As a key member of the team, you will contribute to creating and optimizing our data pipeline infrastructure.Responsibilities:Design and Optimize...

  • Senior Data Architect

    4 weeks ago


    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job OverviewWe are seeking an experienced Senior Data Architect - Cloud Engineer to join our team in Remote India. The successful candidate will play a critical role in supporting and enhancing our Azure Data Lakehouse environment.About the RoleThis is an exciting opportunity to leverage your expertise in designing and implementing efficient data models,...


  • Bengaluru, Karnataka, India LanceSoft, Inc Full time

    Job Description:">">Design, develop, and maintain Ab Initio data pipelines using the latest Ab Initio suite of products (GDE, CoOp, EME, Plans, Control Center).">Leverage advanced Ab Initio concepts like PDL and Meta Programming to create efficient and scalable data solutions.">Work hands-on with HDFS and Hive to manage and process large datasets.">Utilize...


  • Bengaluru, Karnataka, India Dautom Full time

    About the RoleWe are looking for a talented Data Engineering Specialist to join our team at Dautom. As a senior member of our engineering team, you will be responsible for designing and implementing high-performance data pipelines using cutting-edge technologies.The ideal candidate will have a strong background in software engineering, with expertise in...

  • Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Luxoft Full time

    Job OverviewLuxoft is seeking an experienced Data Engineer to join our team as a Cloud Data Pipeline Specialist. In this role, you will be responsible for designing and implementing scalable data pipelines within the Azure ecosystem.ResponsibilitiesDesign, develop, and maintain large-scale data processing pipelines using Azure Data Factory (ADF) and Azure...


  • Bengaluru, Karnataka, India Data Inc Full time

    Overview of the RoleThe position is part of a global initiative to digitize at scale, simplifying the way we deliver change in Bank. To achieve this, we need to align businesses, functions, technology, data, and operations behind a shared vision. This involves providing clear and documented processes between business strategy and the delivery of change...