Senior Data Pipeline Architect

1 week ago

Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

About this role

We're seeking a skilled Data Engineer to design, build and maintain scalable data pipelines and solutions. In this position, you'll work closely with cross-functional teams to ensure the availability, accuracy and performance of data infrastructure.

Key responsibilities:

  • Design, develop and maintain ETL/ELT data pipelines for batch and near real-time processing using tools such as Talend, AWS Glue, Lambda and SQS.
  • Develop and optimize SQL queries and perform data modeling, cleansing, transformation and validation to ensure data quality and accuracy.
  • Write efficient, reusable and scalable ETL scripts primarily using Python, PySpark and SQL. Utilize AWS SDK for Python (Boto3) for automation and integration with AWS services.
  • Collaborate with data analysts, data scientists and business stakeholders to understand data requirements and deliver actionable insights.
  • Build and orchestrate workflows using Talend and AWS Step Functions to automate complex data processes.
  • Monitor and troubleshoot data pipelines and infrastructure using AWS CloudWatch and other monitoring tools to ensure optimal performance and availability.

Required skills and qualifications:

  • Cloud platforms and services: expertise in AWS with hands-on experience in data processing and querying, storage and databases, compute and networking, messaging and orchestration, security and monitoring.
  • Programming and scripting: proficiency in Python, including AWS SDK (Boto3), PySpark, Java and SQL.
  • ETL and workflow orchestration: experience with Talend or similar ETL tools, Apache Airflow or other orchestration tools.

Benefits include the opportunity to work with large datasets, excellent problem-solving skills and strong communication skills. A Bachelor's or Master's degree in Computer Science, Engineering, Information Technology or a related field is required, along with 7+ years of professional experience in data engineering or related roles.

  • Data Architect

    1 week ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Title: Data Architect – AWS & GCP Experience: 11 Years Location: Remote / Hybrid Job Type: Full-timeAbout the Role:We are seeking an experienced Data Architect with deep expertise in AWS and Google Cloud Platform (GCP). The ideal candidate will have a proven track record in designing and implementing scalable, secure, and high-performance data...

  • Senior Data Architect

    1 week ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data ArchitectWe are seeking a skilled leader with strong expertise in designing, building and optimizing data platforms.Main Responsibilities:Develop and drive initiatives to improve the company's data strategy.Design and implement large-scale structured and unstructured data pipelines.Create ETL/ELT processes using SQL and modern big data...

  • Senior Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataArchitect Full time ₹ 15,00,000 - ₹ 25,00,000

    We are looking for a skilled Senior Data Architect to lead the design, development and management of scalable data platforms.Key Responsibilities:Design, develop and maintain end-to-end data pipelines and infrastructureTranslate business requirements into technical solutionsBuild and manage data flows across structured and unstructured data sourcesEnsure...

  • Chief Data Pipeline Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataengineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe're seeking an experienced and passionate Senior Data Engineer to join our growing data & AI team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions using Microsoft Fabric.Key Responsibilities:Design and implement end-to-end data pipelines using Microsoft Fabric.Work with...

  • Senior Cloud Data Architect

    7 days ago

    Chennai, Tamil Nadu, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:We are seeking an experienced Senior Cloud Data Architect to lead our data strategy and implement cutting-edge data solutions.This is a fantastic opportunity for a highly skilled professional to take ownership of our data infrastructure and drive innovation in the field of cloud-based data engineering.The ideal candidate will have extensive...

  • Senior Data Architect

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataArchitect Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data ArchitectThe position of Senior Data Architect is a challenging opportunity to take on a leadership role in the development and implementation of cutting-edge data solutions.As a seasoned professional with extensive experience in designing and implementing large-scale data architectures, you will play a key part in driving our...

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...

  • Data Pipeline Architect

    7 days ago

    Chennai, Tamil Nadu, India beBeeSnapLogic Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Description: As a seasoned professional, you will be responsible for designing and implementing complex data pipelines using SnapLogic. Your expertise in building Ultra pipelines, error handling, scheduling tasks & alerts, Fault Tolerance, and Error Percolation will be invaluable to our organization.Required Skills and Qualifications:Technical Expertise:...

  • Expert Data Pipeline Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a highly skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and managing scalable data pipelines.Key Responsibilities:Design and implement data pipelines using GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Composer, and...

  • AWS Data Pipeline Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataPipelineEngineer Full time ₹ 1,20,00,000 - ₹ 2,10,00,000

    Cloud Engineer Role SummaryWe are seeking an experienced professional to fill the Cloud Engineer position.The ideal candidate will possess strong proficiency in AWS Glue and PySpark, as well as a solid understanding of data lake architecture.This role requires collaboration with cross-functional teams to ensure seamless integration of data pipelines.Key...