Data Pipeline Architect

6 days ago

Coimbatore, Tamil Nadu, India beBeeETLDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

Job Title: ETL Developer

We are seeking a skilled and motivated ETL professional to join our data engineering team.

You will work on designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Main Responsibilities:

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Collaborate with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.

Required Skills and Experience:

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have:

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

About the Role:

This role offers an opportunity to work on building cutting-edge data pipelines and collaborating with cross-functional teams.

  • Data Pipeline Architect

    4 hours ago

    Coimbatore, Tamil Nadu, India beBeeDataPipeline Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a skilled Data Pipeline Architect to design, develop, and deploy scalable data pipelines and ETL processes.Key Responsibilities:Design and implement efficient data pipelines using Databricks.Collaborate with stakeholders to understand data requirements and translate them into technical solutions.Develop and maintain Databricks...

  • Data Pipeline Architect

    2 weeks ago

    Coimbatore, Tamil Nadu, India beBeeDatabricks Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    **Job Overview:**We are seeking a seasoned Databricks Professional to join our organization on a contract basis. The ideal candidate will possess a strong background in designing and implementing data pipelines using Azure Data Factory, developing high-performance Databricks notebooks leveraging SQL and Python, and working with various file formats.**Key...

  • Data Architect

    2 days ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About Us At Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms. Design and implement end-to-end data...

  • Data Architect

    1 week ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About UsAt Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms.Key Responsibilities- Design and implement...

  • Data Pipeline Specialist

    4 days ago

    Coimbatore, Tamil Nadu, India beBeeDataPipeline Full time ₹ 30,00,000 - ₹ 40,00,000

    Key Data Pipeline SpecialistThe role entails designing, developing, and maintaining high-capacity data pipelines that efficiently handle structured and unstructured data.ResponsibilitiesDesign and Build Complex Pipelines: You will be responsible for designing and building complex production-level data pipelines utilizing AWS services such as S3, EC2, RDS,...

  • Data Architect

    2 weeks ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About UsAt Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms.Key ResponsibilitiesDesign and implement ...

  • Data Architect

    7 days ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About Us At Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms. Key Responsibilities Design and implement...

  • Data Architect

    23 hours ago

    Coimbatore, Tamil Nadu, India Hash Agile Technologies Full time

    About UsAt Hash Agile Technologies, we empower businesses with data-driven insights by building scalable, secure, and innovative data solutions. We are looking for a highly skilled Data Architect to design and lead our data ecosystem, ensuring seamless integration, accessibility, and reliability across platforms.Key ResponsibilitiesDesign and implement...

  • Senior Data Solutions Architect

    19 hours ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a highly skilled Data Engineer to design, develop and maintain data pipelines that extract data from APIs, process and store it in the Databricks Lakehouse platform.This role requires deep expertise in data integration, ETL/ELT, APIs and Databricks.The engineer must design and implement end-to-end pipelines from API extraction into...

  • Big Data Architect

    6 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data ArchitectRole Overview:We are seeking a highly skilled and motivated Data Architect to design, develop, and maintain scalable data architectures that process and transform large volumes of structured and unstructured data.The ideal candidate will have a strong background in data architecture, cloud technologies, and data engineering, and will...