Advanced Data Pipeline Architect

3 days ago

Nellore, Andhra Pradesh, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 22,00,000

**Job Summary**

We are seeking a highly skilled ETL Developer to join our team. As an ETL Developer, you will be responsible for designing, developing, and maintaining ETL pipelines using IBM DataStage and AWS Glue/Lambda.

**Key Responsibilities:**

  • Design and develop ETL pipelines for ingestion from varied sources using IBM DataStage and AWS Glue/Lambda.
  • Build and optimize data flows for loading curated datasets into Snowflake.
  • Collaborate with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.

**Required Skills and Experience:**

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

**Nice to Have:**

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

  • Data Pipeline Architect

    6 days ago

    Nellore, Andhra Pradesh, India beBeeData Full time ₹ 80,00,000 - ₹ 1,60,00,000

    Job Title: Data Pipeline ArchitectAbout the Role:We are looking for a seasoned professional to design and develop efficient data pipelines using AWS Glue and DBT.The ideal candidate will have experience with serverless and AWS ecosystem services for data ingestion and transformation, aligning with Gartner's approach on data transformation.This role involves...

  • Building Scalable Data Pipelines

    1 week ago

    Nellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data ArchitectWe are seeking an experienced data architect to lead the design and implementation of our data pipelines using Snowflake, AWS Lambda, Glue, and SQL.Key Responsibilities:Design and implement scalable data pipelines that integrate with multiple internal and external sources.Develop ETL/ELT pipelines using AWS Glue, Lambda, and DBT for...

  • Data Pipeline Architect

    5 days ago

    Nellore, Andhra Pradesh, India beBeeFinancial Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Big Data EngineerWe are seeking a skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing and building robust data pipelines that handle high-volume financial data.Key Responsibilities:Design end-to-end data pipelines for stocks, crypto, and other financial datasets.Integrate third-party APIs and data...

  • Data Pipeline Architect

    7 days ago

    Nellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data EngineerWe are seeking a seasoned Senior Data Engineer to lead the development and implementation of large-scale data pipelines, multi-environment deployments, and modern DevOps/configuration management tools.Main Responsibilities:Design, implement, and manage Continuous Integration/Continuous Deployment (CI/CD) pipelines using GitHub Actions or...

  • Data Pipeline Architect

    6 days ago

    Nellore, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Big Data Engineer Position OverviewThe ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.This role requires a highly skilled professional to design, develop, and optimize large-scale data pipelines and distributed data...

  • Data Pipeline Specialist

    6 days ago

    Nellore, Andhra Pradesh, India beBeeCloudMigrationEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    About the Role:We are seeking a highly skilled Senior Cloud Migration Engineer with extensive experience in large-scale cloud migration projects. This role involves designing, developing, and deploying robust and scalable data pipelines for migrating data from various on-premise sources to cloud platforms.Design and implement efficient cloud migration...

  • Cloud Data Pipeline Architect

    6 days ago

    Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a seasoned data engineer to lead our data pipeline development efforts, focusing on scalability and efficiency.The ideal candidate will design, develop, and optimize ETL/ELT pipelines using Apache Spark, AWS Glue, and Azure Data Factory for seamless data integration.Collaboration with cross-functional teams is essential in building...

  • Azure Cloud Data Architect

    6 days ago

    Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Solutions ArchitectWe are seeking an exceptional Senior Data Solutions Architect to join our team and drive the development of scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.The ideal candidate will have a proven track record of delivering high-quality data solutions across scrum teams, ensuring data governance with...

  • Chief Data Architect

    7 days ago

    Nellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Opportunity:This role is ideal for someone with deep technical expertise in data engineering who thrives in a fast-paced environment.You will play a key role in scaling analytics and reporting capabilities across the full data stack—from ingestion and transformation to orchestration and delivery.Key Responsibilities:Design and implement efficient...

  • Data Pipeline Architect

    6 days ago

    Nellore, Andhra Pradesh, India beBeeBackend Full time ₹ 12,00,000 - ₹ 25,00,000

    Backend Developer NeededWe are looking for a skilled Backend Developer to join our team. The ideal candidate will have experience with Node.js, AWS, and SQL/NoSQL databases.About the RoleDesign and scale APIs, services, and data pipelinesRapidly build POCs and ship features with the teamOptimize for performance, reliability, and cost-efficiencyOwn CI/CD,...