Data Pipeline Architect

1 week ago

Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000
Scalable Data Engineer

We are seeking a highly motivated and talented data engineer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Key Responsibilities:

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
  • Adhere to agile delivery practices, sprint planning, and documentation requirements.

Required Skills and Qualifications:

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have:

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

  • Data Pipeline Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeAirflow Full time ₹ 15,00,000 - ₹ 19,00,000

    Expert Data Pipeline DeveloperAs a seasoned data engineer, you will be responsible for designing and implementing scalable and reliable data pipelines using Airflow.Develop complex workflows using Airflow, ensuring high-quality data flow and meeting business requirements.Collaborate with cross-functional teams to integrate data pipelines with business...

  • Data Pipeline Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 18,00,000

    Cloud Data Engineer OpportunityWe are seeking a highly skilled AWS Cloud Data Engineer to design, build, and maintain scalable data pipelines on AWS. The ideal candidate will have a solid background in cloud-based data solutions, object-oriented programming, reusable code design, and Python development.This role involves building, optimizing, and maintaining...

  • Lead Data Pipeline Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000

    Data Engineer Role">This position involves designing, building, and optimizing data pipelines to ensure that data is clean, secure, and accessible for analytics and operations.">As a member of our team, you will collaborate with talented individuals to solve real-world data challenges and make a direct impact on business decisions.">About Us">We are a...

  • AWS Cloud Data Pipeline Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    As a Data Engineering Specialist, you will play a key role in designing and implementing data pipelines using AWS services.Key Responsibilities:Design and develop scalable data pipelines using AWS services including Glue, PySpark, SQL, Databricks, and Python.Develop and maintain applications using Python and PySpark for data manipulation, transformation, and...

  • Data Architect for Cloud-Based Data Pipelines

    1 week ago

    Kota, Rajasthan, India beBeeETL Full time ₹ 15,00,000 - ₹ 20,00,000

    We are seeking a highly experienced Senior Informatica Engineer to design and develop cloud-based ETL pipelines using Azure services and integrate data into AWS Datalake or other modern data platforms.

  • Cloud Data Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineer – Cloud Solutions ArchitectWe are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data solutions.The ideal candidate will be proficient in GCP services like Pub/Sub, Dataflow, Cloud Storage, and BigQuery, with a foundational understanding of AI/ML workflows...

  • Chief Data Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeData Full time US$ 1,80,000 - US$ 2,50,000

    Advanced Data Systems LeadJob Description:The successful candidate will lead a team of data engineers and architects to design, implement, and deploy high-performance, scalable data systems. This includes developing robust data architectures, designing and optimizing data pipelines, and ensuring compliance with industry standards.Required Skills and...

  • Senior Data Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Opportunity: Senior Data ArchitectWe are seeking an experienced and skilled data architect to join our team. The ideal candidate will be proficient in designing and implementing scalable, high-performance data transformation pipelines using Snowflake and dbt.About the Role:Design and implement scalable ELT pipelines using dbt on Snowflake, following...

  • Chief Data Architect

    2 weeks ago

    Kota, Rajasthan, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    As a seasoned data architect, you will design and implement large-scale data pipelines to fuel business insights and analytics. Your expertise in SQL, Python, or other data engineering languages will ensure high-quality, reliable data for player behavior analysis, game performance, and optimization.Key Responsibilities:Develop and maintain scalable data...

  • Highly Skilled Data Architect Wanted

    2 weeks ago

    Kota, Rajasthan, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Architect RoleAs a Senior Data Architect, you will play a pivotal role in designing and implementing scalable data pipelines using Snowflake and dbt. This position demands an individual with expertise in crafting modular, test-driven ELT pipelines on Snowflake and leveraging dbt best practices to design layered data models.Develop robust data...