Chief Data Pipeline Architect

6 days ago

Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 7,00,000 - ₹ 8,50,000
Senior Data Engineer

We are seeking an accomplished Senior Data Engineer to spearhead the design, optimization, and maintenance of ETL pipelines in a Python & SQL-based ecosystem.

This role demands strong architectural expertise, ensuring that new data pipeline solutions are scalable, maintainable, and aligned with existing designs.

Key Responsibilities:

  • Design and optimize ETL pipelines using Python (Pandas) and SQL for scalability and maintainability.
  • Analyse existing ETL architecture and propose improvements for efficiency, consistency, and performance.
  • Develop new data pipelines while maintaining the integrity of existing workflows and data models.
  • Refactor and modernize legacy ETL processes to fit evolving business needs.
  • Maintain and improve automation scripts in a Linux environment.
  • Ensure high code quality by following best practices in version control (Git) and deployment strategies.
  • Collaborate with stakeholders to define business requirements and translate them into technical solutions.
  • Work within an Agile environment, contributing to sprint planning, retrospectives, and backlog refinement.

Position Requirements:

Technical Expertise
  • 5+ years of experience in ETL development, with a strong focus on data pipeline architecture.
  • Strong Python skills, particularly in Pandas for data manipulation and vectorized processing.
  • 3+ years of experience in designing and maintaining relational databases (PostgreSQL or similar).
  • Expertise in SQL query optimization, indexing, and schema design.
  • Experience with Airflow (or similar orchestration tools) for scheduling and monitoring ETL workflows.
  • Solid experience with Linux environments, including scripting and automation.
  • Hands-on experience integrating multiple data formats (CSV, JSON, Parquet, YAML, etc.).
  • Strong knowledge of data extraction and integration from diverse sources, including databases, SFTP, cloud storage, SharePoint, and API.
  • Git proficiency, preferably GitLab for version control and CI/CD.
Architectural & Problem-Solving Skills
  • Ability to evaluate existing ETL pipelines and propose enhancements for performance and scalability.
  • Strong problem-solving mindset can analyse a business problem, understand the existing data infrastructure, and design the best technical approach.
  • Experience with modular pipeline design to ensure reusable and efficient ETL components.
  • Design and develop ETL/ELT pipelines with an emphasis on incremental data loading to optimize performance and reduce resource consumption.
  • Expertise in data modelling (Kimball), schema design, and building dimension tables for data warehousing.

  • Chief Architect

    1 day ago

    Bengaluru, Karnataka, India Phygital Insights Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Full time|Work From OfficeThis Position is Currently OpenDepartment / Category:ARCHITECTListed on Sep 18, 2025Work Location:BANGALOREJob Descritpion of Chief Architect10+ Years Relevant ExperienceAbout the RoleWe're looking for a seasoned Data Architect with extensive experience in designing, developing, and managing large-scale data solutions. The ideal...

  • Chief Data Pipeline Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,04,000 - ₹ 1,30,878

    Senior Data Engineering LeadWe are seeking an experienced Senior Data Engineering Lead to join our team. In this role, you will lead and mentor data engineering teams to ensure alignment with technical and business objectives.You will be responsible for architecting, designing, and implementing scalable and high-performance data pipelines that meet the needs...

  • Chief Data Architect

    6 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Summary:The role of Chief Data Architect involves overseeing the design, development, and implementation of data architectures to support business strategies. This includes collaborating with cross-functional teams to understand data requirements, designing scalable data pipelines, and developing data governance frameworks.Key Responsibilities:Design and...

  • Chief Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 60,00,000 - ₹ 1,00,00,000

    Job SummaryWe are seeking an experienced Chief Data Architect to design and develop a robust data engineering environment.Key Responsibilities:Data Engineering Development & Optimization: Design, develop, and optimize ETL pipelines using open-source or cloud ETL tools (e.g., Apache Nifi, Talend).Python Scripting for Data Analysis: Write Python scripts to...

  • Chief Data Pipeline Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Senior Data Engineer Position OverviewAs a Senior Data Engineer, you will play a key role in designing and implementing data engineering solutions for our organization.Key Responsibilities:Design and develop complex data pipelines using Snowflake, Apache Beam, or similar technologies.Collaborate with cross-functional teams to identify business requirements...

  • Chief Data Architect

    6 days ago

    Bengaluru, Karnataka, India beBeeDataArchitect Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Description">We are seeking a highly skilled Chief Data Architect to lead our data transformation and integration initiatives. The successful candidate will design, develop, and implement scalable, reliable, and high-performance data processing pipelines to extract, transform, and load data from various systems to the Enterprise Data Warehouse.The ideal...

  • Data Pipeline Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job TitleData Engineering ManagerRole OverviewThe Senior Data Engineer will design, develop, and maintain scalable data pipelines and infrastructure to support data-driven decision-making and advanced analytics. They must have deep expertise in data engineering, strong problem-solving skills, and the ability to collaborate with cross-functional teams to...

  • Chief Data Pipeline Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Cyber Data Pipeline Engineer Job DescriptionAchieving success in this role demands an exceptional blend of technical expertise and collaboration skills. We are seeking a Cyber Data Pipeline Engineer to contribute to various strategic initiatives. Key Responsibilities:

  • Chief Data Pipeline Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,76,96,000 - ₹ 2,14,36,000

    Job DescriptionWe are seeking an experienced Data Engineer to design, build and maintain scalable data pipelines and processing systems.The ideal candidate will have hands-on expertise in distributed data processing, data lake/lakehouse systems, and workflow orchestration using modern tools such as Spark, PySpark, Apache Iceberg, Apache Hudi, and Apache...

  • Strategic Data Pipeline Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 19,96,000 - ₹ 24,64,700

    Job DescriptionWe are seeking a highly skilled Data Engineer to design, develop, and maintain data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform, and then integrate it into Oracle EPM (Enterprise Performance Management).This role requires deep expertise in data integration, ETL/ELT, APIs,...