Data Pipeline Architect

1 day ago

Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

Job Title: Data Engineering Specialist

Experience: 6 to 9 Years

Location: Bangalore / Mumbai (any) (Hybrid WFO)

About the Role:

We are seeking a highly skilled data engineer with hands-on experience in designing, developing, and managing data pipelines and transformation frameworks. The ideal candidate will play a critical role in enabling analytics, reporting, and business intelligence capabilities across the enterprise.

You will be working closely with data architects, platform engineers, and DevOps teams to build scalable, resilient, and efficient data engineering solutions using modern Azure tools and best practices.

Key Responsibilities:

  • Design and build scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (Spark), and Azure Synapse Analytics.
  • Develop and optimize PySpark/Spark SQL jobs for large-scale batch and streaming data transformations.
  • Ingest data from various sources including Apache Kafka, REST APIs, and RDBMS, ensuring real-time or near-real-time availability.
  • Implement data modeling strategies (star schema, snowflake schema) for analytics consumption layers in Synapse or ADLS.
  • Collaborate with DevOps teams to automate deployment using CI/CD pipelines (Azure DevOps, GitHub Actions, etc.).
  • Monitor, troubleshoot, and optimize data workflows for performance, cost-efficiency, and reliability.
  • Follow coding standards, participate in peer reviews, and maintain version-controlled code in Git repositories.
  • Support data quality checks, logging, alerting, and observability mechanisms for production workloads.
  • Participate in sprint ceremonies and contribute to task estimation and delivery planning.

Must-Have Skills:

● 5+ years of experience in data engineering roles.

● Hands-on experience with:

○ Azure Data Factory (ADF) – building pipelines, triggers, linked services.

○ Azure Databricks – building and managing Spark jobs in PySpark.

○ Azure Synapse Analytics – data warehousing, SQL queries, workspace orchestration.

○ Apache Kafka – consuming and processing real-time data streams.

● Strong in SQL, Python, and Spark for data manipulation and transformation.

● Exposure to CI/CD practices (Azure DevOps, Git workflows, build/release pipelines).

● Understanding of data lake architecture and modern data warehousing principles.

Nice-to-Have Skills:

● Experience with Delta Lake, Event Hub, or Azure Stream Analytics.

● Familiarity with infrastructure-as-code tools like Terraform or ARM templates.

● Exposure to Data Governance, Data Quality frameworks, or metadata management.

● Understanding of networking, security, and access control in Azure data services

Preferred Certifications:

● Microsoft Certified: Azure Data Engineer Associate

● Databricks Certified Associate Developer for Apache Spark

● Azure Fundamentals (AZ-900) – optional but helpful

  • Data Pipeline Architect

    1 week ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Senior Data Engineer">">Our company is seeking an experienced Senior Data Engineer to join our team. This role will play a critical part in building scalable data pipelines and integrating healthcare data.">

  • Senior Data Pipeline Engineer

    5 days ago

    Mangalore, Karnataka, India beBeeDataSolutions Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Data Solutions ArchitectAs a highly skilled data solutions architect, you will play a pivotal role in designing and developing scalable data pipelines and ETL processes on Databricks. Your expertise will be instrumental in leading the architecture and implementation of secure, high-performance data solutions.Key Responsibilities:Develop and...

  • Chief Data Pipeline Architect

    5 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer to join our team at a cutting-edge organization.This role is critical to the development of new data products and the modernization of existing platforms in cloud environments, especially AWS.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines and data...

  • Data Pipeline Architect

    1 week ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering group, you will be responsible for designing, developing, and implementing large-scale data processing systems.Key Responsibilities:Design and develop scalable ETL workflows using Python and Pandas.Implement and manage...

  • Cloud Data Pipeline Architect

    7 days ago

    Mangalore, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity: Data Engineer">We are seeking a skilled professional to lead our data engineering efforts.Our team requires someone who can design, develop and maintain robust data pipelines using cloud services such as GCP, AWS or Azure.">Main Responsibilities:Develop scalable data pipelines for real-time data processing.Implement data ingestion,...

  • Data Architect

    2 weeks ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a seasoned Data Architect to lead the design and development of scalable data engineering solutions that drive business growth.">Design, develop, and implement high-performance data processing pipelines using Apache Spark, Streamsets, or similar frameworks.Collaborate with cross-functional teams to integrate data systems and ensure...

  • Data Architect

    7 days ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job DescriptionAs a senior Data Architect, you will be responsible for designing and implementing a scalable data warehouse architecture on PostgreSQL, leveraging open-source technologies. You will work closely with cross-functional teams to deliver high-quality, reliable data solutions that meet business and analytics requirements.Key...

  • Senior Data Pipeline Architect

    7 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Key ResponsibilitiesWe're looking for a skilled professional to lead the development of scalable data pipelines.Design and Implement Efficient ETL ProcessesCollaborate with Data TeamsImplement Data Governance and Security StandardsOptimize Data Workflows for Scalability and PerformanceRequired Skills & QualificationsTo succeed in this role, you'll...

  • Data Architect

    1 week ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 25,00,000

    Job Overview">This Data Architect role involves designing and implementing scalable data pipelines to support increasing data volume and complexity.">Key Responsibilities:Develop and Maintain Data Pipelines: Ensure data pipelines are scalable, efficient, and secure.Collaborate with Cross-Functional Teams: Work closely with analytics, business, and...

  • Cloud Data Architect

    1 week ago

    Mangalore, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Cloud Data ArchitectOur organization is seeking a highly skilled Cloud Data Architect to design, develop, and implement end-to-end data pipelines on cloud platforms.Key Responsibilities:Design and deploy scalable data architectures on cloud infrastructure using Azure Data Factory, Databricks, and Data Lake.Develop and maintain complex data pipelines using...