High Performance Data Pipelines Architect

1 day ago

Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 1,20,50,000 - ₹ 2,53,10,000
Job Opportunity

We are seeking a highly skilled Senior Data Engineer to join our dynamic team.

  • This role focuses on building scalable, high-performance data pipelines using Python and PySpark within the Azure Databricks environment.

The ideal candidate will have expertise in:

  • Data Pipeline Development & Optimization: Design, develop, and maintain scalable ETL/ELT data pipelines using Python and PySpark.
  • Leverage Azure Databricks to process large volumes of structured and semi-structured data efficiently.
  • Implement data quality checks, error handling, and performance tuning across all stages of data processing.

Key Responsibilities:

Data Architecture & Modeling:
  • Contribute to the design of cloud-based data architectures that support analytics and reporting use cases.
  • Develop and maintain data models that adhere to industry best practices and support business requirements.
Cloud Integration (Azure):
  • Integrate and orchestrate data workflows using Azure Data Factory, Azure Blob Storage, and Event Hub where applicable.
  • Optimize cloud compute resources and manage cost-effective data processing at scale.
Collaboration & Stakeholder Engagement:
  • Partner with data analysts, data scientists, and business users to understand evolving data needs.
  • Collaborate with DevOps and platform teams to ensure reliable, secure, and automated data operations.
Documentation & Best Practices:
  • Maintain clear and comprehensive documentation of code, pipelines, and architectural decisions.
  • Contribute to internal data engineering standards and promote best practices for code quality, testing, and CI/CD.

This role requires strong problem-solving skills, attention to detail, and excellent communication skills. If you are passionate about data engineering and want to work with a dynamic team, please apply.

  • High Performance Data Pipelines Specialist

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time US$ 1,80,000 - US$ 2,40,000

    Databricks Lead - High Performance Data PipelinesNTT DATA is seeking a skilled Databricks Lead to join our team. The ideal candidate will have extensive experience in data engineering, with a strong focus on high-performance data pipelines.Job DescriptionWe are looking for a technically hands-on leader who can design and implement scalable data architectures...

  • High-Performance Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer RoleWe are seeking a highly skilled and experienced data engineer to play a critical role in building, optimizing, and maintaining data pipelines and architectures.Key Responsibilities:Design, develop, and optimize scalable data pipelines and workflows using Azure Data Factory, Synapse Pipelines, and Microsoft Fabric.Transform and...

  • High-Performance Backend Developer

    9 hours ago

    Bengaluru, Karnataka, India beBeeBackend Full time US$ 1,60,000 - US$ 2,20,000

    We are building a high-performance backend foundation that powers our entire platform. As a key member of our founding team, you will be responsible for architecting and implementing the systems that handle real-time financial data, low-latency execution pipelines, and mission-critical APIs that integrate with frontend apps, AI engines, and external...

  • Principal Data Pipeline Architect

    11 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,25,000 - US$ 1,75,000

    About the RoleWe are seeking a Senior Data Engineer with expertise in building scalable data pipelines and integrating healthcare data. Reporting to the Head of AI and Engineering, you will design, build, and operate the backbone of our AI-powered applications.You will work closely with software engineers, ML specialists, and clinical partners to bring our...

  • High-Performance Customer Data Architect

    2 days ago

    Bengaluru, Karnataka, India beBeeCloud Full time ₹ 20,00,000 - ₹ 40,00,000

    Customer Data Platforms ArchitectAs a seasoned architect, you will play a pivotal role in the design and implementation of Customer Data Platforms (CDPs) using Salesforce Data Cloud. Your technical expertise will enable seamless integration with diverse technologies, driving business growth through data-driven decision making.Job Description:Oversee the...

  • Data Architect

    4 days ago

    Bengaluru, Karnataka, India Best Infosystems Ltd. Full time

    Data Architect_Full-Time_Trivandrum/Bangalore/Chennai/KochiJob Title: Data ArchitectJob Type: Full-TimeLocation: Trivandrum/Bangalore/Chennai/KochiExp: 10+ yearsJob Description:•Design and implement scalable, secure, and cost-effective data architectures using GCP.•Lead the design and development of data pipelines with BigQuery, Dataflow, and Cloud...

  • Data Pipeline Specialist

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the JobAre you ready for a new career challenge?We are currently seeking an ETL Developer to join our team.The successful candidate will have at least 5 years of experience in designing and implementing data pipelines using Databricks (PySpark/Scala/SQL).The role involves assessing current-state Informatica ETL jobs, mappings, workflows, and...

  • High Salary Senior Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Tredence Inc. Full time

    SNOWFLAKE ARCHITECTLocation: Bengaluru, Pune, Gurugram, Kolkata, Chennai and HyderabadPrimary Roles and Responsibilities:- Working experience in Snowflake; use of Snow SQL CLI, Snow Pipe creation of custom functions and Snowflake stored producers, schema modelling, performance tuning etc.- Expertise in Snowflake data modelling, ELT using Snowflake SQL,...

  • Lead Data Pipeline Specialist

    11 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878

    Job Title: Senior Data EngineerAbout the Role:This is an exceptional opportunity to join our organization as a Senior Data Engineer. As a key member of our data team, you will be responsible for designing, building, and optimizing scalable data pipelines using cloud-native technologies.Key Responsibilities:Design and architect high-throughput ETL pipelines...

  • Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India Best Infosystems Ltd. Full time

    Data Architect_Full-Time_Trivandrum/Bangalore/Chennai/Kochi Job Title: Data Architect Job Type: Full-Time Location: Trivandrum/Bangalore/Chennai/Kochi Exp: 10+ years Job Description: •Design and implement scalable, secure, and cost-effective data architectures using GCP. •Lead the design and development of data pipelines with BigQuery, Dataflow,...