Building Scalable Data Infrastructure

14 hours ago

Gulbarga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,10,00,000
Job Title: Data Engineer

We are seeking an experienced Data Engineer to join our team in designing and building large-scale data pipelines using Google Cloud Platform.

This role requires a strong understanding of data engineering principles, including data acquisition, processing, and storage. The successful candidate will be responsible for developing, testing, and maintaining complex data pipelines using various tools in the Google Cloud Ecosystem such as Python, Data Flow, DataStream, CDC, Cloud Functions, Cloud Run, Pub Sub, BigQuery, Cloud Storage.

The ideal candidate will have 5-7 years of experience in a data engineering role with core GCP experience of 3-4 years. They will be able to work effectively in a fast-paced environment and collaborate with cross-functional teams to deliver high-quality solutions.

  • Develop, construct, test, and maintain large-scale data pipelines using Google Cloud Platform.
  • Build complex datasets based on business requirements.
  • Construct 'big data' pipeline architecture.
  • Identify opportunities for data acquisition via working with stakeholders and business clients.
  • Translate business needs into technical requirements.
  • Leverage various tools in the Google Cloud Ecosystem to integrate systems and data pipelines.
  • Use logs & alerts to effectively monitor pipelines.
  • Use SAP SLT to replicate SAP tables to Google Cloud using SLT.
  • Develop JSON messaging structure for integrating with various applications.
  • Leverage DevOps and CI/CD practices (GitHub, Terraform) to ensure the reliability and scalability of data pipelines.
  • Partition/Cluster and retrieve content in Big Query and use IAM roles & Policy Tags to secure the data.
  • Use roles to secure access to datasets, authorized views to share data between projects.
  • Design and build an ingestion pipeline using Rest API.
  • Recommend ways to improve data quality, reliability, and efficiency.
Requirements:
  • Requires 5-7 years of experience in a data engineering role – with core GCP experience of 3-4 years.
  • Work time will be in CST, until 2 am IST on most days.
  • 1-2 times in a month, the resource might be needed to stretch their work hours by 1-2 hours.

  • Building Scalable Financial Infrastructure

    2 days ago

    Gulbarga, Karnataka, India beBeeBackend Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Backend Software EngineerWe are seeking a highly skilled and experienced Backend Software Engineer to join our team. As a key member of our engineering team, you will play a critical role in designing and implementing scalable and secure systems for global financial transactions.This is an excellent opportunity to build financial infrastructure from the...

  • Building Scalable Data Architectures

    6 days ago

    Gulbarga, Karnataka, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 3,00,00,000

    Senior Data Engineer Job DescriptionAs a Senior Data Engineer, you will play a pivotal role in designing and maintaining cloud-based data architectures. Your primary responsibility will be to build scalable real-time and batch processing workflows using Azure Databricks, PySpark, and Apache Spark.Designing and implementing data pipelines that meet business...

  • Building Scalable Cloud Platforms

    6 days ago

    Gulbarga, Karnataka, India beBeeInfrastructure Full time ₹ 80,00,000 - ₹ 1,45,00,000

    Platform Infrastructure SpecialistWe are seeking an exceptional Platform Infrastructure Specialist to build our infrastructure from the ground up. As our first dedicated specialist, you will have the rare opportunity to make architectural decisions that will impact the company for years to come.Main ResponsibilitiesCreate and implement a cloud platform on...

  • Building Scalable Data Solutions

    6 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 30,00,000

    Job OverviewWe are seeking a skilled professional to join our team as a Data Engineer. As a key member of our data engineering team, you will be responsible for designing, building, and managing scalable data pipelines and enterprise data solutions.Our ideal candidate has a deep understanding of Azure, SQL Server, and Databricks, with experience in ETL/ELT...

  • Building Scalable Cloud Systems

    12 hours ago

    Gulbarga, Karnataka, India beBeeCloudArchitect Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Cloud ArchitectJob Description:We are seeking a seasoned Senior or Lead AWS Cloud Engineer to spearhead the design and development of scalable, secure, and reliable cloud-native applications on AWS.Key Responsibilities:Design and develop scalable, secure, and reliable cloud-native applications on AWS.Lead implementation of containerized...

  • Build Scalable Cloud Platforms

    3 days ago

    Gulbarga, Karnataka, India beBeeCloudEngineer Full time ₹ 1,60,00,000 - ₹ 2,20,00,000

    Cloud Platform Engineer PositionAs a seasoned cloud platform engineer, you will be responsible for designing and building scalable, secure, and reliable cloud platforms.Key Responsibilities:Implement and maintain cloud-based compute offerings based on industry best practices.Manage subscriptions, resource groups, and resources following architecture...

  • Building Scalable Machine Learning Systems

    19 hours ago

    Gulbarga, Karnataka, India beBeeMachineLearningEngineer Full time ₹ 99,99,000 - ₹ 1,99,99,999

    Machine Learning EngineerAbout the Role:We are seeking a highly skilled Machine Learning Engineer to join our team. As a key member of our organization, you will play a crucial role in ensuring the scalability of our machine learning products and credit risk policies within pre-approved/pre-qualified campaigns and live digital journeys.This entails ensuring...

  • Scalable Data Pipeline Expert

    5 days ago

    Gulbarga, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineer RoleWe are seeking a seasoned Senior Data Engineer to spearhead the design, development, and maintenance of scalable data pipelines using PySpark. This is an exceptional opportunity for someone with strong expertise in PySpark and hands-on experience in building robust data pipelines.

  • Scalable Data Solutions Expert

    5 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OpportunityWe are seeking an experienced Data Engineer to design, build, and optimize scalable data solutions using Google Cloud Platform (GCP) and AtScale.The ideal candidate will have hands-on experience in developing robust data pipelines, implementing efficient data models, and ensuring performance optimization for enterprise-level analytics.Key...

  • Strategic Data Infrastructure Leader

    6 days ago

    Gulbarga, Karnataka, India beBeeDataEngineer Full time ₹ 19,62,500 - ₹ 24,35,000

    Senior Data Engineer JobAbout this RoleWe are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data. You will report to the Head of AI and Engineering, responsible for designing, building, and operating the backbone of our AI-powered applications.This is a critical role where you will create scalable...