Developing Scalable Data Pipelines

2 days ago

Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000
Job Title

Senior Data Engineer Opportunity

We are seeking an experienced Senior Data Engineer to join our team and play a crucial role in developing and maintaining large-scale data acquisition pipelines using Google Cloud technologies. The ideal candidate will have a strong background in data engineering, excellent problem-solving skills, and the ability to work collaboratively with stakeholders and clients.

Key Responsibilities:
  • Design and develop efficient, scalable, and secure data pipelines that meet business requirements.
  • Construct complex datasets based on business needs, leveraging tools like Python, Data Flow, and BigQuery.
  • Build 'big data' pipeline architecture and identify opportunities for data acquisition through collaboration with stakeholders and clients.
  • Leverage various tools in the Google Cloud Ecosystem to integrate systems and data pipelines.
  • Use logs and alerts to effectively monitor pipelines and ensure reliability and scalability through DevOps and CI/CD practices (GitHub, Terraform).
  • Partition and retrieve content in Big Query, securing data with IAM roles and Policy Tags.
  • Design and build an ingestion pipeline using Rest API and recommend ways to improve data quality, reliability, and efficiency.
Requirements:
  • Junior DE: 2-3 years of experience in a GCP data engineering role (offshore)
  • Senior DE: 4+ years of experience in a GCP data engineering role (offshore)
  • At least 1 year of hands-on experience in GCP tools (Python, SQL, Data Flow, DataStream, CDC, Cloud Functions, Cloud Run, Pub Sub, BigQuery, Cloud Storage)

  • Build Scalable Data Pipelines

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the RoleOur organization is seeking a Senior Data Engineer to lead data engineering efforts and build scalable, event-driven data pipelines.This critical role involves designing, building, and operating the backbone of AI-powered applications that rely on processing clinical encounter data, retrieving associated medical documents, and delivering...

  • Data Pipeline Specialist

    5 days ago

    Lucknow, Uttar Pradesh, India beBeeSnowflakeDataArchitect Full time ₹ 27,00,000 - ₹ 31,00,000

    Snowflake Data Architect PositionKey Role Summary:We are seeking a highly skilled Snowflake Expert with hands-on experience in Matillion ETL and proven expertise in the Banking, Financial Services domain.The ideal candidate will design, develop, and optimize scalable data pipelines, data models, and analytics solutions on Snowflake, ensuring compliance,...

  • Data Pipeline Architect

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior AWS Data EngineerThis position involves the design, development and deployment of end-to-end data pipelines on AWS cloud infrastructure. The right candidate will have hands-on experience in working with services such as Amazon S3, AWS Glue, AWS Lambda and Amazon Redshift.Responsibilities:Design and develop end-to-end data pipelines on AWS...

  • Chief Data Pipeline Architect

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Data Engineer JobThis is a freelance data engineer position where you will work with us to design and develop large-scale data pipelines.We require an experienced freelancer to collaborate with our teams and clients to deliver robust, scalable, and high-performance data solutions.Key Skills and QualificationsStrong expertise in Python and SQL with...

  • Senior Data Pipeline Specialist

    7 days ago

    Lucknow, Uttar Pradesh, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    About the RoleWe are seeking a skilled Data Engineer to play a key role in designing and developing scalable data pipelines, distributed systems, and ensuring the smooth operation of our data infrastructure.

  • Senior Data Pipeline Architect

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 20,10,000

    We are seeking a skilled professional to design, build, and optimize scalable data pipelines in our organization.Key responsibilities include:Designing, building, and maintaining ETL/ELT pipelines using AWS Glue, Lambda, and DBT.Developing and optimizing data models in Snowflake for scalability and performance.Writing efficient and optimized SQL queries for...

  • Building Scalable Data Systems

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,50,000

    Job Title: Data Solutions ArchitectRole SummaryThe role of a Data Engineer involves designing, building, and optimizing data pipelines and ETL workflows. This includes managing and processing large datasets using Spark, PySpark, and SQL.Key ResponsibilitiesDesign and build scalable data pipelines to meet business requirements.Collaborate with...

  • Chief Data Pipeline Architect

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,50,000 - US$ 2,00,000

    We are seeking a seasoned professional to spearhead the development of scalable data pipelines. The ideal candidate will have expertise in AWS cloud services and Databricks.Key Responsibilities:The Data Engineer will design, develop, and maintain data pipelines using AWS and Databricks.They will build and optimize ETL/ELT processes for large-scale...

  • Senior Data Pipeline Specialist

    4 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    About the RoleWe are looking for skilled professionals to design, develop, and manage data pipelines and transformation frameworks.As a key member of our team, you will work closely with data architects, platform engineers, and DevOps teams to build scalable, resilient, and efficient data engineering solutions using modern tools and best practices.Design and...

  • Senior Data Pipeline Developer

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineering Full time US$ 75,000 - US$ 1,50,000

    As a Data Engineering Specialist, you will play a pivotal role in designing and implementing efficient data pipelines using Azure Data Factory.About the Role:You will collaborate with clients to comprehend their data needs and requirements.You will provide actionable insights through data visualization.Your Key Responsibilities:Design and implement data...