Lead Data Pipeline Specialist

4 days ago

Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

Job Summary:

We are seeking a skilled Data Engineer with a strong background in data pipeline architecture, integration, and transformation techniques.

Key Responsibilities:

  • Design, develop, test, and maintain scalable ETL data pipelines using Python.
  • Work extensively on Google Cloud Platform (GCP) services such as:
    • Dataflow for real-time and batch data processing
    • Cloud Functions for lightweight serverless compute
    • BigQuery for data warehousing and analytics
    • Cloud Composer for orchestration of data workflows (based on Apache Airflow)
    • Google Cloud Storage (GCS) for managing data at scale
    • IAM for access control and security
    • Cloud Run for containerized applications
  • Strong experience in Python for backend or data engineering projects.
  • Strong understanding and working experience with GCP cloud services.
  • Solid understanding of data pipeline architecture, data integration, and transformation techniques.
  • Experience in working with version control systems like GitHub and knowledge of CI/CD practices.
  • Strong experience in SQL with at least one enterprise database.

About the Ideal Candidate:

  • Able to demonstrate 46 years of hands-on experience in Python programming.

What We Offer:

  • A collaborative and dynamic work environment.
  • Opportunities for growth and professional development.

  • Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataPipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an accomplished Data Pipeline Specialist to join our team. The successful candidate will be responsible for creating and maintaining data pipelines that drive business growth.Main ResponsibilitiesCreate and implement new data pipelines to meet business needsEssential Skills and Qualifications4-6 years of experience in designing and...

  • Data Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeeDataPipelineManager Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data Team LeadData Pipeline ManagerWe are seeking a seasoned data pipeline manager with experience in designing and developing robust data pipelines using Azure Databricks, Data Factory, and Datalake.The role involves overseeing the development of complex ETL solutions, managing large volumes of data, and working closely with business teams to...

  • Lead Data Pipeline Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 1,55,52,000 - ₹ 2,59,20,000

    Senior Data EngineerWe seek an expert in designing and maintaining large-scale data pipelines on Azure using Databricks, ADF, and PySpark. This individual will ensure high-quality, secure, and reliable datasets for analytics and customer-facing applications.Key ResponsibilitiesDesign, build, and optimise large-scale data pipelines using Databricks, Delta...

  • Lead Data Pipeline Architect

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 1,20,00,000

    About us:We're a leading provider of personalized digital solutions for healthier food choices, driving lasting behavior change and delivering long-term health outcomes.We guide individuals on a tailored journey to eating well while saving time and money through our platform.Our platform integrates dietary assessments and nutrition counseling with online...

  • Lead/Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Principal Pipeline Specialist

    1 week ago

    Hyderabad, Telangana, India beBeePipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Pipeline Engineering ExcellenceWe seek a skilled pipeline specialist to lead the design and development of complex pipeline systems. Key responsibilities include:Developing and reviewing technical documents, such as specifications, data sheets, drawings, and reports.Evaluating pipeline designs for compliance with industry standards (e.g., ASME B31.4, B31.8,...

  • Lead Cloud Data Engineer

    1 week ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,20,000 - US$ 2,00,000 per year

    Key Responsibilities:Engage directly with clients to understand requirements, provide solution design, and drive successful project delivery.Lead cloud migration initiatives, specifically moving on-premise applications and databases to AWS cloud platforms.Design, develop, and maintain scalable, reliable, and secure data applications in a cloud...

  • Data Pipeline Specialist

    5 hours ago

    Hyderabad, Telangana, India beBeeProfessional Full time ₹ 18,00,000 - ₹ 24,00,000

    Job OverviewThe Software Professional will design, build, and maintain complex data pipelines, datasets, dataflows, integration runtimes, and monitor pipeline executions using Azure Data Factory and Azure Synapse.Key ResponsibilitiesExtract, transform, and load (ETL) data from source systems and process large datasets using Azure Databricks.Develop advanced...

  • Data Pipeline Specialist

    5 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 12,00,000

    Data Engineer PositionWe are seeking a skilled data engineer to join our team. The ideal candidate will be responsible for designing, implementing, and optimizing data pipelines on our platform.Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Azure Synapse, Databricks & Apache Spark (PySpark).Data Integration: Integrate...

  • Cloud Data Pipeline Specialist

    4 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job TitleData Engineer Position OverviewWe are seeking an experienced data engineer to lead the development of scalable data pipelines using Apache Spark and PySpark. The ideal candidate will have a deep understanding of cloud-based data processing and hands-on experience with AWS services.Develop and maintain robust ETL/ELT pipelines on AWS using Spark and...