Senior Backend Data Pipeline Specialist

17 hours ago

Bhavnagar, Gujarat, India beBeeBackend Full time ₹ 1,20,00,000 - ₹ 2,50,00,000
Backend Data Engineer Role

This is a pivotal position within our organization, focused on building data pipelines to connect external systems and interfaces. The ideal candidate will possess expertise in developing crawling/fetch pipelines using API-first approaches.

The responsibilities of this role include parsing/normalization of job postings & CVs, deduplication/delta logic, embeddings/similarity search, integrations with various platforms such as HR4YOU, SerpAPI, and BA job board, batch/stream processing, and telemetry for data quality.

Key Responsibilities:
  • Developing data pipelines to fetch and process information from various sources
  • Parsing and normalizing job postings and CVs for effective analysis
  • Implementing deduplication and delta logic to ensure data accuracy
  • Building embeddings and similarity search capabilities using Azure OpenAI and pgvector
  • Integrating with HR4YOU, SerpAPI, and other platforms to enhance data collection
  • Processing data in batches or streams using Azure Functions and container jobs
  • Maintaining data quality through telemetry and analytics
Requirements:
  • 4+ years of experience in backend/data engineering
  • Proficiency in Python (FastAPI, pydantic, httpx/requests, Playwright/Selenium), TypeScript for smaller services/SDKs
  • Familiarity with Azure (Functions/Container Apps or AKS jobs, Storage/Blob, Key Vault, Monitor/Log Analytics)
  • Experience with messaging queues (Service Bus/Queues) and idempotence/exactly-once semantics
  • Expertise in databases (PostgreSQL, pgvector, query design & performance tuning)
  • Understanding of ETL/ELT patterns, testability (pytest), and observability (OpenTelemetry)

  • Data Pipeline Architect

    8 hours ago

    Bhavnagar, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerAbout the Role:We are seeking a skilled professional to lead the development and operation of data pipelines that support AI-powered applications. As a Senior Data Engineer, you will work closely with software engineers, ML specialists, and clinical partners to design, build, and scale data infrastructure that meets the needs...

  • Senior Data Pipeline Developer

    7 hours ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Senior Data Pipeline DeveloperWe are seeking an experienced data pipeline developer to join our team. As a senior member, you will be responsible for designing and implementing scalable and efficient data pipelines using IBM DataStage and AWS Glue.The ideal candidate will have a strong background in ETL development and experience with cloud-based...

  • Backend Data Support Specialist

    3 days ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 12,00,000 - ₹ 16,00,000

    Job Title: Backend Data Support SpecialistJob Overview:The ideal candidate will excel in data analysis, tracking and presentation development while ensuring seamless daily operations.A strong proficiency in MIS, PowerPoint and Excel is essential for success in this role.Manage MS Suite with easeEfficiently manage ATS tools such as recruitment softwareAnalyse...

  • Senior Backend Infrastructure Specialist

    4 days ago

    Bhavnagar, Gujarat, India beBeeBackendDeveloper Full time ₹ 12,00,000 - ₹ 16,00,000

    Backend Developer Job DescriptionAbout this role:We are looking for a skilled backend developer to join our team. You'll play a key role in transforming raw on-chain data into actionable insights by decoding smart contract events and implementing pricing logic from decentralized exchanges.This is an ideal opportunity for someone who enjoys low-level protocol...

  • Senior Backend

    7 days ago

    Bhavnagar, Gujarat, India RightShop Full time

    Type: Part-Time - Project-Based workRightShop is building the world's most trusted engine of transparent wellness.We're looking for a Backend Engineer with a systems thinker mindset. Someone who understands how and why data flows, where bottlenecks occur, and how to build reliable solutions. This is not a "just code what's asked" role, ideally you'll help...

  • Data Pipeline Architect

    2 weeks ago

    Bhavnagar, Gujarat, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 18,00,000

    Job Title: Data Engineering SpecialistWe are seeking an accomplished professional to lead the design, development, and maintenance of scalable data pipelines.About the Role:As a key member of our team, you will collaborate with cross-functional stakeholders to deliver high-quality data assets that support business-critical analytics and AI/ML...

  • Senior Data Pipeline Specialist

    5 days ago

    Bhavnagar, Gujarat, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000

    Data Engineer Role SummaryThe ideal candidate should have 4+ years of IT experience with a good exposure to diagnosis and troubleshooting complex issues.Key ResponsibilitiesDesign, develop, and deploy data ingestion & ETL pipelines using cloud-based technologies.Work on large-scale datasets using big data analytics platforms.Develop and maintain continuous...

  • Senior Backend Data Specialist

    2 days ago

    Bhavnagar, Gujarat, India beBeeBackendDeveloper Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job OverviewOur organization is seeking a highly skilled Backend Developer to join our team. As a key member of our development team, you will play a crucial role in designing, building, and maintaining scalable data workflows.Key ResponsibilitiesDesign and implement backend services and APIs that integrate seamlessly with SQL databases.Develop and maintain...

  • Backend Infrastructure Specialist

    1 week ago

    Bhavnagar, Gujarat, India beBeeBackend Full time ₹ 40,00,000 - ₹ 80,00,000

    Job OpportunityWe are building a next-generation AI-powered Search & Engagement platform that will transform static search into a guided, personalized experience.About the RoleUsers will see tailored swimlanes of content, recommendations, and engagement features. The Backend Engineer will help design and implement the APIs, data flows, and integrations on...

  • Cloud Data Pipeline Specialist

    4 days ago

    Bhavnagar, Gujarat, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Cloud Data EngineerAs a Cloud Data Engineer, you will play a key role in developing and maintaining large-scale data pipelines on the cloud.Key Responsibilities:Design and develop complex data pipelines using Python, Data Flow, DataStream, CDC, Cloud Functions, Cloud Run, Pub Sub, BigQuery, and Cloud Storage.Utilize DevOps and CI/CD practices to ensure the...