Highly Experienced Data Engineering Professional Wanted

1 day ago

Chennai, Tamil Nadu, India beBeeBackend Full time ₹ 12,00,000 - ₹ 25,00,000
Senior Backend Data Engineer

We are seeking an experienced Senior Backend Data Engineer to develop and maintain scalable data pipelines using API-first approach.

Key Responsibilities:

  • Design and implement crawling/fetch pipelines utilizing Playwright/Requests.
  • Parsing/normalization of job postings & CVs, deduplication/delta logic with seen hash, repost heuristics.
  • Embeddings/similarity search controlling Azure OpenAI, vector persistence in pgvector.
  • Integrate HR4YOU (API/webhooks/CSV import), SerpAPI, BA job board, email/SMTP services.
  • Develop batch/stream processing with Azure Functions/container jobs, retry/backoff, dead-letter queues.
  • Implement telemetry for data quality: freshness, duplicate rate, coverage, cost per 1,000 items.
  • Collaborate with FE for exports (CSV/Excel, presigned URLs) and admin configuration.

Requirements:

  • 4+ years of backend/data engineering experience.
  • Proficiency in Python (FastAPI, pydantic, httpx/requests, Playwright/Selenium), solid TypeScript for smaller services/SDKs.
  • Azure expertise: Functions/Container Apps or AKS jobs, Storage/Blob, Key Vault, Monitor/Log Analytics.
  • Messaging: Service Bus/Queues, idempotence & exactly-once semantics, pragmatic approach.
  • Databases: PostgreSQL, pgvector, query design & performance tuning.
  • Clean ETL/ELT patterns, testability (pytest), observability (OpenTelemetry).

Nice-to-have:

  • NLP/IE experience (spaCy/regex/rapidfuzz), document parsing (pdfminer/textract).
  • Experience with license/ToS-compliant data retrieval, captcha/anti-bot strategies (legally compliant).
  • Working method: API-first, clean code, trunk-based development, mandatory code reviews.
  • Tools/stack: GitHub, GitHub Actions/Azure DevOps, Docker, pnpm/Turborepo (Monorepo), Jira/Linear, Notion/Confluence.

  • Highly Experienced Data Engineering Leader Wanted

    2 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    **Lead Data Engineer Role Summary**This role presents a fantastic opportunity for an experienced professional to take on a leadership position and contribute their expertise to the success of our organization.We are seeking a highly skilled Lead Data Engineer with extensive experience in designing, developing, and implementing large-scale data systems.Job...

  • Senior Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...

  • Highly Experienced Senior Data Engineering Professional Wanted

    2 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    About the Opportunity:We are seeking an experienced Data Engineer with a proven track record of delivering data integration solutions using Azure Data Factory (ADF) and Python.As a key member of our team, you will design and implement data flows to enhance ETL processes, leveraging your expertise in SQL, orchestration, and ETL tools like Matillion. You will...

  • Experienced Data Professional Wanted

    7 days ago

    Chennai, Tamil Nadu, India beBeeDataAnalyst Full time ₹ 12,00,000 - ₹ 15,00,000

    Job Title: Data AnalystWe are currently seeking an experienced data analyst to join our team in Chennai.The ideal candidate should have hands-on experience in SQL, data visualization tools, and statistical analysis, with a strong understanding of database design and business data flows.Key Responsibilities:Extract, clean, transform, and analyze large...

  • Highly Experienced Accounting Professional Wanted

    3 days ago

    Chennai, Tamil Nadu, India beBeeAccountant Full time US$ 70,000 - US$ 94,000

    **Job Overview:**Indevia Accounting Inc. is a leading provider of accounting services with a specialty in the U.S. Quick Service Restaurant (QSR) industry.As a Senior Accounts Officer, you will be responsible for handling client escalations, managing end-to-end accounting for clients, training staff and completing other duties as may be assigned. You will...

  • Highly Skilled Data Engineering Professional Wanted

    7 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewData Engineering Position: We are seeking an experienced Data Engineer to join our team.This position plays a crucial role in designing, developing, and maintaining data pipelines for ingestion, transformation, and storage of large-scale data.The ideal candidate should have a minimum of 5+ years of experience working with Big Data ecosystems...

  • Senior Data Engineer

    22 hours ago

    Chennai, Tamil Nadu, India beBeeDataEngineering Full time US$ 1,34,400 - US$ 1,83,840

    Job Title: Data Engineering ProfessionalAre you a highly skilled data engineering professional looking for a challenging role in a dynamic organization? We are seeking an experienced data engineer to join our team as a Senior Data Engineer. This is an exciting opportunity to work on large-scale data engineering projects, architecting and implementing...

  • Highly Experienced Technical Professional Wanted

    3 days ago

    Chennai, Tamil Nadu, India beBeeSoftwareDeveloper Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Technical Staff MemberWe are looking for a highly experienced professional to work on designing and developing large-scale distributed applications.The successful candidate will have expertise in payment gateway development and integration, as well as experience with NoSQL technology and strong communication and analytical skills.Key Responsibilities:Design...

  • Highly Skilled Data Engineer Wanted

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeEngineering Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Software Engineering RoleWe are seeking a highly skilled professional to join our team.Key Responsibilities:Create and maintain customer data records in ERP systems.Ensure accurate setup of customer profiles including billing/shipping addresses, contact information.Develop efficient solutions to improve data management processes.Analyze and resolve complex...

  • Highly Skilled Data Engineer Wanted

    1 day ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 13,00,000 - ₹ 17,00,000

    \\\Data Engineering Role OverviewAs a Data Engineer, you will play a pivotal role in shaping the future of our data-driven solutions. This is an exciting opportunity to leverage your technical expertise and passion for innovation to drive business growth and success.We are seeking a highly skilled Data Engineer to join our team on a high-profile project. In...