Data Integration Engineer

1 week ago

Cochin, Kerala, India Velodata Global Pvt Ltd Full time

About the job

Designation : Data & Integration Engineer (Python/TypeScript, Azure, Integrations)

Experience : 4 -8 Years

Location : Cochin

Job Summary :

Build data pipelines (crawling/parsing, deduplication/delta, embeddings) and connect external systems and interfaces.

Key Responsibilities :

- Development of crawling/fetch pipelines (API-first; playwright/requests only where permitted)

- Parsing/normalization of job postings & CVs, deduplication/delta logic (seen hash, repost heuristics)

- Embeddings/similarity search (controlling Azure OpenAI, vector persistence in pgvector)

- Integrations: HR4YOU (API/webhooks/CSV import), SerpAPI, BA job board, email/SMTP

- Batch/stream processing (Azure Functions/container jobs), retry/backoff, dead-letter queues

- Telemetry for data quality (freshness, duplicate rate, coverage, cost per 1,000 items)

- Collaboration with FE for exports (CSV/Excel, presigned URLs) and admin configuration

Must Have Requirements :

- 4+ years of backend/data engineering experience

- Python (FastAPI, pydantic, httpx/requests, Playwright/Selenium), solid TypeScript for smaller services/SDKs

- Azure: Functions/Container Apps or AKS jobs, Storage/Blob, Key Vault, Monitor/Log Analytics

- Messaging: Service Bus/Queues, idempotence & exactly-once semantics, pragmatic approach

- Databases: PostgreSQL, pgvector, query design & performance tuning

- Clean ETL/ELT patterns, testability (pytest), observability (OpenTelemetry)

Nice-to-have :

- NLP/IE experience (spaCy/regex/rapidfuzz), document parsing (pdfminer/textract)

- Experience with license/ToS-compliant data retrieval, captcha/anti-bot strategies (legally compliant)

- Working method: API-first, clean code, trunk-based development, mandatory code reviews

- Tools/stack: GitHub, GitHub Actions/Azure DevOps, Docker, pnpm/Turborepo (Monorepo), Jira/Linear, Notion/Confluence

- On-call/support: rotating, "you build it, you run it"

(ref:hirist.tech)

  • Data Integration Engineer

    2 weeks ago

    Cochin, Kerala, India beBeeDataIntegration Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Integration EngineerWe are seeking a skilled Data Integration Engineer to join our team. The ideal candidate will have experience in designing and implementing automated migration pipelines for moving enterprise Kafka workloads to open-source or cloud-native clusters.The successful candidate will be responsible for scripting and provisioning...

  • Data Integration Specialist

    1 week ago

    Cochin, Kerala, India Talent Basket Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Designation: Senior Backend Data & Integration Engineer (Python/TypeScript, Azure, Integrations)Experience: 4 -8 YearsLocation: Cochin(Onsite)Job SummaryBuild data pipelines (crawling/parsing, deduplication/delta, embeddings) and connect external systems and interfaces.Key Responsibilities:• Development of crawling/fetch pipelines (API-first;...

  • Senior Data Integration Specialist

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OpportunityWe are seeking a highly skilled Data Engineer to design, deploy, and manage ETL processes that integrate data from various sources.About the RoleThis is an exciting opportunity for a talented individual with hands-on experience in designing, developing, and maintaining ETL pipelines using Informatica and other tools. The successful candidate...

  • Cloud Data Integration Expert

    2 weeks ago

    Cochin, Kerala, India beBeeDataIntegration Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Azure Integration EngineerUnlock Your Potential in Cloud ComputingThis exciting opportunity involves designing, developing and deploying data integration solutions using Azure Data Factory (ADF), Azure Synapse Analytics, Power BI and SQL Server.Key Responsibilities:Azure Synapse Analytics: Troubleshoot and resolve issues, optimize performance, and tune...

  • Senior Big Data Integration Specialist

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Job Opportunity: Data EngineerWe are seeking a skilled and experienced Apache NiFi Developer/Data Engineer to join our team.The ideal candidate will have a strong background in data engineering, ETL processes, and data integration, with hands-on experience in designing, developing, and managing data pipelines using Apache NiFi.This role involves working...

  • Data Integration Specialist

    2 weeks ago

    Cochin, Kerala, India beBeeReltio Full time ₹ 80,00,000 - ₹ 1,50,00,000

    We are seeking an expert Data Integration Specialist to join our team. The ideal candidate will have hands-on experience implementing, customizing, and supporting Reltio Master Data Management (MDM) solutions.About the RoleThis is a key position that requires a strong understanding of data quality, governance, and scalability for business-critical master...

  • Data Integration Expert

    2 weeks ago

    Cochin, Kerala, India beBeeTalend Full time ₹ 10,00,000 - ₹ 15,00,000

    Talend ETL ProfessionalWe are seeking an experienced Talend ETL expert to join our project team. The ideal candidate will have strong SAP data integration expertise and a proven track record of delivering high-quality solutions.Key Responsibilities:· Design, develop, and deploy Talend ETL workflows to integrate and transform SAP data.· Collaborate with...

  • Data Integration Specialist

    1 week ago

    Cochin, Kerala, India beBeeETLDeveloper Full time ₹ 20,00,000 - ₹ 30,00,000

    About the RoleWe are seeking a skilled Data Integration Specialist to join our team. The ideal candidate will be responsible for designing, developing, and maintaining data integration processes, as well as data warehouse design and modeling, to support our business intelligence initiatives.This role requires proficiency in T-SQL, Azure Data Factory (ADF),...

  • Data Engineer

    2 weeks ago

    Cochin, Kerala, India Panapps International Private Limited Full time

    Job DescriptionWe are seeking a skilled and experienced Apache NiFi Developer/Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, ETL processes, and data integration, with hands-on experience in designing, developing, and managing data pipelines using Apache NiFi. This role will involve working closely with...

  • Data Engineer

    2 weeks ago

    Cochin, Kerala, India Prevalent AI Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    Role purpose Are you passionate about cutting-edge technology and big data? At Prevalent AI, we're at the forefront of building innovative solutions to handle petabyte-scale data. As a Data Engineer, you will play a pivotal role in supporting the development of key data engineering components for our Data Fabric and Exposure Management products.In this...