Data Architect

5 days ago

Kozhikode, Kerala, India beBeeMachineLearning Full time US$ 7,50,000 - US$ 15,00,000

Expertise Sought for Next-Generation Data Intelligence Platform

We are building a leading-edge data intelligence platform leveraging microservices architecture. A senior technical leader is required to oversee the development of our Entity Resolution and Network Generation services.

This role will involve hands-on technical leadership where you will design and implement distributed graph computing solutions processing vast amounts of entities and relationships.

The proposed platform will utilize:

Microservices architecture with Kubernetes orchestration.

Apache Spark for distributed processing.

Elasticsearch for real-time search and fuzzy matching.

Scala as the primary development language.

Data mesh principles with API-first design.

Main Responsibilities

  1. Entity Resolution Service:
  2. Design and develop distributed entity resolution algorithms capable of processing billions of records.
  3. Build blocking strategies (e.g., LSH, canopy clustering) optimized for Spark at scale.
  4. Develop fuzzy matching algorithms leveraging Elasticsearch's capabilities.
  5. Create ML-enhanced matching with explainable AI for match decisions.
  6. Implement incremental resolution supporting real-time and batch modes.
  7. Design APIs for entity lookup with sub-100ms latency requirements.
  1. Network Generation Service:
  2. Architect distributed graph generation pipelines using GraphX/GraphFrames.
  3. Implement graph analytics algorithms (PageRank, community detection, centrality measures).
  4. Design storage strategies for multi-billion edge graphs in Parquet/distributed file systems.
  5. Build temporal graph support for time-evolving networks.
  6. Create high-performance graph serving APIs with complex query capabilities.
  7. Optimize graph partitioning to minimize shuffle and maximize locality.
  1. Ai Model Development:
  2. Build Graph Neural Networks (GNNs): Develop GNN models (e.g., GraphSAGE, GATv2) using PyTorch Geometric or DGL to analyze corporate and transaction networks, detecting fraud rings and risk patterns.
  3. Implement Entity Resolution: Design algorithms for fuzzy matching, semantic matching (Sentence-BERT), and clustering to unify entities across heterogeneous data sources (e.g., CSVs, APIs, PDFs).
  4. Create Risk Scoring Models: Combine rule-based, supervised (XGBoost), and unsupervised (Isolation Forest) methods to generate composite risk scores, optimized for real-time and large data processing.
  5. Advance Composite Ai: Leverage ContexQ's proprietary approach, integrating symbolic AI, vector embeddings, and graph AI for robust entity resolution and network analytics.

Explainable Ai (Xai)

Champion Transparency: Integrate SHAP, LIME, and GNNExplainer to provide clear, interpretable explanations for model predictions, meeting regulatory and ethical standards.

Ensure Fairness: Audit models for bias and fairness, embedding ethical principles into every stage of development.

Cross-Service Responsibilities

Ensure seamless integration between entity resolution and network generation.

Design data lineage tracking across both services.

Implement comprehensive monitoring and observability.

Contribute to API design and service contracts.

Optimize for 10x scale growth.

  • Cloud Data Architect

    15 hours ago

    Kozhikode, Kerala, India beBeeDataArchitect Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Cloud Data Architect Role OverviewWe are seeking a highly skilled and experienced Cloud Data Architect to join our team. The ideal candidate will have a strong background in designing, developing, and managing enterprise data solutions on Google Cloud Platform (GCP).The successful candidate will be responsible for:Designing and architecting end-to-end data...

  • Lead Data Architect

    1 day ago

    Kozhikode, Kerala, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job Title:Data Systems Architect Job Description:We are seeking an experienced Data Systems Architect to lead the design, development, and maintenance of large-scale data systems.Key Responsibilities:Distributed System Expertise: Proven experience in designing and implementing distributed systems is essential. Strong hands-on knowledge of SQL performance...

  • Senior Data Architect – Cloud, Data Lake

    4 weeks ago

    Kozhikode, Kerala, India Cube Hub Inc. Full time

    Job Title: Data Architect – Cloud, Data Lake InfrastructureDuration: 4 monthsPosition is 100% remote, offshore (India)Job Description:Prefer minimum of 5-15 years of experience.Would consider 3-4 years of experience , especially if they have previous healthcare / biotech/ pharma experience.Candidates MUST be fluent in English, big requirement.This project...

  • Chief Data Architect

    7 days ago

    Kozhikode, Kerala, India beBeeDataEngineering Full time US$ 1,20,000 - US$ 1,50,000

    Design and Deliver Innovative Data SolutionsWe are seeking skilled professionals to join our dynamic team and contribute to our mission of harnessing the power of data to drive growth and success.As a key member of our data engineering team, you will play a critical role in designing, implementing, and maintaining scalable data infrastructure to support our...

  • Data Systems Architect

    3 days ago

    Kozhikode, Kerala, India beBeeDataArchitect Full time ₹ 10,00,000 - ₹ 15,00,000

    Job DescriptionA data architect is required to be proactive in designing, building and maintaining data systems. The role also requires cross-functional collaborations and ensuring the highest standards of data quality and performance by extending their expertise in data engineering, data architecture, pipeline creation and big data technologies.The role...

  • Technical architect data/ai

    3 weeks ago

    Kozhikode, Kerala, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TATA Consultancy ServicesThank you for expressing your interest in exploring a career possibility with the TCS Family.Hiring For:- Technical architect data/ai Location: PAN INDIAExperience: 10 to 16yrsRole & responsibilitiesRole Technical architect data/ai (Azure/AWS/Google) Required Technical Skill Set Python, Cloud ML, MLOps,...

  • Highly Skilled Data Systems Architect Wanted

    1 day ago

    Kozhikode, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job TitleData Systems ArchitectRole OverviewWe are seeking a highly skilled data systems architect to join our team. The ideal candidate will have expertise in designing and implementing high-quality, secure, and scalable data pipelines and systems that power reporting, analytics, and operational use cases.This is a remote-first opportunity with occasional...

  • High-Level Data Systems Architect

    22 hours ago

    Kozhikode, Kerala, India beBeeDataArchitect Full time ₹ 10,00,000 - ₹ 12,00,000

    Data Systems ArchitectWe are seeking a skilled Data Systems Architect to design and implement large-scale data systems. The ideal candidate will have expertise in designing, building, and maintaining complex data architectures.Key Responsibilities:System Design: Design scalable data systems on cloud platforms.DBT Experience: Work with DBT to transform and...

  • Enterprise Data Solutions Architect

    1 day ago

    Kozhikode, Kerala, India beBeeData Full time ₹ 9,84,321 - ₹ 12,34,567

    Business Intelligence Analyst: Enterprise Data SolutionsThis critical role supports the migration of enterprise data solutions from HANA to Snowflake.Collaborate with business stakeholders to understand analytics and reporting requirements.Conduct feasibility studies and analyze requirements for technical implementation.Convert functional requirements into...

  • Highly Skilled Data Architect

    3 days ago

    Kozhikode, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job DescriptionWe are seeking an experienced Senior Data Engineer to join our team. As a key member of the data engineering group, you will be responsible for designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.Key Responsibilities:Design, develop, and...