Data Intelligence Architect

13 hours ago

Gandhinagar, Gujarat, India beBeeMachineLearning Full time US$ 2,50,000 - US$ 3,50,000
Senior AI/ML Engineer

About this Role

We're building a next-generation data intelligence platform using a microservices architecture, and we need a senior expert to lead the development of our Entity Resolution and Network Generation services. This is a hands-on technical leadership role where you'll architect and implement distributed graph computing solutions processing billions of entities and relationships.

The Platform

Our cloud-native platform leverages:

  • Microservices architecture with Kubernetes orchestration
  • Apache Spark for distributed processing
  • Elasticsearch for real-time search and fuzzy matching
  • Scala as the primary development language
  • Data mesh principles with API-first design

Core Responsibilities

Entity Resolution Service

Design and implement distributed entity resolution algorithms capable of processing billions of records

Build blocking strategies (e.g., LSH, canopy clustering) optimized for Spark at scale

Develop fuzzy matching algorithms leveraging Elasticsearch's capabilities

Create ML-enhanced matching with explainable AI for match decisions

Implement incremental resolution supporting real-time and batch modes

Design APIs for entity lookup with sub-100ms latency requirements

Network Generation Service

Architect distributed graph generation pipelines using GraphX/GraphFrames

Implement graph analytics algorithms (PageRank, community detection, centrality measures)

Design storage strategies for multi-billion edge graphs in Parquet/distributed file systems

Build temporal graph support for time-evolving networks

Create high-performance graph serving APIs with complex query capabilities

Optimize graph partitioning to minimize shuffle and maximize locality

AI Model Development

Build Graph Neural Networks (GNNs): Develop GNN models (e.g., GraphSAGE, GATv2) using PyTorch Geometric or DGL to analyze corporate and transaction networks, detecting fraud rings and risk patterns.

Implement Entity Resolution: Design algorithms for fuzzy matching, semantic matching (Sentence-BERT), and clustering to unify entities across heterogeneous data sources (e.g., CSVs, APIs, PDFs).

Create Risk Scoring Models: Combine rule-based, supervised (XGBoost), and unsupervised (Isolation Forest) methods to generate composite risk scores, optimized for real-time and large data processing in trillions.

Advance Composite AI: Leverage ContexQ's proprietary approach, integrating symbolic AI, vector embeddings, and graph AI for robust entity resolution and network analytics.

Explainable AI (XAI)

Champion Transparency: Integrate SHAP, LIME, and GNNExplainer to provide clear, interpretable explanations for model predictions, meeting regulatory and ethical standards.

Ensure Fairness: Audit models for bias and fairness, embedding ethical principles into every stage of development.

Cross-Service Responsibilities

Ensure seamless integration between entity resolution and network generation

Design data lineage tracking across both services

Implement comprehensive monitoring and observability

Contribute to API design and service contracts

Optimize for 10x scale growth

  • Data Architect

    13 hours ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    We are seeking a skilled Data Architect to lead the design and development of our data infrastructure.This is a full-time role that involves creating scalable data architectures, developing real-time data processing pipelines using technologies like Kafka, Flink, and Spark. The ideal candidate will have experience with data modeling, ETL processes, and...

  • Cloud Data Architect

    2 days ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Transformative SolutionsWe are crafting a world-class cloud data platform using cutting-edge technologies. As a Cloud Data Architect, you will spearhead the design and development of our data infrastructure, making it scalable, reliable, and cost-efficient.Key Responsibilities:Analyze existing data warehouse implementations to inform migration and...

  • Business Intelligence Specialist

    1 day ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 5,00,000 - ₹ 9,00,000

    Business Intelligence SpecialistThe role of a Business Intelligence Specialist is to analyze data and provide valuable insights that can help inform business decisions.A strong analytical mindset, effective communication skills, and the ability to work independently are highly valued in this position.Developing and maintaining data models to support business...

  • Business Intelligence Professional

    13 hours ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Business Intelligence ExpertWe are seeking a seasoned Business Intelligence expert to join our team.Data Analysis and Reporting:Develop and manage data visualizations using cutting-edge tools such as Tableau, PowerBI, and Excel.Analyze sales trends, market performance, and customer behavior to identify opportunities and risks.Provide actionable insights to...

  • Cloud Architect

    22 hours ago

    Gandhinagar, Gujarat, India beBeeCustomerExperience Full time ₹ 15,00,000 - ₹ 30,00,000

    ">Cloud Architect - Customer Experience Specialist">We are seeking a seasoned Cloud Architect to drive the design and implementation of scalable, high-performance customer experience solutions leveraging Salesforce technologies.">The ideal candidate will bring deep expertise in Salesforce Sales Cloud, Service Cloud, Marketing Cloud, integrations, and...

  • Data Architect and Engineer

    1 day ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 24,00,000

    Job OpportunityWe are seeking a skilled Data Architect and Engineer to join our team.The ideal candidate will have experience designing, developing, and testing ETL processes to ensure data integrity and accuracy.They will collaborate with cross-functional teams to identify business requirements and implement data solutions.The successful candidate will...

  • Chief Data Architect

    1 day ago

    Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 25,12,000

    Job Overview:As a data engineer, you will play a pivotal role in designing and building large-scale data systems that drive business intelligence and advanced analytics initiatives.Key Responsibilities:Develop and maintain ETL/ELT pipelines using AWS Glue, Lambda, and DBT to ensure seamless data integration and transformation.Design and optimize data models...

  • Strategic Data Insights Professional

    3 days ago

    Gandhinagar, Gujarat, India beBeeBusiness Full time ₹ 6,00,000 - ₹ 12,00,000

    Senior Business Intelligence SpecialistWe are seeking a highly skilled Senior Business Intelligence Specialist to join our team. The ideal candidate will have extensive experience in designing and implementing data architectures, as well as providing expert guidance on analytics and data visualization.Key Responsibilities:Develop and maintain complex data...

  • Collibra Data Governance Specialist

    3 weeks ago

    Gandhinagar, Gujarat, India Anblicks Full time

    Job Title: Collibra Data Governance SpecialistLocation: HyderabadJob Type: Full-TimeAbout the Role:We are seeking a highly skilled and experienced Collibra Expert to lead and manage our enterprise-level data governance initiatives using Collibra. This role requires deep expertise in Collibra's platform, including configuration, integration, workflow...

  • Chief Data Strategist

    1 day ago

    Gandhinagar, Gujarat, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    As a data architect, you will be responsible for reimagining scalable data platforms and AI solutions.We are looking for an expert who can help our clients build, optimize, and enablement their data to harness its full potential.The successful candidate will design and implement efficient Databricks solutions, develop data pipelines, architect data lakes,...