Data Intelligence Platform Architect

2 days ago

Amrāvati, Maharashtra, India beBeeAI Full time US$ 1,50,000 - US$ 2,00,000
About the Role

Our organization is creating a next-generation data intelligence platform built on microservices architecture. We are seeking an experienced architect to lead the development of our Entity Resolution and Network Generation services.

The Platform

We leverage cloud-native technology with Kubernetes orchestration, Apache Spark for distributed processing, Elasticsearch for real-time search and fuzzy matching, Scala as our primary development language, Data mesh principles with API-first design.

Key ResponsibilitiesEntity Resolution Service
  • Design and implement scalable entity resolution algorithms capable of processing large datasets.
  • Build efficient blocking strategies (e.g. LSH, canopy clustering) optimized for Spark at scale.
  • Develop fuzzy matching algorithms leveraging Elasticsearch's capabilities.
  • Create ML-enhanced matching with explainable AI for match decisions.
  • Implement incremental resolution supporting real-time and batch modes.
  • Design APIs for entity lookup with sub-100ms latency requirements.
Network Generation Service
  • Architect distributed graph generation pipelines using GraphX/GraphFrames.
  • Implement graph analytics algorithms (PageRank, community detection, centrality measures).
  • Design storage strategies for multi-billion edge graphs in Parquet/distributed file systems.
  • Build temporal graph support for time-evolving networks.
  • Create high-performance graph serving APIs with complex query capabilities.
  • Optimize graph partitioning to minimize shuffle and maximize locality.
Ai Model Development
  • Build Graph Neural Networks (GNNs): Develop GNN models (e.g., GraphSAGE, GATv2) using PyTorch Geometric or DGL to analyze corporate and transaction networks, detecting fraud rings and risk patterns.
  • Implement Entity Resolution: Design algorithms for fuzzy matching, semantic matching (Sentence-BERT), and clustering to unify entities across heterogeneous data sources (e.g., CSVs, APIs, PDFs).
  • Create Risk Scoring Models: Combine rule-based, supervised (XGBoost), and unsupervised (Isolation Forest) methods to generate composite risk scores, optimized for real-time and large data processing.
  • Advance Composite Ai: Leverage proprietary approach integrating symbolic Ai, vector embeddings, and graph Ai for robust entity resolution and network analytics.
Explainable Ai (Xai)
  • Champion Transparency: Integrate SHAP, LIME, and GNNExplainer to provide clear, interpretable explanations for model predictions, meeting regulatory and ethical standards.
  • Ensure Fairness: Audit models for bias and fairness, embedding ethical principles into every stage of development.
Cross-Service Responsibilities
  • Ensure seamless integration between entity resolution and network generation.
  • Design data lineage tracking across both services.
  • Implement comprehensive monitoring and observability.
  • Contribute to API design and service contracts.
  • Optimize for 10x scale growth.

  • Senior Enterprise Data Architect

    1 day ago

    Amrāvati, Maharashtra, India beBeeDataArchitect Full time ₹ 60,00,000 - ₹ 1,25,00,000

    Key Data PositionWe are seeking an expert data architect to develop, govern and scale our data architecture and pipelines.Responsibilities:Design logical and physical data models with collaboration from stakeholders.Develop robust data architectures for warehousing and enterprise platforms.Create scalable workflows using Fivetran and Cloud Composer.Essential...

  • Highly Skilled Artificial Intelligence Professional Wanted

    22 hours ago

    Amrāvati, Maharashtra, India beBeeArtificial Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Artificial Intelligence EngineerWe are seeking a highly skilled Artificial Intelligence Developer with 4–6 years of experience in applying Large Language Models (LLMs) and building AI-driven applications to join our growing team. A significant part of this role involves designing and developing AI Agents within our platform with an initial focus on...

  • Data Architect Opportunity

    17 hours ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Cloud Data Architect RoleWe are seeking an experienced Cloud Data Architect to work remotely on designing, governing, and implementing conceptual, logical, and physical data models across modern cloud platforms.The ideal candidate will have strong experience with SQL Data Warehouses, Databricks Delta Lakehouse, and other MPP databases, combined with domain...

  • Strategic Cloud Data Platform Specialist

    2 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Cloud Data EngineerWe are seeking a skilled Cloud Data Engineer with experience in designing, developing, and maintaining scalable cloud data platforms. The ideal candidate will have expertise in Databricks, AWS, PL/SQL, SQL, and data lake architecture.Key Responsibilities:Design and develop robust data pipelines using ETL tools with ingestion into S3 and...

  • Principal Data Engineering Professional

    1 day ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Business Data ArchitectWe are seeking a skilled Business Data Architect to design, build, and optimize scalable data platforms. The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanced analytics initiatives.Design, build, and maintain ETL/ELT pipelines using...

  • Data Solutions Architect

    4 days ago

    Amrāvati, Maharashtra, India beBeeSolution Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We are seeking a seasoned architect to drive the design and delivery of cutting-edge data solutions. You will work closely with cross-functional teams to understand project requirements, develop high-level architectures, and oversee full lifecycle implementations—ensuring that solutions are scalable, secure, and aligned to business objectives.This role...

  • AI Solutions Architect

    1 day ago

    Amrāvati, Maharashtra, India beBeeArtificialintelligence Full time US$ 1,20,000 - US$ 1,60,000

    AI Solutions ArchitectThis is a pivotal role where you will spearhead the development of cutting-edge AI solutions to drive business growth.Design and implement machine learning models for complex business problems.Create scalable AI pipelines for data ingestion, preprocessing, training, testing, deployment, and monitoring.Collaborate with cross-functional...

  • Senior Data Solutions Architect

    1 day ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 40,00,000

    Job DescriptionAs a Technical Lead at datae2e-technologies, you will be responsible for leading a team of talented data engineers to design and deliver scalable data pipelines and analytics platforms.The ideal candidate will have deep expertise in Big Data platforms, Microsoft Azure, and Databricks. You will oversee the development of end-to-end data...

  • Chief Artificial Intelligence Architect

    12 hours ago

    Amrāvati, Maharashtra, India beBeeEngineering Full time US$ 10,00,000 - US$ 15,00,000

    Senior GenAI Engineer RoleWe are seeking an experienced GenAI Engineer with expertise in Prompt Engineering, Retrieval-Augmented Generation (RAG), and Vector Search Systems to integrate GenAI into our engineering ecosystem.This is a contractor role where you will design intelligent assistants & agents to augment CI/CD workflows, knowledge retrieval, incident...

  • Network Architect

    3 weeks ago

    Amrāvati, Maharashtra, India Programming Full time

    Job Title: Network Architect (Network Traffic Intelligence & Flow Data Systems)Location: Pune, India (with Travel to Onsite)Experience Required:8+ years in network traffic monitoring and flow data systems, with 2+ years hands-on experience in configuring and deploying nProbe Cento in high-throughput environments.Overview:We are seeking a specialist with...