Senior Distributed Graph Computing Professional

Job OverviewWe are seeking a highly skilled professional to lead the development of distributed graph computing solutions processing billions of entities and relationships for a next-generation data intelligence platform using a microservices architecture.The role is a hands-on technical leadership position focused on architecting and implementing distributed graph computing solutions processing billions of entities and relationships for a next-generation data intelligence platform using a microservices architecture.The engineer is expected to be performance-obsessed, think in distributed systems, optimize for latency and throughput, and act as a self-directed technical leader who makes architectural decisions and implement them.Main Responsibilities:Design and implement distributed entity resolution algorithms to process billions of records.Build blocking strategies (e.g., LSH, canopy clustering) optimized for Spark at scale.Create AI/ML based advanced matching with explainable AI (XAI).Implement incremental resolution supporting real-time and batch modes.Design APIs for entity lookup with sub-100ms latency requirements.Assist the team in building dynamic entity resolution with embedded graph knowledge of GNN and GT (Graph transformers).Architect distributed graph generation pipelines using libraries such as GraphX/GraphFrames.Implement graph analytics algorithms such as PageRank, community detection, and centrality measures.Design storage strategies for multi-billion edge graphs in the cloud.Build temporal graph support for time-evolving networks.Create high-performance graph serving APIs with complex query capabilities.Optimize graph partitioning to minimize shuffle and maximize locality.Build Graph Neural Networks (GNNs) and models using neighbour sampling techniques such as GraphSAGE to analyse corporate and transaction networks for fraud and risk patterns.Implement AI based Entity Resolution algorithms using techniques such as fuzzy matching, semantic matching (Sentence-BERT), and clustering across heterogeneous data sources.Create Risk Scoring Models by combining rule-based, supervised and unsupervised methods, optimized for real-time and large data processing.Champion model transparency and fairness by integrating state-of-art models evaluation techniques such as SHAP, LIME, bias detection and monitoring model drifts.Ensure seamless integration between entity resolution and network generation services.Implement comprehensive monitoring and observability.Contribute to API design and service contracts.RequirementsTo succeed in this role, you should have:3+ years of experience in distributed computing and big data systems.2+ years' experience in entity resolution and graph analytics at scale.Strong understanding of blocking algorithms, probabilistic record linkage, and similarity measures.Ability to design and optimise large Apache Spark workloads.Ability to implement graph algorithms in distributed computing frameworks such as Apache Spark, Dask or equivalent with optimised queries and storage formatsFamiliar with Elasticsearch or equivalent for fast search and pattern matching.Understanding of AI/ML applications to entity resolution.Preferred ExperienceWe would love to see experience in:Master or PhD in Computer Science or related field with focus on graphs/entity resolution.Experience with Architecture Design and implementation of large scaled Graph databases (Neo4j, Amazon Neptune, JanusGraph) or equivalent.Track record in building systems processing millions of entities/edges.Experience in designing microservices architectures, fault-tolerant, scalable systems, and API design.Experience with real-time stream processing (Kafka, Flink, Apache Beam, Apache Spark Streaming etc…)Basic experience with Banking compliances (Financial Crime, Fraud)Why Work with Us?This is a unique opportunity for the right candidate to solve intricate technical challenges, push the boundaries of what's possible with distributed graph computing and build a platform that fights financial crime with:Impact: You will shape the architecture of an essential platform for enterprise decision-making.Innovation: The role offers the chance to work on advanced network graph technologies, big data pipelines, and cutting-edge AI/ML technologies.Growth: You will influence core engineering and play a central role in building the early team.Learning: We provide comprehensive onboarding on our ecosystem. It offers continuous learning opportunities and mentorship from senior Data Analytics & AI practitioners.Flexibility & Equity: This is your opportunity to be more than just an employee and become an entrepreneur in the true sense where you have significant upside as our organization grows into the future.],