Data Intelligence Engineer

5 days ago

Vadodara, Gujarat, India beBeeArtificialintelligence Full time US$ 2,00,000 - US$ 2,50,000

About the Role

We are seeking a senior expert to lead the development of our Entity Resolution and Network Generation services within a next-generation data intelligence platform built using microservices architecture.

The Platform

The cloud-native platform leverages:

  • Microservices architecture with Kubernetes orchestration
  • Apache Spark for distributed processing
  • Elasticsearch for real-time search and fuzzy matching
  • Scala as the primary development language
  • Data mesh principles with API-first design

Core Responsibilities

Entity Resolution Service

  • Design and implement distributed entity resolution algorithms capable of processing billions of records
  • Build blocking strategies (e.g. LSH, canopy clustering) optimized for Spark at scale
  • Develop fuzzy matching algorithms leveraging Elasticsearch's capabilities
  • Create ML-enhanced matching with explainable AI for match decisions
  • Implement incremental resolution supporting real-time and batch modes
  • Design APIs for entity lookup with sub-100ms latency requirements

Network Generation Service

  • Architect distributed graph generation pipelines using GraphX/GraphFrames
  • Implement graph analytics algorithms (PageRank, community detection, centrality measures)
  • Design storage strategies for multi-billion edge graphs in Parquet/distributed file systems
  • Build temporal graph support for time-evolving networks
  • Create high-performance graph serving APIs with complex query capabilities
  • Optimize graph partitioning to minimize shuffle and maximize locality

AI Model Development

  • Build Graph Neural Networks (GNNs): Develop GNN models (e.g., GraphSAGE, GATv2) using PyTorch Geometric or DGL to analyze corporate and transaction networks, detecting fraud rings and risk patterns.
  • Implement Entity Resolution: Design algorithms for fuzzy matching, semantic matching (Sentence-BERT), and clustering to unify entities across heterogeneous data sources (e.g., CSVs, APIs, PDFs).
  • Create Risk Scoring Models: Combine rule-based, supervised (XGBoost), and unsupervised (Isolation Forest) methods to generate composite risk scores, optimized for real-time and large data processing in trillions.
  • Advance Composite AI: Leverage proprietary approach, integrating symbolic AI, vector embeddings, and graph AI for robust entity resolution and network analytics.

Explainable AI (XAI)

  • Champion Transparency: Integrate SHAP, LIME, and GNNExplainer to provide clear, interpretable explanations for model predictions, meeting regulatory and ethical standards.
  • Ensure Fairness: Audit models for bias and fairness, embedding ethical principles into every stage of development.

Cross-Service Responsibilities

  • Ensure seamless integration between entity resolution and network generation
  • Design data lineage tracking across both services
  • Implement comprehensive monitoring and observability
  • Contribute to API design and service contracts
  • Optimize for 10x scale growth

  • Gen AI Engineer

    9 hours ago

    Vadodara, Gujarat, India beBeeDataIntelligence Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Gen AI EngineerAbout the RoleWe're seeking a highly skilled Data Intelligence Platform Specialist to join our team.Design and implement scalable data intelligence platforms that integrate structured and unstructured data sources.Collaborate with cross-functional teams to deploy AI solutions into production environments.Build and maintain data...

  • Space Intelligence Engineer

    2 days ago

    Vadodara, Gujarat, India beBeeMachineLearning Full time ₹ 2,00,00,000 - ₹ 3,00,00,000

    At Little Place Labs, we're revolutionizing the way satellites process and deliver intelligence from orbit.We specialize in satellite edge computing, developing cutting-edge solutions that process data directly on board, enabling real-time analysis and rapid decision-making.As a key member of our development team, you'll work on the space data analytics...

  • Artificial Intelligence Engineer

    3 weeks ago

    Vadodara, Gujarat, India Apidel Technologies Full time

    Responsibilities :- Design and develop scalable and robust AI models and algorithms.- Implement and optimize machine learning and deep learning models for various applications.- Develop and deploy AI-powered applications and services.- Integrate AI solutions with existing systems and platforms.- Conduct research and experimentation to explore new AI...

  • High Performance Artificial Intelligence Engineer

    2 days ago

    Vadodara, Gujarat, India beBeeArtificialIntelligence Full time ₹ 1,50,00,000 - ₹ 2,40,00,000

    AI/ML Engineers drive innovation in software development, leveraging machine learning to create cutting-edge solutions. As an AI Engineer, you will play a pivotal role in developing AI-native software that enhances creativity and productivity for teams and enterprises.Key Responsibilities:Design and develop AI-enabled products fully enabled by Large Language...

  • Azure Cloud Data Engineer

    2 days ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Cloud Data Solutions EngineerLocation: RemoteNotice Period: Immediate Starters OnlyExperience: 4–6 YearsWe are looking for an experienced professional to work in a hybrid development + support role. You will be responsible for designing, building, and maintaining data & analytics solutions using cloud-based technologies with a strong focus on...

  • Business Intelligence Analyst

    9 hours ago

    Vadodara, Gujarat, India beBeeDataScientist Full time ₹ 15,00,000 - ₹ 22,50,000

    Senior Business Intelligence AnalystSeeking a detail-oriented and analytical individual to contribute to our team as a Senior Business Intelligence Analyst. The ideal candidate will possess excellent problem-solving skills, with the ability to derive insights from complex data sets.

  • Business Intelligence Developer with Data Governance Expertise

    2 days ago

    Vadodara, Gujarat, India beBeeDataGovernance Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Summary">We are seeking a skilled Business Intelligence developer to join our team. As a key player in our organization, you will be responsible for designing and implementing end-to-end data models and pipelines in SAP Datasphere.Key Responsibilities:">Design and implement data models and pipelines in SAP Datasphere.Integrate SAP and non-SAP sources...

  • Artificial Intelligence Engineer

    3 weeks ago

    Vadodara, Gujarat, India Successive Digital Full time

    About Us:Successive Digital, a digital transformation company, offers a comprehensive suite of solutions, including digital strategy, product engineering, CX, Cloud, Data & AI, and Generative AI services. We help companies continuously optimize business and technology that transform how they connect with customers and grow their business. Our team of...

  • Data Engineer

    7 days ago

    Vadodara, Gujarat, India Allata Full time US$ 1,50,000 - US$ 2,00,000 per year

    Allata is a global consulting and technology services firm with offices in the US, India, and Argentina. We help organizations accelerate growth, drive innovation, and solve complex challenges by combining strategy, design, and advanced technology. Our expertise covers defining business vision, optimizing processes, and creating engaging digital experiences....

  • Data Engineer

    3 weeks ago

    Vadodara, Gujarat, India Kyara Consulting Full time

    Job Title Data EngineerExperience 1 5 to 2 YearsLocation Shyaml cross road AhmedabadEmployment Type Full-time We are looking for a Data Engineer with 1 5 to 2 years of experience who has expertise in Python and data scraping The ideal candidate will be responsible for building and optimizing data pipelines extracting and processing data from various...