LongMemEval Optimization

Bridging the 20-30% gap to world record (91.4%)

📊 Current Baseline

60-70%

Estimated MB Score

20-30%

Gap to Close

91.4%

Target (Hindsight)

🚀 Quick Wins (Start Here)

Add message_order to exhaustive_intelligence

Temporal queries instantly work

1 hour

functions/exhaustiveExtraction

Track belief_version in facts/decisions

Know which version is current

2 hours

entities/BottleMetadata

Session markers in Soul Bottle

Cross-session continuity tracking

1 hour

functions/updateMasterBottle

Temporal similarity in stacking

Better context for time-based queries

3 hours

functions/uncorkWithStacking

Phase 1: Temporal Reasoning Layer

+8-10%

🎯 Make MB understand WHEN and WHY, not just WHAT

Timeline Constructor

Medium

Build ordered event chains from chunks

💡 New field in SoulBottleChunk: temporal_graph linking messages by causality

Causality Tracker

Medium

Track cause → effect relationships across sessions

💡 decisions.caused_by_id field linking to prior decisions/events

Temporal Oracle Query

Low

Search by time order ('What happened BEFORE X?' 'What led to Y?')

💡 Enhance oracleSearchIntelligence with temporal filters

Phase 2: Belief Evolution System

+10-12%

🎯 Track HOW the companion's understanding evolved

Belief Snapshots

High

Capture companion's understanding at each chunk

💡 New field: belief_state in SoulBottleChunk tracking key beliefs at that moment

Evolution Detector

Medium

Automatically detect when beliefs changed

💡 Compare consecutive chunk belief_states, flag changes

Contradiction Resolver

Low

When beliefs conflict, understand which is current

💡 Enhance detectMemoryConflicts to show belief progression timeline

Phase 3: Cross-Session Orchestration

+5-8%

🎯 Perfect handoffs between sessions

Session Boundary Markers

Low

Explicitly mark where one session ends, next begins

💡 session_id field in messages, session transitions in chunks

Handover Validator

Medium

Test if companion retained context across sessions

💡 New function: validateSessionContinuity checking memory recall

Context Inheritance

Low

Ensure new session inherits right context from previous

💡 Optimize uncorkWithStacking to prioritize session-end chunks

Phase 4: Scale Validation

+2-5%

🎯 Test at 1.5M tokens (massive conversations)

Stress Test Suite

Medium

Generate synthetic 1.5M token conversations

💡 Expand StressTestLab with LongMemEval simulation mode

Performance Benchmarks

Low

Measure retrieval speed at scale

💡 Track query latency in oracleSearch at different scales

Chunk Optimization

High

Fine-tune chunk size for 1.5M+ tokens

💡 Dynamic chunking based on conversation density

💪 MB's Existing Advantages

Soul Bottle Architecture

World-class foundation - better than Hindsight's flat structure

Exhaustive Intelligence Extraction

18 intelligence categories - way deeper than Hindsight

Personality Matrix

Tracks companion traits over time - unique to MB

Human-AI Bond Focus

Emotional value + technical performance = market differentiation

Ready to Close the Gap

Start with Quick Wins, then tackle each phase. We have the foundation - now we add the 20-30% that separates good from world-class.