InFeeo
Language

TRACE: open-source hierarchical memory for LLM agents, 82.5% on MemoryAgentBench’s EventQA using gpt-oss-20B [P](reddit.com)

×
Link preview TRACE: open-source hierarchical memory for LLM agents, 82.5% on MemoryAgentBench’s EventQA using gpt-oss-20B [P] Built a memory system called TRACE that organizes agent conversation history into a topic tree (branches + summaries) instead of flat RAG chunks, and benchmarked it on MemoryAgentBench (ICLR 2026), specifically the EventQA accurate-retrieval task. Its a pypi package: pip install trace-memory Results (F1): • TRACE (gpt-oss-20B): 82.5% • TRACE (gpt-oss-120B): 83.8% • Mem0 (GPT-4o-mini, paper’s official number): 37.5% • MemGPT/Letta (GPT-4o-mini, paper’s official number): 26.2% Ran gpt-oss locally, so this is an open-weights model against MemGPT/Mem0 on GPT-4o-mini, not an apples-to-apples same-backbone test (I don’t have the money for open ai tokens). I tried to get Mem0 running on gpt-oss-20B directly for fairness, but its fact-extraction step needs strict JSON output and gpt-oss’s responses didn’t parse cleanly (known issue, not gpt-oss specific. Same bug shows up with Gemini/Mistral too). Letta needs a full server setup so I skipped it. Full JSON logs from both runs are in the repo if you want to dig into the methodology yourselves. GitHub: https://github.com/husain34/TRACE submitted by /u/PsychologicalDot7749 [link] [Kommentare] reddit.com · reddit.com
Built a memory system called TRACE that organizes agent conversation history into a topic tree (branches + summaries) instead of flat RAG chunks, and benchmarked it on MemoryAgentBench (ICLR 2026), specifically the EventQA accurate-retrieval task. Its a pypi package: pip install trace-memory Results (F1): • TRACE (gpt-oss-20B): 82.5% • TRACE (gpt-oss-120B): 83.8% • Mem0 (GPT-4o-mini, paper’s official number): 37.5% • MemGPT/Letta (GPT-4o-mini, paper’s official number): 26.2% Ran gpt-oss locally, so this is an open-weights model against MemGPT/Mem0 on GPT-4o-mini, not an apples-to-apples same-backbone test (I don’t have the money for open ai tokens). I tried to get Mem0 running on gpt-oss-20B directly for fairness, but its fact-extraction step needs strict JSON output and gpt-oss’s responses didn’t parse cleanly (known issue, not gpt-oss specific. Same bug shows up with Gemini/Mistral too). Letta needs a full server setup so I skipped it. Full JSON logs from both runs are in the repo if you want to dig into the methodology yourselves. GitHub: https://github.com/husain34/TRACE submitted by /u/PsychologicalDot7749 [link] [Kommentare]

Log in Log in to comment.

No comments yet.