dehora

Plan: Hybrid search over memory index

Context

The indexing pipeline (chunker → embeddings → sqlite-vec) is built and working. Now we need the query side: hybrid search combining FTS5 keyword search with sqlite-vec vector KNN, fused via Reciprocal Rank Fusion (RRF). This replaces the fixed recent-sessions loading with relevance-based retrieval at conversation start, and gives the agent a memory_search tool for mid-conversation lookups.

Approach: FTS5 + sqlite-vec, zero new dependencies

FTS5 is built into SQLite. A standalone FTS5 table (chunks_fts) is populated alongside vec_chunks during indexing. Content-sync mode (content=vec_chunks) won't work because SQLite doesn't fire triggers on virtual table writes. Duplicating the text in FTS5 is fine — these are small personal memory files.

---

dehora

Upgrade xcode cli tools to the most recent (this takes a while):

$ softwareupdate --list
$ softwareupdate --install "Command Line Tools for Xcode-13.4"

Watch pyenv not be able to install anything (pyenv/issues/2143), because of what seems to be a mismatch across Python, Clang and Monterey (issue45405).

Update pyenv to pick up patched versions that work (3.10.3+, 3.9.13, 3.9.9, 3.8.13, 3.7.13).

dehora

Keybase proof

I hereby claim:

• I am dehora on github.
• I am dehora (https://keybase.io/dehora) on keybase.
• I have a public key ASCMcCHXvRlJKpB5ioCcRqJBvT0NT2XNZh4VR8JnJCnf0Qo

To claim this, I am signing this object:

dehora

subl /Users/bdehora/Library/Application\ Support/JetBrains/Toolbox/apps/PyCharm-P/ch-1/181.4445.28/PyCharm.app/Contents/helpers/pydev/_pydev_bundle/pydev_ipython_console_011.py


class PyDevIPCompleter(IPCompleter):

    def __init__(self, *args, **kwargs):
        """ Create a Completer that reuses the advanced completion support of PyDev
            in addition to the completion support provided by IPython """
        IPCompleter.__init__(self, *args, **kwargs)
        # Use PyDev for python matches, see getCompletions below

dehora

package services

import java.security.Security

import org.slf4j.LoggerFactory

object DnsCache {

  private val logger = LoggerFactory.getLogger("DnsCache")

dehora

package spike;

import java.util.concurrent.TimeUnit;
import nakadi.Cursor;
import nakadi.LoggingStreamObserverProvider;
import nakadi.LoggingStreamOffsetObserver;
import nakadi.NakadiClient;
import nakadi.Response;
import nakadi.StreamConfiguration;
import nakadi.StreamProcessor;

dehora

package foo;

import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.TemporalAccessor;
import java.time.temporal.TemporalQuery;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

dehora

package org.zalando.fahrschein.domain;

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.ByteArrayOutputStream;
import java.util.Collections;
import org.junit.Test;

import static org.junit.Assert.assertFalse;

dehora

HeilmeierQuestions

• What are you trying to do? No jargon. What is the problem? Why is it hard?

• How is it done today, and what are the limits of current practice?

• What's new in your approach and why do you think it will be successful?

• Who cares?

dehora

	 RequestMessage
	 +---------------+---------------+---------------+---------------+
	 |  Size (int32)                                                 |
	 +---------------+---------------+---------------+---------------+
	 |  ApiKey (int16)                  |  ApiVersion (int16)        |
	 +---------------+---------------+---------------+---------------+
	 |  CorrelationId (int32)                                        |
	 +---------------+---------------+---------------+---------------+
	 |   ClientId (str, 0..)                                            
	     ...