Closing Step 3: A Report Card, and the Right to Forget
Building My Personal AI, Step 3 (part 2)
When I wrote about giving my AI a memory, I ended with a list of what was still missing. A proper test. A way to tell it things. A way to take things back.
Those are done now. So is Step 3.
A report card
A few good answers are a start, not a score. So I wrote my own test.
I put nine real files in the vault: some of my own financial statements, an NDA, a one-page framework I use for my day, and four books. That came to 1,015 passages. About 900 of them were from the books.
That mix was on purpose. I wanted to know if a question about one page of my own would still find that page, with nine hundred passages of philosophy and strategy around it.
Then I wrote seven questions I already knew the answers to. One of them had no answer in any file.
| Test | Result |
|---|---|
| Right file in the top 3 | 6 of 6 |
| Right file at #1 | 5 of 6 |
| Answer right | 4 of 6 |
| Said "Not in my memory" when it should | 1 of 1 |
| Answer time | 3.4 seconds on average (12.6 for the first, while the model loaded) |
Search found the right file every time. The honesty question passed: it said "Not in my memory."
Two answers were wrong.
The right file, the wrong answer
Both misses had the same shape. The right file came back, but the answer needed facts spread across several pages. The AI reads the best eight passages. For a question like "the total across the whole year," eight wasn't enough.
That's useful to know. The problem isn't finding. It's reading enough of what was found.
There are fixes: read sixteen passages instead of eight, or read the pages next to each match. I'm leaving it at eight for now and will tune it when a real use needs it.
Two kinds of memory
Documents are only half of what I'd want a personal memory to know.
My files hold what other people wrote about me. Statements, contracts, books. Most of what would make an answer mine isn't in any file. My golf handicap. A decision I made and why. Something a friend told me. A rule I keep.
So now I can just tell it.
remember "My golf handicap is 14"
It saves my words, with the date, in a plain file I can open and read. The next question finds it.
The right to forget
The other half is taking things back.
forget --note "handicap"
The line is gone from the file, and gone from the search index. Asked again, it says "Not in my memory."
I wanted forgetting to mean gone, not hidden. So the index overwrites deleted text instead of just unlinking it. To be honest, my tests couldn't show this extra step was needed: on a small index, the old text disappeared anyway. I kept it as insurance. It doesn't reach backups. That's a separate question, for later.
There are two other limits, and they're on purpose. It only remembers what I tell it to, never on its own. And it refuses anything that looks like a card number, an account number or a password.
That's where BuWe shows up again. The machine carries the understanding. I decide what it knows.
What went wrong this time
My upload to GitHub landed one folder too deep. The new code went into a folder inside a folder, and the Studio kept running the old version. The first command said, in effect, "remember? Never heard of it."
The fix took five minutes. Finding it took longer than it should have, because everything looked fine.
What Step 3 leaves open
| Not yet | Why |
|---|---|
| Backup | The vault doesn't hold anything important yet. We're still building |
| Reading scans | One old tax form is a picture with no text. Reading images comes later |
| Asking from my phone | Today I ask on the Studio itself |
| Sixteen passages | Parked until a real use needs it |
Next
Step 4: Values. A short set of rules the AI follows, and what it may never do on its own.
If memory is what it knows, values are how it behaves with what it knows.
The how, with every number: