Android Police — Android — Sat, 29 Aug 2026 14:15:10 GMT
Summary
The author describes how their mobile Obsidian vault became unmanageable after a few hundred notes, with overlapping tags and orphaned daily logs that full-text search couldn't surface because they no longer remembered the notes existed. To close the gap, they adopted retrieval-augmented generation (RAG) via an Android app called LocalRAG!, which indexes Markdown vault files and runs an on-device model—about a 3GB download that works fully offline—unlike Obsidian plugins such as Smart Connections and RAG Chat, which require a desktop-hosted Ollama server. The app answers questions with numbered, tappable citations, and on its first query it returned three notes about the same story setting written over the course of a year, which the author then linked manually in Obsidian. LocalRAG! offers a free plan (five questions per day with the on-device model), Standard at $4.99/month and Pro at $9.99/month with one-week trials, or full access via a personal Anthropic API key, while AnythingLLM is cited as a free, open-source on-device alternative. The author stresses that the tool only finds connections; they still read each note and decide themselves whether a link belongs.
I have spent years building a second brain in Obsidian on my phone, and I trust it completely. Then I started failing, and it was entirely my fault.
I was writing a note about a setting for a book I have worked on for years. Halfway through, I noticed the idea was familiar, and that was because I had created the same place before.
I tried to find a related note by checking my links, but I found nothing. So I searched manually, and after an hour I found the note. I wrote it more than a year earlier and then forgot it.
The entire time, Obsidian stored my notes exactly how it was designed to. I needed something else to remove the headache of organizing them.
I set the vault up carefully, and Obsidian ordered my chaotic thoughts for the first year or so. But somewhere past a few hundred notes, my system started drifting apart.
I found two tags covering the same topic and one label attached to three unrelated projects. A couple of minutes of checking turned up more broken things than I can write down.
Most of my daily logs don't even have a category. They're stored in the vault as loose text, connected to nothing I had organized on purpose.
Obsidian's full-text search would have found that setting note in seconds if I had known to look for it, but I didn't know it existed.
That is the gap I wanted to close. I needed something that could read the whole vault and tell me what connected without me naming the thing first.
Retrieval-augmented generation does that. A RAG app indexes your documents, then answers questions by pulling the relevant passages and showing you where each one came from.
This is where Obsidian continues to take the cake for me. Since it stores every note as a plain Markdown file on the device, any app with file access can read a vault.
When people say "local LLM," they usually imagine a model running on a desktop. I had the same thought, and it decided where I looked first.
Obsidian has plugins for local LLMs. Smart Connections and RAG Chat both index a vault and answer questions against it without sending notes to a cloud provider.
However, neither one runs a model. They send requests to a server, usually Ollama, hosted at a fixed address on your desktop. On my phone, I would need a machine running at home and a route back to it every time I asked a question.
I had used PocketPal for offline LLM access, so I knew a phone could run a model on its own. I wanted that, plus document indexing, rather than a chat window.
LocalRAG! handles 25 file formats, including Markdown. It runs either through Anthropic's Claude in the cloud or through a bundled on-device model.
The local model needs a download of about 3GB. After that, extraction, indexing, search, and answer generation all happen on the phone with no internet connection needed.
I added my vault through the app's file picker and asked about the story setting first, since that's what started all of this. The app returned three notes about the same place, written across a full year.
Every answer carried numbered citations I could tap. I opened each source note, read it, and added the links to Obsidian by hand. I worked through that folder in a couple of days, then started on the others.
LocalRAG! runs a free plan at five questions per day, and that plan includes the on-device model. Standard costs $4.99 per month, while Pro costs $9.99, and both come with a one-week trial.
I pay for Standard. Five questions a day is not enough for a full vault audit, though it is enough to test whether the approach suits you.
You can also skip the subscription by bringing your own Anthropic API key, which unlocks every model and bills you through Anthropic.
If price is a blocker, AnythingLLM runs on-device RAG for free, and it is open source. I chose LocalRAG! because tapping a citation opens the exact passage behind it, and that's what lets me repair links one note at a time.
The on-device model is small, and it's noticeable. It answers questions about my notes quite well, but I still need a full assistant for more complex tasks.
Nothing here organizes my vault for me, and I would be careful of any tool that offers to do so. What changed is that I can ask my notes what I have written and get an answer with the source attached.
I still read the note, still decide whether the link belongs, and still type it in myself. Obsidian rewards deliberate work, and I have not stopped doing that work. I have just stopped losing years of it to my own memory.
Original article: https://www.androidpolice.com/paired-local-llm-with-obsidian-mobile-eliminated-note-sorting-headaches/ Source: Android Police