Zum Inhalt springen
Scattered document nodes whose light paths converge on a single tag — as a glowing low-poly illustration.
All posts
Series: NextSearchPart 3 of 4
·Self-HostingNextcloudAPI

What full text doesn't know

A matter rarely lives in one folder. Tags record what no file says — and the API sets them by the hundred instead of one click at a time.

Search finds what a file says. Tags hold on to what it doesn't.

The first two parts were about making the collection searchable at all and getting the stack onto a server of your own. After that there is an index over ten or twenty thousand documents, and full-text search answers questions that used to be half an hour of clicking through folders. A few questions it still doesn't answer — and not for lack of better search terms.

Tags in detail

docs/tags.md covers visibility, permissions and the automation; docs/api.md covers the HTTP interface, with an OpenAPI file to import into Postman, Insomnia or Bruno.


The problem: a matter rarely lives in one folder

An example of the kind that turns up in every grown archive. Something was wrong with the roof, and over eighteen months it became a matter: the original quote sits in the project folder, the final invoice in Accounting/2023, the handover record as a scan in the inbox folder, the email thread with the roofer as an .eml export, plus two letters from the insurer — and those live in a second Nextcloud, because insurance paperwork has always been kept elsewhere.

Seven documents, five places, two instances. Searching for "roof tiles" finds four of them and forty others that happen to use the words. The handover record it doesn't find at all; the words aren't in there.

No better search term fixes that, because the missing piece isn't in any of the files: that these seven belong together. The facets from part one are no help either — instance, folder, file type, year, size describe where something sits and what it is. Not what it belongs to.

That is the gap tags fill. They are the one facet somebody puts there on purpose.


A word you assign yourself

Two kinds of tag, and the difference isn't cosmetic.

Instance tags belong to the installation. An administrator maintains them under Administration › Tags, and everyone sees the same list. That is the shared vocabulary — "Contract", "Warranty", "Approved".

Private tags belong to whoever created them. Anybody can make one, straight from the tag picker on a document. Nobody else sees them, an administrator included.

On top of that sits the scope, and it decides who sees an assignment: with global everyone who has access to the document, with personal only whoever made it.

Why those are two separate axes becomes clear at the first "To check". Two people work on the same archive and both need a pile of "look at this later". If the assignment were global, each would see the other's pile and have to guess which one is theirs. If each created their own private tag, the filter panel would carry the same word twice. The combination that works is an instance tag with personal scope: shared word, separate piles. The reverse case is "Approved" — visible to everyone, but with assign_policy: admin only administrators may set it.

Scope is fixed once the tag exists. Flipping it later would retroactively change who has been seeing what: a "To check" that was personal yesterday and global today exposes two hundred assignments without anyone being asked. So an update refuses the field. Whoever got it wrong creates a new tag — awkward, but visible.


"New": what came in since Monday

The second question full-text search walks straight past is the simplest one there is: what has actually arrived?

Sorting by date only half helps, because a file's date is its modification date. A scan from 2019 that somebody uploaded yesterday sits far down the list, although it is brand new to the index. And that is usually the question you have.

So every installation ships one system tag, new. It hangs itself on every document indexed for the first time and falls off again after a set time — seven days by default. That makes "arrived lately" a filter instead of a sort order.

Three things keep it useful:

The initial crawl of a folder tags nothing. On the first run the entire stock is new, and "New" on twenty thousand documents says nothing. The first completed run draws the baseline; from the second run on, new files get the tag.

A rebuild does not fire it again. Each document records its first successful indexing exactly once. Otherwise the whole collection would be "new" after every reindex. That changed the rebuild itself along the way: it used to drop the document rows and let the crawl write them again. With tags hanging on those rows that would throw away other people's handiwork, so the rows now stay and only their state is reset.

And emptying the index resets the baseline, because after that everything starts over anyway.

The sweep that clears expired assignments runs every five minutes. To see a changed shelf life immediately instead of waiting:

make artisan CMD="nextsearch:tags-sweep"

The system tag cannot be deleted, and its handle, scope and automation are fixed. Name, colour, shelf life and the on/off switch belong to whoever runs the instance.


The backlog: one search, one call

A vocabulary helps from the day you introduce it. Yesterday's collection isn't tagged by it, and that is the actual pile of work. The seven documents from the roof matter you click by hand. But when the same firm has worked for you for years and everything touching them should carry one tag, that is a few hundred hits in short order. At that point clicking stops being an option.

For that there has been an HTTP interface under /api/v1 since 0.3.0. Requests authenticate with a personal key from User settings › API keys, as a bearer token. The plain text is shown exactly once; the database keeps only its SHA-256 hash.

The important part of the permission model: the key carries no permissions of its own. A request does exactly what its owner can do in the interface — the same folders, the same tags, the same administrator rights or lack of them. There is deliberately no service account that may do more than a person. And a key cannot mint another key; that only works over a signed-in session.

The routine that has proven itself runs through the interface: click the filters together until the result list is right, and only then pour that same search into a tag over the API. The interface and the API use the same search, so they return the same thing.

KEY=nxs_…
HOST=https://search.example.org

# 1. Find everything that mentions the firm — across both instances
UUIDS=$(curl -sG -H "Authorization: Bearer $KEY" \
  --data-urlencode "q=meiners roofing" \
  --data-urlencode 'filters={"year":["2023","2024"]}' \
  --data-urlencode "per_page=100" \
  "$HOST/api/v1/documents" | jq -c '[.hits[].uuid]')

# 2. Tag them in one call
curl -s -X POST -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d "{\"documents\":$UUIDS,\"add\":[\"roof-warranty\"]}" \
  "$HOST/api/v1/documents/tags"
{ "updated": 63, "skipped": [], "unknown": [] }

Up to 500 documents per call, up to 50 tags each in add and remove. Beyond that you page with page; total_pages comes back with the search. Tags are addressed by their slug or their uuid — and a slug is only unique per owner. If you have a private tag contract next to an instance tag of the same name, the slug hits your own. When it matters, use the uuid.

Whatever no search term catches — the handover record above, for instance — stays handwork. Three to add by hand after sixty-three went through in one call: that's fine.


When something doesn't work, nothing works halfway

Two error cases that deliberately end differently, and the difference is the interesting part.

An unknown or disallowed tag fails the whole call with 422. Nothing is dropped quietly. A typo in a slug is a typo you want to fix — not something you notice three weeks later because half the collection stayed unmarked. A call that reports success did everything it was given.

The bulk call goes the other way. Documents you have no access to come back as skipped, uuids that match nothing as unknown, and the call runs through regardless. With five hundred documents out of a wide search, a folder you were never granted is the normal case, not a reason to leave the other 480 alone. The response does say which ones they were.

The full text never travels, by the way. Search delivers the highlighted snippet only, and <mark> is the sole markup in it — everything else is escaped. Whoever wants the file opens it in NextSearch, where that is a visible act.


What the interface deliberately does not do

No file contents, no downloads, no index administration, no user management. Those stay in the interface, where the consequences are visible.

Nor is there a second way past that: everything under /api without the v1 belongs to the interface, runs on the session cookie and changes without notice. It isn't a contract and isn't meant to become one. If you're missing something, an issue serves you better than helping yourself there.

The limit is 120 requests per minute per key, adjustable with API_RATE_LIMIT; X-RateLimit-Remaining rides along in every response. The full description sits in the repository as an OpenAPI file at docs/api/openapi.yaml.


Conclusion

Full-text search answers questions about content. Belonging and state are not questions about content — somebody has to set those, and after that they are a filter like any other. Tags do exactly that, with a visibility model you can explain in two sentences and a system tag that answers the most common question on its own.

The API is the tool for the pile that is already there. Click the search together, fire once, done. For everyday use the interface stays faster.

Both ship in NextSearch 0.3.0.


Call to action

NextSearch is open source and lives on GitHub: https://github.com/McGo/NextSearch

To try tags without touching a Nextcloud of your own: make demo spins up a throwaway instance, make demo-seed drops sample files into it.

The next part covers the same toolbox for assistants — NextSearch is getting an MCP server that lets a client search and set tags, with the same permissions and without anyone passing a key around. That ships with 0.4.0.