It keeps a record of every item it sent, compares that record with what the index holds and with what the source still has, and reports the differences before it acts on them.
Removing documents is the one step that can make search worse than doing nothing, so it is the step with the most conditions.
De-indexing is the last step of a run and it is conditional. An interrupted run, a partial one, or one where at least half the items failed keeps every document where it was.
The run is recorded with its status and notes, and the next run picks up from there.
On a cron you choose (03:00 by default) Dumont lists what the target holds and compares it with the source. Missing or stale documents are queued again.
Index inventory is read from Turing; on Solr and Elasticsearch the audit is not available yet.
Documents that left the source but are still indexed go into a dated orphan report. A person applies it or discards it; an untouched report expires after 24 hours.
Nothing is removed from the index because a crawler did not happen to see it today.
For each source, the share of documents that carry each field, and how that changed since the last run. A field that stops arriving is a number on a screen, not a support ticket.
An optional summary of the coverage report is written by the LLM configured in Turing.
Each document can carry where each of its fields came from, so a wrong value in search leads back to the extractor and the source item that produced it.
Plus a validation that compares what Dumont recorded as sent with what the index actually holds.
Run outcomes are exposed as dumont_run_* metrics on a separate management port, never the public one. A run that starts warning posts JSON to the webhook you configure.
Last status, finish time, duration and document count per source, plus a run counter.
On the management port only, never the public one, tagged by source and provider. The question an on-call engineer asks first is whether last night's run happened.
# GET http://dumont:30131/actuator/prometheus
dumont_run_last_status{source="wknd",provider="turing"} 1
dumont_run_last_finished_seconds{source="wknd",provider="turing"}
dumont_run_last_duration_seconds{source="wknd",provider="turing"}
dumont_run_last_documents{source="wknd",provider="turing"}
dumont_runs_total{outcome="complete",source="wknd",provider="turing"}
# alert when the nightly run is stale
time() - dumont_run_last_finished_seconds > 26 * 3600One container, a console in English or Portuguese, and a dry scan that shows what would be indexed before anything is. Apache 2.0, with the roadmap in the repository.
docker run -p 30130:30130 ghcr.io/openviglet/dumont-ce:latest