Viglet Dumont
Get started
Connectors

Six kinds of source, one way to run them

Every connector has a console screen, REST endpoints for create, update, export and import, and a JSON export package seeded on first boot. They all go through the same queue, checksums, safe finish and audit.

Adobe Experience Manager

On-prem, AMS and AEM as a Cloud Service

Pages, Content Fragments and assets, each read the way its content kind wants. Extraction is declared per content type, so one recipe serves every site that shares the template.

  • Author servlets on-prem, delivery APIs on Cloud Service with no author credential
  • A changed Experience or Content Fragment reindexes every page that embeds it
  • Changes arrive by Adobe I/O Events, an AEM-side bundle, or a journal poller

Edge Delivery Services

aem.live sites, from their query index

Reads the site's query-index.json (and its sitemap, if you want it), takes meta tags and body text by CSS selector, and derives facets from the URL path.

  • Checksum-based incremental runs
  • Index, reindex and dry scan from the console
  • A WKND EDS sample in the marketplace

Databases (JDBC)

A query, a key, and a typed field map

One SQL query and its primary key become documents. Each column maps to a field with a type, and can be multi-valued, a facet, mandatory or an embedding input.

  • Reads file paths or file contents from columns
  • Strips HTML from the fields you name
  • A custom class when a mapping is not enough

Web crawler

Sites you do not own the CMS of

Start points, allow and deny lists, and CSS selectors per attribute. It reads robots.txt and sitemaps, honours Crawl-delay, and backs off when a host answers 429 or 403.

  • Parallel workers with a per-host rate limit
  • Re-crawls with ETag and Last-Modified, skipping what did not change
  • A crawl that finds nothing does not empty the index

Files and documents

A directory, read with Apache Tika

Walks a directory and extracts the text of each document with Apache Tika, so PDFs, Office files and the rest become searchable alongside the pages that link to them.

  • Title, URL, date, size, extension and content per file
  • Images keep their file URL as the image field
  • The Celestial Gallery sample indexes a folder of astrophotography

Structured catalogs and JSON APIs

Paged JSON, with request budgets

Ingests catalog JSON that is already normalised, or pages through a JSON API under a daily, weekly or monthly request budget, so a rate-limited source is never overrun.

  • Declared as data: discover, extract, map
  • A parity gate before a source moves to a native recipe
  • Built for catalogs the generic crawler reads badly
Targets

Where the documents go

One property chooses the engine. Schema provisioning and the index audit need the target to answer back, which today only Turing does.

Viglet Turing ES

dumont.indexing.provider=turing

The default. Dumont sends the field schema before the first document and audits what Turing holds.

Apache Solr

dumont.indexing.provider=solr

Writes straight to a Solr collection.

Elasticsearch

dumont.indexing.provider=elasticsearch

Writes straight to an index, with optional basic auth.

Not sure how to model a source?

Source inference takes a URL and drafts the source and its fields with the LLM configured in Turing. A dry run shows the coverage and a diff; nothing is indexed until someone confirms the draft.

A source nobody has written yet

Connectors implement one Java interface in connector-commons. Plugins can be uploaded, started, stopped and disabled at runtime, without rebuilding the connector.

Point it at one source this afternoon.

One container, a console in English or Portuguese, and a dry scan that shows what would be indexed before anything is. Apache 2.0, with the roadmap in the repository.

Install it Start from a sampleRead the source
$docker run -p 30130:30130 ghcr.io/openviglet/dumont-ce:latest