What is Viglet Dumont?

Viglet Dumont is an open-source data extraction platform. It connects your content (wherever it lives) to Viglet Turing for indexing and search.

Think of Dumont as the bridge between your content sources and the search engine. It crawls websites, queries databases, scans file systems, reads AEM repositories, and pulls from WordPress, then delivers every document to Turing through a reliable, asynchronous pipeline.

What can you do with Dumont?

Crawl websites

Point the Web Crawler at a starting URL and let it discover and extract content from your entire site: pages, articles, documentation. It handles authentication, URL filtering, locale detection, and incremental updates.

Index databases

The Database Connector runs any SQL query against Oracle, PostgreSQL, MariaDB, MySQL, or any JDBC-compatible database and turns each row into a searchable document. Complex joins, custom transformations, and batch processing are all supported.

Scan file systems

The FileSystem Connector walks a directory tree, extracts text from PDFs, Word documents, spreadsheets, presentations, and images (via OCR), and indexes everything with full metadata: file size, extension, modification date.

Connect to AEM

The AEM Connector indexes content from Adobe Experience Manager author and publish instances, supporting content fragments, delta tracking, locale mapping, and custom extension points for your specific AEM setup.

Integrate with WordPress

The WordPress Connector is a WordPress plugin that indexes posts, pages, and custom post types directly into Turing. It keeps the index in step with every edit, turns taxonomies and custom fields into facets, and answers the site’s search from Turing, block themes included.

Send to any search engine

Beyond Turing, Dumont can deliver content directly to Apache Solr or Elasticsearch via pluggable indexing adapters, no Turing required.

How it works at a glance

graph LR
    WEB["🌐 Websites"]
    DB["🗄️ Databases"]
    FS["📁 File Systems"]
    AEM["📦 AEM"]
    WP["📝 WordPress"]

    DUM["Dumont\n(Connectors + Pipeline)"]

    TUR["Turing\n(Search + GenAI)"]

    WEB --> DUM
    DB --> DUM
    FS --> DUM
    AEM --> DUM
    WP --> DUM

    DUM -->|"REST API\n+ Message Queue"| TUR

Content flows from its original source through Dumont connectors, into an internal message queue for reliable processing, and out to the search engine, where it becomes immediately searchable.

Key concepts

These are the main building blocks you will work with in Dumont. You do not need to understand all of them before getting started: come back to each one as you need it.

Concept	What it is	Learn more
Connector	A component that extracts content from a specific source type (web, database, files, AEM, WordPress)	Connectors
Indexing Plugin	The output adapter that delivers documents to a search engine, Turing (default), Apache Solr, or Elasticsearch	Indexing Plugins
Job Item	A single document being processed through the pipeline, with fields, metadata, and an action (index, de-index, update)	Core Concepts
Batch Processor	Groups individual job items into configurable batches (default: 50) for efficient queue delivery	Core Concepts
Message Queue	Apache Artemis (embedded): decouples connector extraction from indexing delivery for reliability and throughput	Architecture
Processing Strategy	Priority-based rules that determine how each document is handled, index, re-index, de-index, ignore, or skip unchanged	Core Concepts
Indexing Rules	Regex-based filters that skip documents matching specific attribute patterns before they reach the queue	Core Concepts

Where to go next

Not sure where to start? Here is a suggested path depending on what you want to do:

I want to understand how Dumont works → Read Core Concepts first, then Architecture.

I want to install Dumont → Go to the Installation Guide.

I want to crawl a website → Go to Web Crawler Connector.

I want to index a database → Go to Database Connector.

I want to index files from a directory → Go to FileSystem Connector.

I want to connect AEM to Turing → Go to AEM Connector.

I want to index WordPress content → Go to WordPress Connector.

I want to send content to Solr or Elasticsearch directly → Go to Indexing Plugins.

I want to contribute or build from source → Go to the Developer Guide.