- Amazon S3 Source
- Box Business Source
- Catalog Source
- Confluence Cloud Source
- Confluence Self-Hosted Source
- Database Source
- Dropbox Business Source
- Exchange Enterprise Source
- File System Source
- Generic REST API Source
- Gmail for Work Source
- Google Drive for Work Source
- Jira Software Cloud Source
- Jira Software Self-Hosted Source
- Jive Cloud Source
- Jive Server Source
- Khoros Community Source
- Microsoft Dynamics 365 Source
- OneDrive for Business Source
- OTCS Source
- Push Source
- RSS Source
- Salesforce Source
- ServiceNow Source
- SharePoint Online Source
- SharePoint Online Legacy Source
- SharePoint Server Source
- Sitecore Source
- Sitemap Source
- Twitter Source
- Web Source
- YouTube Source
- Zendesk Source
- Connector Building Best Practices
Add or Edit a Database Source
A Database source allows members of the Administrators and Content Managers built-in groups to retrieve and make searchable the content of a local database via the Coveo On-Premises Crawling Module (see Coveo On-Premises Crawling Module).
Your company developer created a custom database to manage the parts used in your facilities, their location in your warehouse, and purchase orders. You decide to index data regarding purchase orders only so that your buyers can find this content via your Coveo-powered search page.
As an administrator or a content manager, you can add the content of a local database to a Coveo organization. In a Coveo-powered search interface, the source content is accessible to either everyone, the source creator only, or specific users as determined by source permissions (see Content Security).
Source Key Characteristics
|Content update operations||Refresh|
|Rescan||Takes place every day by default.|
|Content security options||Determined by source permissions|
Add or Edit a Database Source
Before you start, ensure that the Coveo On-Premises Crawling Module is installed on a server that has access to the database of which you want to retrieve the content. Both servers must use the same time zone to prevent indexing timing issues.
If the crawling module is running in a different time zone than the database, you must add a parameter by modifying the source JSON configuration from the Coveo Administration Console during the completion process.
In the Add/Edit a Database Source subpage, the Configuration tab is selected by default. It contains your source’s general and authentication information, as well as other parameters.
If you have not already installed the Coveo On-Premises Crawling Module on a server that has access to the database of which you want to retrieve the content, click Download Crawling Module to do so.
Enter a name for your source.
A source name can’t be modified once it’s saved, therefore be sure to use a short and descriptive name, using letters, numbers, hyphens (
-), and underscores (
_). Avoid spaces and other special characters.
Enter your database connection parameters. Since the connection string syntax differs from one database type to another, you might want to see The Connection Strings Reference for details.
Since connection strings aren’t encrypted, they should never contain credentials in plain text. You can hide the password and user ID in the connection string by respectively introducing the
@pwd tokens. The Database source internally replaces the tokens with the information provided in the Authentication section.
You must provide either both tokens or none. If you don’t provide tokens, but add source credentials, the behavior will be the same as before, meaning that the credentials will be used to impersonate the process running the queries.
Basic connection string:
Data Source=mydatabase.mycompany.com;Initial Catalog=MyDatabase;User Id=companyUser;Password=MyPassword
Hiding password and user ID using tokens:
Data Source=mydatabase.mycompany.com;Initial Catalog=MyDatabase;User Id=@uid;Password=@pwd
Enter the table or view object names (
<Mapping type="name">) that you intend to index. You will later define them in the database configuration.
Select the software driver that provides access to your database.
To get a complete list of available drivers and therefore of supported database types, use the Crawling Module REST API
GET call on
/api/odbc/drivers (see Crawling Module REST API Reference and Getting the Available Drivers for an ODBC Source)
Paired Crawling Module
If your source is a Crawling Module source and if you have more than one Crawling Module linked to this organization, select the one with which you want to pair your source. If you change the Crawling Module instance paired with your source, a successful rebuild is required for your change to apply.
Character Optical Recognition (OCR)
Check this box if you want Coveo Cloud to extract text from image files or PDF files containing images. OCR-extracted text is processed as item data, meaning that it’s fully searchable and will appear in the item Quick View. See Enable Optical Character Recognition for details on this feature.
When adding a source, if you have more than one logical (non-Elasticsearch) index in your organization, select the index in which the retrieved content will be stored (see Leverage Many Coveo Indexes). If your organization only has one index, this drop-down menu isn’t visible and you have no decision to make.
Enter the Username and Password of a dedicated user account that has access to the content you want to index. See Source Credentials Leading Practices. Then, select one of the following options from the drop-down list:
Select this option to authenticate users based on the credentials stored in the connection string.
Active Directory on-premises
Select this option to secure your database content using the Active Directory identities in the XML configuration.
To enforce this feature, you must:
If you selected Active Directory on-premises, you must also fill the Active Directory username and Active Directory password fields that appear since this option won’t work with only the dedicated user credentials. If you selected Native, skip to “Content Security” Tab.
If you leave the Active Directory username and Active Directory password fields blank, your current (dockerless) crawling module credentials will be automatically used.
Active Directory on-premises doesn’t affect the authentication of the Database source.
Active Directory Username and Active Directory Password
Enter credentials to grant Coveo Cloud access to your Active Directory.
Expand Well-Known SIDs
Check this box if you want the users included in your Active Directory well-known security identifiers to be granted access to the indexed content. Expect an increase in the duration of the security identity provider refresh operation. Supported well-known SIDs are:
Domain Users, and
If your entire content is secured with the
Authenticated users well-known, it’s more cost-effective resource-wise to index it with a source whose content is accessible to everyone than to expand the well-known with a source that indexes permissions.
Check this box to use a TLS protocol to retrieve your security identities. If you do, we strongly recommend selecting StartTLS if you can. Since LDAPS is a much older protocol, you should only select this value if StartTLS is incompatible with your environment.
By default, Coveo Cloud retrieves the email address associated to each security identity from the
“Database Configuration” Section
In the XML configuration box, enter a XML-formatted configuration consisting of queries against your database to retrieve and copy the data from record fields to Coveo Cloud fields. This configuration should contain only read queries, or else you could make unwanted changes to your database.
“Content Security” Tab
Select who will be able to access the source items through a Coveo-powered search interface. For details on this parameter, see Content Security.
In the Access tab, determine whether each group and API key can view or edit the source configuration (see Resource Access):
In the Access Level column, select View or Edit for each available group.
On the left-hand side of the tab, if available, click Groups or API Keys to switch lists.
Finish adding or editing your source:
When you want to save your source configuration changes without starting a build/rebuild, such as when you know you want to do other changes soon, click Add Source/Save.
To add the source content or to make your changes effective, on the Sources page, you must click Start initial build or Start required rebuild in the source Status column.
When you’re done editing the source and want to make changes effective, click Add and Build Source/Save and Rebuild Source.
Back on the Sources page, you can review the progress of your source addition or modification.
Once the source is built or rebuilt, you can review its content in the Content Browser.
Optionally, consider editing or adding mappings.
You can only manage mapping rules once you build the source (see Refresh, Rescan, or Rebuild Sources).
See XML Configuration Further Configuration for options you can implement to make your source more efficient.
If you encounter database timeout errors, you may want to edit the source JSON configuration and set the
CommandTimeouthidden parameter value to
600seconds in the
If the issue persists, you can either increase the parameter value or use paged SQL queries instead.
Consider subscribing to deactivation notifications to receive an alert when a Crawling Module component becomes obsolete and stops the content crawling process.