Add or Edit a SharePoint Online Source

Index Content Coveo Relevance Cloud Developer System Administrator Product Documentation

In this article

Source Key Characteristics
About Authentication and Site Access
Requirements
Add or Edit a SharePoint Online Source
Safely apply content filtering
Update a SharePoint Online Access Token
Required privileges
What’s Next?

Members with the required privileges can index SharePoint Online or OneDrive content and make it searchable. In a Coveo-powered search interface, the source content is accessible to either everyone, some specific users and groups, or the same users and groups as in the content system.

Notes

To retrieve SharePoint on-premises content, you must create a SharePoint Server source.
The item modifications that are retrievable during a source rescan are determined by the options selected when adding or editing the SharePoint Online source in the Additional content section.
Following a refresh operation, deleted discussion lists are excluded from your SharePoint Online source content, but replies to the original discussion message will only be excluded following the next rescan operation. This is a known issue caused by a limitation of SharePoint Online.
To decrease indexing times, you can configure your SharePoint Online source so that if a change is detected in the folder when indexing, it won’t re-index a list folder.

Leading practice

The number of items that a source processes per hour (crawling speed) depends on various factors, such as network bandwidth and source configuration. See About crawling speed for information on what can impact crawling speed, as well as possible solutions.

Source Key Characteristics

Features	Supported	Additional information
SharePoint Online version	Latest cloud version
Searchable content types		Sites, sub-sites, user profiles, personal websites, lists, list items, list item attachments, document libraries, document sets, documents, web parts, and microblog posts and replies.
Content update operations	Refresh		Takes place every hour by default. A rescan or rebuild is required to take account of deleted user profiles.
Rescan		Takes place every week by default. Extracts all of the data and indexes the following: modified permissions, new items, existing items with a modified date greater than the date in the index, and existing items with a computed entity tag^[1] different than the one in the index.
Rebuild
Content security options	Same users and groups as in your content system
Specific users and groups
Everyone

Features

Supported

Additional information

SharePoint Online version

Latest cloud version

Searchable content types

Sites, sub-sites, user profiles, personal websites, lists, list items, list item attachments, document libraries, document sets, documents, web parts, and microblog posts and replies.

Content update operations

Refresh

Takes place every hour by default. A rescan or rebuild is required to take account of deleted user profiles.

Rescan

Takes place every week by default. Extracts all of the data and indexes the following: modified permissions, new items, existing items with a modified date greater than the date in the index, and existing items with a computed entity tag^[1] different than the one in the index.

Rebuild

Content security options

Same users and groups as in your content system

Specific users and groups

Everyone

About Authentication and Site Access

A SharePoint Online source uses the OAuth 2.0 authorization protocol to access your SharePoint Online site content, and the source must authenticate through an Azure Active Directory application. You can choose to authenticate the Azure Active Directory application via an SSL certificate using app-only permissions, or using a delegated SharePoint Online user account (crawling account).

Note

It’s important to know the difference between site access, indexed items, and user access to content.

Authenticating through a certificate or a delegated crawling account gives Coveo authorization to crawl site content in your SharePoint Online tenant. The sites and items that are actually indexed by your SharePoint Online source depends on your source Content to Include setting. User access to the indexed items through a Coveo-powered search interface depends on your source Content Security setting.

The authentication method you choose depends mainly on your individual needs and corporate policy. The following sections highlight some of the main features of the two authentication methods.

Note

You must enter authentication credentials to be able to save a new SharePoint Online source configuration. Before creating a new SharePoint Online source:

Determine the authentication method you want to use.
Perform the tasks outlined in the Requirements section.

Certificate Authentication

If you choose to authenticate using a certificate, you must create a certificate and the Azure Active Directory application, and then add the certificate to the application and your source (see Certificate Authentication Requirements).

Some of the main features of certificate authentication are:

Provides a higher throttling rate limit than delegated authentication, and is recommended for indexing large amounts of data.
Grants the source with permission to crawl all site content, personal sites, and user profiles in your SharePoint Online tenant without having to provide individual access to each site. The content that is actually crawled and indexed depends on your source Content to Include setting.
Provides easier setup, as you don’t need to create and manage a crawling account in SharePoint Online, assign the account appropriate roles and permissions, and grant the account access to content.

Note

Coveo recommends using certificate authentication for your SharePoint Online source.

Delegated Authentication

For delegated authentication, the Azure Active Directory application is automatically created in your SharePoint tenant when you create the source, and is linked to the permissions of the crawling account that you create (see Delegated Authentication Requirements).

Some of the main features of delegated authentication are:

Provides a way to give the crawling account, and by association your source, access to crawl only specific sites and user profiles.
Provides a way to grant the crawling account with minimal permissions when accessing site content.

Requirements

Once you’ve determined the authentication method to use, you must perform the following before creating your SharePoint Online source.

Certificate Authentication Requirements

Create an SSL Certificate

Your SharePoint Online source uses the SSL certificate to authenticate the Azure Active Directory application in order to crawl your SharePoint Online tenant.

Create a CA-signed certificate using a trusted certificate authority (recommended), or a self-signed certificate using the method of your choice.

Note

The certificate file format must be .cer, .pem, or .cert. You’ll need the certificate file when adding the certificate to your Azure Active Directory application.
Export the certificate as a password-protected .pfx file. Depending on how you created the certificate file, the .pfx file may be created for you automatically.

Note

You’ll need the .pfx file and password when creating your SharePoint Online source.

Create the Azure Active Directory Application

The Azure Active Directory Application that you create for use with your source grants Coveo the permissions to crawl your SharePoint Online tenant. Create the application, assign the required permissions, and add your certificate to the application as follows:

Access your Azure portal with an administrator account, and create (register) an Azure Active Directory application.

Notes

Select Accounts in this organizational directory only for the Supported account type option when creating the application.
Once you register the application, you’re taken to the application Overview page in Azure. Take note of the Application (client) ID and Directory (tenant) ID as you’ll need them when creating your SharePoint Online source.

Grant the Azure Active Directory application the required crawling permissions as follows:
1. If you’re currently on your application’s page in Azure, proceed to the next step. Otherwise, access your Azure portal with an administrator account, click App registration, and then click the application you created previously.
2. Click API permissions.
3. If the User.Read permission is added by default, click the permission, and then click Remove permission.
4. For each of the required permissions below, click Add a permission, and then in the Microsoft APIs tab:
  - Click Microsoft Graph, click Application permissions, and then add the following permissions:
    
    Directory.Read.All
    
    Group.Read.All
    
    Sites.Read.All
  - Click SharePoint, click Application permissions, and then add the following permissions:
    
    Sites.FullControl.All
    
    User.Read.All
Once you’ve added all the required permissions, grant tenant-wide admin consent to the application.

Note

You must have the appropriate user role to consent on behalf of the organization.
Add your certificate to the Azure Active Directory application.

Delegated Authentication Requirements

Create a SharePoint Online user account (crawling account) with appropriate roles and permissions.
Grant the crawling account permission to access sites.

Domain Name System Records Configuration for Microsoft 365

Regardless of the chosen authentication method, if you’re using custom domains in SharePoint Online, you must configure your Domain Name System (DNS) records for Microsoft 365.

Access the Domains page of your Office 365 admin center.
Select your corporate domain (not company.onmicrosoft.com) check box.
On the domain page, in the DNS records section, take note of the DNS records.
Configure the DNS records in your DNS host provider.
On the domain page, in the DNS records section, click Check health to ensure that the DNS records were correctly configured.

Azure Application Permissions

A SharePoint Online source uses the OAuth 2.0 authorization protocol. To work with Microsoft APIs (CSOM and REST), Coveo must authenticate via an Azure Active Directory application that has the proper permissions. The access token is then limited to these permissions, which are necessary to successfully crawl SharePoint Online.

Note

For certificate authentication, you must create the Azure Active Directory application and assign the required permissions. For delegated authentication, the Azure Active Directory application is automatically created in your SharePoint Online tenant with the required permissions when you create your SharePoint Online source, and appears as SharePoint Online Connector in your Azure portal’s Enterprise applications page.

You can use the information in the following table to either review the permissions that are automatically assigned to the application when using delegated authentication, or to view a description of the permissions that you must grant the application when using certificate authentication.

Notes

The application permissions determine what Coveo can access when crawling your SharePoint Online tenant. The sites and items that are actually crawled and indexed by your SharePoint Online source depends on your source Content to Include setting. User access to the indexed items through a Coveo-powered search interface depends on your source Content Security setting.
Coveo is a verified publisher for the Azure application.
You must provide admin consent for all of the required application permissions.

API Permission Justification

SharePoint

Sites.FullControl.All (application permission for certificate authentication)

AllSites.FullControl (delegated permission)

Allows Coveo to retrieve permissions of crawled items, such as sites, users, lists, and documents. When combined with the Same users and groups as in your content system content security source option, it allows Coveo to secure your content by replicating your SharePoint permission system. Microsoft doesn’t offer enough granularity for Coveo to use a permission with fewer privileges.

Note

When using delegated authentication, Coveo will never have more privileges than the crawling account, as the crawling account permissions take precedence. Coveo would only have the complete set of AllSites.FullControl privileges if the crawling account also has the same level of privileges.

User.Read.All

Grants Coveo the permission to crawl user profiles.

Note

For certificate authentication, Coveo crawls all user profiles in your SharePoint Online tenant. You cannot choose to grant Coveo access to crawl only specific user profiles. For delegated authentication, Coveo crawls only the user profiles that the crawling account permissions allow.

Microsoft Graph

Sites.Read.All

Grants Coveo the permission to crawl site content.

Note

For certificate authentication, this permission simply grants permission to crawl site content. The site content that is actually crawled and indexed depends on your source Content to Include setting. For delegated authentication, Coveo has permission to crawl only the sites that the crawling account permissions allow. Like for certificate authentication, the site content that is actually crawled and indexed depends on your source Content to Include setting.

Directory.Read.All

Coveo requires this permission to fetch:

The Directory Role and Directory Role Members (see List members).

All users in Office 365, which is necessary to determine which users are in built-in groups such as Everyone (see List users and Coveo Management of Security Identities and Item Permissions).

Note

The Azure documentation shows that the least privileged permission to retrieve the list of users in a group is actually User.ReadBasic.All, but since Directory.Read.All is already required for other operations, User.ReadBasic.All doesn’t appear in the list of required permissions.

Group.Read.All

Coveo uses this permission to obtain the ID of a group (represents an Azure Active Directory group, which can be an Office 365 group, or a security group), and then a list of the group members (see Get group and List members).

You must provide tenant-wide admin consent for the permissions in the Azure Active Directory application that’s used to authenticate your source.

Notes

To provide admin consent from the Azure portal, you must have the appropriate user role.
You won’t be able to index your SharePoint Online content until you provide admin consent.

Certificate authentication: Provide admin consent for the application permissions from your Azure portal. Typically, you provide consent when creating the Azure Active Directory application for use with Coveo, but you can do so at a later time (see Grant admin consent in App registrations).

Delegated authentication: Provide admin consent directly from your SharePoint Online source panel when creating your source (requires SharePoint Global Admin credentials), or from your Azure portal after creating your source (see Grant tenant-wide admin consent in Enterprise apps).

Note

The Azure Active Directory application that’s automatically created in your SharePoint Online tenant after you create your source appears as SharePoint Online Connector in your Azure portal’s Enterprise applications page.

Add or Edit a SharePoint Online Source

This section details how to add or edit a SharePoint Online source.

Determine the authentication method you want to use.
Perform the tasks outlined in the Requirements section.

On the Sources (platform-ca | platform-eu | platform-au) page, do one of the following:

To create a new source, click Add source, and then click SharePoint Online.
To edit an existing source, click your SharePoint Online source, and then click Edit in the Action bar.

Leading practice

It’s best to create or edit your source in your sandbox organization first. Once you’ve confirmed that it indexes the desired content, you can copy your source configuration to your production organization, either with a snapshot or manually.

See About non-production organizations for more information and best practices regarding sandbox organizations.

Specify your source settings on the Add/Edit a SharePoint Online Source subpage. Refer to the following sections for detailed information on the source settings:
Note

You can save your source settings at any time by clicking Add and build source/Add source, or Save and rebuild source/Save.
Build or rebuild your source.

"Configuration" Tab

On the Add/Edit a SharePoint Online Source subpage, the Configuration tab is selected by default. It contains your source general and content information, as well as other parameters.

General Information

Source Name

Enter a name for your source.

Leading practice

A source name can’t be modified once it’s saved, therefore be sure to use a short and descriptive name, using letters, numbers, hyphens (-), and underscores (_). Avoid spaces and other special characters.

Optical Character Recognition (OCR)

If you want Coveo to extract text from image files or PDF files containing images, check the appropriate box. OCR-extracted text is processed as item data, meaning that it’s fully searchable and will appear in the item Quick View. See Enable optical character recognition for details on this feature.

Note

Contact Coveo Sales to add this feature to your organization license.

"Authentication" Section

You can authenticate the source to access your SharePoint Online content using a certificate or a delegated user account.

Note

Your source authentication access token can potentially expire and become invalid. See Update a SharePoint Online Access Token for information on what causes the access token to expire and how to update an expired access token.

Select whether to use Certificate or OAuth2 (delegated) authentication.

Specify the corresponding settings:

For Certificate authentication:
1. Enter your SharePoint Online Tenant name or tenant address.
  Examples
  SharePoint Online tenant name: mycompany
  
  SharePoint Online tenant address: https://mycompany.sharepoint.com
2. Enter your SharePoint Online Tenant id.
3. Enter the Client id for the Azure Active Directory application that you created for your source.
4. Enter your Certificate password.
5. Click Certificate file to upload your .pfx certificate.

For OAuth2 (delegated) authentication:

Click Authorize account.
Enter your SharePoint Online Tenant name or tenant address, and then click Sign In.
Examples
- SharePoint Online tenant name: mycompany
- SharePoint Online tenant address: https://mycompany.sharepoint.com
Provide admin consent using SharePoint Online user credentials that have the Global Admin role by following the steps detailed here, or proceed to the next step if you wish to provide consent from your Azure portal after creating your source.

Note

You can switch your source to the crawling account after you provide admin consent.
1. Enter the Email and Password of a SharePoint account with the Global Admin role.
2. Select Consent on behalf of your organization.
3. Click Accept.
4. To switch the source to the crawling account, click Authorize account again, enter your SharePoint Online Tenant name, click Sign In, and then proceed to the next step.

Enter the Email and Password of the crawling account that you created earlier and that has access to the desired SharePoint Online content, and then click Sign in.

Note

When you create two SharePoint Online sources retrieving content with the same tenant, they share their security providers, which increases the speed of the security identities refresh operation. You must, however, use the same limited administrator credentials for both sources.

"Content to Include" Section

Specify the content that your source indexes and makes searchable to users in a Coveo-powered search interface.

Note

For implementations using the Quickview component in a Coveo JavaScript Search Framework result template, and ASPX list items in SharePoint Online, quick view is supported only for ASPX list items (pages) of type Wiki, Publishing, Modern, and Web Part.

Select whether you want to index SharePoint Online or OneDrive content:

OneDrive: Index only the document libraries in OneDrive, including the My Files content, of users' personal sites. If you want to index all content of users' personal sites (all site collections), in addition to document libraries, select SharePoint Online, and then choose the Personal sites option. However, if you only want to index user documents, we recommend using the OneDrive option to limit the crawling scope.

SharePoint Online: You can choose to index all or specific SharePoint Online sites, lists, all content in user personal sites, or user profiles.

Notes

User access to the indexed items through a Coveo-powered search interface depends on your source Content Security setting. Personal/OneDrive documents and folders are private unless they are shared with others.
You can configure your source to ignore specific SharePoint Online list template types when indexing items.

Specify the corresponding options.

For OneDrive, your source indexes the OneDrive documents for the users to which the source has access. You can select Folders to also index folders and document sets.

Note

For certificate authentication, your source has access to all user content. For delegated authentication, the crawling account must be set as an owner in all personal sites that you want to index.

For SharePoint Online:

Select the content to retrieve:

All sites

For certificate authentication, all sites in your SharePoint Online tenant will be indexed and searchable. For delegated authentication, only the sites that the crawling account is allowed to access will be indexed and searchable.

Note

This option corresponds only to top-level site collections and their associated content. It does not include personal-site content.

Hub sites

You can choose to index the content of all sites that are associated with a SharePoint hub site. This includes all the associated site’s subsites and lists.

Note

For delegated authentication, the crawling account must have access to the hub site and the associated sites. If the crawling account has access only to a subset of the associated sites, just those sites will be indexed and searchable.

In the URL field, enter the URL corresponding to the desired hub site. Each URL must include the protocol and tenant name.

Example

Sites https://site:8080/sites/support and https://site:8080/sites/hr are associated with your SharePoint Online hub site (https://site:8080/sites/Main), so you enter https://site:8080/sites/Main in the URL field to index the content of both associated sites.

Specific items

You can choose to make only certain items searchable, such as specific sites, lists, websites, and subsites, by entering the corresponding URLs in the URL field. Each URL must include the protocol and tenant name.
Notes

For delegated authentication, the crawling account must have access to the specified sites.

A specific folder in a list isn’t supported.
Examples
- For a specific site: https://site:8080/sites/support
- For a specific website: https://site:8080/sites/support/subsite
- For a specific list: https://site:8080/sites/support/lists/contacts/allItems.aspx
Personal sites

You can choose to index only the content of personal sites, which includes site collections and OneDrive documents, from your SharePoint Online tenant.

Note

For delegated authentication, the crawling account must be set as an owner in all personal sites that you want to index.
User profiles

You can choose to index only the user profiles in your SharePoint Online tenant.

Note

For delegated authentication, the crawling account must be set as an owner in the personal sites for the user profiles that you want to index.

If you selected All sites, Hub sites, Specific items, or Personal sites, under Additional content, select whether to index the following:

Folders

Select this option to index list folders and document sets.

Unapproved items

Select this option to retrieve unapproved items, which are items with a Draft or Pending approval status, from lists where moderation is activated. If an unapproved version exists for an item that is already Approved, your source indexes the unapproved item instead of the approved item. As a result, the unapproved item appears in Coveo search results. If this option is disabled, your source indexes only Approved items.

Example

In a list where moderation is active, a document named Meeting Notes is Approved and indexed by Coveo. This document version is 1.0. However, a coworker edits Meeting Notes, thereby creating version 1.1, and the document status becomes Draft. Then, your SharePoint Online source is rescanned. If Unapproved items is enabled in your source, version 1.0 is deleted from the Coveo index and is replaced with the draft version 1.1. If Unapproved items is disabled in your source, Coveo indexes version 1.0 as version 1.1 is not yet Approved.

In lists where moderation is deactivated, Coveo indexes the latest version of an item, be it Approved, Draft, or Pending. In this case, this option doesn’t apply.

Note

For SharePoint lists that require documents to be checked out before editing, Coveo doesn’t index a document while it’s checked out regardless of the Unapproved items option and the list moderation setting in SharePoint. If a checked out item is checked in and its status changes to Draft or Pending, the unapproved item is indexed only if the Unapproved items option is enabled in your source or if moderation is deactivated for the list.

"Filters" Section

Use this section to include or exclude content from specific pages based on URL expressions. You can also configure your source to ignore specific SharePoint list template types when indexing items.

Note

You can view your URL expressions in the addressPatterns attribute of your source JSON configuration panel.
Inclusion and Exclusion filters for Sharepoint Online sources are not case sensitive when using a Regex (regular expression). For example,(company-(dev|staging)).*html.?$ will match http:// ComPanY-dev/important/document.html without adding any additional symbols to account for case sensitivity. Inclusion and Exclusion filters are case sensitive when using Wildcard expressions.

Inclusion Filters

Your source indexes only the pages that match a URL expression specified in this section.

Note

The index URL(s) for your source must be part of the inclusion filter scope, otherwise the corresponding content won’t be indexed. For example, if you entered https://site:8080/sites/support for the Specific items SharePoint Online option, that URL must match one of your filter expressions to index the corresponding content. If a source URL redirects to another URL, both URLs must be part of the inclusion filter scope.

Inclusion filters for Sharepoint Online sources are not case sensitive when using a Regex (regular expression). For example,(company-(dev|staging)).*html.?$ will match http:// ComPanY-dev/important/document.html without adding any additional symbols to account for case sensitivity. Inclusion filters are case sensitive when using Wildcard expressions.

Enter a URL expression to apply as the inclusion filter.
Select whether the URL expression uses a Wildcard or a Regex (regular expression) pattern.

Leading practice

You can test your regexes to ensure that they match the desired URLs with tools such as Regex101.

You can customize regexes to meet your use case focusing on aspects such as:

Case insensitivity
Capturing groups
Trailing slash inclusion
File extension

For example, you want to index HTML pages on your company staging and dev websites without taking the case sensitivity or the trailing slash (/) into account, so you use the following regex:

(?i)^.*(company-(dev|staging)).*html.?$

The regex matches the following URLs:

http://company-dev/important/document.html/
http://ComPanY-DeV/important/document.html/ (because of (?i), the case insensitive flag)
http://company-dev/important/document.html (with or without trailing / because of .?)
http://company-staging/important/document.html/ (because of dev|staging)

but doesn’t match the following ones:

http://besttech-dev/important/document.html/ (besttech isn’t included in the regex)
http://company-dev/important/document.pdf/ (only html files are included)
http://company-prod/important/document.html/ (prod isn’t included in the regex)

Example

The www.mycompany.com website you crawl contains versions in several languages and you want to have one source per language. For the US English source, if the source URL is www.mycompany.com/en-us/welcome.html, the inclusion filter would be www.mycompany.com/en-us/*.

Exclusion Filters

Your source ignores content from pages that match a URL expression specified in this section.

Note

When you specify an exclusion filter, the index URL(s) for your source must not be part of the exclusion filter scope, otherwise the corresponding content won’t be indexed. For example, if you entered https://site:8080/sites/support for the Specific items SharePoint Online option, and that URL matches one of your filter expressions, the corresponding content won’t be indexed. If a source URL redirects to another URL, both URLs must not be part of the exclusion filter scope.

Enter a URL expression to apply as the exclusion filter.

Notes

Exclusion filters also apply to shortened and redirected URLs.
By default, if pages are only accessible via excluded pages, those pages will also be excluded.
Exclusion filters for Sharepoint Online sources are not case sensitive when using a Regex (regular expression). For example,(company-(dev|staging)).*html.?$ will match http:// ComPanY-dev/important/document.html without adding any additional symbols to account for case sensitivity. Exclusion filters are case sensitive when using Wildcard expressions.

Select whether the URL expression uses a Wildcard or a Regex (regular expression) pattern.

Examples

There’s no point in indexing the search page of your website, so you exclude its URL:

www.mycompany.com/en-us/search.html
You don’t want to index ZIP files that are linked from website pages:

www.mycompany.com/en-us/*.zip

List Template Types to Ignore

You can configure your SharePoint Online source to ignore specific SharePoint list template types when indexing items.

Enter the list template types to ignore by adding a separate entry for each template type.

Example

You don’t want your source to index DocumentLibrary and Tasks template-type items. Therefore, you enter the following:

Leading practice

Once configured, the list template types to ignore appear in the ListTypesToIgnore parameter of your source JSON configuration panel.

"ListTypesToIgnore": {
   "sensitive": false,
   "value": "DocumentLibrary, Tasks"
 },

"Content Security" Tab

Select who will be able to access the source items through a Coveo-powered search interface. For details on this parameter, see Content security.

Note

When using the Same users and groups as in your content system content security option, you can map Microsoft 365 email aliases to their corresponding primary email addresses so that your repository’s content permissions are respected when a user logs in to a Coveo search interface using an email alias.

When using the Everyone content security option, see Safely Apply Content Filtering for information on how to ensure that your source content is safely filtered and only accessible by intended users.

"Access" Tab

In the Access tab, set whether each group (and API key, if applicable) in your Coveo organization can view or edit the current source.

For example, when creating a new source, you could decide that members of Group A can edit its configuration while Group B can only view it.

See Custom access level for more information.

Completion

Finish adding or editing your source:

When you want to save your source configuration changes without starting a build/rebuild, such as when you know you want to do other changes soon, click Add source/Save.

Note

On the Sources (platform-ca | platform-eu | platform-au) page, you must click Launch build or Start required rebuild in the source Status column to add the source content or to make your changes effective, respectively.

When you’re done editing the source and want to make changes effective, click Add and build source/Save and rebuild source.

Back on the Sources (platform-ca | platform-eu | platform-au) page, you can follow the progress of your source addition or modification.

Once the source is built or rebuilt, you can review its content in the Content Browser.

Note

If you selected Specific Items or User Profiles in the Content to Include section, some additional items will appear in the Content Browser. To retrieve user profiles, Coveo must dig through your SharePoint Online instance, including your My Site host site collection and the documents it contains. The items it encounters in the process are retrieved as well and therefore appear in the Content Browser.

Optionally, consider editing or adding mappings once your source is done building or rebuilding.

Safely apply content filtering

The best way to ensure that your indexed content is seen only by the intended users is to enforce content security by selecting the Same users and groups as in your content system option. Should this option be unavailable, select Specific users and groups instead.

However, if you need to configure your source so that the indexed source content is accessible to Everyone, you should adhere to the following leading practices. These practices ensure that your source content is safely filtered and only accessible by the appropriate users:

Configure query filters: Apply filter rules on a query pipeline to filter the source content that appears in search results when a query goes through that pipeline.
Use condition-based query pipeline routing: Apply a condition on a query pipeline to make sure that every query originating from a specific search hub is routed to the right query pipeline.
Configure the search token: Authenticate user queries via a search token that’s generated server side that enforces a specific search hub.

Following the above leading practices results in a workflow whereby the user query is authenticated server side via a search token that enforces the search hub from which the query originates. Therefore, the query can’t be modified by users or client-side code. The query then passes through a specific query pipeline based on a search hub condition, and the query results are filtered using the filter rules.

Configure query filters

Filter rules allow you to enter hidden query expressions to be added to all queries going through a given query pipeline. They’re typically used to add a field-based expression to the constant query expression (cq).

Example

You apply the @objectType=="Solution" query filter to the pipeline to which the traffic of your public support portal is directed. As a result, the @objectType=="Solution" query expression is added to any query sent via this support portal.

Therefore, if a user types Speedbit watch wristband in the search box, the items returned are those that match these keywords and whose objectType has the Solution value. Items matching these keywords but having a different objectType value aren’t returned in the user’s search results.

To learn how to configure query pipeline filter rules, see Manage Filter Rules.

Note

You can also enforce a filter expression directly in the search token.

Use condition-based query pipeline routing

The most recommended and flexible query pipeline routing mechanism is condition-based routing.

When using this routing mechanism, you ensure that search requests are routed to a specific query pipeline according to the search interface from which they originate, and the authentication is done server side.

To accomplish this:

Apply a condition to a query pipeline based on a search hub value, such as Search Hub is Community Search or Search Hub is Agent Panel. This condition ensures that all queries that originate from a specific search hub go through that query pipeline.
Authenticate user queries via a search token that’s generated server side and that contains the search hub parameter that you specified in the query pipeline.

Configure the search token

When using query filters to secure content, the safest way to enforce content security is to authenticate user queries using a search token that’s generated server side. For instance, when using this approach, you can enforce a search hub value in the search token. This makes every authenticated request that originates from a component use the specified search hub, and therefore be routed to the proper query pipeline. Because this configuration is stored server side and encrypted in the search token, it can’t be modified by users or client-side code.

Implementing search token authentication requires you to add server side logic to your web site or application. Therefore, the actual implementation details will vary from one project to another.

The following procedure provides general guidelines:

Note

If you’re using the Coveo In-Product Experience (IPX) feature, see Implement advanced search token authentication.

Authenticate the user.
Call a service exposed through Coveo to request a search token for the authenticated user.
Specify the userIDs for the search token, and enforce a searchHub parameter in the search token.

Note

You can specify other parameters in the search token, such as a query filter.

For more information and examples, see Search token authentication.

Update a SharePoint Online Access Token

Your SharePoint Online source uses the OAuth 2.0 authorization protocol to access your SharePoint Online site content via an Azure Active Directory application that has the required permissions (see About Authentication and Site Access).

The access token is linked to the certificate or SharePoint Online user account (crawling account) that you specified in your source configuration, and you must update the access token manually if it’s no longer valid. An invalid access token occurs when:

(certificate authentication) the certificate expires
(delegated authentication) the SharePoint Online crawling account’s credentials (email and/or password) are modified

An Authentication issue error appears for your source on the Sources (platform-ca | platform-eu | platform-au) page when your SharePoint Online source access token is no longer valid.

Note

A source Authentication issue error may also appear due to configuration or connectivity issues. If the certificate hasn’t expired, or the crawling account’s credentials haven’t changed, verify the following:

(certificate authentication) the certificate is assigned to the Azure Active Directory application that you created for use with your source, and the Azure Active Directory application has the required permissions (see Create the Azure Active Directory Application)
(delegated authentication) the crawling account has sufficient privileges
the Azure Active Directory is running (see Azure Status)

To update the access token

For certificate authentication:
1. Create a new certificate.
2. Add your certificate to the Azure Active Directory application that you created for use with your source.
3. On the Sources (platform-ca | platform-eu | platform-au) page, click your SharePoint Online source, and then click Edit in the Action bar.
4. In the Authentication section, click Certificate file to upload your new certificate.
5. Enter the Certificate password.
6. Click Save or Save and rebuild source.
For delegated authentication:
1. On the Sources (platform-ca | platform-eu | platform-au) page, click your SharePoint Online source, and then click Edit in the Action bar.
2. In the Authentication section, click Authorize Account.
3. Enter your SharePoint Online Tenant name or tenant address, and then click Sign In.
4. Enter the Email and Password of the crawling account, and then click Sign in.
5. Click Save or Save and rebuild source.

Required privileges

You can assign privileges to allow access to specific tools in the Coveo Administration Console. The following table indicates the privileges required to view or edit elements of the Sources (platform-ca | platform-eu | platform-au) page and associated panels. See Manage privileges and Privilege reference for more information.

Note

The Edit all privilege isn’t required to create sources. When granting privileges for the Sources domain, you can grant a group or API key the View all or Custom access level, instead of Edit all, and then select the Can Create check box to allow users to create sources. See Can Create ability dependence for more information.

Actions

Service

Domain

Required access level

View sources, view source update schedules, and subscribe to source notifications

Content

Fields

View

Sources

Organization

Edit sources, edit source update schedules, and view the View Metadata page

Content

Fields

Edit

Sources

Content

Source metadata

View

Organization

What’s Next?

Schedule source updates.

1. The entity tag is the version identifier of an item and is calculated using the item metadata.

Add or Edit a SharePoint Online Source

Add or Edit a SharePoint Online Source

Source Key Characteristics

About Authentication and Site Access

Certificate Authentication

Delegated Authentication

Requirements

Certificate Authentication Requirements

Create an SSL Certificate

Create the Azure Active Directory Application

Delegated Authentication Requirements

Domain Name System Records Configuration for Microsoft 365

Azure Application Permissions

Provide Admin Consent

Add or Edit a SharePoint Online Source

"Configuration" Tab

General Information

Source Name

Optical Character Recognition (OCR)

"Authentication" Section

"Content to Include" Section

"Filters" Section

Inclusion Filters

Exclusion Filters

List Template Types to Ignore

"Content Security" Tab

"Access" Tab

Completion

Safely apply content filtering

Configure query filters

Use condition-based query pipeline routing

Configure the search token

Update a SharePoint Online Access Token

Required privileges

What’s Next?