Add or Edit a SharePoint Server Source

Members of the Administrators and Content Managers built-in groups can include SharePoint on-premises content and make it searchable. In a Coveo-powered search interface, the source content is accessible to either everyone, the source creator only, or specific users as determined by source permissions (see Content Security). By default, a SharePoint Server source is refreshed every six hours and rescanned every week to retrieve SharePoint item changes (addition, modification, or deletion). A source rescan or rebuild is necessary to capture deleted user profiles.

To retrieve SharePoint Online content, you must create a SharePoint Online source.

Source Features Summary

Features Supported Additional information
SharePoint version 2019, 2016, 2013, 2010, Foundation 2013, and Foundation 2010  
Searchable content types Sites, sub-sites, public user profiles1, personal websites1, lists, list items, list item attachments, document libraries, document sets, documents, web parts2, and microblog posts and replies.
Content update Refresh

Rescan or rebuild is required to retrieve deleted user profiles.

Rescan  
Rebuild  
Content security options Determined by source permissions On-premises Active Directory permission systems aren't supported with SharePoint Server sources of the On-Premises type. If you use the Crawling Module, however, Active Directory is supported.
Source creator  
Everyone  

Note 1: Not available in Microsoft SharePoint Foundation.

Note 2: Not all web parts are available in Microsoft SharePoint Foundation 2010 (see Web Parts in SharePoint Foundation).

Requirements

Active Directory Federation Services

When your SharePoint environment uses ADFS as a trusted identity provider, the ADFS service endpoint URL paths must be enabled (see Enable the ADFS Service Endpoint URL Path).

SharePoint Account Permissions

When you want to include SharePoint content, you must create a specific SharePoint account to be used by the source only. Otherwise, you need to also change the source Password value each time the account password changes to prevent authentication errors (see Username and Password).

  1. Access your SharePoint tenant with an administrator account.

  2. On your SharePoint tenant:

    1. Select or create a user account for the source to use when retrieving your SharePoint content. See the following table to identify the required type of user for your web application enabled authentication.

      SharePoint environment SharePoint web application enabled authentication User type User format
      Classic Windows Windows account

      domain\username

      or

      username@domain.com
      Claims Windows Windows account
      ADFS ADFS SSO
      Okta Okta SSO username@domain.com
    2. Grant appropriate SharePoint permissions to the SharePoint account to ensure it has access to the content that you want to make searchable.

      The following table presents the minimal required permissions that the source account must have to perform specific actions.

      Action to perform Minimal required permission
      Content and security indexing, source refresh, and site collection discovery

      Full Read policy for each web application to make searchable (see Add the Full Read Policy to All SharePoint Tenant Web Applications).

      Personal site, public user profile, and social tags indexing

      When including personal sites or public user profiles, the account used as source credentials must not have a personal site on the SharePoint server being included to prevent failures when attempting to retrieve the list of personal sites.

Add or Edit a SharePoint Server Source

Before you start, ensure that your SharePoint instance meets the source requirements.

When selecting the type of source you want to add, select the SharePoint Server option with the appropriate content retrieval method, depending on whether you need to use the Coveo On-Premises Crawling Module to retrieve your content.

“Configuration” Tab

In the Add/Edit a SharePoint Server Source subpage, the Configuration tab is selected by default. It contains your source general and authentication information, as well as other parameters.

General Information

Source Name

Enter a descriptive name for your source.

Use a short and descriptive name, using letters, numbers, - and _ characters, and avoid spaces and other special characters.

URL

Enter one or more URLs corresponding to the desired site collection, lists, websites, and subsites to make searchable. Each URL must include the protocol and tenant name.

  • For a specific web application: https://site:8080/

  • For a specific site collection: https://site:8080/sites/support

  • For a specific website: https://site:8080/sites/support/subsite

  • For a specific list: https://site:8080/sites/support/lists/contacts/allItems.aspx

    A specific folder in a list isn’t supported.

Scope

In the drop-down menu, select the option for the content type matching the URLs you specified. By default, Web application is selected.

Available options are the following:

Value Content to make searchable
Web application All site collections of the specified web application
Site collection All web sites of the specified site collection
Web and sub webs Only the specified web site and its sub webs (also known as subsites)
List Only the specified list or document library
Paired Crawling Module

If your source is a Crawling Module source and if you have more than one Crawling Module linked to this organization, select the one with which you want to pair your source. If you change the Crawling Module instance with which your source is paired, a successful rebuild is required for your change to apply.

Character Optical Recognition (OCR)

Check this box if you want Coveo Cloud to extract text from image files or PDF files containing images. OCR-extracted text is processed as item data, meaning that it’s fully searchable and will appear in the item Quick View.

Since the OCR feature is available at an extra charge, you must first contact Coveo Sales to add this feature to your organization license. You can then enable it for your source.

Index

When adding a source, if you have more than one logical (non-Elasticsearch) index in your organization, select the index in which the retrieved content will be stored (see Leverage Many Coveo Indexes). If your organization only has one index, this drop-down menu isn’t visible and you have no decision to make.

  • To add a source storing content in an index different than default, you need the View access level on the Logical Index domain (see Privilege Management and Logical Indexes Domain).

  • Once the source is added, you can’t switch to a different index.

Content Security

In the Content Security tab, select who will be able to access the source items through a Coveo-powered search interface. For details on this parameter, see Content Security.

“Authentication” Section

In the Authentication section, you must provide authentication information so that Coveo can access the content you want to make searchable. In the drop-down menu, select the identity provider you use to manage identities in your SharePoint site. Options are:

  • Active Directory On-Premises (available when using the Crawling Module only)

  • Windows (NTLM or Kerberos)

  • ADFS under claims (simple or chained ADFS identity provider)

  • Okta

Depending on the option you choose in the drop-down menu, you must fill or check some of the following boxes.

Username and Password

The username and password of a dedicated SharePoint administrator account that has access to the content to include or, if using Okta, the username of an Okta administrator account.

ADFS Server URL

The URL of an ADFS server trusted by SharePoint.

https://adfs01.subdomain.example.com

SharePoint Trust Identifier

The SharePoint server relying party trust identifier.

https://subdomain.example.com:44626/_trust

To find your relying party trust identifier:

  1. Access the AD FS 2.0 Management Console (Windows Start menu > All Programs > Administrative Tools > AD FS 2.0 Management).

  2. In AD FS 2.0 Management Console, under Trust Relationships, select Relying Party Trusts.

  3. In the Relying Party Trusts list, find the row for SharePoint. The ADFS relying party trust identifier is the value in the Identifier column.

ADFS Trust Identifier

The relying party trust identifier of the ADFS server acting as an intermediate.

http://adfs01.subdomain.example.com/adfs/services/trust

Identity Provider Server URL

The URL of the identity provider used in SharePoint to authenticate users.

https://adfs2012.subdomain.example.com

You can edit the identity provider server URL in the ADFS settings (see Enable the ADFS Service Endpoint URL Path).

Okta Realm

The SharePoint trusted identity provider realm provided in your Okta application configuration (see Using Okta as a Trusted Identity Provider).

urn:okta:sharepoint:exknuavz9hbOItwsS8e7

Okta Sign in URL

The URL to which users should be redirected in order to authenticate with Okta (see Using Okta as a Trusted Identity Provider).

https://dev-782461.oktapreview.com/app/appname/sso/wsfed/passive

Active Directory Username and Active Directory Password

Enter credentials to grant Coveo Cloud access to your Active Directory.

Expand Well-Known SIDs

Check this box if you want the users included in your Active Directory well-known security identifiers to be granted access to the indexed content. Expect an increase in the duration of the security identity provider refresh operation. Supported well-known SIDs are: Everyone, Authenticated Users, Domain Admins, Domain Users, and Anonymous Users.

If your entire site collection is secured with the Everyone or Authenticated users well-known, it’s more cost-effective resource-wise to index it with a SharePoint Server source whose content is accessible to everyone than to expand the well-known with a source that indexes permissions.

Enable TLS

Check this box to use a TLS protocol to retrieve your security identities. If you do, Coveo Cloud strongly recommends selecting StartTLS if you can. Since LDAPS is a much older protocol, you should only select this value if StartTLS is incompatible with your environment.

Email Attributes

By default, Coveo Cloud retrieves the email address associated to each security identity from the mail attribute. Optionally, you can specify additional or different attributes to check. Should an attribute contain more than one value, Coveo Cloud uses the first one.

“Content to Include” Section

In the Content to Include section, consider changing the default settings to make additional content searchable.

User Profiles

Check this box to index public SharePoint user profiles.

Personal Sites

When the Scope is Web application, check this box to include SharePoint personal sites.

“Crawling Settings” Section

In the Crawling Settings section, the Reindex all child items on UpdateShallow option allows you to reindex the children of an item that has been updated. This ensures that, if the metadata of the child items contains parent item information, this information stays up to date. However, checking this box significantly impacts the source refresh time. Therefore, if you don’t check it, we recommend scheduling source rescans so that the child items are eventually updated as well.

You change your SharePoint site name. In the metadata of the child items, the site name appears under spsitename. If the box isn’t checked, the children aren’t reindexed and keep an outdated spsitename until the next source rescan or rebuild. However, if the box is checked, the children are updated along with the parent SharePoint site item.

“Access” Tab

In the Access tab, determine whether each group and API key can view or edit the source configuration (see Understanding Resource Access):

  1. In the Access Level column, select View or Edit for each available group.

  2. On the left-hand side of the tab, if available, click Groups or API Keys to switch lists.

If you remove the Edit access level from all the groups of which you’re a member, you won’t be able to edit the source again after saving. Only administrators and members of other groups that have Edit access on this resource will be able to do so. To keep your ability to edit this resource, you must grant the Edit access level to at least one of your groups.

Completion

  1. Complete your source addition or edition:

    • Click Add Source/Save when you want to save your source configuration changes without starting a build/rebuild, such as when you know you want to do other changes soon.

      On the Sources page, you must click Start initial build or Start required rebuild in the source Status column to add the source content or make your changes effective, respectively.

      OR

    • Click Add and Build Source/Save and Rebuild Source when you’re done editing the source and want to make changes effective.

      Back on the Sources page, you can review the progress of your SharePoint Server source addition or modification.

    Once the source is built or rebuilt, you can review its content in the Content Browser.

  2. Optionally, consider editing or adding mappings.

    You can only manage mapping rules once you build the source (see Refresh, Rescan, or Rebuild Sources).

Additional Adjustments

  1. If your source retrieves your content through the Crawling Module and if access to its content is secured with an Active Directory security identity provider, you must edit the JSON configuration of the security identity provider associated to this source to provide additional information. In the security identity provider JSON configuration, add the following code snippet, in which you replace <HOSTNAME> with either your Active Directory server IP address or domain name to use to connect to your Active Directory.

     "Hostname": {
       "value": "<HOSTNAME>"
    

    If you provide a domain name, ensure that your DNS server can resolve this name.

  2. Moreover, if you checked the Enable TLS box, edit the JSON configuration of your SharePoint source, SharePoint security identity provider, and Active Directory security identity provider to add the following:

     "AlwaysTrustCertificates": {
       "value": "true"
     }
    

    See Edit a Source JSON Configuration and Edit a Security Identity Provider for detailed instructions.

What’s Next?

Recommended Articles