Create a SharePoint Online Crawling Account With Permissions

This is for:

Developer System Administrator

This article describes how to create a crawling account with appropriate roles and permissions when using the User delegated access using OAuth 2.0 method for your SharePoint Online source. It also details how to grant the crawling account access to crawl specific sites.

Note

When using the app authentication using certificate method for your SharePoint Online source, creating a crawling account and providing access to sites isn’t required.

When you create a SharePoint Online source using User delegated access using OAuth 2.0, an Azure Active Directory application is automatically created in your Azure tenant with the appropriate application permissions. The Azure Active Directory application is authenticated using the crawling account, and site and content access is limited to the crawling account’s permissions.

Note

The Azure Active Directory application that’s automatically created in your SharePoint Online tenant after you create your source appears as SharePoint Online Connector in your Azure portal’s Enterprise applications page.

A SharePoint administrator can grant the crawling account permission to access individual sites. Alternatively, the administrator can add the crawling account to a group that either has access to certain sites with the appropriate permission level, or that hasn’t been granted Site Admin (previously called Site Collection Administrator) permissions by the administrator. Depending on your company’s internal security policy, you can choose to grant access to all or specific sites and profiles.

Create a SharePoint Online Account With Appropriate Roles and Permissions

To index SharePoint Online content, you must create a SharePoint Online account (crawling account), which will only be used for the source, that has access to the content you want to make searchable. If you allow Coveo to retrieve your content through your personal account, you’ll need to also update the source access token each time you change the account password to prevent authentication errors.

  1. Access your Azure Portal with an administrator account.

  2. In Azure, create an account with the following roles:

    Role Description

    Application Administrator

    This role allows the user to provide consent for the Azure Active Directory application permissions from the Azure portal.

    If you don’t want the crawling account to have that role, you need to consent with a user that has the Global Admin role before logging in with the crawling account when creating your source, or you can provide consent with a Global Admin user from the Azure portal after creating your source.

    SharePoint Administrator

    This role is needed for the site URLs autodiscovery, and is required if you choose to index Hub site URLs or All sites in your SharePoint Online source. This role simply provides permission to scan for URLs when using Hub site URLs or All sites. The specific sites that your crawling account has access to based on its permissions and is set up in the next step.

    If you don’t want the crawling account to have that role, you must select Specific URLs, Personal sites, or User profiles, for your SharePoint Online source.
    Notes

    • For some permissions, the Required Admin parameter is set to true, and therefore the user must have a limited administrator account with the following roles: Application Administrator and SharePoint Administrator.

    • To crawl all sites, the crawling account must have the above mentioned roles and must also be a Site Admin.

  3. Access your SharePoint Online tenant with an account that has the SharePoint Administrator role, and then grant appropriate SharePoint Online permissions to the crawling account to ensure it has access to all the content that you want to index.

    The following table presents the minimal required permissions that the account must have to perform the specified action.

    Important

    If you specified sites to crawl and you didn’t grant the minimal permissions, the crawler will stop. If you selected All sites for your SharePoint Online source, it will skip sites that the crawling account can’t see.
    To crawl Minimal required permission

    Site content (with security indexing)

    Set the crawling account as a Site Admin (previously called Site Collection Administrator) for all SharePoint Online sites that you want to crawl (see Add Permission for a Specific Site).

    Site content (without security indexing)

    We recommend that you set the crawling account as a Site Admin for every site you want to crawl to avoid permission misconfiguration. If you don’t want the crawling account to be a site admin, it requires the following minimal permissions for every site that you want to crawl. For permission information see Permission levels in SharePoint:

    • Site permissions:

      • View Pages - View pages in a Web site.

      • Open - Allows users to open a Web site, list, or folder in order to access items inside that container.

    • List permissions:

      • View Items - View items in lists and documents in document libraries.

      • Open Items - View the source of documents with server-side file handlers.

      • View Versions - View past versions of a list item or document.

    Personal sites and user profiles

    Set the crawling account as an Owner of all personal sites, which includes a user’s OneDrive content, that you want to crawl (see Add Permission for a Personal Site).

    Note

    User access to the indexed items through a Coveo-powered search interface depends on your source Content Security setting. Personal/OneDrive documents and folders are private unless they are shared with others.

Add Permission for a Specific Site

You can grant the crawling account permission to access a specific site either by setting the crawling account as a Site Admin for the site, or by adding the crawling account, or a group to which the crawling account is a member, as a Group Owner for the site.

Note

If you don’t provide the crawling account with permission to sites, your source can only crawl anonymous sites that are public and unrestricted.

  1. Create the crawling account with the proper roles and permissions.

  2. Add the crawling account as a site admin or group owner for the sites that you want to crawl.

Add Permission for a Personal Site

You can grant the crawling account permission to access a specific user’s personal site in SharePoint Online by setting the crawling account as a site collection owner. To index the content of personal sites, your SharePoint Online source must be set to retrieve either OneDrive or Personal sites content.

To give the crawling account permission to crawl a user’s personal site, see Add and remove admins for a user’s OneDrive.

Alternatively, you can use a PowerShell script to facilitate the task of granting permissions to the crawling account for all personal site collections.

Grant the Owner Permission Using SharePoint Online Management Shell

You must regularly perform the following procedure when you want to grant permissions for site collections of new users.

  1. Install SharePoint Online Management Shell.

  2. Download the zipped COVEOSPO.PS1 script file to the server where the SharePoint Online management shell was previously installed.

  3. Unzip the file.

  4. In the Windows menu, select Start > All Programs > SharePoint Online Management Shell.

  5. Load the COVEOSPO.ps1 script, for instance with the following command: $> . C:\script\COVEOSPO.ps1.

  6. Run the Set-COVEOSPOMySitesOwner and Set-COVEOSPOSitesAdmin cmdlets. The following table lists the parameters supported by each of the cmdlets:

    Parameter and definition Set-COVEOSPOMySitesOwner Set-COVEOSPOSitesAdmin

    AdminSiteUrl
    Specifies the URL of the SharePoint Online tenant.

    check

    check

    AdminUsername
    Specifies the username of the SharePoint Online global administrator used to connect to the SharePoint server. This user will be added to the sites collection administrators (for the Set-COVEOSPOMySitesOwner cmdlet) or the personal sites administrators (for the Set-COVEOSPOSitesAdmin cmdlet) if the NewAdminUsername parameter is empty.

    check

    check

    AdminPassword
    Specifies the password of the SharePoint Online global administrator used to connect to the SharePoint server.

    check

    check

    UsersDomainName
    Specifies the domain of the users from which to retrieve personal sites.

    check

    x

    NewAdminUsername
    Specifies the username of one or more SharePoint Online users and/or group(s) to be added in the sites collection administrators (for the Set-COVEOSPOMySitesOwner cmdlet) or the personal sites administrators (for the Set-COVEOSPOSitesAdmin cmdlet). If not set, the user specified in the AdminUsername` parameter will be added.

    Notes

    • You can add users and groups at the same time by separating values with comma.

    • You must find the group ID to add the associated users in the sites collection or personal sites administrators.

    check

    check

    Remove
    This parameter is a switch that, when included in the script, removes the user(s) and/or group(s) specified in the NewAdminUsername parameter (instead of adding them) from the sites collection administrators (for the Set-COVEOSPOMySitesOwner cmdlet) or the personal sites administrators (for the Set-COVEOSPOSitesAdmin cmdlet).

    check

    check
Examples

  • $> Set-COVEOSPOMySitesOwner -AdminSiteUrl https://acme-admin.sharepoint.com -AdminUsername admin@acme.onmicrosoft.com -AdminPassword password -UsersDomainName acme.onmicrosoft.com

  • $> Set-COVEOSPOSitesAdmin -AdminSiteUrl https://acme-admin.sharepoint.com -AdminUsername globaladmin@acme.onmicrosoft.com -AdminPassword password -NewAdminUsername "user@acme.onmicrosoft.com", "c:0-.f|rolemanager|s-1-5-21-2644810858-3409521387-2709630237-4818302"

Find a SharePoint Group ID

  1. Repeat the xref:add-permission-for-a-specific-site,Add Permission for a Specific Site] procedure for a group up to the step where you open the site collection owners panel.

  2. Access the source code of the panel by pressing F12 or by right-clicking, and then selecting Inspect (Google Chrome) or View Page Source (Firefox).

  3. In the window that appears, in the source code, prior to displaytext='GroupName', copy the value of the key parameter (key='GroupID').

    Getting the group ID from the source code

    You can now paste the group ID in the NewAdminUsername parameter to add/remove the group members in/from the sites collection or personal sites administrators.