Push API tutorial 2: Manage secured content

This is for:

Developer System Administrator

In the previous tutorial, you created a public Push source in which you added, and then deleted, a simple item.

In this tutorial, you’ll configure your Push source to make it secure and create its associated security identity provider. You’ll then perform a small mock content update, followed by a small mock security identity update.

Note

This tutorial assumes that the permissions in the fictional content repository are entirely determined at the item level.

Prerequisites

You need the setup described in the Push API Tutorials article.

Step 1: Secure your Push source content

With a secured content Push source, you can manage items with permission models. The index uses a permission model to determine whether a user can see a given item in their query results.

  1. Access the Sources (platform-ca | platform-eu | platform-au) page in the Coveo Administration Console.

  2. Click your Push source, and then click Edit in the Action bar.

  3. On the Edit a Push source page, select the Content security tab.

  4. Select Same users and groups as in your current permission system.

  5. Click Save.

Step 2: Create a security identity provider

The security identity provider of a secured source is a Coveo organization module whose main purpose is to crawl a specific secured enterprise system on a regular basis to retrieve its latest security identity relationships and forward this data to the security identity cache (see Security identity cache and provider). A security identity provider allows the items in a secured source to reference existing and valid security identities in their permission models. This ensures that only the users who are allowed to see a specific item in its original secured repository can see this item in their query results. See Typical Coveo secured search for more details.

You need to use the Security Cache API Create a security provider for an organization HTTP request to create the security identity provider.

Note

The Security Cache API Create a security provider for an organization HTTP request requires the Edit access level on the Security identities domain, which isn’t granted by an API key created specifically for a Push source.

To create the security identity provider

  1. Click the relevant link to open the Security Cache API Swagger UI Create a security provider for an organization HTTP request page associated with your Coveo organization region:

  2. Click Authorize in the top-right corner of the page, and then log in to the Coveo Platform using your Coveo organization administrator account.

  3. In the Create a security provider for an organization HTTP request section, click Try it out.

  4. Under Parameters, set the organizationId value to your Coveo organization ID.

  5. Under Request body, paste the following:

    {
  "id" : "Push API Tutorial Security Identity Provider",
  "displayName" : "Push API Tutorial Security Identity Provider",
  "nodeRequired": false,
  "type": "EXPANDED",
  "referencedBy": [
    {
      "id": "<SOURCE_ID>", 1
      "type": "SOURCE"
    }
  ],
  "cascadingSecurityProviders": {
      "Email Security Provider": {
        "id": "Email Security Provider",
        "type": "EMAIL"
    }
  }
}
    1 Replace <SOURCE_ID> with the ID of your secured Push source.

  6. Click Execute.

You should receive a 200 OK response, and the Response body section should contain information about the newly created security identity provider.

Note

Initially, your security identity provider has no means of automatically crawling its intended target secured content repository. You must eventually implement this behavior with your own code, but that’s beyond the scope of this tutorial.

Step 3: Add items to your source

You’ll now send three Push API requests to add items to your Push source, using the Add or update an item request each time.

  1. In your HTTP client, create a new PUT request using the Push API endpoint corresponding to your deployment region.

    US East region 
    PUT https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    PUT https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    PUT https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    PUT https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the first HTTP request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables and payload have already been set for the three requests.

    • Set the organizationId and sourceId path parameters to the values you noted in the Push API Tutorials setup article.

    • Use your API key as the Authorization header accessToken value.

    • Set the documentId query parameter value to file://folder/. This URI doesn’t exist, but that doesn’t matter as you’re only pushing dummy items for this tutorial.

    • Use the following JSON as your request body. Its permissions property indicates that any anonymous user can see the item in their query results.

       {
   "data": "",
   "permissions": [
     {
       "allowAnonymous": true
     }
   ]
 }

  3. Send the request. You should receive a 202 response.

  4. For the second request, use the same endpoint and headers as in the first request. Complete the request configuration as follows:

    • Set the documentId query parameter value to file://folder/item1.txt.

    • Use the following JSON as your request body. Its permissions property indicates that only users asmith@example.com and bjones@example.com can see the item in their query results. You’ll define these security identities in step 4.

       {
   "data": "Item 1 raw textual data",
   "permissions": [
     {
       "allowAnonymous": false,
       "allowedPermissions": [
         {
           "identity": "asmith@example.com",
           "identityType": "User"
         },
         {
           "identity": "bjones@example.com",
           "identityType": "User"
         }
       ]
     }
   ]
 }

  5. Send the request.

  6. For the third request, use the same endpoint and headers as in the previous requests. Complete the request configuration as follows:

    • Set the documentId query parameter value to file://folder/item2.txt.

    • Use the following JSON as your request body. Its permissions property indicates that only the user asmith@example.com can see this item in their query results. Because bjones@example.com is listed in both allowed and denied permissions, this user is effectively denied access to the item in their query results.

       {
   "data": "Item 2 raw textual data",
   "permissions": [
     {
       "allowAnonymous": false,
       "allowedPermissions": [
         {
           "identity": "asmith@example.com",
           "identityType": "User"
         },
         {
           "identity": "bjones@example.com",
           "identityType": "User"
         }
       ],
       "deniedPermissions": [
         {
           "identity": "bjones@example.com",
           "identityType": "User"
         }
       ]
     }
   ]
 }

  7. Send the request.

You may have to wait several minutes before these three items become available in your source.

Note

You’ll learn how to push batches of items in Tutorial 3. This tutorial also covers how to include metadata and complex permission models.

Step 4: Add security identities to your security identity provider

The permission models of the second and third items you pushed in step 3 reference two security identities: asmith@example.com and bjones@example.com. However, these security identities haven’t been defined in the security identity provider you created in step 2. As a result, after the Coveo indexing pipeline processes these items, the unknown security identities will cause errors. The users won’t be able to see the items in their query results.

To fix this, you’ll now add the asmith@example.com and bjones@example.com security identities to the security identity provider using an Add or update a security identity request for each.

  1. In your HTTP client, create a new PUT request using the Push API endpoint corresponding to your deployment region.

    US East region 
    PUT https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    PUT https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    PUT https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    PUT https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    The Coveo Push API Tutorials collection comes with the securityIdentityProviderId variable set to the id value specified in the body of the security identity provider creation request performed earlier. All the necessary variables and payload have already been set for this request.

    • Set the organizationId and providerId path parameter values. The providerId value should be Push API Tutorial Security Identity Provider, that is, the id value in the body of the security identity provider creation request performed earlier.

    • Use your API key as the Authorization header accessToken value.

    • Use the following JSON as your request body to define the asmith@example.com user security identity.

       {
   "identity": {
     "name": "asmith@example.com",
     "type": "USER"
   }
 }

  3. Send the request.

  4. For the second request, use the same endpoint and headers as in the previous request.

  5. Use the following JSON as your request body to define the bjones@example.com user security identity.

     {
   "identity": {
     "name": "bjones@example.com",
     "type": "USER"
   }
 }

  6. Send the request.

You should receive a 202 response for both requests.

Note

You’ll learn how to define batches of security identities including aliases, groups, and granted identities in the next tutorial.

Step 5: Validate the indexing status of the items

As a reminder, the table below summarizes the information and effective permissions on the items you pushed in step 3.

documentId data permissions

file://folder/

""

Anonymous check

file://folder/item1.txt

"Item 1 raw textual data"

Anonymous x
asmith@example.com check
bjones@example.com check

file://folder/item2.txt

"Item 2 raw textual data"

Anonymous x
asmith@example.com check
bjones@example.com x

The Coveo Administration Console Content Browser is a powerful tool that lets you emulate the query results that security identities would see in a search interface. You’ll now use the Content Browser to validate the successful indexing of the three items, both in terms of their content and permissions.

  1. Access the Sources (platform-ca | platform-eu | platform-au) page.

  2. On the Push source row, click Open in Content Browser.

    Show source items in the Content Browser

    Note that with the View all content option enabled in the Content Browser, you can see all items in the results.

    The View all content Content Browser option | Coveo

    As a Coveo organization administrator, you have the Allowed access level on the View all content domain and the privilege to bypass item permission models.

  3. On the Security Identities (platform-ca | platform-eu | platform-au) page, click your Push API security identity provider, and then click Browse identities in the Action bar.

  4. Click bjones@example.com, and then click Search as in the Action bar.

    You’re now impersonating bjones@example.com and seeing the two items that this user would see in a search interface.

    Searching as bjones in the Content Browser | Coveo

You can repeat the last steps to validate that asmith@example.com can see all three items.

Step 6: Delete the items

To begin the next tutorial with a clean source, you’ll now send a Delete an item and optionally its children request to delete the three items you added in this tutorial.

  1. In your HTTP client, create a new DELETE request using the Push API endpoint corresponding to your deployment region.

    US East region 
    DELETE https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId}&deleteChildren=true|false HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    DELETE https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId}&deleteChildren=true|false HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    DELETE https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId}&deleteChildren=true|false HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    DELETE https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents?documentId={documentId}&deleteChildren=true|false HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables have already been set for this request.

    • Set the organizationId and sourceId path parameter values.

    • Use your API key as the Authorization header accessToken value.

    • Set the documentId query parameter to file://folder/ and the deleteChildren query parameter to true. This deletes the item whose documentId is the one you specified and all items whose documentId values are nested under the documentId of the parent. In this case, the file://folder item and all other items matching file://folder/* in the target source will be deleted.

  3. Send the request.

You should receive a 202 response. Shortly after, the Sources (platform-ca | platform-eu | platform-au) page should show your Push source with No items in the Content column.

Step 7: Disable the security identities

By default, when you perform a Push API request that includes the optional orderingId parameter, the service automatically sets orderingId to the current number of milliseconds since the Unix epoch. Each individual Push API request receives its own timestamp value. The item orderingid field value, which you see when inspecting the item in the Content Browser, reflects the timestamp of the last Push API request that modified it.

You can compare the mandatory orderingId parameter value used in a Delete old items or Delete old security identities HTTP request to the optional orderingId parameter value used in other Push API HTTP requests.

To begin the next tutorial with a clean security identity provider, you’ll now disable the security identities using the Delete old security identities request.

  1. In your HTTP client, create a new DELETE request using the Push API endpoint corresponding to your deployment region.

    US East region 
    DELETE https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/olderthan?orderingId={orderingId} HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    DELETE https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/olderthan?orderingId={orderingId} HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    DELETE https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/olderthan?orderingId={orderingId} HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    DELETE https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/olderthan?orderingId={orderingId} HTTP/1.1
​
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    You need to set the Coveo Push API Tutorials collection orderingId variable value to the 13-digit current number of milliseconds since the Unix epoch.

    • Set the organizationId and providerId path parameters. The providerId value should be Push API Tutorial Security Identity Provider, that is, the id value in the body of the security identity provider creation request performed earlier.

    • Use your API key as the Authorization header accessToken value.

    • Set the orderingId query parameter value. Use the 13-digit current number of milliseconds since the Unix epoch. You can get it from https://currentmillis.com/.

  3. Send the request.

Afterward, refresh your security identity provider.

What’s next?

Proceed to Push API tutorial 3: Using batch updates to manage complex secured content.