Push API tutorial 3: Use batch updates to manage complex secured content

This is for:

Developer System Administrator

In the previous tutorial, you configured a secured Push source along with its associated security identity provider, and you learned how to push simple secured items and security identities individually. You also learned how to delete an item and its children, as well as how to delete items or security identities based on the orderingId value of the last request that was performed on them.

In this tutorial, you’ll create fields in your index. You’ll then use Push API batch update requests, first to simulate a security identity update with different types of security identities, and then to push items that include coherent metadata and complex permission models. Finally, you’ll use the Coveo Administration Console to validate whether everything is working as expected.

Prerequisites

If you completed all steps of the previous tutorial, you should already have the required setup for this tutorial, that is:

  • A security identity provider.

  • A secured Push source.

The source needs to be empty. If necessary, use the Delete old items Push API HTTP request to delete all remaining items. The Delete old items request works the same way as the request you used to disable the security identities in step 7 of tutorial 2.

Step 1: Create fields

Fields let you store and leverage item metadata, allowing you to create facets, define custom query result sort criteria, etc.

Fields are index-wide data containers, which implies that any source can populate a given field with item metadata. For example, the title standard field in your index typically gets populated by item metadata from many sources.

By default, a Push source automatically attempts to map incoming item metadata to a field of the same name in your index. Therefore, the names of the fields you’re about to create are the same as the metadata keys you’ll use when pushing items to your source. These fields use the mmeps prefix.

Tip

While prefixing a field doesn’t change the fact that it’s an index-wide container, it allows you to quickly identify which fields are tied to a specific source.

  1. On the Fields (platform-ca | platform-eu | platform-au) page, click Add field.

  2. Enter the following information for your first field:

    Field Name: mmepsyear

    Type: String

    Description: The year in which the movie was released.

  3. Click Add field.

  4. Repeat the process to add the following fields:

    Field name Type Description Options

    mmepssummary

    String

    A summary of the movie.

    mmepsisseries

    String

    Whether the movie is part of a series.

    mmepsmpaafilmrating

    String

    The MPAA film rating for the movie. Can be 'G', 'PG', 'PG-13', 'R', 'NC-17', or 'UR'.

    Facet check

    mmepsbudget

    Decimal

    The budget of the movie, in million USD.

    mmepscriticsaverage

    Integer 64

    The average score that was given to the movie by critics, on a discrete scale of 1 to 100.

    Use cache for computed fields check

    mmepsdirector

    String

    The director of the movie.

    Facet check
    Sortable check
    Free text search check
    Use cache for sort check
    Use cache for nested queries check

    mmepscastoverview

    String

    The list of principal actors that appear in the movie.

    Multi-value facet check
    Free text search check
    Use cache for nested queries check

    mmepsthumbnaillink

    String

    The URI of the thumbnail image for the movie.

    mmepstrailerlink

    String

    The URI of the movie trailer.
Note

Some fields have the Free text search property set to true. For example, since the mmepsdirector field is free text searchable, an end user could submit a query such as Alfred Hitchcock from a search box and get matching results. However, since the mmepsmpaafilmrating field isn’t free text searchable, an end user would have to submit a field query such as @mmepsmpaafilmrating==G to retrieve matching items.

Step 2: Add a batch of items

You’ll now use Push API batch requests to add several items at once to your Push source. This a three-step process:

  1. Creating the file container.

  2. Uploading the batch of items to the container.

  3. Pushing the container to the Push source.

Create the file container

You’ll send a Create a file container request.

  1. In your HTTP client, create a new POST request using the Push API endpoint corresponding to your deployment region.

    US East region 
    POST https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    POST https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    POST https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    POST https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables have already been set for this request.

    • Set the organizationId path parameter value.

    • Use your API key as the Authorization header accessToken value.

  3. Send the request.

  4. Store the uploadUri, fileId, x-amz-server-side-encryption, and Content-Type values from the response. You’ll need these values in the next steps.

    Postman

    The Coveo Push API Tutorials collection includes a post-response script that automatically sets the uploadUri, fileId, serverSideEncryption, and contentType variable values required for the next steps.

Upload the batch of items to the container

  1. In your HTTP client, create a new PUT request.

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables and payload have already been set for this request.

    • Set the request endpoint to the uploadUri value you stored in the previous step.

    • Add the x-amz-server-side-encryption and Content-Type keys, and the values you stored for each in the previous step, to the request headers.

    • Set the request body to the JSON provided below. Here are details about the JSON structure:

      • Each element in the addOrUpdate array represents a movie item to add to your source. The documentId is the unique identifier of an item.

      • Every movie item contains standard metadata (for example, documenttype), as well as custom metadata (for example, mmepsdirector, mmepsyear). This metadata will populate fields in your index according to the default Push API mapping behavior.

      • Since you’re pushing items to a secured content source, each item must also have a permission model. The item permission models can have multiple permission levels, each of which can have several permission sets. You’ll define the security identities on which these permission models rely in step 3.

        More details on the item permissions

        Permission levels are always evaluated sequentially. As soon as a security identity is explicitly allowed by all permission sets of a lower permission level, it can’t be denied by any higher permission levels. See Permission levels for more details.

        In the JSON below, the first permission level is based on the default MPAA film rating. The second permission level is based on some other form of film rating, should a movie not yet have been rated by the MPAA. The third permission level contains parental approval information, and it’s further divided into two permission sets (one for each parent).

        For example, the movie whose title is 1900, the last item in the JSON, is unrated by the MPAA. That is, it doesn’t have any mmepsmpaamovierating metadata.

        As a result, the first permission level, which is based on the MPAA film rating, only allows adults (Grace, Henry, and yourself). Permissions for young children, pre-teens, and teenagers are still unknown at this level.

        The second permission level is based on some other form of film rating. It explicitly denies young children (Alice and Bob) and pre-teens (Chloe and David). Permissions for teenagers (Erin and Frank) are still unknown at this level.

        The third permission level, which contains parental approval information, indicates that:

        • Chloe, a pre-teen, is explicitly allowed by both permission sets, that is, both of her parents.

          However, since pre-teens were explicitly denied in the previous permission level, Chloe won’t be able to see the 1900 movie entry in her query results.

        • Erin, a teenager, is also explicitly allowed by both permission sets.

          Since teenager permissions were unknown in previous permission levels, Erin will be able to see the 1900 movie entry in her query results.

        • Frank, also a teenager, is explicitly denied in the first permission set and explicitly allowed in the second one.

          Since he isn’t allowed in all of the permission sets at this permission level (that is, one of his parents doesn’t want him to see this movie entry), Frank won’t be able to see the 1900 movie entry in his query results.

        In conclusion, the only users who are allowed to see the 1900 movie entry in their query results are Erin, Grace, Henry, and yourself.

      {
  "addOrUpdate": [
    {
      "documenttype": "Movie Entry",
      "filename": "Gone-with-the-Wind.html",
      "sourcetype": "Push",
      "title": "Gone with the Wind",
      "mmepsdirector": "Victor Fleming",
      "mmepsyear": "1939",
      "mmepssummary": "A manipulative woman and a roguish man conduct a turbulent romance during the American Civil War and Reconstruction periods.",
      "mmepsisseries": false,
      "mmepsmpaafilmrating": "G",
      "mmepsbudget": 3.9,
      "mmepscriticsaverage": 97,
      "mmepscastoverview": [
        "Thomas Mitchell",
        "Barbara O'Neil",
        "Vivien Leigh",
        "Evelyn Keyes",
        "Ann Rutherford"
      ],
      "mmepsthumbnaillink": "https://images-na.ssl-images-amazon.com/images/M/MV5BYWQwOWVkMGItMDU2Yy00YjIzLWJkMjEtNmVkZjE3MjMwYzEzXkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY1000_CR0,0,652,1000_AL_.jpg",
      "clickUri": "https://www.youtube.com/watch?v=8mM8iNarcRc",
      "data": "<html><head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.style-summary{word-wrap: break-word;}.style-title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.g-rating{color: black; background: white;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style></head><body> <div class='left-column'> <img src='https://images-na.ssl-images-amazon.com/images/M/MV5BYWQwOWVkMGItMDU2Yy00YjIzLWJkMjEtNmVkZjE3MjMwYzEzXkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY1000_CR0,0,652,1000_AL_.jpg' width='200'> </div><div class='right-column'> <div class='style-title'> <h1>Gone with the Wind</h1> </div><div class='director-year'> <p>Victor Fleming - 1939</p></div><div class='duration'> <p>3h58</p></div><div class='rating'> <span class='g-rating'>G</span> </div><div class='summary'> <p>A manipulative woman and a roguish man conduct a turbulent romance during the American Civil War and Reconstruction periods.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=8mM8iNarcRc'>View trailer</a> </div></div></body></html>",
      "fileExtension": ".html",
      "documentId": "file://movie-entries/1939/Gone-with-the-Wind.html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Everyone",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": []
            }
          ]
        }
      ]
    },
    {
      "documenttype": "Movie Entry",
      "filename": "The-Man-Who-Knew-Too-Much.html",
      "sourcetype": "Push",
      "title": "The Man Who Knew Too Much",
      "mmepsdirector": "Alfred Hitchcock",
      "mmepsyear": "1956",
      "mmepssummary": "A family vacationing in Morocco accidentally stumble on to an assassination plot and the conspirators are determined to prevent them from interfering.",
      "mmepsisseries": false,
      "mmepsmpaafilmrating": "PG",
      "mmepsbudget": 1.2,
      "mmepscriticsaverage": 91,
      "mmepscastoverview": [
        "James Stewart",
        "Doris Day",
        "Brenda de Branzie",
        "Bernard Miles",
        "Ralph Truman"
      ],
      "mmepsthumbnaillink": "https://images-na.ssl-images-amazon.com/images/M/MV5BNjQ2OWFhMjItYmFmNC00OGQzLWE1ZTMtZGQwZDM3YWUzMDgxL2ltYWdlL2ltYWdlXkEyXkFqcGdeQXVyMDI2NDg0NQ@@._V1_SY1000_CR0,0,656,1000_AL_.jpg",
      "mmepstrailerlink": "https://www.youtube.com/watch?v=Qw0OYSBI71U",
      "data": "<html><head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.summary{word-wrap: break-word;}.title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.pg-rating{color: black; background: yellow;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style></head><body> <div class='left-column'> <img src='https://images-na.ssl-images-amazon.com/images/M/MV5BNjQ2OWFhMjItYmFmNC00OGQzLWE1ZTMtZGQwZDM3YWUzMDgxL2ltYWdlL2ltYWdlXkEyXkFqcGdeQXVyMDI2NDg0NQ@@._V1_SY1000_CR0,0,656,1000_AL_.jpg' width='200'> </div><div class='right-column'> <div class='title'> <h1>The Man Who Knew Too Much</h1> </div><div class='director-year'> <p>Alfred Hitchcock - 1956</p></div><div class='duration'> <p>2h</p></div><div class='rating'> <span class='pg-rating'>PG</span> </div><div class='summary'> <p>A family vacationing in Morocco accidentally stumble on to an assassination plot and the conspirators are determined to prevent them from interfering.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=Qw0OYSBI71U'>View trailer</a> </div></div></body></html>",
      "fileExtension": ".html",
      "documentId": "file://movie-entries/1956/The-Man-Who-Knew-Too-Much.html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Adults",
                  "identityType": "Group"
                },
                {
                  "identity": "Teenagers",
                  "identityType": "Group"
                },
                {
                  "identity": "PreTeenagers",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": []
            }
          ]
        },
        {
          "name": "Parental Approval Permissions",
          "permissionSets": [
            {
              "name": "First Parent Decision",
              "allowedPermissions": [
                {
                  "identity": "alice@example.com",
                  "identityType": "User"
                }
              ],
              "deniedPermissions": []
            },
            {
              "name": "Second Parent Decision",
              "allowedPermissions": [],
              "deniedPermissions": [
                {
                  "identity": "alice@example.com",
                  "identityType": "User"
                }
              ]
            }
          ]
        }
      ]

    },
    {
      "documenttype": "Movie Entry",
      "filename": "Spartacus.html",
      "sourcetype": "Push",
      "title": "Spartacus",
      "mmepsdirector": "Stanley Kubrick",
      "mmepsyear": "1960",
      "mmepssummary": "The slave Spartacus leads a violent revolt against the decadent Roman Republic.",
      "mmepsisseries": false,
      "mmepsmpaafilmrating": "PG-13",
      "mmepsbudget": 12.0,
      "mmepscriticsaverage": 87,
      "mmepscastoverview": [
        "Kirk Douglas",
        "Laurence Olivier",
        "Jean Simmons",
        "Charles Laughton",
        "Peter Ustinov"
      ],
      "mmepsthumbnaillink": "https://images-na.ssl-images-amazon.com/images/M/MV5BMjc4MTUxN2UtMmU1NC00MjQyLTk3YTYtZTQ0YzEzZDc0Njc0XkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY999_CR0,0,660,999_AL_.jpg",
      "mmepstrailerlink": "https://www.youtube.com/watch?v=tFLCEDLCSHA",
      "data": "<html><head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.summary{word-wrap: break-word;}.title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.pg-13-rating{color: white; background: purple;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style></head><body> <div class='left-column'> <img src='https://images-na.ssl-images-amazon.com/images/M/MV5BMjc4MTUxN2UtMmU1NC00MjQyLTk3YTYtZTQ0YzEzZDc0Njc0XkEyXkFqcGdeQXVyNjU0OTQ0OTY@._V1_SY999_CR0,0,660,999_AL_.jpg' width='200'> </div><div class='right-column'> <div class='title'> <h1>Spartacus</h1> </div><div class='director-year'> <p>Stanley Kubrick - 1960</p></div><div class='duration'> <p>3h17</p></div><div class='rating'> <span class='pg-13-rating'>PG-13</span> </div><div class='summary'> <p>The slave Spartacus leads a violent revolt against the decadent Roman Republic.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=tFLCEDLCSHA'>View trailer</a> </div></div></body></html>",
      "documentId": "file://movie-entries/1960/Spartacus.html",
      "fileExtension": ".html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Adults",
                  "identityType": "Group"
                },
                {
                  "identity": "Teenagers",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": [
                {
                  "identity": "YoungChildren",
                  "identityType": "Group"
                }
              ]
            }
          ]

        },
        {
          "name": "Parental Approval Permissions",
          "permissionSets": [
            {
              "name": "First Parent Decision",
              "allowedPermissions": [
                {
                  "identity": "bob@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "david@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "chloe@example.com",
                  "identityType": "User"
                }
              ],
              "deniedPermissions": []
            },
            {
              "name": "Second Parent Decision",
              "allowedPermissions": [
                {
                  "identity": "bob@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "david@example.com",
                  "identityType": "User"
                }
              ],
              "deniedPermissions": []
            }
          ]
        }
      ]
    },
    {
      "documenttype": "Movie Entry",
      "filename": "Dangerous-Liaisons.html",
      "sourcetype": "Push",
      "title": "Dangerous Liaisons",
      "mmepsdirector": "Stephen Frears",
      "mmepsyear": "1988",
      "mmepssummary": "A scheming widow and her manipulative ex-lover make a bet regarding the corruption of a recently married woman.",
      "mmepsisseries": false,
      "mmepsmpaafilmrating": "R",
      "mmepsbudget": 14.0,
      "mmepscriticsaverage": 74,
      "mmepscastoverview": [
        "Glenn Close",
        "John Malkovich",
        "Michelle Pfeiffer",
        "Swoosie Kurtz",
        "Keanu Reeves"
      ],
      "mmepsthumbnaillink": "https://images-na.ssl-images-amazon.com/images/M/MV5BMDZmODc0MWQtYWVjOS00OGFiLWExNjItOTMyN2Q5ZTFmYzUzXkEyXkFqcGdeQXVyMTAwMzUyOTc@._V1_.jpg",
      "mmepstrailerlink": "https://www.youtube.com/watch?v=FbB2oBlP2uI",
      "data": "<html> <head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.summary{word-wrap: break-word;}.title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.r-rating{color: white; background: red;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style> </head> <body> <div class='left-column'> <img src='https://images-na.ssl-images-amazon.com/images/M/MV5BMDZmODc0MWQtYWVjOS00OGFiLWExNjItOTMyN2Q5ZTFmYzUzXkEyXkFqcGdeQXVyMTAwMzUyOTc@._V1_.jpg' width='200'> </div><div class='right-column'> <div class='title'> <h1>Dangerous Liaisons</h1> </div><div class='director-year'> <p>Stephen Frears - 1988</p></div><div class='duration'> <p>1h59</p></div><div class='rating'> <span class='r-rating'>R</span> </div><div class='summary'> <p>A scheming widow and her manipulative ex-lover make a bet regarding the corruption of a recently married woman.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=FbB2oBlP2uI'>View trailer</a> </div></div></body></html>",
      "documentId": "file://movie-entries/1988/Dangerous-Liaisons.html",
      "fileExtension": ".html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Adults",
                  "identityType": "Group"
                },
                {
                  "identity": "Teenagers",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": [
                {
                  "identity": "YoungChildren",
                  "identityType": "Group"
                },
                {
                  "identity": "PreTeenagers",
                  "identityType": "Group"
                }
              ]
            }
          ]
        },
        {
          "name": "Parental Approval Permissions",
          "permissionSets": [
            {
              "name": "First Parent Decision",
              "allowedPermissions": [],
              "deniedPermissions": []
            },
            {
              "name": "Second Parent Decision",
              "allowedPermissions": [],
              "deniedPermissions": []
            }
          ]
        }
      ]
    },
    {
      "documenttype": "Movie Entry",
      "filename": "The-Evil-Dead.html",
      "sourcetype": "Push",
      "title": "The Evil Dead",
      "mmepsdirector": "Sam Raimi",
      "mmepsyear": "1981",
      "mmepssummary": "Five friends travel to a cabin in the woods, where they unknowingly release flesh-possessing demons.",
      "mmepsisseries": true,
      "mmepsmpaafilmrating": "NC-17",
      "mmepsbudget": 0.35,
      "mmepscriticsaverage": 70,
      "mmepscastoverview": [
        "Bruce Campbell",
        "Ellen Sandweiss",
        "Richard DeManincor",
        "Betsy Baker",
        "Theresa Tilly"
      ],
      "mmepsthumbnaillink": "https://images-na.ssl-images-amazon.com/images/M/MV5BODc2MmVjZmUtNjAzMS00MDNiLWIyM2YtOGEzMjg0YjRhMzRmXkEyXkFqcGdeQXVyMTQxNzMzNDI@._V1_UX182_CR0,0,182,268_AL_.jpg",
      "mmepstrailerlink": "https://www.youtube.com/watch?v=jgw4ZdC2XTE",
      "data": "<html><head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.summary{word-wrap: break-word;}.title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.nc-17{color: white; background: black;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style></head><body> <div class='left-column'> <img src='https://images-na.ssl-images-amazon.com/images/M/MV5BODc2MmVjZmUtNjAzMS00MDNiLWIyM2YtOGEzMjg0YjRhMzRmXkEyXkFqcGdeQXVyMTQxNzMzNDI@._V1_UX182_CR0,0,182,268_AL_.jpg' width='200'> </div><div class='right-column'> <div class='title'> <h1>The Evil Dead</h1> </div><div class='director-year'> <p>Sam Raimi - 1981</p></div><div class='duration'> <p>1h25</p></div><div class='rating'><span class='NC-17'>NC-17</span> </div><div class='summary'> <p>Five friends travel to a cabin in the woods, where they unknowingly release flesh-possessing demons.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=jgw4ZdC2XTE'>View trailer</a> </div></div></body></html>",
      "documentId": "file://movie-entries/1981/The-Evil-Dead.html",
      "fileExtension": ".html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Adults",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": [
                {
                  "identity": "YoungChildren",
                  "identityType": "Group"
                },
                {
                  "identity": "PreTeenagers",
                  "identityType": "Group"
                },
                {
                  "identity": "Teenagers",
                  "identityType": "Group"
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "documenttype": "Movie Entry",
      "filename": "1900.html",
      "sourcetype": "Push",
      "title": "1900",
      "mmepsdirector": "Bernardo Bertolucci",
      "mmepsyear": 1976,
      "mmepssummary": "The epic tale of a class struggle in twentieth century Italy, as seen through the eyes of two childhood friends on opposing sides.",
      "mmepsisseries": false,
      "mmepsmpaafilmrating": "UR",
      "mmepsbudget": 9.0,
      "mmepscriticsaverage": 85,
      "mmepscastoverview": [
        "Robert De Niro",
        "Gérard Depardieu",
        "Dominique Sanda",
        "Francesca Bertini",
        "Laura Betti"
      ],
      "mmepsthumbnaillink": "https://upload.wikimedia.org/wikipedia/en/thumb/0/07/1900_Bertolluci.jpg/220px-1900_Bertolluci.jpg",
      "mmepstrailerlink": "https://www.youtube.com/watch?v=uGyH6Tt6qwY",
      "data": "<html><head> <style>body{font-family: 'Lato', 'Helvetica Neue', Helvetica, Arial, sans-serif, sans-serif; font-size: 15px; color: #373737;}a{text-decoration: none; color: #009DDC; cursor: pointer;}a:hover{text-decoration: underline;}.left-column{float: left; width: 200px; padding: 5px 15px;}.right-column{overflow: hidden; padding: 5px 10px 10px 15px;}.summary{word-wrap: break-word;}.title h1{color: #67768B; text-transform: uppercase; white-space: normal; margin: 0px;}.director-year{color: #67768B;}.rating{width: 60px; text-align: center; border-width: 1px; border-style: solid; border-radius: 5px;}.rating span{margin: 0px; border-radius: 5px; display: block;}.rating span.ur{color: white; background: blue;}.duration{font-size: 12px;}.trailer{border-bottom: 1px solid #BCC3CA; padding: 16px 0px 16px 0px;}</style></head><body> <div class='left-column'> <img src='https://upload.wikimedia.org/wikipedia/en/thumb/0/07/1900_Bertolluci.jpg/220px-1900_Bertolluci.jpg' width='200'> </div><div class='right-column'> <div class='title'> <h1>1900</h1> </div><div class='director-year'> <p>Bernardo Bertolucci - 1976</p></div><div class='duration'> <p>5h17</p></div><div class='rating'><span class='ur'>UR</span> </div><div class='summary'> <p>The epic tale of a class struggle in twentieth century Italy, as seen through the eyes of two childhood friends on opposing sides.</p></div><div class='trailer'> <a href='https://www.youtube.com/watch?v=uGyH6Tt6qwY'>View trailer</a> </div></div></body></html>",
      "documentId": "file://movie-entries/1976/1900.html",
      "fileExtension": ".html",
      "permissions": [
        {
          "name": "MPAA Default Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [
                {
                  "identity": "Adults",
                  "identityType": "Group"
                }
              ],
              "deniedPermissions": []
            }
          ]
        },
        {
          "name": "Additional Rating Permissions",
          "permissionSets": [
            {
              "allowAnonymous": false,
              "allowedPermissions": [],
              "deniedPermissions": [
                {
                  "identity": "YoungChildren",
                  "identityType": "Group"
                },
                {
                  "identity": "PreTeenagers",
                  "identityType": "Group"
                }
              ]
            }
          ]
        },
        {
          "name": "Parental Approval Permissions",
          "permissionSets": [
            {
              "name": "First Parent Decision",
              "allowedPermissions": [
                {
                  "identity": "chloe@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "erin@example.com",
                  "identityType": "User"
                }
              ],
              "deniedPermissions": [
                {
                  "identity": "frank@example.com",
                  "identityType": "User"
                }
              ]
            },
            {
              "name": "Second Parent Decision",
              "allowedPermissions": [
                {
                  "identity": "chloe@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "erin@example.com",
                  "identityType": "User"
                },
                {
                  "identity": "frank@example.com",
                  "identityType": "User"
                }
              ],
              "deniedPermissions": []
            }
          ]
        }
      ]
    }
  ],
  "delete": []
}

  3. Send the request.

Push the container to the Push source

You’ll send an Add, update, and/or delete a batch of items HTTP request.

  1. In your HTTP client, create a new PUT request using the Push API endpoint corresponding to your deployment region.

    US East region 
    PUT https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    PUT https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    PUT https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    PUT https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/documents/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables have already been set for this request.

    • Set the organizationId and sourceId path parameter values.

    • Use your API key as the Authorization header accessToken value.

    • Set the fileId query parameter value to the fileId value you obtained when you created the file container.

  3. Send the request.

You should receive a 202 response. You may have to wait several minutes before these items become available in your source.

Step 3: Add a batch of security identities

Whenever you use the Push API to add, update, or delete items in a secured Push source, you should ensure that the security identity definitions to which these items refer in their permission models are up to date in the security identity cache of your Coveo organization. Otherwise, certain users may not be able to see some of the items for which they should have access. Even worse, other users may be able to see items for which they should no longer have access.

In this step, you’ll use batch security identity update requests to define several security identities at once. The items that you added in your secured Push source in step 2 refer to these security identities in their permission models.

Adding a batch of security identities uses a three-step process similar to the one used to add a batch of items, that is:

  1. Creating the file container.

  2. Uploading the batch of security identities to the container.

  3. Pushing the container to the security identity provider.

Create the file container

You’ll send a Create a file container request.

  1. In your HTTP client, create a new POST request using the Push API endpoint corresponding to your deployment region.

    US East region 
    POST https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    POST https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    POST https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    POST https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/files HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables have already been set for this request.

    • Set the organizationId path parameter value.

    • Use your API key as the Authorization header accessToken value.

  3. Send the request.

  4. Store the uploadUri, fileId, x-amz-server-side-encryption, and Content-Type values from the response. You’ll need these values in the next steps.

    Postman

    The Coveo Push API Tutorials collection includes a post-response script that automatically sets the uploadUri, fileId, serverSideEncryption, and contentType variable values required for the next steps.

Upload the batch of security identities to the container

  1. In your HTTP client, create a new PUT request.

  2. Configure the request.

    Postman

    Set the Coveo Push API Tutorials collection myEmailAddress variable value to the email address you use when authenticating to the Coveo Platform.

    • Set the request endpoint to the uploadUri value you stored in the previous step.

    • Add the x-amz-server-side-encryption and Content-Type keys, and the values you stored for each in the previous step, to the request headers.

    • Set the request body to the JSON provided below. Here are details about the JSON structure:

      • The JSON defines nine users (Alice, Bob, Chloe, David, Erin, Frank, Grace, Henry, and yourself), four groups (YoungChildren, PreTeenagers, Teenagers, and Adults), and one granted identity (Everyone). Every user belongs to a single group and each group also has the Everyone granted identity.

      • The JSON also defines an alias to your own security identity in Coveo. For this to work, you must replace <MyEmailAddress> towards the end of this JSON with the actual email address with which you authenticate to Coveo.

        Important

        Typically, you should define an alias relationship between each user security identity and its corresponding security identity in the Email Security Provider.

        This means that you’ll normally define each user in the mappings array, as the yourself user is defined in the JSON below, when pushing a batch of security identities rather than in the members array.

        		 
      {
  "members": [
    {
      "identity": {
        "name": "Everyone",
        "type": "GROUP"
      }
    },
    {
      "identity": {
        "name": "alice@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "bob@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "chloe@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "david@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "erin@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "frank@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "grace@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "henry@example.com",
        "type": "USER"
      }
    },
    {
      "identity": {
        "name": "YoungChildren",
        "type": "GROUP"
      },
      "members": [
        {
          "name": "alice@example.com",
          "type": "USER"
        },
        {
          "name": "bob@example.com",
          "type": "USER"
        }
      ],
      "wellKnowns": [
        {
          "name": "Everyone",
          "type": "GROUP"
        }
      ]
    },
    {
      "identity": {
        "name": "PreTeenagers",
        "type": "GROUP"
      },
      "members": [
        {
          "name": "chloe@example.com",
          "type": "USER"
        },
        {
          "name": "david@example.com",
          "type": "USER"
        }
      ],
      "wellKnowns": [
        {
          "name": "Everyone",
          "type": "GROUP"
        }
      ]
    },
    {
      "identity": {
        "name": "Teenagers",
        "type": "GROUP"
      },
      "members": [
        {
          "name": "erin@example.com",
          "type": "USER"
        },
        {
          "name": "frank@example.com",
          "type": "USER"
        }
      ],
      "wellKnowns": [
        {
          "name": "Everyone",
          "type": "GROUP"
        }
      ]
    },
    {
      "identity": {
        "name": "Adults",
        "type": "GROUP"
      },
      "members": [
        {
          "name": "grace@example.com",
          "type": "USER"
        },
        {
          "name": "henry@example.com",
          "type": "USER"
        },
        {
          "name": "yourself@example.com",
          "type": "USER"
        }
      ],
      "wellKnowns": [
        {
          "name": "Everyone",
          "type": "GROUP"
        }
      ]
    }
  ],
  "mappings": [
    {
      "identity": {
        "name": "yourself@example.com",
        "type": "USER"
      },
      "mappings": [
        {
          "name": "<MyEmailAddress>", 1
          "type": "USER",
          "provider": "Email Security Provider"
        }
      ]
    }
  ],
  "deleted": []
}
      1 Replace <MyEmailAddress> with the email address you use when authenticating to the Coveo Platform.

  3. Send the request.

You should receive a 202 response.

Push the container to the security identity provider

You’ll send an Add, update, and/or delete a batch of security identities HTTP request.

  1. In your HTTP client, create a new PUT request using the Push API endpoint corresponding to your deployment region.

    US East region 
    PUT https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Canada region 
    PUT https://api-ca.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Ireland region 
    PUT https://api-eu.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}
    Australia region 
    PUT https://api-au.cloud.coveo.com/push/v1/organizations/{organizationId}/providers/{providerId}/permissions/batch?fileId={fileId} HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer {accessToken}

  2. Configure the request.

    Postman

    In the Coveo Push API Tutorials collection, all the necessary variables have already been set for this request.

    • Set the organizationId and providerId path parameter values.

    • Use your API key as the Authorization header accessToken value.

    • Set the fileId query parameter value to the fileId value you obtained when you created the file container.

  3. Send the request.

You should receive a 202 response.

Step 4: Validate item field values, effective permissions, and data

You’ll now use the Coveo Administration Console to verify that the metadata of the items you just pushed has populated the fields in your index as expected, and that the permission models of your items are being evaluated correctly.

  1. On the Sources (platform-ca | platform-eu | platform-au) page.

  2. On the Push source row, click Open in Content Browser.

    Show source items in the Content Browser

    The six items that you pushed in your source should appear in the result list.

  3. Click the result whose title is 1900, and then click Properties in the Action bar.

    A panel should open in which the Fields tab is selected by default. You should see a list of fields and their values for this item.

  4. You can now validate that the fields you created in step 1 were properly populated with the item metadata you pushed in step 2.

    Item 1900 under the Fields tab

    Note that the title field was also populated, even though you didn’t create this field in step 1; title is a default field.

    Item 1900 under the Fields tab

  5. In the header of the panel, select the Permissions tab.

    You should see the list of permissions for the 1900 item. As explained in step 2:

    • The only allowed users are Erin, Grace, Henry, and yourself.

    • The YoungChildren and PreTeenagers groups, as well as the user Frank, are explicitly denied.

      Item 1900 under the Permissions tab

  6. In the header of the panel, select the Permission details tab.

    You should see that this item has three permission levels. The first two levels have only one permission set, whereas the last one has two. As explained in step 2:

    • Permission Level 0 (that is, Mpaa default permissions) explicitly allows the Adults group.

    • Permission Level 1 (that is, Additional rating permissions) explicitly denies the YoungChildren and PreTeenagers groups.

    • Permission Level 2 (that is, Parental approval permissions) explicitly allows the user Erin, and denies the user Frank.

      Item 1900 under the Permission details tab

  7. In the header of the panel, select the Quick view tab.

    The HTML and CSS that was provided as the value of the data property for this item should render as expected.

    Item 1900 under the Fields tab

You can validate the other items that you added in your secured Push source the same way.