Index Optimizely content
Index Optimizely content
This article guides you on how to use the generic GraphQL API source, which is particularly suitable for Optimizely Graph endpoints, to optimize your indexing strategy. This approach ensures that you not only collect but also effectively utilize your content for enhanced search experiences.
Create a new GraphQL API source
When creating a GraphQL API source in the Coveo Administration Console, you must provide a JSON configuration that details what to crawl to retrieve the desired content and how to retrieve each type of item.
This configuration has a single property, Services, whose value must be an array of objects.
This reference article defines the parameters to include in your JSON configuration for the Optimizely Graph endpoint.
The examples below show how to configure the source to index the items ArticlePage where each article will be a document in the Coveo index available for search and recommendations.
Examples
Example 1 - Basic setup
{
"Services": [
{
"Url": "https://cg.optimizely.com/content",
"authentication": {
"username": "@username",
"password": "@password",
"forceBasicAuthentication": "true"
},
"Endpoints": [
{
"Path": "v2",
"Method": "GET",
"ItemPath": "data.ArticlePage.items",
"SkippableErrorCodes": "404",
"ItemType": "Articles",
"Uri": "https://optimizely-saas-cms.netlify.app/%[_metadata.url.default]",
"ClickableUri": "%[_metadata.url.base]%[_metadata.url.default]",
"Title": "%[articleTitle]",
"ModifiedDate": "%[_metadata.lastModified]",
"Body": "%[articleBody.html]",
"QueryParameters": {
"query": "query MyQuery { }"
}
}
]
}
]
}Example 2 - Using the ArticlePage object with pagination
{
"Services": [
{
"Url": "https://cg.optimizely.com/content",
"authentication": {
"username": "@username",
"password": "@password",
"forceBasicAuthentication": "true"
},
"Endpoints": [
{
"paging": {
"pageSize": 2,
"offsetType": "item",
"TotalCountKey": "data.ArticlePage.total"
},
"Path": "v2",
"Method": "GET",
"ItemPath": "data.ArticlePage.items",
"SkippableErrorCodes": "404",
"ItemType": "Articles",
"Uri": "https://optimizely-saas-cms.netlify.app/%[_metadata.url.default]",
"ClickableUri": "%[_metadata.url.base]%[_metadata.url.default]",
"Title": "%[articleTitle]",
"ModifiedDate": "%[_metadata.lastModified]",
"Body": "%[articleBody.html]",
"QueryParameters": {
"query": "query MyQuery { ArticlePage(where: { _metadata: { status: { eq: \"Published\" } } } skip:@offset limit:@pageSize ) { items { articleBody { html } articleTitle _metadata { url { base default } lastModified status locale } articleHeroImage { url { default } } articleAuthors } total }}"
}
}
]
}
]
}Example 3 - Using the ArticlePage object, field mapping and pagination
{
"Services": [
{
"Url": "https://cg.optimizely.com/content",
"authentication": {
"username": "@username",
"password": "@password",
"forceBasicAuthentication": "true"
},
"Endpoints": [
{
"paging": {
"pageSize": 2,
"offsetType": "item",
"TotalCountKey": "data.ArticlePage.total"
},
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT(%[my_locale] == \"en\")"
},
"Path": "v2",
"Method": "GET",
"ItemPath": "data.ArticlePage.items",
"SkippableErrorCodes": "404",
"ItemType": "Articles",
"Uri": "https://optimizely-saas-cms.netlify.app/%[_metadata.url.default]",
"ClickableUri": "%[_metadata.url.base]%[_metadata.url.default]",
"Title": "%[articleTitle]",
"ModifiedDate": "%[_metadata.lastModified]",
"Body": "%[articleBody.html]",
"Metadata": {
"author": "%[articleAuthors[0]]",
"ec_images": "%[articleHeroImage.url.default]",
"originalUrl": "%[_metadata.url.base]%[_metadata.url.default]",
"my_locale": "%[_metadata.locale]"
},
"QueryParameters": {
"query": "query MyQuery { ArticlePage(where: { _metadata: { status: { eq: \"Published\" } } } skip:@offset limit:@pageSize ) { items { articleBody { html } articleTitle _metadata { url { base default } lastModified status locale } articleHeroImage { url { default } } articleAuthors } total }}"
}
}
]
}
]
}Configure "Services"
"Url" section
The Url value is the Optimizely Execute GraphQL query Url without the endpoint v2.
The GraphQL API source configuration Url will be as follows:
"Url": "https://cg.optimizely.com/content""Authentication" section
If your Optimizely Graph endpoint requires authentication, this section ensures the right connection is established using the credentials provided. Enter the required authentication information in the Authentication section of the GraphQL source configuration panel. Use the Optimizely App key as username and Secret as password.
The recommendation is to use the @username and @password placeholders to retrieve the values specified in the GraphQL API source configuration panel.
"Url": "Optimizely Endpoint Name",
"authentication": {
"username": "<@username_PLACEHOLDER>",
"password": "<@password_PLACEHOLDER>",
}Configure "Endpoints"
"Paging" section
Pagination is a technique used in web development and APIs to divide a large dataset into smaller, more manageable chunks called pages. When configuring the GraphQL API source, pagination allows users to navigate through the data by requesting and displaying one page at a time.
In the context of the GraphQL API source, pagination is being used to retrieve data from the API sequentially, where each request fetches one page of data. This approach helps in optimizing performance and reducing the amount of data transferred between the client and the server.
When building the query, include the total field to indicate the total number of pages.
This is calculated by the server-side GraphQL implementation to provide the total count of items matching the query’s filters.
In the source configuration, include the number of items per page, the offsetType to specify to Coveo how your API paginates its content and the count field that specifies the total expected number of items through the data object.
Then, in your GraphQL query, make sure to include tokens @pageSize and @offset.
Coveo will replace @pageSize with the value of the pageSize parameter of your paging configuration.
Similarly, @offset will be replaced with the value extracted from the response, depending on the paging method selected in offsetType.
"paging": {
"pageSize": 2,
"offsetType": "item",
"TotalCountKey": "data.ArticlePage.total"
},"QueryParameters": {
"query": "query MyQuery { ArticlePage(where: { _metadata: { status: { eq: \"Published\" } } } skip:@offset limit:@pageSize ) { items { articleBody { html } articleTitle _metadata { url { base default } lastModified status locale } articleHeroImage { url { default } } articleAuthors } total }}"
}"IndexingAction" section
This section determines whether to ignore or retrieve an item when a condition is met.
When the condition resolves to true, the Coveo crawler applies the specified action to the item.
Possible actions are Retrieve and Ignore, that is, the crawler can either index the item or ignore it.
An item ignored with IndexingAction isn’t indexed and therefore not visible in the Content Browser or a search interface.
In the provided example, we only index items where my_locale equals "en".
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT(%[my_locale] == \"en\")"
},"ItemType" section
Whether you’re using Optimizely Graph alone or with an existing Optimizely CMS, you need to define the type of item you want to pass from Optimizely into the Coveo index. In the example of this article, Content Type property is used. In CMS, you’ll have Content Templates that define the document you want to make discoverable through search. In the provided example, the configuration of the query fetches articles. Therefore, the GraphQL API source configuration will be as follows:
"ItemType": "Articles""Body" section
The body (the full text) of the text you want to index. All the text is free-text searchable and also available in the quickview/preview of the index. In the provided example, we assign the HTML body:
"Body": "%[articleBody.html]""Metadata" section
Each key represents the metadata name of the item, while its value is the value path (simple path or JSONPath) in the JSON response.
Typically, the value path consists of one or more dynamic values, since a static, hardcoded value would result in an identical piece of metadata for all items. However, you could choose to use a hardcoded value so that the corresponding Coveo field is filled even if the API doesn’t provide this information.
When you build your source, Coveo retrieves the desired metadata. You can then review a summary of this metadata in the Administration Console and use it to create mapping rules for your source.
Don’t forget to create the field in the Administration Console related to each metadata name and define if those fields are Facet, Multi-value facet, and Sortable.
This section allows us to associate specific properties of each item in the JSON response with a specific field in Coveo.
In the provided example, we use the first author, set the hero image as ec_images, and set the originalUrl and my_locale.
"Metadata": {
"author": "%[articleAuthors[0]]",
"ec_images": "%[articleHeroImage.url.default]",
"originalUrl": "%[_metadata.url.base]%[_metadata.url.default]",
"my_locale": "%[_metadata.locale]"
},"Query Parameters" section
Configure the query to get content type objects Configure the desired query that defines the items and fields. Each item in the Graph query will be a document in the Coveo index that will be a part of search or recommendations, while each field will be either the body of the document or the field of the document populated with appropriate metadata.
Test your graph query with Optimizely’s Graph explorer:
https://cg.optimizely.com/app/graphiql?auth=<YOUR_SECRET_SINGLE_KEY>See the following example query:
Note: When configuring the query, ensure the following fields are passed with each item.
The documents indexed by Coveo, which are available as search results, must have a URL associated with them.
In the query above, the field URL is composed of a static value and the default url ("https://optimizely-saas-cms.netlify.app/%[_metadata.url.default]").
Include _modified or Changed fields in your query to ensure that the index picks up the changes in a timely manner and the freshest Optimizely Graph content resides in the index.
If the Url returned in the Query field doesn’t return the desired properties, consider using an indexing pipeline extension (IPE) or opt to index content with a Sitemap source instead.
Very useful
Not really
Very useful
Not really