Manage stop word rules

This is for:

Developer System Administrator

Stop words are words that are filtered out of a query in order to give more importance to keywords. This provides users with more relevant search results. The stop word list for a Coveo organization index is empty by default, but with the required privileges, you can add stop word rules in the Coveo Administration Console to define words that should be ignored when they appear in user queries.

Stop words are defined independently for each query pipeline. Depending on your specific needs, you may want to add a list of words to ignore from queries. For instance, you might ignore industry-specific words that appear in many of your items and offer little differentiation between items in search results.

Example

On a medical website, words such as cause, symptoms, and treatment tend to appear frequently in articles across the site. These words are often used in queries but they don’t help differentiate between articles on the site.

You therefore create a stop word rule for each of these words.

Coveo Platform | Stop word rule panel

So when a user searches for headache or migraine treatment, the query sent to the index is headache migraine.

The search results will be more relevant as the focus is on the keywords headache and migraine rather than the generic term treatment.

Note

Query pipeline stop word rules aren’t applied to the query that’s used by RGA, CPR, SE, and Smart Snippet models. These models always use the raw basic query expression (q) entered by the user without any transformations or modifications.

Prerequisites

Before creating a rule, make sure that you have the following:

Once you meet these requirements, you can create a rule on the Query Pipelines (platform-ca | platform-eu | platform-au) page. To test the rule, use the A/B test feature to compare the results of the rule with the results of the original pipeline.

Order of execution

The following diagram illustrates the order of execution of query pipeline features:

Diagram showing order of execution | Coveo

Access the "Stop words" subtab

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline you want to modify, then click Edit components in the Action bar.

  2. On the page that opens, select the Search terms tab.

  3. On the Search terms tab, on the left side of the page, select Stop words.

Add or edit stop word rules

  1. Access the Stop words subtab.

  2. Do one of the following:

    • To create a new rule, click Add stop word rule.

    • To edit an existing rule, select the rule you want to edit, and then click Edit in the Action bar.

  3. In the Add/Edit a stop word rule panel, under Stop words, add or edit the words that will be ignored when they appear in queries. Type one or more words separated by commas.

    Coveo Platform | Comma-separated words in stop words panel

  4. Under Condition (optional), do one of the following:

  5. Click Add rule or Save.

Your rule is now active.

Duplicate stop word rules

You can duplicate stop word rules to use as a starting point for new rules within the same query pipeline.

  1. Access the Stop words subtab.

  2. Select the rules you want to duplicate, and then click Duplicate in the Action bar. The duplicated rules appear at the bottom of the list.

  3. Edit the duplicated rules as required.

Change the order of stop word rules

You can move stop word rules in the list to change the order in which they appear.

Note

When you move a stop word rule, only the order of execution of stop word rules is adjusted. Moving the rule doesn’t affect the order of execution of stop word rules relative to other query pipeline components.

  1. Access the Stop words subtab.

  2. Select the rule you want to move, and then click Move up or Move down in the Action bar to move the rule in the list.

Copy stop word rules to another pipeline

You can copy one or more stop word rules from one query pipeline to another.

  1. Access the Stop words subtab.

  2. Select the rules you want to copy, and then click Copy to in the More menu.

  3. In the Copy rule to another pipeline panel, select the query pipeline to which you want to copy the rule, and then click Copy.

The rule is copied to the selected query pipeline and is immediately active.

Delete stop word rules

  1. Access the Stop words subtab.

  2. Select the rules you want to delete, and then click Delete in the More menu.

  3. In the confirmation prompt, click Delete.

Your deleted rules will stop being effective immediately in the target pipeline.

Tip

  • To avoid undesired search behaviors, delete unused query pipeline rules, as they can impact search relevance.

  • Consider using the Groups & Campaigns feature to make query pipeline rules only apply for a specific time period.

Leading practices

When managing stop word rules, consider the following recommendations and tips:

  • The index assigns a semantic value to every word by considering their frequency in the index. Frequent words in indexed items are considered to carry less meaning. Consequently, the index already attributes minimal ranking weight for the occurrence of these words in search results. Therefore, add stop word rules only for specific use cases.

    Note

    You might consider adding stop word rules to exclude bad keywords from queries to not impact the Coveo ML model learning process. However, if end users perform queries containing only banned words, the model learning process could be affected depending on the returned search results.

    For more information on the management of the blocklist words, see Blocklist.

  • Apply conditions

    Typically, stop word rules should only apply when a certain condition is fulfilled.

    In general, you should associate the rule, and/or the query pipeline it’s defined in, to a condition. Without a condition, the stop word rule would apply to all queries.

  • Consider using the Partial match feature:

    For implementations where users tend to enter lengthy natural language queries, using the Partial match feature can simplify search results by eliminating the need for exhaustive stop word definitions. This option lets you define a minimum number of keywords to be found in a search result before it’s returned. As a result, the index favors the most important keywords and stop words become optional.

  • Consider how stop words affect ART models:

    When creating stop word rules, the words specified in the rule don’t get passed to the ART models, which means that the Automatic Relevance Tuning (ART) models don’t learn from these words and won’t associate them with user interactions.

    However, it’s important to note that these models also have their own stop word list that can be managed through the Advanced model configuration API. These stop words help reduce the impact of common words when generating recommendations, however, a model only starts excluding them after accumulating sufficient interaction data.

    Therefore, regularly review ML model training data to ensure that stop word rules aren’t adversely affecting the quality of the recommendations.

  • Test your stop words:

    When creating stop word rules, you should always perform tests to ensure that your stop words don’t negatively impact the search experience in cases other than the one you’re trying to improve.

Stop word special cases

Stop word rules remove the specified words from queries only when they appear in combination with other keywords. They aren’t removed when they’re the only word in the query, because otherwise the removal could create an invalid expression.

Coveo indexes all words contained in your source items, including the stop words. This allows Coveo to manage exceptions and keep stop words in the query in the following cases:

  • Stop words within a phrase search.

    Example

    A user is looking for an item that contains a very specific phrase and encloses the phrase between double-quotes in the search box:

    "in the plan for year 2025"

    The words in, the, and for are stop words, but since the phrase is enclosed in double-quotes, all keywords in this query are sent to the index, so only items containing these keywords in the same order and as contiguous occurrences are returned.

  • A query containing only stop words.

    Example

    A user searches for:

    to be or not to be

    If all the keywords of this query are stop words, they’re all kept and sent to the index.

  • A stop word is an argument of the NOT or NEAR operators.

    Examples

    • A user searches for:

      how NEAR:10 export, meaning the user is looking for items containing both how and export occurring within ten words from each other.

      The word how has been set as a stop word. However, since it’s an argument of the NEAR operator, it will be kept to return items containing both how and export within ten words from each other.

    • A user searches for:

      (NOT how export), meaning the user is looking for items containing export but not how.

      The word how has been set as a stop word.

    If how is a stop word, because it’s an argument of the NOT operator, and the NOT operator has precedence over the implicit AND operator, it will be kept to return items containing export but not how.

  • A stop word also has a plural form.

    Example

    An online store sells a variety of wristwatches. Since the majority of their products contain the word watch or watches, searching for these words doesn’t provide relevant results, as customers are looking for either specific brands or types of watches. Therefore, you add both watch and watches to your stop words list to ensure that the search results are more relevant. You must add both forms since the plural form isn’t automatically removed when the singular form is a stop word.

Required privileges

The following table indicates the privileges required to view or edit stop word rules. Learn more about the Privilege reference or how to manage privileges.

Action Service - Domain Required access level

View stop word rules

Organization - Organization
Search - Query pipelines

View

Edit stop word rules

Organization - Organization

View

Search - Query pipelines

Edit