Adding and Managing Query Pipeline Filters

Members of the Administrators and Relevance Managers built-in groups can use query pipeline filters to easily configure the scope for queries going through a given pipeline. The Filters tab allows you to enter hidden query expressions to be added to all queries going through the related query pipeline. While this feature permits the addition of any query syntax expression to any part of the query expression (i.e., q, aq, cq, dq, or lq), query pipeline filters are typically used to add a field-based expression to the constant query expression (cq).

A support agent authenticates on a customer service search interface. A query pipeline filter rule adds the expression NOT @source=="Marketing" to ensure that marketing items don’t appear when the end-user is identified as a support agent. The cq is then always NOT @source=="Marketing" when support agents send queries.

The list of filters in a pipeline is empty by default. Filter rules are defined independently for each pipeline.

The following diagram highlights the position of Filter rules in the overall order of execution of query pipeline features.

Apply filter rules

Filters, by themselves, do not protect the security of filtered content. We strongly advise against creating a source whose content is accessible to everyone and using a pipeline filter to exclude sensitive information.

In the following cases, sensitive content from a source whose content is accessible to everyone could be exposed:

  • A colleague not understanding the reason for the filter could modify or remove the filter.

  • Using other pipelines not having a similar filter from other search interfaces or directly from the API.

You can ensure security by enforce the search hub at the search token level (see Search Token Authentication). Moreover, search hubs defined on the client-side that are used as conditions in pipelines do not safeguard the security of your filters.

Access the “Filters” Tab of a Pipeline

  1. Log in to the Coveo Platform as a member of a group with the required privileges to manage query pipeline components in the target Coveo Cloud organization.

  2. In the main menu on the left, under Search, select Query Pipelines.

  3. On the Query Pipelines page, click the query pipeline in which you find the filter rules you want to manage, and then in the Action bar, click Edit components.

  4. On the selected pipeline page, select the Filters tab.

Leading Practices

Use Inclusion Filters

When creating filters, specify what the pipeline should include rather than exclude (e.g., @source==(“Source 1”, “Source 2”) rather than NOT @source==(“Source 3”)) to prevent new source content or content types in your organization index from appearing in search results.

If you only use an exclusion filter (e.g., NOT @source==(“Source 3”)), you must modify the filter expression when new content is added to your organization index (e.g., NOT @source==(“Source 3”, "Source 4")). To provide a controlled deployment of new sources into search results, use an inclusion filter (e.g., @source==(“Source 1”, “Source 2”)), which allows content to be included in an additive manner, rather than subtraction. This means content only appears in search results after the filter is updated, not immediately after indexing.

You have three sources (Source 1, Source 2, Source 3) in your index and you want only Source 1 and Source 2 content to appear in the search results of a specific tab:

  • An exclusion filter (e.g., NOT @source==(“Source 3”)), requires you to modify the filter expression if you retrieve content from Source 4 (e.g., NOT @source==(“Source 3”, "Source 4")).

  • An inclusion filter (e.g., @source==(“Source 1”, “Source 2”)), requires no further changes even if you index Source 4 content.

Check Your Sources Mappings

Filter rules filter out from the search results all items that don’t match the hidden query expression on which the rule is based.

Since you will typically use filter rules to add a field-based expression to the cq, all items that must fall within the scope of the filter rule must be mapped with the matching metadata values in order to be displayed in the search results (see Adding and Managing Source Mappings). In other words, items that don’t match the defined query expression won’t appear in the search results.

Therefore, you should be 100% certain that all items that must match the filter expression are consistently mapped with the proper metadata.

If you’re not 100% certain that the mappings are totally accurate, you should consider using query ranking expression rules instead.

Manage the Filters of a Query Pipeline

Members with the privilege to view query pipelines (i.e., the View all or the Custom access level on the Query Pipelines domain) can review filter rules in read-only mode (see Privilege Management and Query Pipelines Domain).

  1. Access the “Filters” tab of a pipeline.

  2. In the Filters tab, you can add a new filter rule:

    1. In the upper-right corner, click Add Rule to open the Add a Filter Rule subpage.

    2. In the Query Parameter drop-down menu, select the query parameter in which the filter will be included in an AND expression.

    3. In the Content matching this expression input, you can either:

      When creating cq filters, avoid including dynamic content in the input. Otherwise, you risk filling up the cache with useless data, which can have a negative impact on performance.

      You want to remove the current case record in the search results when your agents perform a query, so you enter the following expression:

      QPL-FilterRuleExb

    4. On the right-hand side, under Condition, you can optionally select a query pipeline condition in the drop-down menu or create a new one.

    5. Under User Note, optionally enter information that will help you and your colleagues manage the rule in the future.

    6. Click Add Rule. All changes are now effective.

  3. You can perform other actions on rules (see Adding and Managing Query Pipeline Rules and Rule Conditions From Tabs).

  4. On the search interface(s) for your Coveo Cloud organization, test your filter rule to ensure that it provides the desired outcome. Make adjustments when needed.

Manage Filter Rules With Code

Advanced users can also enter a rule that respects the Query Pipeline Language (QPL) syntax for filters (see Query Parameters - cq and Filter - Query Pipeline Feature):

  1. In the Filters tab, click Action-Menu, and then select Add rule with code.

  2. In the Add a Rule With Code dialog, in the form editor box, enter a filter rule in the following form:

    filter [query_parameter] expression

    where you replace [query_parameter] with q, aq, cq, dq, or lq.

    If you base a filter rule on the content of one or more context keys, you should associate this rule to a condition asserting that those context keys have values. Otherwise, you risk injecting invalid content in your query expression (see the Leading Practice box in Filter - Query Pipeline Feature).

  3. Click Add With Code.

Reference

“Filter Expression Editor” Subpage

When clicking Edit, you access the Filter Expression Editor subpage to add query expressions.

In the Filter Expression Editor subpage, users can either select the “Basic Mode” or “Advanced Mode” to define query expressions.

Basic Mode

To define query expressions using the "Basic Mode"

  1. In the Select field drop-down menu, type or select the field that must be targeted by the query expression.

  2. In the Select comparator drop-down menu, select one of the available comparators.

  3. (If available1) In the Add a value drop-down menu, type or select the values that must be filtered according to the field selected in step 1.

  4. Optionally, in the Add rule drop-down menu, select one of the available operators to add another query expression to your filter rule.

  5. In the right-hand section of the Filter Expression Editor subpage, you can preview the results triggered by your filter rule.

  6. Click Done to add the expression to the “Content matching this expression” input.

1: The Add a value drop-down menu isn’t available if the user selected the is defined or is not defined comparators in step 2.

Advanced Mode

To define query expressions using the "Advanced Mode"

  1. In the input box, enter a query expression using the Coveo Cloud query syntax.

  2. In the right-hand section of the Filter Expression Editor subpage, you can preview the results triggered by your filter rule.

  3. Click Done to add the expression to the “Content matching this expression” input.

Required Privileges

The following table indicates the required privileges to view and edit elements of the Query Pipelines page and associated panels (see Privilege Management and Privilege Reference).

Action Service - Domain Required access level
View filter rules

Search - Query pipelines

View
Edit filter rules

Search - Query pipelines

Edit
Recommended Articles