Manage filter rules

This is for:

Developer System Administrator

A query pipeline filter is a hidden query expression that can be added to queries going through the related pipeline. It helps define the scope of the search results displayed to users. The Filters section of the Query Pipelines (platform-ca | platform-eu | platform-au) page lets members with the required privileges create and manage filter rules.

By default, the list of filters in a pipeline is empty. Without filters, all items matching the query are returned in the search results. Filter rules are defined independently for each pipeline and are typically used to add a field-based expression to the constant query expression (cq). They also permit the addition of any query syntax expression to any part of the query expression (q, aq, cq, dq, and lq).

These parameters are useful in customizing the search experience, allowing for the override of default values for every query that matches a defined condition.

Example

A support agent authenticates on a customer service search interface. A query pipeline filter rule adds the expression NOT @source=="Marketing" to ensure that marketing items don’t appear when the end user is identified as a support agent. The cq is then always NOT @source=="Marketing" when support agents send queries.

Prerequisites

Before creating a rule, first make sure that you have the following:

Once you meet these requirements, you can create a rule on the Query Pipelines (platform-ca | platform-eu | platform-au) page. To test the rule, use the A/B test feature to compare the results of the rule with the results of the original pipeline.

Access the "Filters" subtab

To manage filter rules, you must first access the Filters subtab in a query pipeline.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline in which you want to manage filter rules, and then click Edit components in the Action bar.

  2. On the page that opens, select the Advanced tab.

  3. In the Advanced tab, on the left side of the page, select Filters.

Add or edit filter rules

  1. Access the Filters subtab in the query pipeline you want to manage, and then do one of the following:

    • To add a new filter rule, click Add a filter rule.

      Note

      • If you’re adding a filter rule for the first time, you can click Add rule with code to define the rule using the appropriate QPL syntax.

      • If you’re adding a filter rule to a list of existing rules, you can click dots, and then click Add rule with code.

    • To edit an existing filter rule, click the rule you want to edit, and then click Edit in the Action bar.

      Note

      You can also click More, and then select Edit code to edit the rule using the appropriate QPL syntax.

  2. In the Add/Edit a filter rule panel, under the Query Parameter dropdown menu, select the query parameter in which the filter will be included in an AND expression.

  3. In the Content that matches section, depending on the mode, use the dropdown menus or the input to provide the desired filter expressions.

  4. In the Condition section, you can optionally select a query pipeline condition in the dropdown menu or create a new one.

  5. Under Description, optionally enter information that will help you manage the rule in the future.

  6. Click Add rule or Save.

Duplicate filter rules

When creating filter rules in a query pipeline, you may want to create a new rule that’s similarly configured to an existing one. An efficient way to do this is to duplicate the existing rule and then modify it as needed.

  1. Access the Filters subtab in the query pipeline you want to manage.

  2. In the Filters subtab, select each checkbox next to the rules you want to duplicate within the same pipeline.

  3. In the Action bar, click Duplicate.

The duplicated filter rules appear at the bottom of the list in the pipeline component tab.

Copy filter rules to another pipeline

When you have more than one query pipeline that serve different purposes but require similar rules, you may want to copy filter rules from one pipeline to another.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline from which you want to copy query pipeline rules, and then click Edit components in the Action bar.

  2. On the page that opens, select the Advanced tab.

  3. In the Advanced tab, on the left side of the page, select Filters.

  4. In the Filters subtab, select each checkbox next to the rules you want to copy to another pipeline.

  5. In the Action bar, click More, and then click Copy to…​.

  6. In the dialog that appears, select the target pipeline to which you want to copy the rules, and then click Copy.

Delete filter rules

You can delete filter rules from a query pipeline.

  1. Access the Filters subtab in the query pipeline you want to manage.

  2. In the Filters subtab, select each checkbox next to the rules you want to delete.

  3. In the Action bar, click More, and then Delete.

  4. Click Delete to confirm.

Change the rule order

Query pipeline rules are executed in the order in which they appear on the page until a condition is satisfied.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline for which you want to manage the rules' execution order, and then click Edit components in the Action bar.

  2. On the page that opens, select the Advanced tab.

  3. In the Advanced tab, on the left side of the page, select Filters.

  4. In the Filters subtab, click the rule whose position you want to change.

  5. In the Action bar, click Move up or Move down to change the position of the rule.

Leading practices

Consider the following leading practices when creating filter rules:

Don’t use filter rules to exclude sensitive information

Filters, by themselves, don’t prevent the exposure of filtered content. We strongly advise against creating a source whose content is accessible to everyone and using a pipeline filter to exclude sensitive information.

In the following cases, sensitive content from a source whose content is accessible to everyone could be exposed:

  • A filter rule based on the aq query parameter is set on a query pipeline that contains an ART model for which the Match the advanced query option is disabled.

  • Using other pipelines not having a similar filter from other search interfaces or directly from the API.

  • A colleague not understanding the reason for the filter could modify or remove the filter.

You can ensure security by enforcing the search hub at the search token level (see Search token authentication). Moreover, search hubs defined on the client-side that are used as conditions in pipelines don’t safeguard the security of the filtered content.

Use inclusion filters

When creating filters, specify what the pipeline should include rather than exclude (for example, @source==("Source 1", "Source 2") rather than NOT @source==("Source 3")) to prevent new source content or content types in your organization index from appearing in search results.

If you only use an exclusion filter you must modify the filter expression whenever new content is added to your organization index. To provide a controlled deployment of new sources into search results, use an inclusion filter, which allows content to be included in an additive manner. This means content only appears in search results after the filter is updated, not immediately after indexing.

Example

You have three sources (Source 1, Source 2, Source 3) in your index and you want only Source 1 and Source 2 content to appear in the search results of a specific tab:

  • An exclusion filter (for example, NOT @source==("Source 3")), requires you to modify the filter expression if you retrieve content from the newly created Source 4 (for example, NOT @source==("Source 3", "Source 4")).

  • An inclusion filter (for example, @source==("Source 1", "Source 2")), requires no further changes even if you index Source 4 content.

Check your source mappings

Filter rules filter out from the search results all items that don’t match the hidden query expression on which the rule is based.

Since you’ll typically use filter rules to add a field-based expression to the cq, all items that must fall within the scope of the filter rule must be mapped with the matching metadata values in order to be displayed in the search results (see Manage source mappings). In other words, items that don’t match the defined query expression won’t appear in the search results.

Therefore, you should be 100% certain that all items that must match the filter expression are consistently mapped with the proper metadata.

If you’re not 100% certain that the mappings are totally accurate, you should consider using query ranking expression rules instead.

Apply filter rules conditionally

Typically, filter rules should only apply when a certain condition is fulfilled.

In general, you should ensure that this is the case by associating such a rule, and/or the pipeline it’s defined in, to a global condition.

Moreover, when basing a filter rule on the value of one or more $context keys, you should use a condition to assert that these keys contain values. Otherwise, you risk injecting invalid expressions in the target part of the query expression (that is, aq, cq, etc.).

Reference

"Content that matches" section

When creating a rule, under the Content that matches section, you must define filter expressions to target specific content.

You can define filter expressions using either the "Basic mode" or "Advanced mode".

Basic mode

To define filter expressions using the Basic mode:

  1. In the Select field dropdown menu, use the Filter box to find and select the field that must be targeted by the filter expression.

  2. In the Select a comparator dropdown menu, select one of the available comparators.

  3. If applicable, in the Select a field value dropdown menu, enter or select the values that must be filtered according to the field selected in step 1.

  4. (Optional) Click Add to add another filter expression to the rule. Note that when adding multiple filter expressions to a rule, new filter expressions are appended to the initial expression using the and logic, meaning that only content matching all filter expressions will be affected by the rule.

Advanced mode

To define filter expressions using the Advanced mode, enter a query expression using the query syntax.

QPL syntax

Use the following query pipeline language (QPL) syntax to define a statement expressing the filter feature:

filter <queryExpressionPart> <expressions>

<queryExpressionPart>

A string indicating the part of the query expression to which one or more query expressions should be appended. Must be one of:

  • aq: The advanced part

  • cq: The constant part

  • dq: the disjunction part

  • lq: the large part

  • q: the basic part

Notes

<expressions>

A comma-separated list of query expressions (for example, foo`, @bar=="baz"). Each of these query expressions will be appended to the current value of the target `<queryExpressionPart> using an implicit AND operator (that is, a whitespace character).

Important

The result set of the constant part (cq) of the combined query expression is kept in a special cache.

Consequently, you should avoid creating filter statements that add dynamic content to the cq, otherwise you risk filling the cache with useless data, which might have a negative impact on performance.

Order of execution

The following diagram illustrates the overall order of execution of query pipeline features:

diagram showing order of execution

Required privileges

By default, members with the required privileges can view and edit elements of the Query Pipelines (platform-ca | platform-eu | platform-au) page.

The following table indicates the required privileges for members to view or edit filter rules (see Manage privileges and Privilege reference).

Action Service - Domain Required access level

View filter rules

Organization - Organization
Search - Query pipelines

View

Edit filter rules using the Basic mode

Organization - Organization
Content - Fields

View

Search - Query pipelines

Edit

Edit filter rules using the Advanced mode

Organization - Organization

View

Search - Query pipelines

Edit