Filter - Query Pipeline Feature
A query pipeline statement expressing the filter
query pipeline feature adds one or more query expressions to the advanced (aq
), constant (cq
), disjunction (dq
), large (lq
), or basic (q
) part of the query expression before the query is sent to the index.
Typically, a statement expressing the filter
feature should only apply when a certain condition is fulfilled.
In general, you should ensure that this is the case by associating such a statement, and/or the pipeline it’s defined in, to a global condition.
Moreover, when basing a filter
statement on the value of one or more $context
keys, you should use a condition to assert that those keys contain values. Otherwise, you risk injecting invalid expressions in the target part of the query expression (i.e., aq
, cq
, etc.).
Condition
when $context[favoriteBookGenre] matches /*.\S.*/
Statement
filter aq `@bookgenre==$context.favoriteBookGenre`
In the Coveo Administration Console, you can manage statements expressing the filter
feature from the Filters tab (see Manage Filter Rules).
The following diagram highlights when statements expressing the filter
feature are applied in the query pipeline (see Order of Execution of Query Pipeline Features).
Filters, by themselves, don’t protect the security of filtered content. We strongly advise against creating a source whose content is accessible to everyone ("sourceVisibility": "PUBLIC"
) and using a pipeline filter to exclude sensitive information.
In the following cases, sensitive content from a source whose content is accessible to everyone ("sourceVisibility": "PUBLIC"
) could be exposed:
-
A colleague not understanding the reason for the filter could modify or remove the filter.
-
Using other pipelines not having a similar filter from other search interfaces or directly from the API.
You can ensure security by enforcing the search hub at the search token level (see Search Token Authentication). Moreover, search hubs defined on the client-side that are used as conditions in pipelines do not safeguard the security of your filters.
Syntax
Use the following query pipeline language (QPL) syntax to define a statement expressing the filter
feature:
filter <queryExpressionPart> <expressions>
<queryExpressionPart>
A string indicating the part of the query expression to which one or more query expressions should be appended. Must be one of:
-
aq
: The advanced part -
cq
: The constant part -
dq
: the disjunction part -
lq
: the large part -
q
: the basic part
-
The combined query expression is:
((q AND aq) OR dq) AND cq
. -
When an Automatic Relevance Tuning (ART) (i.e.,
topClicks
) model is configured and ready in the query pipeline, the Coveo Machine Learning (Coveo ML) Intelligent Term Detection (ITD) feature extracts relevant keywords from the large part of the query expression (lq
). Otherwise, the Search API convertslq
to a partial match expression (see thelqPartialMatchKeywords
andlqPartialMatchThreshold
query parameters). In both instances, the subset of keywords processed fromlq
is ultimately injected in the basic part of the query expression (q
).
<expressions>
A comma-separated list of query expressions (e.g., `foo`, `@bar=="baz"`
). Each of those query expressions will be appended to the current value of the target <queryExpressionPart>
using an implicit AND
operator (i.e., a whitespace character).
The result set of the constant part (cq
) of the combined query expression is kept in a special cache.
Consequently, you should avoid creating filter
statements that add dynamic content to cq
, otherwise you risk filling the cache with useless data, which might have a negative impact on performance.
Example
You create a global condition with the following QPL definition:
Global condition:
when $searchHub is "Community"
In an empty query pipeline named Testing Filter
, you create three distinct statements, each expressing the filter
feature, with the following QPL definitions:
Statement 1:
filter cq `@language==en`
Statement 2:
filter aq `@source=="Public Content"`
Statement 3:
filter dq `@date=today`
You associate each of those three statements to the global condition you created before.
A user performs a query with the following payload:
{
"aq": "@tags==(\"Query pipeline\", \"Search API\")",
"context": {
"userRole": "Developer"
},
"cq": "@filetype==html",
"pipeline": "Testing Filter",
"q": "filter statement",
"searchHub": "Community"
}
Since this query goes through the Testing Filter
query pipeline and satisfies the condition of each statement in that pipeline, those statements apply. As a result, the different parts of the query expression are modified as follows:
-
cq
(i.e.,$constantQuery
) becomes@filetype==html @language==en
. -
aq
(i.e.,$advancedQuery
) becomes@tags==("Query pipeline", "Search API") @source=="Public Content"
. -
dq
(i.e.,$disjunctionQuery
) becomes@date=today
. -
q
(i.e.,$query
) remainsfilter statement
.
The combined query expression (i.e., ((q AND aq) OR dq) AND cq
) means that to be included in the result set of this query, an item must mandatorily:
-
Be an HTML file (because of the original
cq
query parameter). -
Be in English (because of Statement 1, which modifies the
cq
expression).
In addition, this item must:
-
Match the
filter statement
basic query expression (because of theq
query parameter). -
Be tagged as being about query pipelines or the Search API (because of the original
aq
query parameter). -
Reside within the source named Public Content (because of Statement 2, which modifies the
aq
expression).OR
-
Have been updated today (because of Statement 3, which modifies the
dq
expression).