Query Suggestions

Query Suggestions - What is it?

Query Suggestions is a well-known UI experience that many of the best websites offer. You’ve most likely encountered it: as users type a query, a dropdown box appears with suggestions of more specific - and more popular - queries.

Providing suggestions helps users type longer queries. For mobile users, it reduces the amount of typing required to execute a search. But it’s not just about typing less. It’s far more powerful - it helps users formulate the best search. Good suggestions are proven queries that ensure best results: the quality of the suggestions is as important as the results they generate.

At Algolia, we use Query Suggestions to extend the conversational search experience. It’s as if the user, with every keystroke, is conversing with the search bar, discussing with it exactly what to send the search engine to obtain the best results. For this to work, the suggested queries need to be instantly responsive and intuitively relevant; if either one of these is missing, the conversation is over.

How it works

Query Suggestions precedes the actual search:

• As the user types, an initial search processes each keystroke, searching through an index of carefully curated suggestions. We speak below about how we curate these suggestions, but essentially it is based on Algolia’s Search Analytics feature.

• With every keystroke, the user has an opportunity to select from a small set of similar queries that start with the same prefix. The user can of course ignore the suggestions and keep typing; with every keystroke, a new list of suggestions appears.

• When a user selects a suggestion, that action sends the suggestion as a query off to the main index.

Check out our demo.

All Enterprise customers have access to Query Suggestions from their dashboard. Please reach out to the Algolia Success Team at support@algolia.com if you have any problems accessing Query Suggestions on your dashboard.

Getting Started with Query Suggestions

To get started with Query Suggestions, you’ll need to think about two very different subjects:

• The Query Suggestions index: How to populate, configure, and maintain the index behind the suggestions.
• Your front-end implementation: How to integrate Query Suggestions into your front-end solution.

We’ll focus on the first one - that of configuring the Query Suggestions index - as this is the essence of Query Suggestions, to create an index of good suggestions based on analytics.

If you want to learn how to build a great front-end for Query Suggestions, we offer guidance below.

Query Suggestions is a separate Index

Query Suggestions uses a dedicated index to suggest the most relevant/most used queries. This is not a new API endpoint, nor an API feature, but a clever use of an index. Also note that this is different from your primary index that contains your data. This is a separate index whose only content is previous searches from your users, ordered by popularity, as we see below.

Configuring the index with the Dashboard

Query Suggestions is available in the Dashboard to all Enterprise customers, but you need to enable it first. Please reach out to the Algolia Success Team at support@algolia.com to enable this feature on your dashboard.

You can use the Query Suggestions section of our dashboard to manage its data and configuration.

• Create your index
• Import data
• Setup attributes or categories
• Configure settings like languages, minimum length for prefix
• Enable/Disable data sources
• Status updates
• Manage banned expressions

Query Suggestions - An Example of How it Works

Query Suggestions is entirely data-driven, suggesting the best-matched searches based on a comparison between the letters and words being typed and a collection of stored queries.

The best-matching algorithm is pretty straight-forward: best suggestions are based on popularity, which is the number of times the query has been previously typed in. Essentially, queries that have the highest counts - that is, which have been made most often by users in the past - are the ones that appear at the top of the list of query suggestions.. Here’s an example:

1
2
3
4
{
  "query": "iphone X",
  "popularity": 255
}

Whatever the source of data, the suggestion engine uses a popularity score to determine which queries to suggest. Every input uses the same format for its suggestions: the query + the number of times it has been used. Note that if the same query comes in from more than one source, we add the values together and use the total as the score for that query.

An example

There are 2 important aspects to keep in mind with query suggestions:

• The content of suggestions - which suggestions appear
• The order of the suggestions - which appear at the top and which appear below

Take a look at these suggestions for the query “iph”:

• iphone
• iphone 6
• iphone 8
• iphone charger

These suggestions seem appropriate and relevant: “iph” suggests “iphone”, and the most popular queries for iphone are either models or accessories. Good so far.

However, iphone X is missing and 6 is higher than 8. This is most likely due to the current analytics data, which relies on popularity.

• From a content point of view, X is only a recent release, so it is not yet popular enough to be included as a suggestion.
• From an order point of view, 6 has been on the market longer than 8, so it appears above 8, even though 8 might have been the better, more up-to-date / relevant suggestion.

The solution? We regenerate your suggestions index every 24 hours, and take into account only the last 30 days of analytics data. So if more people search for “iphone 8” than “iphone 6”, the former will end up on top. This is still biased by the initial position though - that’s why we let our customers upload external analytics data to feed and improve the popularity scores of their suggestions.

Building your query suggestions index

All suggestions are stored in a separate, single Query Suggestions Index. This index is different from your main index.

There are 3 sources that can populate the Query Suggestions index:

• Algolia Analytics
• your own datasets of suggestions, like Google Analytics or derived from your own database
• filters & facets from your main indices

You can use one source, or combine several sources together.

Algolia Analytics

Algolia Analytics happens behind the scene. You only agree to use Algolia’s Search Analytics (setting true or false in your configuration), and we do the rest - capture the necessary metrics and build the analytic-based suggestion index. While we capture the data in real-time, we rebuild the index once every 24 hours.

Query Suggestions is a perfect use case for Analytics data. We base our suggestions on Popular Searches.

External data / Your own suggestions

Analytics data is only one source. When you get started, you’ll need some kind of starting data. You can either upload your own suggestions, or we can use your main index’s filters and facets to help build your suggestions index.

To do this, you can place your own suggestions in a JSON format and upload the content. This data can be made up of what you consider ideal queries; or you can feed in analytics data from other sources, including Google Analytics. It just needs to follow the same format as Algolia’s own analytics data.

Here is an example of JSON file you can send:

1
2
3
4
5
6
7
8
9
10
[
  {
    "query": "iphone",
    "count": 10031
  },
  {
    "query": "samsung",
    "count": 731
  }
]

The above JSON format can be easily uploaded via the dashboard, within the Query Suggestions section.

Using your index’s searchable facets

You can also use your main index to generate query suggestions by using facets. You send the query suggestions engine attributes that are set up as searchable facets. These will be resolved into very precise suggestions. For example, if you send the engine “brand + color”, you can generate a query for every brand + color combination:

• “nike red”
• “nike blue”
• “adidas red”
• “Adidas blue”
• …

These queries will be sent along with their respective scores. Normally, they come in with low scores, being based on the amount of items that match those search terms.

This method is used primarily when you have no initial data - neither analytics nor your own suggestions. In the long run, these facet-based suggestions become less relevant (that is, less popular) than your analytics-based ones.

You’ll need to have large activity to justify using Query Suggestions. We also have a 30000 limit of suggestions by the Algolia analytics. So large customers will want to supplement the data with Google or other kinds of analytics.*

Note: the analytics on your replicas will be taken into account.

Filtering-out unwanted queries

You need to be careful: by default, every search is captured and uploaded. Therefore, Query Suggestions can be misled - the data can be mistaken, irrelevant, or even completely embarrassing! One company, for example, once received a flood of phony, inappropriate searches, which, due to their apparent popularity, scored higher than other searches. This meant that other users began to see these phony searches. It’s a real concern that you want to address.

We have 4 systems in place to limit such inappropriate suggestions and to prevent fraud.

Minimum number of results required

By default, the suggestions index only includes queries that have returned at least 5 results. This limit is configurable, yet keep in mind that you’ll need an Algolia Admin to assist you.

Minimum number of letters required

We do not suggest queries that contain less than the minimum amount of letters configured. The default is 4 but this can be overridden.

IP restrictions

We apply a distinct IP per query: if an IP sends many requests for the same query, it will always count as 1. This creates a barrier to an accidental or abusive flood of requests from the same IP.

Blacklisted words

Perhaps the most powerful way to tell the engine what not to do, is to use our banned list functionality. With blacklisting, you can remove any unwanted queries. There are two ways to do this:

• A list of exact matches: you can eliminate full queries by creating a list of partial words, full words, and phrases that the engine should ignore. The query must find an exact match in the list to be ignored.
• You can also create one or more expressions that look for certain inappropriate or unwanted patterns in queries. Here, you can implement partial matching: any query that contains a blacklisted expression is ignored.

Displaying Query Suggestions

Query Suggestions vs. Autocomplete

Query Suggestions is essentially an autocomplete UI experience.

Algolia already offers an Autocomplete library as an alternative to the InstantSearch libraries, in which both display as-you-type search results. Autocomplete displays its results in a dropdown box, usually with a short text and small thumbnail images. InstantSearch displays its results directly on the page, with room for full-size images, more descriptive text, and a variety of filters and categories.

Query Suggestions is a different kind of autocomplete: it also uses a dropdown box and changes its contents as the user types, but it does not display results. Instead, it displays suggested queries that are executed only when the user selects one of them.

Combining InstantSearch with Query Suggestions

A particularly advanced and useful UI combines the power of Query Suggestions with InstantSearch results. This would mean that, as the user types, two areas of the screen change: (1) queries are suggested while (2) new results appear. Seeing both can help users make a decision. Check out our InstantSearch + Query Suggestions demo to see how this could look.

Combining suggestions and results in the same drop box

Another choice is to combine suggestions and results within the same dropdown box. This is an especially good UI for mobile, which simplifies the UX by having suggestions + results. Check out our mobile android guide and mobile ios guide.

Adding scope

A useful enhancement to the list of suggestions is to add categories / scope into the results. So instead of, or in addition to, suggesting queries, you suggest filters. For example:

If someone types in “iphone”, the standard approach is to present useful queries: “iphone 7” “iphone charger” “iphone X cases” “iphone X speakers”

But you can also present categories:

iphone

• in phones
• in accessories
• in electronics

Or a combination of both.

This can be done by configuring Query Suggestions with categories from your main index.

Once your categories are set up, Algolia will return the suggestions with additional category information. It will not do any kind of logic for you. So it’s up to you to choose:

• how to display the category-based suggestions;
• what to do when a category-based suggestions is clicked, selected, hovered, … . In most cases you’ll want to redirect to an InstantSearch page with a pre-filled query and add a default filter on the chosen category.