Tuning SAP Commerce Cloud's Solr Search

SAP Commerce Cloud ships with Solr product search out of the box, and Solr is a very capable search platform. So why is the search experience so often disappointing?

In 15+ years working with SAP Commerce Cloud (formerly Hybris), I've spent a lot of that time tuning and optimizing its Solr-based product search. Poor search quality is often the result of running on default settings without investing the time needed to customize it, combined with a few outdated design decisions in the Solr integration.

This is the start of a series where I share the fixes I've applied across many projects — practical, opinionated, but grounded in real client work.

Tip 1: Fuzzy and Wildcard Search

With fuzzy search, words match not only when they're identical, but also when they're similar (within a certain Levenshtein distance), accounting for typos and spelling variations. A fuzzy search for the misspelled word “sweter” manages to find “sweater”. This sounds great at first glance, but in reality it makes the search unpredictable and causes a lot of unwanted matches.

Turn fuzzy search off entirely for all fields and use the Solr spellchecker to detect typos instead.

The same applies to prefix wildcards. Don't put a wildcard in front of the search term, like *searchterm*. Use the wildcard only at the end: searchterm*. Would you expect to find products with “blink” when searching for “ink”, or in German “Schwein” when searching for “Wein”? That's why in almost all cases you're better off turning the prefix wildcard off and searching for ink* rather than *ink*. On top of that, a search for *ink* is far more expensive than ink* — this lies in the nature of how the Lucene index is structured.

If you're worried about not finding German compound words, look into a decompounder.

Tip 2: Field Boosting

SAP Commerce Cloud uses arbitrary numbers for field boosts — values like 25, 50, 100. A field boost is a multiplier of the calculated BM25 score.

If one of the search terms matches your product title, the BM25 score might be something like 3.5. Apply a field boost of 25 and we get 87.5 as the final score for this match. Now take a match on a second field with a much lower score, say 2.0, but where that field has a boost of 50 — the final score becomes 100.

The field boost is dominating the ranking, while the actual match quality, the BM25 score, barely matters. It should be the other way around: the quality of the match should drive the ranking, with the boost only tilting it slightly. That's why the different boosts need to stay close together. What matters is the ratio between them, not the absolute number. When the spread is large (25 vs 100), a weak match in a heavily boosted field beats a strong match in a lightly boosted one — which is exactly backwards.

So keep the weights small — factors like 1.2 or 1.5, with slightly higher values for stronger match types such as phrase matches. You can also use values below 1, like 0.75, for a field with less significance.

What works well is a fixed ratio between exact match, wildcard match, and phrase match. For example, a wildcard match carries 30% less weight than an exact match, and a phrase match carries double the weight of an exact match. If your product-name field has a base weight of 1, then exact match stays at 1, wildcard at 0.7, and phrase at 2.

Tip 3: Use ValueResolvers, not ValueProviders

SAP Commerce ships with both ValueProviders and ValueResolvers for populating custom index fields in Solr. The interface FieldValueProvider and the class AbstractPropertyFieldValueProvider have been deprecated since versions 5.5 and 5.2 respectively — more than 11 years ago.

ValueResolvers are the way to go, and no new code should be implemented using ValueProviders. In some cases it's even worth porting existing code from a ValueProvider to a ValueResolver. Here's why ValueResolvers are superior:

1. Localized fields need no special handling: the ValueResolver is called for each configured language, with the correct language already in the session. You don't have to deal with it in your code, and the same applies to currencies. The interface is cleaner, with less boilerplate to write.
2. In a ValueResolver you have direct access to Solr's InputDocument, so you're not limited to adding a single attribute — you can add multiple fields. Say you want to index gross and net price, both sitting on the current price row: you call your PriceService only once in a single ValueResolver, get the price row, and add both prices. With a ValueProvider you'd have to implement two and call your PriceService in each. It has some limitations, but it works fine for something like two prices.
3. You have an IndexerBatchContext available. This context is created before the first product of the current batch is indexed and discarded afterwards. IndexerBatchContext.attributes is a general-purpose HashMap. It can cache attributes between products of the same batch, or — more importantly — between different ValueResolver calls for the same product. If you want to index 20 different classification attributes, fetch the features for a product once and store them on the IndexerBatchContext. The remaining 19 ValueResolvers then just read from a HashMap instead of fetching from the classification system, which we all know is expensive.
4. Don't use CommerceClassificationPropertyValueProvider — it will severely cripple your index performance. Implement your own ValueResolver instead, and make sure the classifications are only fetched once per product, e.g. by caching them on the IndexerBatchContext.attributes mentioned above.

Tip 4: Synonyms

One of the reasons synonym handling gets complex is the support for multi-term synonyms. As soon as you add support for multi-term synonyms — like “wifi” to “wi fi”, or “real estate” to “property” — you have to deal with a graph structure and not a flat stream of tokens anymore. Dealing with a graph in your query, and potentially also during indexing, is not trivial and comes with a lot of catches.

But let's put these technical complexities aside for now and focus on a couple of best practices around how to maintain synonyms in SAP Commerce Cloud (SCC).

Synonyms in SCC are very simple: you add a new SolrSynonymConfig, link it to a language, put e.g. “couch” into the from-field and “sofa” into the to-field, click Export to Solr, and you have added a new synonym.

But what about the other direction? We have added couch → sofa. Will a search for “sofa” now also find a couch? No, it will not — what we've configured is a uni-directional synonym.

The good news: you don't have to maintain synonyms in both directions manually. You can leave the to-field empty and put a comma-separated list into the from-field, like couch, sofa. SCC will then expand this to couch → sofa and sofa → couch during the synonyms export to Solr.

This works with larger groups too, and also with multi-term synonyms. Just extend the list: couch, sofa, chaise lounge. With that, you have configured a group of three bi-directional synonyms, one of which is a multi-term.

The one thing to keep in mind is that the from-field should not be longer than 5–6 terms, since the number of combinations grows fast — especially if you do the synonym expansion at query time.