How automated merchandising tools can improve online product discovery

A modern merchandising system can provide a guided product discovery experience to shoppers, which is essential for retailers with large online catalogs. These systems and tools allow merchandisers to improve product listing relevancy by controlling product matching and ranking when a shopper sees a large number of similar items. With automated merchandising tools based on precise customer targeting, powerful ranking rules, business signals, and customer data, merchandisers no longer need to manually sort products in categories or create curated results for hundreds of popular search phrases.

Before we dive into further details, here are the topics we will cover in the remainder of the post:

• What does relevance mean in online commerce, and why is it challenging to nail down
• What are the merchandising strategies and automated tools that can help improve product discovery relevance:
  • Merchandising bias: using business signals to add drivers to search result relevance
  • Referring: redirecting customers to different content where appropriate
  • Curation: making sure that products are grouped and ordered purposely by category and relevance
  • Targeting: designing specific customer experiences based on the context of the situation
  • Experimentation: A/B testing and data-driven decision-making
  • Governance and transparency: merchandisers should be able to easily preview the effects of merchandising rules to prevent or resolve conflicts between them
• Why a modern merchandising platform is important for every large retailer

That being said, let’s examine relevance and its place in the e-commerce world.

When defining a mission for product discovery teams, businesses often state that they should “provide the shortest path towards the most relevant products in the catalog”. While this goal is very clear on the surface, accomplishing it requires a deep understanding of relevance, a key concept which is pretty challenging to pinpoint. Academic papers written on information retrieval often define relevance as “bringing users the content which satisfies their information need as specified by search query”. Yet, in e-commerce practice, relevance also relies on implicit signals, such as the knowledge of the customer, the context of the particular customer engagement, and the general business context.

For example, when a customer is searching for a “cocktail dress”, which of the 800 matching cocktail dresses in the catalog should we show first? Do we know the customer’s size? Their favorite designers? Color preferences? Price sensitivity? All these questions are important to determining the relevance of a product to a particular customer. On the business side, maybe we need to push high inventory items to the top of the results page to ensure that we don’t sell out during large sale events? Or maybe we have a deal with some designer to promote their products this week? In a broad sense, the product discovery system should strike a balance between a customer’s intent to buy a particular product and a business’s intent to sell that product.

So, what are some specific merchandising tools that can help merchandisers maintain this delicate balance in relevance?

Let’s take a look at the distribution of search queries on a typical e-commerce site. In most cases, we see a common, long-tail distribution picture:

This distribution shows that 50% of the query traffic is coming from a relatively small set of queries, called the “head”. The other 50% of query traffic comes from a huge number of rare or unique queries, which form a long tail of query distribution. Head queries often involve the most valuable items within an e-commerce catalog, such as the most popular product types, brands, and styles. Therefore, it’s no wonder that head phrases are responsible for the bulk of the retailer’s revenue.

Merchandisers almost universally focus their efforts on guiding customers through the search experience associated with head queries. The long tail, on the other hand, is largely left for search algorithms to address. For these queries, natural language processing, query classification, ontologies, concept search, normalizations, sophisticated ranking, and other information retrieval and machine learning techniques can provide a solid relevance basis.

Head queries often present significant challenges for traditional information retrieval and ranking systems. One of the reasons for this is that head queries are typically short and simple phrases that match a large number of products. For example, a customer’s query for “handbag” doesn’t provide enough context for search engines. From the search engine perspective, all handbags in the catalog are equally relevant to the customer. These are the situations when search engines need an additional signal coming from the merchandiser or from the customer’s history to determine relevance. Since we can’t expect that the majority of customers have an interaction history deep enough to provide additional relevance signals, merchandisers often stand on their own in the quest for relevance.

To solve this problem of search engine “indifference”, merchandisers should be able to inject their bias into product scoring formulae based on business relevance. This consists of product and SKU business metrics, such as newness, popularity, inventory levels, SKU availability, review ratings, sales and returns numbers, and many others. The combination of these business signals provides an additional signal which will re-rank equally relevant products by their business value. This technique is especially valuable for category browse, when all products belonging to a category are equally relevant from the standpoint of a search engine.

Referring is another powerful tool for merchandisers. The customer experience can often be optimized by redirecting customers from a keyword search to a particular category, landing page, or even mini-shop created around a specific designer or style of the product type the customer is looking for. Sometimes, it is desirable to just send customers directly to the product page. This may happen when a customer is searching explicitly for a single product (by SKU, model number, or marketing ID). In other cases, we can detect that the customer is looking for a product which is not carried by the shop, and suggest an alternative instead of showing zero search results. This suggesting applies not only to a specific product, but also to whole product types and categories. For example, if the customer searches for “Hawaiian shirts”, and the retailer provides none, they should suggest colorful button-down shirts as a reasonable substitute. In all of these cases, the merchandiser should have a clear way to detect customer intent and configure the appropriate referring action.

At the heart of an effective online catalog, there is a carefully tuned category hierarchy which reflects how customers think about a retailer’s assortment of products. Managing this hierarchy and assigning products to categories manually is a tedious task for merchandisers. Modern merchandising platforms should therefore provide tools for automated product categorization which take category definition and category inheritance rules into account. New products should automatically become a part of a proper category based on their attributes and business metrics, such as: newness, price range, time-on-sale, and sales numbers. For example, merchandisers should be able to easily create categories such as “Gifts under $50”, “Most popular items”, or “New arrivals”.

Another powerful technique is to further organize the presentation of category browse or search results by grouping products by an attribute such as: color, style, purpose, or occasion.

Grouping products into collections like outfits, gift baskets, dinnerware, bedding sets, or tool sets will help to cross-sell products and further inspire customers.

In the operations of online retailers, there is often a need to override “natural relevancy” decisions made by the search engine from a keyword search perspective. There can be issues with product attribution, which causes the search engine to return bizarre results, or with special business constraints. For example, this might not allow results from several designers to be mixed together, even if the products match the user query. Other times, there is a need to pin some featured products on top of the search results, or take an opportunity to cross-sell a complementary product. Another situation for overriding natural relevancy decisions is when there’s a specific trendy term or marketing campaign name which can not be handled directly by the search engine.

Additionally, merchandisers and marketers should be able to easily create timed campaigns for seasonal and trendy products by boosting the corresponding product attributes, or by creating dynamic landing pages.

A merchandising platform is only as good as its targeting. Merchandisers should be able to design a customer experience based on the detailed customer and business context of the interaction. This context may include the customer query, a current and requested category, the detected customer intent, customer preferences, and other available signals.

For the merchandiser, it is important to avoid surprising results, and maintain a consistent customer experience throughout the navigation journey. For example, let’s say that a customer is looking for Hurley brand shoes.

There are many ways to arrive at essentially the same result set. The customer could search for “Hurley shoes”, or search for “Hurley” and select the “shoes” product type facet, or go to the shoes category and select “Hurley” in the brand facet. Sometimes, an additional query term does not change the result set. For example, queries for “Samsonite luggage” and “Samsonite” will essentially yield the same result.

From a merchandising perspective, the customer experience on a result page should mostly depend on the product assortment in terms of product ranking, facet selection, and featured products selection. It is more productive for the merchandiser to analyze the result set, rather than trying to figure out how to recreate the same result set. Thus, it should be possible to use characteristics of the result set in making merchandising decisions. For example, merchandisers should be able to configure the selection of shoes-related facets regardless of how a set of shoes was discovered. This can be done through keyword search, category browse with filters, or a combination of both. To achieve this unification of experience, the merchandising platform should support “result-based context”, in addition to “request-based context”.

The modern approach to merchandising calls for a certain amount of discipline, and a data-driven approach when it comes to merchandising decisions. The merchandising platform should seamlessly integrate with customer experimentation systems to allow for data-driven decision-making and A/B testing. In general, the customer allocation for experimentation should be treated as yet another interaction context parameter, and be allowed to tweak the customer navigation experience within the experimental group.

The large-scale implementation of a merchandising platform is likely to result in thousands of business rules, managed by dozens of merchandisers. The governance and management of these merchandising rules requires special support from the merchandising platform. It should be possible to easily search and dissect merchandising rules, and come up with usable analytics for them, such as highlighting rules which never or rarely fire, or frequently cause rule conflicts.

Merchandising platforms should offer instant help in rule creation, such as suggesting proper attribute names and making recommendations. For example, when selecting facets to show in a given situation, the platform should be able to recommend the most common and most salient attributes per product type.

In many cases, merchandisers should allow the platform to go on auto-pilot. This makes sense for decisions such as selection of the facets, featured products, and default product ranking. However, it should be easy to make a single override when needed.

Most importantly, merchandisers should be able to immediately preview the results of rule changes, and the platform should be able to clearly explain what rules have fired for any given interaction, and how they affect the customer experience. With a sufficient number of rules, conflict between rules becomes a frequent situation, so the platform should have a simple, yet powerful model to resolve such conflicts before they become an issue for customers. Transparency for rule changes and behavior is needed in these situations, and is therefore vital for a merchandising platform.

With modern merchandising systems, department stores can more easily provide a guided product discovery and selection experience through their expansive catalogs. Even better, these tools are largely automated, greatly reducing the time spent configuring results pages manually. Merchandising is important for any large department store with a massive online catalog, making these tools of great value.

Now that we’ve laid out the overview for what merchandising tools are required by large retailers, our next post in the series will be a technical explanation of how to build a merchandising platform that contains all these tools. If you are interested in learning more about this topic, and our experience with building and running sophisticated large-scale merchandising platforms, don’t hesitate to contact us — we will be happy to help.