Algorithmic pricing, part I: The risks and opportunities

AI has great potential to transform or even disrupt enterprise operations, and price management is among the most important business functions that directly affect enterprise profitability and efficiency. The impact of artificial intelligence (AI) technologies on the price management process is not always straightforward. For instance, basic machine learning methods are widely used for price elasticity analysis, and their benefits are well understood. However, more advanced and emerging methods, such as natural language processing, image classification, and reinforcement learning, are less commonly used in enterprise operations, and their applicability to price management is less obvious. Even so, as we will discuss later, these methods can deliver substantial benefits in certain pricing use cases.

This is the first post in a series of articles where we will explore how the increasing adoption of data science methods and AI technologies impacts price management processes in retail and consumer goods manufacturing companies. In this series of posts, we will use the term “algorithmic pricing” to refer to a price management process with fully or partially automated decision-making through statistical analysis, machine learning, or other AI methods and techniques.

In the first article, we will attempt to quantify the risks and opportunities introduced by algorithmic pricing by looking at industry trends and statistics, examining known case studies, and relaying facts about the adoption of algorithmic methods by medium and large retailers, including Amazon.

Algorithmic pricing is a broad term that generally refers to automated decision-making in the area of price management using rule-based or self-learning algorithms. In most retail and consumer goods applications, this generic concept typically translates into some combination of the following features and capabilities:

• Dynamic pricing. Automated decision-making enables more frequent and precise changes of various components of the price waterfall, including base prices, promotions, and special offers. These changes help pricing accommodate competitor moves, inventory turnover and clearance goals, and demand spikes in a profit-optimal way.
• Personalization. The algorithmic approach helps to segment or personalize certain elements of the price waterfall, such as discounts and special offers. It also accommodates differences in consumer tastes, price sensitivities, and other statistical information on potential buyers.
• Differentiated pricing strategies. Data-driven methods help to optimize pricing strategies for different products and categories depending on the product’s influence on consumer value perception, similarity to other products, and other factors.
• Optimality guarantees. The data-driven approach helps to determine near-optimal pricing parameters and discover missed opportunities. For example, an algorithmic promotion management system can suggest new promotions that can improve the performance of the baseline promotion calendar.
• Advanced internal and external signals. Algorithmic pricing often relies on the statistical analysis of transactional, inventory, and catalog data as well as external signals, such as competitor pricing, to improve the quality of decisions and react to trends. In some cases, it uses natural language processing (NLP) and image recognition technologies to leverage textual and visual data, such as product descriptions and images.
• Integration with merchandising and inventory management. Algorithmic pricing incorporates stock level and selling velocity data into price optimization, so the price can be increased or decreased to avoid stockouts and overstocks. On the other hand, the fundamental capabilities that are generally needed for algorithmic pricing, such as demand modeling, are often applied to other use cases, such as assortment and inventory optimization.

These features and capabilities can be implemented and used differently depending on strategic considerations and other factors. In other words, algorithmic pricing is not a pricing strategy, but rather a toolkit for executing a pricing strategy efficiently. We will defer a detailed discussion of the relationship between algorithmic pricing and pricing strategies until the next post in this series, and instead focus on the big picture of algorithmic pricing potential and adoption in the remainder of this post.

The idea that an enterprise can benefit from making more intelligent and dynamic pricing decisions using some sort of algorithm is not new. The history of algorithmic price management can be traced back to 1985, when the first yield management system was introduced by American Airlines. The airline company developed this system for a good reason: it had come under heavy pressure from low-cost airlines that were emerging at that time, and the optimality of ticket price setting was a matter of survival. Once the dynamic price and seat inventory management system was launched, it not only turned out to be a critically important factor for the company’s survival, but also shook the entire airline market. The low-cost airlines were hit very hard because American could now match or beat their prices. This process culminated in the bankruptcy of American’s main low-cost competitor, People Express, a year after the yield management system was launched. Donald Burr, CEO of People Express, commented on the failure of the company [1]:

We were a vibrant, profitable company from 1981 to 1985, and then we tipped right over into losing $50 million a month. We were still the same company. What changed was American’s ability to do widespread Yield Management in every one of our markets. … We did a lot of things right. But, we didn’t get our hands around Yield Management and automation issues. … [If I were to do it again,] the number one priority on my list every day would be to see that my people got the best information technology tools.

Nowadays, the incremental revenue delivered by yield management systems in airlines is comparable to the margins; in other words, algorithmic pricing is literally a means of survival.

Using an airline as a model for retail and manufacturing might seem odd, but this old story demonstrates the disruptive potential of algorithmic operations and suggests a very timely question: if a significant share of players in a market adopt an algorithmic approach to price, promotion, and inventory management, can late adopters even survive this change? This adoption occurred early in the airline industry, and one of the reasons for this was the early digitization of the airline reservation systems, which created a perfect environment for decision automation and dynamic optimization. Other industries, such as retail, have different digital transformation trajectories, and algorithmic operations might not have fully hit them yet. We will attempt to analyze the trajectory of algorithmic pricing for the retail sector in the sections that follow.

Algorithmic pricing technologies and the transparency of the Internet have had a major impact on the pricing behavior of retailers and even the US economy as a whole. Although it is not always possible to directly observe and measure the adoption of algorithmic methods by enterprises, a considerable research effort was made to quantify the effects of algorithmic pricing based on available data, as well as to reverse engineer the algorithms used by companies such as Amazon and Uber. [2]

The frequency of price changes in multi-channel retailers has been increasing over the past 10 years.[3] The average duration for regular prices has fallen from 6.7 months in 2008–2010 to 3.6 months in 2014–2017, and the duration of posted (promotional) prices followed the same pattern, decreasing from 5 months in 2008 to 3 months in 2017. As shown in Figure 1, the increase in the frequency of price changes is particularly high in retail sectors in which online retailers have large market shares, such as electronics and household goods. Moreover, there is evidence that this effect is particularly strong for goods that are actively sold on Amazon, one of the earliest and most advanced adopters of algorithmic pricing, and its pressure is spread across the entire industry through the automatic price matching implemented by many retailers in other ways.[3:1] The industry-average trend for both regular and posted prices is shown in Figure 2.

Figure 1. Monthly duration of regular price changes by retail sector[3:2]

Figure 2. Monthly duration of regular and posted price changes. [3:3]

Prices are also becoming more uniform, that is, less differentiated by the buyer’s geographical location. This process is largely driven by online retailers (particularly Amazon), which tend to have more uniform prices than multi-channel retailers. It is estimated that uniform pricing causes retail chains to underharvest anywhere from 3 to 10 percent of variable profits relative to a benchmark of flexible prices.[4]

The above trends create a risk of a downward spiral, where a retailer or brand loses its position as a decision-maker and passively follows external changes through automatic price matching or markdowns. There is strong evidence indicating that this process can be harmful for the firms. For example, Nielsen reported that around 67 percent of U.S retail promotions do not even break even, and the average efficiency of trade promotions is getting worse over time. [5] However, the same report indicates that there is a large spread in promotion efficiency, with the best promoters performing five times better than the least efficient ones.

It can be concluded that as competition heightens and leading retailers master algorithmic methods, the overall role of algorithmic pricing increases. Some companies will be harmed by it, while other companies will benefit from it, depending on their ability to use the new methods efficiently. The companies that attempt to use individual techniques (such as automatic price matching) without properly integrating these methods into their strategies and value propositions are likely to be among the losers in this game. The companies that take a broad perspective on algorithmic methods and leverage algorithms and data to create engaging and valuable customer experiences will most likely be the winners.

We next turn to the case studies of algorithmic pricing to better understand how retailers use it, and the typical benefits it provides. We will first focus on the case studies of medium-size retailers, and then examine the techniques used by Amazon.

There is no shortage of case studies connected with algorithmic pricing in retail in recent years. We provide six case study summaries in Table 1 to highlight several popular use cases and the typical results in terms of revenue and margin uplift. On average, the implementation of algorithmic pricing for one or two use cases drives an increase of 3 to 5 percent in sales and 4 to 11 percent in margins. It is not surprising that the results are particularly impressive for retailers with dynamic environments, such as flash sales or online-only retailers. For example, Rue La La, an apparel flash sales retailer, achieved an average revenue uplift of 9.7 percent by implementing an advanced price optimization system that jointly optimizes prices for all products in a category using predictive modeling and numerical methods.

Table 1. Example case studies from medium-size retailers.

Amazon is known to be one of the earliest, most advanced, and most successful adopters of algorithmic pricing and algorithmic operations in general. This was certainly one of the factors that contributed to Amazon’s tremendous success and thirtyfold growth in market value over the last decade, from 30 billion in 2008 to 1 trillion in 2018.

Although the price management algorithms used by Amazon are not disclosed and most likely keep changing over time, there have been a number of attempts to analyze Amazon’s price setting logic and quantify its benefits by examining posted prices and product availability patterns.

First, it should be noted that Amazon does not just use pricing algorithms internally; rather, it is designed as an open algorithmic marketplace. Amazon provides third-party sellers with an API (Amazon Marketplace Web Service), which allows them to receive near real-time updates on competitive pricing, recommended inventory and pricing levels, new product opportunities, and so on. This interface helps sellers make dynamic pricing and sourcing decisions that can also be executed in near real time. [10] These capabilities are important because 18 percent of Amazon’s revenue comes from third-party seller services, which includes commissions, fulfillment, and shipping fees, and more than 3,000 sellers join the marketplace every single day. [11][12]

Second, Amazon uses complex algorithms to manage the marketplace and arbitrate sellers, which can be revealed using statistical analysis of the data scraped from the Amazon website. For example, all product pages on Amazon have a Buy Box, which contains the product price, shipping information, seller name, and buttons to change the quantity or purchase the product (Figure 3). If a product is offered by multiple sellers, Amazon selects the primary seller to be displayed in the Buy Box. The analysis of thousands of Buy Box instances reveals the complexity of the seller selection algorithm, which takes multiple factors into account, such as the price, number of feedbacks, and average ratings, as shown in Figure 4. [13] This algorithm has a major impact on the marketplace and seller performance, as more than 80% of all sales on Amazon go through the Buy Box. [14]

Figure 3. An example of a Buy Box on Amazon.

Figure 4. Relative importance of features in winning the Buy Box. [13:1]

Next, there is strong empirical evidence showing that third-party sellers that use algorithmic pricing techniques are generally more successful than non-algorithmic sellers. [13:2] Although the sellers who use algorithmic techniques do not explicitly identify themselves, it is generally possible to detect them based on changing price patterns and analyze their performance:

• On average, algorithmic sellers have slightly higher amounts of positive feedback. This is a surprising finding in light of the numerous surveys arguing that dynamic pricing is hated by most consumers.
• The rank of algorithmic sellers on the New Offers page is significantly higher than the rank of non-algorithmic sellers.
• Algorithmic sellers are active on Amazon for significantly longer periods of time than non-algorithmic sellers.

Finally, Amazon itself is an algorithmic seller, which seems to compete with third-party sellers in a relatively fair way. As illustrated in Figure 4, “being the Amazon fulfilled product” gives only a minor advantage in winning the Buy Box, and Amazon’s chances to win vary depending on the sophistication of other sellers. Amazon falls out of the top five sellers for just 4 percent of products when none of the competing sellers uses algorithmic pricing, but this figure jumps to 12 percent when algorithmic pricing is used by at least one third-party seller of a competing product. [13:3]

As a seller, Amazon is known to use a large variety of algorithmic pricing techniques. The most well known are dynamic pricing and automatic price matching, where the price can change several times a day following competitor prices, as illustrated in Figure 5. Other techniques employed by Amazon include charging higher prices for Prime members compared to non-Prime customers, increasing prices for customers who select Free One-Day Shipping, and adjusting prices depending on the time of the day or customer traffic. [15] Amazon sets prices automatically but has an anomaly detection process to intercept risky changes (e.g., the magnitude of price changes is above a certain threshold) and send them on for manual review and approval.

Figure 5. Example of Amazon following the lowest price of other sellers but with a premium. [13:4] (2014)

Although the above list of techniques is far from complete, it gives some idea of how sophisticated, ubiquitous, and diverse algorithmic pricing can be when it comes to industry leaders.

A comparative analysis of price changing patterns across large retailers indicates that they are gradually adopting the algorithmic methods pioneered by Amazon, but with a significant time lag. Amazon started to use dynamic pricing much earlier than other retailers. In 2012, Amazon was implementing around 270,000 price changes per day; meanwhile, Walmart and Best Buy were implementing only about 50,000 price changes per month. In 2013, Amazon sharply increased its price change frequency to 2.5 million price changes every day, while Walmart and Best Buy stayed at approximately the same level. [16] Amazon continues to change prices more frequently than other large retailers, but the gap has become smaller as competitors adopt dynamic pricing, as illustrated in Figure 6. [17]

Figure 6. Dynamic pricing by category: The percentage of tracked assortments of each retailer that changed price at least once a day (does not include third-party sellers). [17:1] (2017)

In our work with some of the largest US retailers, we have also observed the increasing usage of predictive models for various pricing and inventory use cases, such as the optimization of promotion calendars over the last few years. We discuss these insights in more detail in the following sections.

Advanced methods of revenue management have demonstrated disruptive potential in many industries, including airlines, hotels, and car rentals. The trajectory of algorithmic pricing in the retail industry seems to be different from many of these examples, but the analysis of price setting patterns indicates that the ability to manage prices and assortments dynamically using intelligent algorithms and diverse signals is becoming increasingly important. It can certainly contribute toward the pricing “race to the bottom,” but companies that properly integrate algorithmic pricing into their strategies and combine it with other strengths (e.g., customer intelligence) can actually create a competitive advantage.

1. R. G. Cross, Revenue Management: Hardcore Tactics for Market Domination, Broadway Books, New York, NY, 1997 ↩︎

2. L. Chen, Measuring Algorithms in Online Marketplaces, May 2017 ↩︎

3. A. Cavallo, More Amazon Effects: Online Competition and Pricing Behaviors, September 2018 ↩︎↩︎↩︎↩︎

4. S. DellaVigna, M. Gentzkow, Uniform Pricing in US Retail Chains, August 2017 ↩︎

5. Nielsen, Cracking the Trade Promotion Code, November 2014 ↩︎

6. Antuit, Food Retailer Deploys Offer Optimization to Increase Customer Loyalty & Sales, 2018 ↩︎

7. G. Benmark, S. Klapdor, M. Kullmann, and R. Sundararajan, How Retailers can Drive Profitable Growth through Dynamic Pricing, March 2017 ↩︎↩︎↩︎

8. K. Ferreira, B. Lee, D. Simchi-Levi, Analytics for an Online Retailer: Demand Forecasting and Price Optimization, 2015 ↩︎

9. H. Langer, How Personalization is Changing Retail Pricing, 2015 ↩︎

10. Amazon Marketplace Web Service Documentation, https://developer.amazonservices.com/↩︎

11. A. Levy, Amazon’s Sellers are Going Global, Helping the Company Generate Big Profits, April 2018 ↩︎

12. J. Kaziukenas, One Million New Sellers on Amazon, October 2018 ↩︎

13. L. Chen, A. Mislove, C. Wilson, An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace, April 2016 ↩︎↩︎↩︎↩︎↩︎

14. D. Taft, Amazon Buy Box: The Internet’s $80 Billion Sales Button. eWeek, October 2014 ↩︎

15. I. Steffens, Amazon Prime and “Free” Shipping, March 2018 ↩︎

16. Profitero, Amazon makes more than 2.5 million daily price changes, December 2013 ↩︎

17. A. Cheris, D. Rigby and S. Tager, Dreaming of an Amazon Christmas?, November 2017 ↩︎↩︎