Considering A Recommendation Engine? Read This First.
It pays big to get your recommendation engine algorithm and delivery right.
What’s the secret sauce? How do they actually work? What do you need to know before building your own? There are two types of recommendation engines based on two distinct pieces of logic.
1. Collaborative Filtering
Algorithms that make recommendations using existing and historic customer information. This logic rests on the idea that similar types of customers make similar purchases. This approach is known as Collaborative Filtering.
The thinking behind this method is that if a customer behaves similarly to another customer then you can use the purchasing behaviour of one to recommend products to the other. Hence, collaborative.
The advantage of this approach is that it doesn’t rely on the attributes of the products themselves to make recommendations. If you’re selling homewares, product attributes may be relatively easy to deploy, but if you’re Netflix and your product is multimedia, asking the algorithm to “understand” a product as complex as a movie is far more difficult.
The disadvantage of using a collaborative filtering approach is that it requires an existing, rich database of customer information that includes browsing and purchase histories, wish list contents, email marketing activities and the like.
To work effectively, this algorithm will need to ingest enough information to build customer profiles and then to filter them into groups into ‘like’ groups based on clusters of similar behaviours, activities, and preferences. It will then use the purchase histories of similar customers to recommend products to others in the group.
This 'cold start' challenge of collaborative filtering means that it doesn't suit companies who are just starting out acquiring customers or established companies who have insufficient customer data.
2. Content Based Filtering
Algorithms that make recommendations using product information. This logic rests on the idea that customers will purchase products that are similar or adjacent. These engines that make recommendations based on attributes of the products being sold. This is known as Content Based Filtering.
The thinking behind this method is that if a customer purchases a particular item then they are likely to purchase other similar or related items. The algorithm builds a user preference profile based on their purchasing and browsing behaviour and recommends products with shared attributes.
The advantage of this approach is that it doesn’t require a large database of customer profile or other data to work. It can be rolled out on a clean system so long as the product attributes and relationships have been mapped properly.
The disadvantage of this approach is that it will miss those (many) instances where customers buy products that are seemingly unrelated. It also narrows the pool of potential recommendations to those that are similar to the product being purchased.
3. Hybrid Approach
A hybrid approach can be most effective. Research has (unsurprisingly) shown that an recommendation engine that uses elements of both content and collaborative filtering can have a much bigger impact on sales revenue than each one separately. A hybrid approach can be used to mitigate the disadvantages of both approaches separately and provide more accurate recommendations.
More than that, the engines used most often as examples of recommendation engine excellence are all hybrid systems – like Netflix and Amazon.
Amazon’s calls its hybrid approach “item to item collaborative filtering” and uses it to personalise each customer’s experience of the Amazon site. The engine is based on what a customer has bought in the past, what they have in their shopping cart, their wish list, products they’ve rated and liked, and what similar customers have viewed and purchased.
A recommendation engine has the potential to put some serious rocket fuel behind sales numbers. It can also fall flat, which is disappointing (but also preventable). As with any technical investment, decisions need to be made in the design of the engine that impact on cost, build time, and performance.
Here are a few considerations that can have a big impact both on both technical and commercial performance:
1. The structure of the hybrid approach.
Depending on the use case, the hybrid approach can be built to optimise for accuracy by beginning with either collaborative or content based filtering and moving on to the other, by assigning weightings or rankings to the outputs of one approach over the other, or by running them not in sequence but in parallel.
2. What data the algorithm will use to calculate affinity.
This is not a problem for companies with thousands or even hundreds of thousands of customers, but when there are millions, decisions need to be made that will allow the algorithm to perform at peak and instantly. Solutions include using an evolving subset of customers, defined by characteristics such as recency and activity, to base the collaborative filtering aspect of the engine on.
3. How recommendations will be delivered to customers.
In-situ site recommendations should be supplemented with email marketing campaigns that are based on the algorithm. Ideally, email marketing campaigns that are based on product categories should be ranked by performance. If a customer qualifies for more than one campaign, the best performing campaign should be deployed first.
Fundamentally, the purpose of the recommendation engine is to grow sales. The Return on Investment (ROI) of building a recommendation engine should be considered in the early stages. It may make most sense to take a staged approach, or it may make sense (if the data is there) to build a hybrid from the outset and stage the roll-out from on-site to email. Most importantly, metrics should be put in place to measure its success and a baseline established to compare performance.
If your business is ready to start scoping out what your recommendation engine may look like, make sure you find the right technical and business partner who understands not only the technical aspects of design, build, and implementation – but also approaches the problem commercially with a focus on measurable growth in revenue.