Recommendation engines are powerful algorithms that help dictate many of our day-to-day decisions as consumers, from purchasing shoes on Amazon, to picking a song on Pandora, to selecting a show on Netflix.

 

Research has shown that effective algorithms powering these systems are instrumental in reducing customer churn and driving revenue, with Netflix valuing its recommendation platform at $1 billion per year. But how do these engines work, and what information do you need before implementing a successful system for your organization? Data Scientist Tuck Ngun from DataScience.com sheds some light.

 
The Necessary Foundation

Before you get started determining which recommendation engine is right for your team, it’s important to have a substantial data repository available to draw upon. Without that, your recommendation engine will lack necessary context, and the accuracy of the system will likely suffer as a result. Once you have a significant amount of high quality data, and enough support from your data science and engineering teams to manage and maintain the model, you are ready to start thinking about the pros and cons of content-based filtering versus collaborative filtering.

 
Content-Based Filtering

Like its name suggests, content-based filtering is derived from the content of the item itself. For example, if you are buying a piece of clothing, the content of the clothing would be the color, material, and designer. Pandora uses this same approach, albeit slightly more abstractly, for music: the streaming service classifies the “content” of each song using over 450 attributes. A content-based recommendation engine will generate preferences by combining the profiles of items in a user’s history and measuring the distance between a user’s profile and the items your organization would like to suggest. This approach may be the right choice if your organization is still in an early stage, since item profiles do not rely on historical data, and accurate recommendation engines can still be formed without a highly active user base. But if you already have a strong base and want to target your prospective customers with nuanced recommendations that factor additional variables and inconsistencies in the buyer’s journey, you may want to consider collaborative-based filtering.

 
Collaborative-Based Filtering

A collaborative filtering model leverages similarity between your users to make suggestions  rather than focusing on concrete details about a particular item. Typically these systems use rating or purchase history to compare users to each other. The distance between pairs of users is calculated, and a particular user is matched to others that are “close” to him or her (i.e. have similar tastes). For a clothing company employing this model, a user would be shown a Balmain jacket if their transaction history was similar to another user who went on to purchase that jacket. In contrast, with a content-based system, that recommendation would have been made based solely on that user’s history, without considering the behavior of others. Collaborative filtering models are more effective in curating effective recommendations for people with seemingly idiosyncratic preferences, so it can result in a more diverse set of suggestions. Keep in mind, though, that the additional ambiguity that this engine allows also introduces increased variability, and less interpretable results. Discrepancies in ratings (ie: some people are much tougher critics than others) will need to be addressed as well.

 
Measuring the Impact of Your Engine

Regardless of the filtering system you use, your organization will be employing a recommendation engine with the end goals of improving user experience, reducing churn, and boosting revenue. To ensure you are meeting those goals, you’ll want to focus on asking the right questions. These include:

  • Are recommendations fine-tuned for users’ needs, or are they too broad?
  • Are users modifying their behavior in response to the information presented?
  • Do users like getting these suggestions and find them helpful?

Building a recommendation model will be a big investment for your organization, both financially and technically. But if employed mindfully with the bottom line in mind, it can catapult your business to a whole new level.

Read the full report from Dr. Tuck Ngun here.