11 Jan ”We think you might like this”
Our thirst for online shopping and engagement with multi-media streaming sites seems insatiable. The success of the likes of Amazon, Netflix, Spotify, and YouTube, demonstrates this major growth area, and Covid-19 has served to accelerate our consumer behaviour from offline to online. It seems ironic, that whilst our offline interactions have become ever more de-personalised through social distancing requirements, limited personal interactions and the wearing of face masks, it is our online interactions which have become increasingly more personalised. The more we consume and interact online, the more data we create. This data is the currency of the digital world. Recommender systems leverage this data to tailor the content of websites in real time to match individual predicted user preferences and needs.
But, do recommender systems influence our actual behaviour? Do they enrich our lives, or simply reinforce our own behaviour? And, how does this technology benefit organisations, customers and society?
These were just some of the questions raised in Dr. Mark Graus’s Recommender session, delivered as part of the Digital Strategy module for the MaastrichtMBA programme in November. Mark is an Assistant Professor on Data Science in Marketing at Maastricht University. His background in Human-Technology Interaction combines machine learning with fundamental psychological theory. It is this combination of expertise, which enabled a dynamic and nuanced debate on the recommender session, and yes, it got personal!
One size does not fit all
A personalised online experience is the antithesis to a one size fits all customer approach. For a successful personalised user experience, businesses need to adapt what they show people based on what they know about them. When online businesses do this well, it can generate significant income streams and create competitive advantage. All recommender systems are personalised systems and the algorithms used to determine this feedback fall into two main categories: content-based filtering and collaborative-based filtering.
Content-based filtering infers a user’s preference from product attributes of items they like. So, for example, when you buy a blue shirt on Esprit’s website, you might be asked if you would also like to look at several other similar types of shirt. The feedback used is explicit, attempting to predict relevance to the consumer in terms of attributes. This type of filtering is very transparent, easy to understand and straightforward to implement. However, the notion is that it is not very reliable, mainly because what people say, isn’t necessarily what they do. A psychological human quirk, which can befuddle an algorithm based solely on content-based feedback.
Collaborative-based filtering, on the other hand, infers a user’s preference from users like you. It matches products to what other people, similar to you, have purchased in the past and then recommends certain products. Unlike content-based filtering, it can be used without any metadata, just requiring user item interactions. It is allegedly a more reliable system, but more complicated to implement. Mark suggests that in order for businesses to achieve success with these systems of feedback, they should practise a combination of both content and collaborative-based filtering.
Assessing the system
Evaluating how well the system is working is a crucial step in the personalised digital process. Taking the right action or making adjustments as a result of this, can have significant and immediate impact on the success of the system. Mark outlined three main approaches to assess how well the system is doing. These are online, offline, and user studies. Within each of these systems, there is a varying level of difficulty, reliability and depth of insight. Mark cited key pointers from Ron Kohavi’s practical guide as a useful source when doing controlled experiments on the web. The key takeaway from Mark’s insights on evaluating the recommender systems was to always define your objectives as this determines what type of evaluation to do.
The three key challenges with recommender systems are around the technical, legal and ethical issues. From a technical perspective, big data, cold start problems and predictive modelling, were cited as the main challenges. In essence, the more data we have, the better the predictions will be. Mark acknowledged that 80% of the problems with recommender systems are what is known in industry speak as a ‘cold start problem’, meaning when you have a new system, a new user or new item, you don’t have any or enough data to infer a user’s preferences to make recommendations and so it’s difficult to get started from scratch.
The legal challenges revolve around the more thorny issue of consent and transparency. Both aspects have important consequences for personalisation. The data protection regulation (GDPR), gives consumers ownership of their data and control of how that data can be used. The consequence is that there is less data available, so a smarter use of data is needed. Mark suggests a more nuanced approach could be taken by companies to demonstrate both the advantages and costs of privacy to the consumer.
The filter bubble: a question of ethics
Are we just living in our own bubble? Recommender systems serve to tailor our online experiences to better match our interests. They exist to give us a better service, and so increase revenue for the company, but the algorithms used to personalise our experience can also reinforce our behaviour, creating what is known as a ‘negative feedback loop’ or ‘filter bubble’. The ethical issues raised by this are hard to ignore. Mark uses his experience of watching cycling workout videos on YouTube to illustrate this. YouTube infers he likes watching videos on cycling and so shows him more cycling workout videos, which results in him watching more cycling workout videos. Sound familiar? Our observable behaviour creates a certain reaction, which then reinforces that behaviour. Much like the echo chamber idea where ideas and beliefs are repeated and shared within a closed system. The result limits our experiences, interactions and perspectives. What impact does this have on our world view and is this what we want from a personalised service?
This may seem a far cry from whether we buy that blue shirt recommended to us on Esprit’s website, but the ethical challenges of personalised systems illicit a philosophical reaction. Mark acknowledges that one of the main limitations of personalised systems is the focus on the ‘observable’ impact on behaviour, e.g. do customers purchase more, whereas the inclusion of a more user-centric model incorporating user psychology and user feedback would allow for a broader, more balanced personalised model. Perhaps this would allow recommender systems to enrich people’s lives as well as deliver on the promise of a more personalised service. Mark believes this could be the case; let’s hope so, after all, his expertise comes highly recommended.