Personalization and Content-based Recommender Systems

Personalization is a big trend today. There is so much information available that we need to find new ways to filter, categorize and display data that is relevant.

Recommender systems guide users in a personalized way to interesting objects in a large set of possible options.

Content-based systems try to recommend items similar to those a given user has liked in the past.

The basic process performed by a content-based recommender systems consists of matching up the attributes of a user profile in which preferences and interests are stored, with those of an object. These attributes have been previously collected and is subjected to analysis and modelling with the intent to arrive at a relevant result.

The recommendation process is performed in 3 steps:

  1. The Content Analyzer: When information has no structure, it is the Content Analyzer’s role to provide the structure necessary for the next processing steps. Data items are analyzed by feature extraction techniques to shift item representation from the original information to the target one. This representation is the input for the next 2 steps.
  2. The Profile Learner: This module collects data from the Content Analyzer and tries to generalize it, building a user profile. The generalization strategy is usually performed using machine learning techniques, which are able to infer a model of user interests.
  3. The Filtering Component: This module uses the user profile to suggest relevant items by matching the profile representation to the items being recommended.

The process begins with the “Content Analyzer” extracting features (keywords, concepts, etc.) to construct an item representation. A profile is created and updated for the active user and reactions to the items collected in some way and stored in a repository. These reactions, called feedback or annotations, in combination with the related item description are exploited during the learning of a model to predict the relevance of a newly presented item. Users can also provide initial information to build a profile without the need of feedback.

Generally feedback can be positive and negative and two types of techniques can be used to determine that feedback; implicit and explicit.

Explicit feedback can be obtained by gathering likes/dislikes, ratings and comments, while implicit feedback is derived form monitoring and analyzing the user’s activities.

The “Profile Learner” generates a predictive model utilizing supervised learning algorithms and then stored to be later used by the “Filtering Component”. Users tastes are likely to change over time, so its important to keep information up-to-date to feed back into the “Profile Learner”.

Amongst the advantages of the Content-based recommendation systems are:

  • User independence since recommendations are based solely on the users ratings
  • Transparency since how the systems works in making a particular recommendation can be described in function of content and descriptions; and
  • New item which is capable of being recommended that has not yet been rated by a user.

Content-based recommendation systems also have disadvantages:

  • Limited Content: There is a natural limit in the number and type of features that can be associated with the objects they recommend, therefore the information collected might not be sufficient to define a particular user’s interests.
  • Over-Specialization: Content-based recommendation systems have no way to recommend something unexpected. The system is limited to ranking a number of items based on score and matching them to the user’s profile, solely based on similarities to items that he has already provided positive feedback on. This drawback is also known as “serendipity” problem, showing the tendency of the system to limit its degree of novelty.



E-commerce and The End of Search

Most of us consider the Internet a bucket of miscellaneous tidbits, and the modern search engine our personal assistant. But is that analogy correct? You open your browser, bringing up the Google homepage, then enter whatever term you happen to be looking for at the time and bingo. You get a list of results you then have to “search” through to find what you are looking for. So in fact you are searching through the results of what Google searched for.

Google co-founder Larry Page once described the “perfect search engine” as something that “understands exactly what you mean and gives you back exactly what you want”, far from what Google is today.


A recent study titled “Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips” by researchers at Columbia, Harvard and Wisconsin-Madison universities studied whether the Internet has become our primary transitive memory source–basically an external memory system. These are the conclusions reached by the four controlled experiments in the study:

1) People share information easily because they rapidly think of computers when they find they need knowledge (Expt. 1).

2) The social form of information storage is also reflected in the findings that people forget items they think will be available externally, and remember items they think will not be available (Expts. 2 and 3).

3) Transactive memory is also evident when people seem better able to remember which computer folder an item has been stored in than the identity of the item itself (Expt. 4).

The effect on whether or not we choose to commit certain information to memory when we know the information is readily available on the computer is what is relevant here. We store specific things in specific places, like food in the fridge, but who remembers what is specifically in the fridge?

It is completely natural for people to minimize what needs to be encoded into memory by organizing and then encoding the location of the information, rather than the information itself. This is where the traditional search engine falls short of meeting the basic cognitive needs of humans.

The emergence of the mobile device has been remarkable and Apple’s vision in this space has changed the way people access information. There is data to support the notion that people are not mirroring desktop behavior on mobile devices.

People are not searching on smartphones as much as they do on desktops. Steve Jobs attributes this to the availability of mobile apps and the desktop lacking an app store. In reality, the availability of app, or the lack thereof, is not really the central point. What’s important is information is being categorized, compartmentalized and organized for consumption, and delivered more efficiently through mobile devices. This is clearly a step in the right direction in delivering more relevant and timely information to the user.

Artificial Intelligence will play a major role in the next wave of innovation, starting with Evolving and Adaptive Fuzzy Systems as classification algorithms and then matching the wants with the needs of the user. A recent example of this is an application that gives personalized restaurant recommendations called Alfred—it is all recommendations and no direct search.

GiftWoo takes the next step forward in the e-commerce space in a vertical market. Until now going online to find a gift for your better half involves a search, which results in thousands of choices. Currently, e-commerce websites are designed to deliver a high number of choices, rather than the “right choice” for the consumer. GiftWoo will give the buyer the unique and perfect gift they seek without the searching, by initially building a profile for the gift recipient, then utilizing a proprietary algorithm to match the ideal gift to the profile.