Recommendations & Collaborative Filtering

Data-based recommendations have really revolutionized marketing and web services, making patterns out of the massive amount of information collected about people in order to give them relevant ads, products, friends and music as well as whole host of other things.

Amazon, for example, tracks my browsing history and buying habits to give me a list of products that I’d hopefully be interested in, and usually their algorithm is spot on. I’ve no doubt that recommendations have contributed greatly to their success.

Likewise, Last.fm indexes my music collection and tracks what I listen to in order to give me recommendations about music I haven’t listened to. Like Amazon, they’re usually give pretty good recommendations.

Delicious is an example of a site that doesn’t make the most of the data it collects. With the amount of sites I’ve bookmarked with their service I’m sure they know what I’m interested in and give me relevant recommendations. Perhaps the data processing power needed is what’s holding them back?

ReadWriteWeb has got a good article on Collaborative Filtering and makes an important point about The Wisdom of Crowds which suggest that “as communities grow, not only does a large (diverse, independent, etc.) community make better decisions than a handful of editors, but the larger a community gets, the better its decisions will be”.

Once you scale past more than a dozen users it soon becomes unpractical to make manual recommendations to people, and computers are pretty good making them themselves. A machine doesn’t need to be ’self aware’ or to actual listen to the music to know what you like. No, what actually usually happens is much more low level – users are grouped based on their listening habits (or whatever else it happens to be), and then users are suggested music based on what other people in their group are listening to.

The fact that it’s so level, means that for a lot of recommendations you don’t need data specific algorithms or code, it’s enough to have a relationship between two entities to get recommendations.

At Made by Many I’ve been working on a plugin for Rails called acts_as_recommendable that makes adding recommendations to your Rails sites a piece of cake! If you have a relationship between books and users, for example, acts_as_recommendable will show you which users are similar and which books a user would probably like to read.

The code is based on the example from the book ‘Programming Collective Intelligence‘ which I really recommend getting. it reveals how collective intelligence can be used in a very practical way with a lot of helpful examples.