The collaborative system we created did not receive anywhere near enough use to be able to determine whether or not the type of collaborative filtering implemented by the system is effective at helping users find interesting articles. Even given that limitation, however, there are still conclusions we can draw from our experience designing and implementing a collaborative filtering system for Usenet Net News.
Our primary conclusion is that it is both possible and feasible to create a system that will efficiently transport some types of vote information from the users who create it to users who will use it to filter their Net News. Our design and implementation represent an existence proof of such a system that provides robustness, ease of integration with existing Net News clients and servers, and low overhead on the communications network. Relating to this primary conclusion are a number of secondary ones that follow from it.
Our first supporting conclusion is that there exist some groups of users, specifically the ones at sites like PARC, who are willing to take some amount of time and explicitly vote for and against the Net News articles they read. Although there was no tangible reward offered for doing so, over half the users of the nn news reader at PARC cast votes. This bodes well for the future success of collaborative filtering systems as it indicates that for whatever reason, users are willing to contribute their expertise to a system which advances a global good.
Our second supporting conclusions relates to the specific domain of Net News. Since a collaborative system necessarily involves transporting information from one user to another, meta-issues such as privacy and social conventions must be addressed by the system. This can be particularly challenging on Net News where many users exhibit strong convictions over what is and is not acceptable practice. Our resolution to these problems was to avoid making changes to the Net News system itself or the data it contains. To make users feel comfortable with disclosing information about which news articles they read, we provide a method of anonymous voting that still contributes significantly to the effectiveness of the collaborative filtering system.
Our third and final supporting conclusion was finding that as in many groupware systems, critical mass is the show stopper. Until the system is well in use, there are uncertain rewards for any user who participates. With a system such as ours, we can define critical mass as having been reached in a newsgroup when users derive benefit from the votes cast by other users, and in turn cast votes themselves. Our data indicates that our system at PARC never reached critical mass in any newsgroup.