As I was sorting my contacts into neat little Google+ circles (talking to self: ‘this one, ah… he’s someone I could talk to about personal things, but I never do, oh, what the heck, let’s call him a “friend”‘), I was reminded that I never blogged about the related paper, soon to be poster-presented at this year’s ICWSM, that I wrote with Debra Lauterbach, Edwin Teng, and Mark Ackerman. Incidentally, Debra is currently a UX researcher at Google and no doubt lent a hand and some brains to Google+.
So, about this friend-rating paper. Previously, Edwin, Debra, and I had written a paper “I rate you. You rate me. Should we do so publicly?”, showing that person-to-person ratings differ by whether they are given publicly (anonymously or not) or privately. In this follow-up paper, we dove in depth into CouchSurfing’s friendship and trust ratings, both analyzing the millions of ratings, conducting a survey, and also Skype interviews. What were we after? Well, initially we just wanted to understand whether the higher alignment of public ratings A->B and B->A, as well as the lack of negative public feedback, were due to reciprocity (or fear of reciprocal action). But the study led us to understand quite a bit more about the nature of trust and friendship. This rather unique data set captures quantitative ratings of trust and friendship on a very large scale. Our results may very well be relevant to products such as Google+ because trust and friendship are sometimes used synonymously (e.g. you can “trust” your friends with intimate details you post on social media sites), but we thought we’d check that.
OK, to cut to the chase. Recap of the “I rate you. You rate me.” paper.
1) The same person will give higher Epinions ratings to other users’ reviews when identifying herself (as opposed to when she chooses to remain anonymous). But this is only true for rating other people. Product reviews on Amazon are no more favorable (though a bit longer) if a person identifies themselves than if they use a pen name.
2) When ratings are shown publicly (Epinions, CouchSurfing), and there is potential for reciprocity, the ratings are more aligned (i.e. A’s rating of B tends to mirror B’s rating of A).
3) On CouchSurfing, women rate other women more highly than they do men (on both trust and friendship), but men rate men and women about equally.
OK, so here is what we learned in the in-depth CouchSurfing paper.
4) Trust and friendship are not always synonymous. The heatmap below shows how often trust/friendship pair ratings are given. What you can see is that trust tends to increase with the closeness of the friendship (e.g. best friends are almost always highly trusted), but high trust can be allocated to individuals who are not one’s closest friends (of course all of this might be context specific to CouchSurfing, but I do believe it holds more generally).
4) When ratings are not aligned (e.g. A indicates B is a good friend, but B categorizes A as an acquaintance), there is a bit of awkwardness (for both A and B), but most do not attach much importance to numerical ratings. Friendship ratings (which are shown publicly) are more aligned (ρ ~ 0.7) than trust ratings (ρ ~ 0.3), which are not shown to the other person.
Connecting this to Facebook lists and Google+ circles, I’d say they are private for a reason. Though I wonder if A could eventually figure out that they are in B’s acquaintance as opposed to friend circle once they hear from C about something that B shared with his friends. Or is there enough content flooding our way that we don’t mind what we’re missing?
5) Negative ratings are seldom given publicly in part because the individual being rated can reciprocate. Also there is the sense that even if one doesn’t have a good experience with someone, there is a chance that someone else would click well, so why ruin the chance of that happening. As with any categorical system that doesn’t quite capture what individuals want to express, CS users work around it: they insert comments into the text of the reference that have subtle signals, e.g.
I’ve gone pretty keen on what certain references mean, and you can tell a. . . you-were-a-nice-person-reference: ”[She] was great. She was very hospitable. She’s a great host.” That can mean in a sense you might be kind of boring.
Or users simply don’t leave a reference:
I either neglect to reference, or write a “positive” response but in a neutral tone.
6) This makes textual references (and their number) far more important than numerical levels to those who are judging whether they would like to get to know/spend time with someone based on their ratings:
6) We also checked whether trust is more a property of a node, and friendship that of an edge, e.g. a given individual would always be perceived as highly trustworthy, but their friendship ratings would depend on the characteristics of each tie. We found only very, very weak support for this, in that the per-individual normalized variance in trust ratings was only a bit smaller than the variance in friendship ratings.
7) Some other neat things, you could find out if you read in the paper 🙂
8 ) Something we couldn’t fit into the paper was the users’ tendency to not update friendship and trust status as the relationship evolved. This could be a function of CS’s primary purpose being to help people find one another as opposed to stay in touch, so that there is little practical utility in making such updates. On the other hand, one might rather carefully groom one’s Google+ circles and Facebook lists, lest one ends up inadvertently over or under sharing because of out-of-date designations. It will be interesting to see to what extent users are interested in and capable of thinking through what each individual means to them and where they “fit” in their information sharing sphere.