Rating Friends Without Making Enemies

 papers  Comments Off on Rating Friends Without Making Enemies
Jun 302011

As I was sorting my contacts into neat little Google+ circles (talking to self: ‘this one, ah… he’s someone I could talk to about personal things, but I never do, oh, what the heck, let’s call him a “friend”‘), I was reminded that I never blogged about the related paper, soon to be poster-presented at this year’s ICWSM, that I wrote with Debra Lauterbach, Edwin Teng, and Mark Ackerman. Incidentally, Debra is currently a UX researcher at Google and no doubt lent a hand and some brains to Google+.

So, about this friend-rating paper. Previously, Edwin, Debra, and I had written a paper “I rate you. You rate me. Should we do so publicly?”, showing that person-to-person ratings differ by whether they are given publicly (anonymously or not) or privately. In this follow-up paper, we dove in depth into CouchSurfing’s friendship and trust ratings, both analyzing the millions of ratings, conducting a survey, and also Skype interviews. What were we after? Well, initially we just wanted to understand whether the higher alignment of public ratings A->B and B->A, as well as the lack of negative public feedback, were due to reciprocity (or fear of reciprocal action). But the study led us to understand quite a bit more about the nature of trust and friendship.  This rather unique data set captures quantitative ratings of trust and friendship on a very large scale. Our results may very well be relevant to products such as Google+ because trust and friendship are sometimes used synonymously (e.g. you can “trust” your friends with intimate details you post on social media sites), but we thought we’d check that.

OK, to cut to the chase. Recap of the “I rate you. You rate me.” paper.

1) The same person will give higher Epinions ratings to other users’ reviews when identifying herself (as opposed to when she chooses to remain anonymous). But this is only true for rating other people. Product reviews on Amazon are no more favorable (though a bit longer) if a person identifies themselves than if they use a pen name.

2) When ratings are shown publicly (Epinions, CouchSurfing), and there is potential for reciprocity, the ratings are more aligned (i.e. A’s rating of B tends to mirror B’s rating of A).

3) On CouchSurfing, women rate other women more highly than they do men (on both trust and friendship), but men rate men and women about equally.

OK, so here is what we learned in the in-depth CouchSurfing paper.

4) Trust and friendship are not always synonymous. The heatmap below shows how often trust/friendship pair ratings are given. What you can see is that trust tends to increase with the closeness of the friendship (e.g. best friends are almost always highly trusted), but high trust can be allocated to individuals who are not one’s closest friends (of course all of this might be context specific to CouchSurfing, but I do believe it holds more generally).

friendship and trust ratings on CouchSurfing

4) When ratings are not aligned (e.g. A indicates B is a good friend, but B categorizes A as an acquaintance), there is a bit of awkwardness (for both A and B), but most do not attach much importance to numerical ratings. Friendship ratings (which are shown publicly) are more aligned (ρ ~ 0.7) than trust ratings (ρ ~ 0.3), which are not shown to the other person.
Connecting this to Facebook lists and Google+ circles, I’d say they are private for a reason. Though I wonder if A could eventually figure out that they are in B’s acquaintance as opposed to friend circle once they hear from C about something that B shared with his friends. Or is there enough content flooding our way that we don’t mind what we’re missing?

5) Negative ratings are seldom given publicly in part because the individual being rated can reciprocate. Also there is the sense that even if one doesn’t have a good experience with someone, there is a chance that someone else would click well, so why ruin the chance of that happening. As with any categorical system that doesn’t quite capture what individuals want to express, CS users work around it: they insert comments into the text of the reference that have subtle signals, e.g.

I’ve gone pretty keen on what certain references mean, and you can tell a. . . you-were-a-nice-person-reference: ”[She] was great. She was very hospitable. She’s a great host.” That can mean in a sense you might be kind of boring.

Or users simply don’t leave a reference:

I either neglect to reference, or write a “positive” response but in a neutral tone.

6) This makes textual references (and their number) far more important than numerical levels to those who are judging whether they would like to get to know/spend time with someone based on their ratings:

How important CouchSurfing ratings are to users

6) We also checked whether trust is more a property of a node, and friendship that of an edge, e.g. a given individual would always be perceived as highly trustworthy, but their friendship ratings would depend on the characteristics of each tie. We found only very, very weak support for this, in that the per-individual normalized variance in trust ratings was only a bit smaller than the variance in friendship ratings.

7) Some other neat things, you could find out if you read in the paper 🙂

8 ) Something we couldn’t fit into the paper was the users’ tendency to not update friendship and trust status as the relationship evolved. This could be a function of CS’s primary purpose being to help people find one another as opposed to stay in touch, so that there is little practical utility in making such updates. On the other hand, one might rather carefully groom one’s Google+ circles and Facebook lists, lest one ends up inadvertently over or under sharing because of out-of-date designations.  It will be interesting to see to what extent users are interested in and capable of thinking through what each individual means to them and where they “fit” in their information sharing sphere.


Sir David Attenborough and Wandering Albatross Chick

Photo © Ben Osborne.

After having watched many hours of nature shows, seemingly most of them narrated by Sir David Attenborough, I find myself haunted not by their incredible natural beauty, but by scenes whose tragedy was accentuated by Attenborough’s calm commentary. For example, I can’t look at a coot without recalling how a mother intentionally kills one of her chicks in “Life of Birds”.  When I see a polar bear I recall a bear hunting a baby seal. The narrative goes something like “If the mother cannot catch this seal, her two hungry cubs will starve to death”. Then she doesn’t catch the seal. A flock of flamingos or flamingo-like birds is flying up from a muddy lake. Except a few of them can’t lift off, their legs caked in mud. The longer the birds flail about, the larger the weights become. A mother lemur (or some other kind of swinging-in-the-trees primate) is moving in the forest with her young who are performing amazing acrobatics while jumping from branch to branch. Then one of the youngsters falls and lies injured. The mother calls to him, and he cries back, but after some time, she continues on, with his cries growing fainter with distance. It makes me wonder if the cameraman ever sneaks back to free the flamingos from their shackles, or throw a steak to the polar bear, or put a cast on the little one’s broken leg. And you, Sir David Attenborough, do you shed a tear when you’re editing the material later on? Or has nature hardened you long ago?

A couple of years ago, I played a DVD (I forget which one exactly), a present. I watched as a young elephant became separated from his herd in some extremely arid part of the planet. As the camera zooms out, you get an aerial view of the little elephant running all alone in the wrong direction, and you hear “He has no chance. Without the herd guiding him to water, he will die of thirst”.  I turned off the TV and ejected the DVD. For a few years I watched no more nature documentaries.

But with new incentives to watch nature shows,  I’ve been searching for ones that may soften the realities of the natural world a bit. And so I came to watch the strangely-titled 1974 film “Animals are beautiful people” (by the time I reached the scene with animals getting drunk off of fermented fruit, I realized I’d seen it before decades ago). In one of the scenes, a piglet is chased by a cheetah. It manages to get away by finding a hiding hole. But it’s separated from its family. However, the next day its search is fruitful and it is reunited with mom. Which is unlike any of the outcomes I’ve seen in other nature documentaries. Must be fiction. But I’d really rather remember it, and forget the coots.


This is just a hypothesis: gender differences in risk taking contribute to differences in participation in science and engineering disciplines. I have no data, just a wish to speculate from personal experience.

Individual tasks in math, science and engineering are risky. One might not find a proof to a theorem, an individual experiment or design will likely not work, etc. In these tasks, one risks failure even after investing considerable effort, i.e. the return on effort is not guaranteed. It turns out that the gender gap in intellectual risk taking is well known. I also wonder if the macho environment that is sometimes associated with these disciplines is not so much a function of their being male dominated, but rather that bullishness goes along with taking risks. I remember well a fellow Caltech undergraduate boasting about spending all night trying to solve a problem in his general relativity class that the instructor later revealed was an unsolved problem. If it’s a choice between boasting or despairing quietly, boasting seems like the option that can keep one going.

In the long run, however, it seems to be the long run that matters. Could it be a coincidence that so many of the women in math, science and engineering I know happen to regularly run marathons or triathlons? And that some of the men who used to boast about solving hard problems are now managing risk algorithmically for financial and IT firms (OK, I admit, I don’t know what the latter signifies)? Perhaps the biggest risk is in not pursuing a science  or engineering education, which is sure (ahem) to be interesting and which is in constant demand by employers. And if one can boost young women’s willingness to apply themselves to problems where they risk not being able to figure out an answer, by e.g. building their overall confidence, then by all means, let’s.


Today, I really enjoyed a performance given by Uri Alon. He lectured on the importance of nurturing young scientists, but he also sang. He sang “Sundays at the lab”, “Dear Author” (an editor’s letter and referee reports), “Scooped” (what you’d imagine), and my favorite, a tribute to Mike, a mentor in his past, to the tune of Leonard Cohen’s Suzanne. Every song touched my heart, though they were not about the heart, or at least not about heartbreak, ok maybe the “dear author” and “scooped” songs were about heartbreak… But you know what I mean. Most of what you hear on the radio is about falling in love or   getting one’s heart broken. And while that was certainly part of my life, oh, well over a decade ago,  the only song that is “current” with my love life is  Shania Twain’s “Still the one”.

Why is music so preoccupied with romance? Is romance the one thing all of us, including musicians, have experience with? I do have my weakness for certain love ballads, but I also really enjoy the old folksongs, poems, war and peace songs sung by die Zupfgeigenhansel, songs about kitties and peaches,  the color blue, and sabotage. I wonder if the digital distribution of music will allow for a greater diversity of song subjects, or if we will continue to listen to musicians’ experiences with relationship drama and partying, just as we have an abundance of novels and movies whose protagonists are (screen)writers.

© 2011 ladamic's blog Suffusion theme by Sayontan Sinha