[lojban] Re: A Lojban Game

[Seth Gordon <lojban-out@lojban.org>]

> 3. Ranking of players is performed by summing values of translations
> made by this player.

One of these weeks I will finally be done with my rant about Why Every 
Community Rating System On The Web Is Fundamentally Flawed, but in the 
meantime, this will have to suffice:

In statistical jargon, ratings are an ordinal measurement, not an 
interval or ratio measurement.  You know that a four-star movie is 
better (in the reviewer's opinion) than a three-star movie, and a 
three-star movie is better than a two-star movie, but the stars convey 
no information about the _magnitude_ of the differences among the three 
movies.  By comparison, temperature is an interval measurement: if it's 
15 degrees Celsius in Boston, 16 degrees in New York, and 30 degrees in 
Miami, you know that the New York-Miami distinction is far more 
important than the Boston-New York distinction.

So treating the ratings of different evaluators as some kind of virtual 
currency that can be accumulated is wrong.  The proper way to aggregate 
rankings is by taking their median.

If we're aggregating the ratings of translations made by a players in 
this game, someone who has received the set of ratings {average, 
average, good, good, good, good, very-good, very-good} is obviously more 
likely to be a good translator than someone who has only received the 
rating set {good}, even though the two sets of ratings have the same 
median.  There's a statistical test for distinguishing these cases, but 
my copy of _Practical Nonparametric Statistics_ is at home so I forget 
what it's called.


To unsubscribe from this list, send mail to lojban-list-request@lojban.org
with the subject unsubscribe, or go to http://www.lojban.org/lsg2/, or if
you're really stuck, send mail to secretary@lojban.org for help.