From nobody@digitalkingdom.org Tue Jan 10 07:15:16 2006 Received: with ECARTIS (v1.0.0; list lojban-list); Tue, 10 Jan 2006 07:15:17 -0800 (PST) Received: from nobody by chain.digitalkingdom.org with local (Exim 4.60) (envelope-from ) id 1EwLD5-0003vt-CK for lojban-list-real@lojban.org; Tue, 10 Jan 2006 07:14:55 -0800 Received: from snowdon.metacarta.com ([65.77.47.18] helo=metacarta.com) by chain.digitalkingdom.org with esmtps (TLS-1.0:DHE_RSA_3DES_EDE_CBC_SHA1:24) (Exim 4.60) (envelope-from ) id 1EwLD1-0003vk-1d for lojban-list@lojban.org; Tue, 10 Jan 2006 07:14:54 -0800 Received: from localhost (silene.metacarta.com [65.77.47.24]) by metacarta.com (Postfix) with ESMTP id 16620518191 for ; Tue, 10 Jan 2006 10:14:46 -0500 (EST) Received: from metacarta.com ([65.77.47.18]) by localhost (silene.metacarta.com [65.77.47.24]) (amavisd-new, port 10024) with ESMTP id 25395-03 for ; Tue, 10 Jan 2006 10:14:41 -0500 (EST) Received: from [65.77.47.138] (baxter.metacarta.com [65.77.47.138]) by metacarta.com (Postfix) with ESMTP id E207C51817E for ; Tue, 10 Jan 2006 10:14:40 -0500 (EST) Message-ID: <43C3CF5F.5050105@ropine.com> Date: Tue, 10 Jan 2006 10:14:39 -0500 From: Seth Gordon User-Agent: Mozilla Thunderbird 0.6 (X11/20040530) X-Accept-Language: en-us, en MIME-Version: 1.0 To: lojban-list@lojban.org Subject: [lojban] Re: A Lojban Game References: <19310326537.20060110120948@mail.ru> In-Reply-To: <19310326537.20060110120948@mail.ru> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at metacarta.com X-Spam-Score: -2.6 (--) X-archive-position: 11031 X-ecartis-version: Ecartis v1.0.0 Sender: lojban-list-bounce@lojban.org Errors-to: lojban-list-bounce@lojban.org X-original-sender: sethg@ropine.com Precedence: bulk Reply-to: lojban-list@lojban.org X-list: lojban-list > 3. Ranking of players is performed by summing values of translations > made by this player. One of these weeks I will finally be done with my rant about Why Every Community Rating System On The Web Is Fundamentally Flawed, but in the meantime, this will have to suffice: In statistical jargon, ratings are an ordinal measurement, not an interval or ratio measurement. You know that a four-star movie is better (in the reviewer's opinion) than a three-star movie, and a three-star movie is better than a two-star movie, but the stars convey no information about the _magnitude_ of the differences among the three movies. By comparison, temperature is an interval measurement: if it's 15 degrees Celsius in Boston, 16 degrees in New York, and 30 degrees in Miami, you know that the New York-Miami distinction is far more important than the Boston-New York distinction. So treating the ratings of different evaluators as some kind of virtual currency that can be accumulated is wrong. The proper way to aggregate rankings is by taking their median. If we're aggregating the ratings of translations made by a players in this game, someone who has received the set of ratings {average, average, good, good, good, good, very-good, very-good} is obviously more likely to be a good translator than someone who has only received the rating set {good}, even though the two sets of ratings have the same median. There's a statistical test for distinguishing these cases, but my copy of _Practical Nonparametric Statistics_ is at home so I forget what it's called. To unsubscribe from this list, send mail to lojban-list-request@lojban.org with the subject unsubscribe, or go to http://www.lojban.org/lsg2/, or if you're really stuck, send mail to secretary@lojban.org for help.