From mark@kli.org Thu Aug 27 10:06:12 2009 Received: with ECARTIS (v1.0.0; list llg-members); Thu, 27 Aug 2009 10:06:12 -0700 (PDT) Received: from pi.meson.org ([66.134.26.207]) by chain.digitalkingdom.org with esmtp (Exim 4.69) (envelope-from ) id 1MgiQK-0001t9-7n for llg-members@lojban.org; Thu, 27 Aug 2009 10:06:11 -0700 Received: (qmail 21021 invoked from network); 27 Aug 2009 13:06:01 -0400 Received: from nagas.meson.org (192.168.1.101) by pi.meson.org with SMTP; 27 Aug 2009 13:06:01 -0400 Message-ID: <4A96BCF9.1020305@kli.org> Date: Thu, 27 Aug 2009 13:06:01 -0400 From: "Mark E. Shoulson" User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: llg-members@lojban.org Subject: [llg-members] Re: Motion: Lojban Certification Program References: <925d17560908261436r7364022dn735a98e5851cc7f9@mail.gmail.com> <200908270711.14068.phma@phma.optus.nu> In-Reply-To: <200908270711.14068.phma@phma.optus.nu> Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-archive-position: 758 X-ecartis-version: Ecartis v1.0.0 Sender: llg-members-bounce@lojban.org Errors-to: llg-members-bounce@lojban.org X-original-sender: mark@kli.org Precedence: bulk Reply-to: llg-members@lojban.org X-list: llg-members Pierre Abbat wrote: > On Wednesday 26 August 2009 17:36:00 Jorge Llambías wrote: > >> On 8/26/09, Matt Arnold wrote: >> So as a friendly amendment I suggest that the tests should not be >> mainly or even at all about translation. >> > > I second the amendment. > > I got a reply from d'Armond Speers about the Klingon tests. They are entirely > written, and there are three levels. The questions are held in a database, > and questions are picked at random when a test is generated for someone. > Several of us know how to write databases with web or other front ends. Everyone else, take a look at http://higbee.cots.net/Holtej//klingon/KLCP/index.htm if you haven't already, for info on the KLI program. I think a few points of Eppcott's proposal need tweaking (this was simpler in the KLI, because it was all done by one person. Design by committee is going to affect things negatively, I fear). I think a written-only test is likely best; it's easier to administer and to archive the results (and there will be fewer arguments about misunderstandings, mis-hearings, etc). A translation-based test leaves a lot of room for uncertainty about how good or bad a translation is. Questions (or at least most of them) should be more closed-ended, mostly multiple-choice and fill-in-the-blanks. Maybe a few with sentences with subtle weirdnesses in the grammar, where a sumti might or might not be in the "obvious" place, and the questions ask about the details, etc. Eppcott's idea of "merit badges" for different selma'o or constructions, etc, sounds too complicated to be workable, to me. It would involve *lots* of effort in designing the tests, and probably more interest than we're likely to have for a while. Might be fun as an adjunct to the formal testing program, but I think it's too much to plan out to start. Similarly, seven levels might be too many. The KLI has three; really four but nobody has ever designed the test for the fourth. http://higbee.cots.net/Holtej//klingon/KLCP/guidelines.htm gives details on what needs to be known for each level and also the structure of the tests (absolutely read this if you haven't already). We will need to produce a similar document so people will know what they have to know. The lowest level can't be "free prize in every box": it should take non-trivial effort to get to the lowest level too. Check the guidelines document: for taghwI' rank, you have to know 500 vocabulary words, most if not all of the verb and noun suffixes, nine most common verb prefixes, pronouns, exclamations, some question-words, and other stuff. It may not be feasible for us really to test past the current levels of experience. I think we should design maybe three or four levels based on vocabulary and grammar expertise (the higher levels starting to hit weird places like mekso), maybe coming close to the level of top speakers today. But for higher levels, the expertise starts to be more a matter of style and speed and expressiveness, something that is very hard to measure (which is why we never made a Grammarian-level test for Klingon). Maybe the names of the ranks should reflect that there's more to go, but we can defer designing the higher levels until we HAVE more people at those higher levels and we can understand what they mean. In-person tests are necessary to guard against use of reference materials, etc, but are less practicable with Lojban than with Klingon (the KLI conference is better-attended and better-publicized than Logfest, and more of an official part of the KLI). Maybe we can work something out with an honors system, perhaps with a witness/proctor also testifying (it's slightly harder to get someone else to lie for you). Also doesn't have to be at Logfest, just if you can get together with A Trusted Lojbanist In Your Time Zone. Hmm. Well, needs work. There should probably be a Certification Committee to work the details rather than doing it all in the meeting. ~mark