Sender: lojban@googlegroups.com
Received-SPF: pass (google.com: domain of adamlopresto@gmail.com designates 2607:f8b0:400c:c08::236 as permitted sender) client-ip=2607:f8b0:400c:c08::236;
MIME-Version: 1.0
References: <b8da37b0-bc48-417f-ad27-6ba85424a312@googlegroups.com>
 <3c86d96b-e0ea-af6b-2ee8-51d4e0741fe5@gmail.com> <c239f7c9-39b7-4bf4-af60-b97c0df89603@googlegroups.com>
 <cdb14652-aa96-4fcf-9c42-7de68b70f22a@googlegroups.com> <CABM8VqdMfn8Do4SJR=OEVZVJdX+Lz5awCjkL4FrhMvP4t+xDEw@mail.gmail.com>
 <00784DD2-C6DC-45F9-9DDE-E2B64BD6A1CB@free.fr>
In-Reply-To: <00784DD2-C6DC-45F9-9DDE-E2B64BD6A1CB@free.fr>
From: Adam Lopresto <adamlopresto@gmail.com>
Date: Fri, 28 Jul 2017 15:51:44 +0000
Message-ID: <CABM8VqdQfRtZ2qbTH9H=WjQEzMWmjrfFBTN53uev3k+dVE3CKA@mail.gmail.com>
Subject: Re: [lojban] Spaces in jbovlaste
To: lojban@googlegroups.com
Content-Type: multipart/alternative; boundary="001a1144f3f811667e055562a9a4"
Reply-To: lojban@googlegroups.com
Precedence: list
Mailing-list: list lojban@googlegroups.com; contact lojban+owners@googlegroups.com
X-Spam_score: -2.0
X-Spam_score_int: -19
X-Spam_bar: --

--001a1144f3f811667e055562a9a4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

jbovlaste should already be filtered to contain only Lojban, and there are,
broadly, three types of Lojban words:
cmevla are everything that ends in a consonant
brivla all contain a consonant cluster and end in a vowel
cmavo optionally start with a single consonant, and consist entirely of
vowels and apostrophes after that.

So, I think you could filter all cmavo clusters by looking for anything
that matches /.+[^aeiou'].*[aeiou]/ but doesn't match /[^aeiou'][^aeiou']/.
Contains a non-vowel somewhere after the first letter, ends in a vowel, and
doesn't contain a consonant cluster.

At least, that seems like a good start.

On Fri, Jul 28, 2017 at 10:43 AM Sukender <sukender@free.fr> wrote:

> Tanks for the clarification. I didn't even imagine that this big random
> compound cmavo would be valid! You made my evening! ;-)
>
> About {lonu} entry, I clearly agree. But I can't filter all them out... O=
r
> can I? If you get any (simple) idea of rule for that, then go ahead!
>
> By the way, I already filtered out a few words. I indeed found some of
> huge length (even a weird one about Macarena!). As it may be spam, I adde=
d
> an arbitrary rule that throws away all that have more than 22 characters.
> Maybe a finer rule has to be found...
>
> Cheers,
>
>
> --
> Sukender
>
>
>
>
> Le 28 juillet 2017 17:33:14 CEST, Adam Lopresto <adamlopresto@gmail.com>
> a =C3=A9crit :
>>
>> If you're going to allow cmavo to be combined arbitrarily (which is
>> probably appropriate), then there's no reason for {lonu} to have its own
>> entry. So I'd suggest not adding any cmavo clusters.
>>
>> And {lonulonucalo} can be grammatical, you just need the right text afte=
r
>> it. {lonulonucalo nu jamna kei mi damba cu nandu mi cu se zungi mi}, "I
>> feel guilty that it was hard for me to fight during the war." As you sai=
d,
>> a fully grammar checker would be needed to really get things right, and
>> that's a separate problem.
>>
>> On Fri, Jul 28, 2017 at 6:54 AM <sukender1@gmail.com> wrote:
>>
>>> coi la .ilmen.
>>>
>>> I just applied your idea (added split entries) and added merged
>>> entries... And I also found a very simple way to add compound cmavo!
>>> Indeed:
>>>
>>>    - I created a script that splits jbovlaste entries into cmavo and
>>>    non-cmavo, by using a simple regex (using rules listed in the CLL, c=
hapter
>>>    4.2)
>>>    - Then I tagged all cmavo with a flag "C", and added the Hunspell
>>>    rule "CCC*" (~=3D "CC+"), which means you can "glue" 2 or more cmavo=
 together.
>>>
>>> Of course, this will allow un-grammatical things such as "lonulonucalo"=
,
>>> but once again this is not the spell-checker role.
>>>
>>> I tried your example "calonu". It seems the "lonu" entry exists, so my
>>> dictionary inteprets that as a "normal word" (=3D non-simple-cmavo) ins=
tead
>>> of a "compound cmavo". But all following combinations are now valid :
>>>
>>>    - ca, lo, nu
>>>    - lo nu, lonu, ca lo, calo
>>>    - ca lonu, calo nu, calonu
>>>
>>> Only calo & calonu are detected as a compound (remember "lonu" is an
>>> entry), but anyway that works as expected.
>>> Experimental cmavo support will be added soon.
>>>
>>> Do you know other rules that could be great integrating?
>>> Please test ( https://github.com/Sukender/lojban-spell-check-dist ) and
>>> give feedback! ki'e
>>>
>>> I still have issues with dots in LibreOffice (.i .a and such)... And
>>> some words of "le cmalu noltru" are not recognized yet. Is there any ot=
her
>>> word source I can use?
>>>
>>> co'o
>>>
>>> --
>>> Sukender
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "lojban" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to lojban+unsubscribe@googlegroups.com.
>>> To post to this group, send email to lojban@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/lojban.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "lojban" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lojban+unsubscribe@googlegroups.com.
> To post to this group, send email to lojban@googlegroups.com.
> Visit this group at https://groups.google.com/group/lojban.
> For more options, visit https://groups.google.com/d/optout.
>

--=20
You received this message because you are subscribed to the Google Groups "=
lojban" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to lojban+unsubscribe@googlegroups.com.
To post to this group, send email to lojban@googlegroups.com.
Visit this group at https://groups.google.com/group/lojban.
For more options, visit https://groups.google.com/d/optout.

--001a1144f3f811667e055562a9a4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">jbovlaste should already be filtered to contain only Lojba=
n, and there are, broadly, three types of Lojban words:<div>cmevla are ever=
ything that ends in a consonant</div><div>brivla all contain a consonant cl=
uster and end in a vowel</div><div>cmavo optionally start with a single con=
sonant, and consist entirely of vowels and apostrophes after that.</div><di=
v><br></div><div>So, I think you could filter all cmavo clusters by looking=
 for anything that matches /.+[^aeiou&#39;].*[aeiou]/ but doesn&#39;t match=
 /[^aeiou&#39;][^aeiou&#39;]/. Contains a non-vowel somewhere after the fir=
st letter, ends in a vowel, and doesn&#39;t contain a consonant cluster.</d=
iv><div><br></div><div>At least, that seems like a good start.=C2=A0</div><=
/div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Fri, Jul 28, 2017 a=
t 10:43 AM Sukender &lt;<a href=3D"mailto:sukender@free.fr">sukender@free.f=
r</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin=
:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Tanks for the=
 clarification. I didn&#39;t even imagine that this big random compound cma=
vo would be valid! You made my evening! ;-)<br>
<br>
About {lonu} entry, I clearly agree. But I can&#39;t filter all them out...=
 Or can I? If you get any (simple) idea of rule for that, then go ahead!<br=
>
<br>
By the way, I already filtered out a few words. I indeed found some of huge=
 length (even a weird one about Macarena!). As it may be spam, I added an a=
rbitrary rule that throws away all that have more than 22 characters. Maybe=
 a finer rule has to be found...<br>
<br>
Cheers,</div><div><br>
<br>
-- <br>
Sukender</div><div><br>
<br>
<br><br><div class=3D"gmail_quote">Le 28 juillet 2017 17:33:14 CEST, Adam L=
opresto &lt;<a href=3D"mailto:adamlopresto@gmail.com" target=3D"_blank">ada=
mlopresto@gmail.com</a>&gt; a =C3=A9crit :<blockquote class=3D"gmail_quote"=
 style=3D"margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);p=
adding-left:1ex">
<div dir=3D"ltr">If you&#39;re going to allow cmavo to be combined arbitrar=
ily (which is probably appropriate), then there&#39;s no reason for {lonu} =
to have its own entry. So I&#39;d suggest not adding any cmavo clusters.<di=
v><br></div><div>And {lonulonucalo} can be grammatical, you just need the r=
ight text after it. {lonulonucalo nu jamna kei mi damba cu nandu mi cu se z=
ungi mi}, &quot;I feel guilty that it was hard for me to fight during the w=
ar.&quot; As you said, a fully grammar checker would be needed to really ge=
t things right, and that&#39;s a separate problem.</div></div><br><div clas=
s=3D"gmail_quote"><div dir=3D"ltr">On Fri, Jul 28, 2017 at 6:54 AM &lt;<a h=
ref=3D"mailto:sukender1@gmail.com" target=3D"_blank">sukender1@gmail.com</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">coi l=
a .ilmen.<div><br></div><div>I just applied your idea (added split entries)=
 and added merged entries... And I also found a very simple way to add comp=
ound cmavo!</div><div>Indeed:</div><div><ul><li>I created a script that spl=
its jbovlaste entries into cmavo and non-cmavo, by using a simple regex (us=
ing rules listed in the CLL, chapter 4.2)</li><li>Then I tagged all cmavo w=
ith a flag &quot;C&quot;, and added the Hunspell rule &quot;CCC*&quot; (~=
=3D &quot;CC+&quot;), which means you can &quot;glue&quot; 2 or more cmavo =
together.<br></li></ul></div><div>Of course, this will allow un-grammatical=
 things such as &quot;lonulonucalo&quot;, but once again this is not the sp=
ell-checker role.</div><div><br></div><div>I tried your example &quot;calon=
u&quot;. It seems the &quot;lonu&quot; entry exists, so my dictionary intep=
rets that as a &quot;normal word&quot; (=3D non-simple-cmavo) instead of a =
&quot;compound cmavo&quot;. But all following combinations are now valid :<=
/div><div><ul><li>ca, lo, nu<br></li><li>lo nu, lonu, ca lo, calo</li><li>c=
a lonu, calo nu, calonu<br></li></ul>Only calo &amp; calonu are detected as=
 a compound (remember &quot;lonu&quot; is an entry), but anyway that works =
as expected.<br>Experimental cmavo support will be added soon.<br></div><di=
v><br></div><div>Do you know other rules that could be great integrating?</=
div><div>Please test (=C2=A0<a href=3D"https://github.com/Sukender/lojban-s=
pell-check-dist" target=3D"_blank">https://github.com/Sukender/lojban-spell=
-check-dist</a>=C2=A0) and give feedback! ki&#39;e</div><div><br></div><div=
>I still have issues with dots in LibreOffice (.i .a and such)... And some =
words of &quot;le cmalu noltru&quot; are not recognized yet. Is there any o=
ther word source I can use?</div><div><br></div><div>co&#39;o</div></div><d=
iv dir=3D"ltr"><div><br></div><div>--=C2=A0</div><div>Sukender</div></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban+unsubscribe@googlegroups.com" target=3D"_b=
lank">lojban+unsubscribe@googlegroups.com</a>.<br>
To post to this group, send email to <a href=3D"mailto:lojban@googlegroups.=
com" target=3D"_blank">lojban@googlegroups.com</a>.<br>
Visit this group at <a href=3D"https://groups.google.com/group/lojban" targ=
et=3D"_blank">https://groups.google.com/group/lojban</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" targ=
et=3D"_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div>

<p></p>
</blockquote></div></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban+unsubscribe@googlegroups.com" target=3D"_b=
lank">lojban+unsubscribe@googlegroups.com</a>.<br>
To post to this group, send email to <a href=3D"mailto:lojban@googlegroups.=
com" target=3D"_blank">lojban@googlegroups.com</a>.<br>
Visit this group at <a href=3D"https://groups.google.com/group/lojban" targ=
et=3D"_blank">https://groups.google.com/group/lojban</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" targ=
et=3D"_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban+unsubscribe@googlegroups.com">lojban+unsub=
scribe@googlegroups.com</a>.<br />
To post to this group, send email to <a href=3D"mailto:lojban@googlegroups.=
com">lojban@googlegroups.com</a>.<br />
Visit this group at <a href=3D"https://groups.google.com/group/lojban">http=
s://groups.google.com/group/lojban</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

--001a1144f3f811667e055562a9a4--