Envelope-to: lojban-list-archive@lojban.org
Delivery-date: Sun, 29 Aug 2021 07:13:15 -0700
Sender: lojban@googlegroups.com
Received-SPF: pass (google.com: domain of shunpiker@gmail.com designates 2a00:1450:4864:20::631 as permitted sender) client-ip=2a00:1450:4864:20::631;
MIME-Version: 1.0
References: <20210827021139.GO309000@gmail.com> <B6AB9786-1025-4F5E-A196-C83556B8EDC6@gmail.com>
 <20210828040224.GS309000@gmail.com>
In-Reply-To: <20210828040224.GS309000@gmail.com>
From: Riley Lynch <shunpiker@gmail.com>
Date: Sun, 29 Aug 2021 10:12:56 -0400
Message-ID: <CAB-HkawLxFuQZT8_vuZf8wmudq=kBkY=y5yC=OWom2fNyvQANw@mail.gmail.com>
Subject: [lojban] Re: Help parsing Lojban from Python? (Hey, Riley! :)
To: Robin Lee Powell <robinleepowell@gmail.com>
Cc: lojban@googlegroups.com
Content-Type: multipart/alternative; boundary="000000000000d7920905cab35044"
Reply-To: lojban@googlegroups.com
Precedence: list
Mailing-list: list lojban@googlegroups.com; contact lojban+owners@googlegroups.com
X-Spam_score: -2.5
X-Spam_score_int: -24
X-Spam_bar: --

--000000000000d7920905cab35044
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I'm going to be spending most of the day driving, but before I do that,
I'll try to address a few questions here, and then can follow up by mail or
IRC.

(1) I want to confirm that camxes-py is the preferred Python option
these days

I'm not aware of other parsers in python. I specifically developed the
parser because I wanted a python implementation to complement your java
implementation and Masato and Ilmen's javascript parsers.

I notice now that Randall Holmes has developed a Python PEG parser for
Loglan.

(2) I want to be able to run =E2=80=A6  it in a direct, straightforward way=
 =E2=80=A6 and
tree should contain an obvious python representation of the
parse tree
(5) Make a mode that collapses productions with only one child

Running will be the easy part. The representation of the parse tree raises
some interesting questions.

For camxes-py, I created a transformation of the parse tree which
replicated the output of Ilmentufa. I did this so that I could run against
the test corpus that you set up for java camxes and verify not only that
the python parser could accept the same corpus as the java and javascript
parsers, but that it was comprehending the same structures.

That said, the output exposes a lot of the mechanics of the parser
specification and obscures the semantics. Ideally, I'd like for the test
suites to target compatibility with a semantically-structured
representation of the parse. There's been some work on Ilmentufa to
post-process the parse tree into something more palatable. Have you taken a
look at that?

(3) Update to most-recent parsimonious; it currently breaks on 0.8.1, but
works on 0.6.2

I wrote against the most recent version of parsimonious at that time. Glad
to see work has continued. I remember the author was working on some
performance enhancements, and one problem with camxes-py in its current
form is that it is slow.

(4) Update to Python 3

I agree that this should be done.


On Sat, Aug 28, 2021 at 12:02 AM Robin Lee Powell <robinleepowell@gmail.com=
>
wrote:

> Feel free to come find me on Libera IRC, or suggest a preferred chat
> option for you.
>
> The stuff I want is actually quite simple, though:
>
> (1) I want to confirm that camxes-py is the preferred Python option
> these days
>
> (2) I want to be able to run "run" (see
> https://github.com/teleological/camxes-py/blob/master/camxes.py#L89
> ) or something like it in a direct, straightforward way, i.e.:
>
>           import camxespy
>           tree =3D camxespy.run("mi klama", transformer=3D'camxes-morphol=
ogy')
>
> , and tree should contain an obvious python representation of the
> parse tree.
>
> This requires, AFAICT (I don't actually know Python very well) that
> camxes-py have a library struture to it that it doesn't currently
> have and that the options be configurable in some way other than
> OptionParser.
>
> I can actually do all that myself, but I'm not really a pythonista
> and what I do won't be idiomatic at all.
>
> Stretch goals:
>
> (3) Update to most-recent parsimonious; it currently breaks on
> 0.8.1, but works on 0.6.2
>
> (4) Update to Python 3, but I'm perfectly capable of making a PR for
> this myself.
>
> (5) Make a mode that collapses productions with only one child, i.e.
> make the output look like this (in terms of productions not syntax):
>
>         rlpowell@stodi> echo "mi klama" | camxes -f
>         Flat layout requested.
>          text=3D(  sentence=3D(  CMAVO=3D(  KOhA=3D( mi )  )  BRIVLA=3D( =
 gismu=3D(
> klama )  )  )  )
>
> Instead of this:
>
> root@66324b4aed4b:/src# python camxes.py "mi klama"
>
> ["text",["text_1",["paragraphs",["paragraph",["statement",["statement_1",=
["statement_2",["statement_3",["sentence",[["terms",["terms_1",["terms_2",[=
"abs_term",["abs_term_1",["sumti",["sumti_1",["sumti_2",["sumti_3",["sumti_=
4",["sumti_5",["sumti_6",["KOhA_clause",[["KOhA","mi"]]]]]]]]]]]]]]],["CU"]=
],["bridi_tail",["bridi_tail_1",["bridi_tail_2",["bridi_tail_3",["selbri",[=
"selbri_1",["selbri_2",["selbri_3",["selbri_4",["selbri_5",["selbri_6",["ta=
nru_unit",["tanru_unit_1",["tanru_unit_2",["BRIVLA_clause",[["BRIVLA",["gis=
mu","klama"]]]]]]]]]]]]]],["tail_terms",["VAU"]]]]]]]]]]]]]]]
>
> , but as I said before this is not hard to do after the fact once
> you have the parse tree.
>
>
> On Fri, Aug 27, 2021 at 11:15:54PM -0400, Riley Martinez-Lynch
> wrote:
> > Robin, I'd be happy to make whatever changes are needed to make it
> > work. I don't see the CLI interface as an essential part of the
> > interface, and if I can do something to make it easier to access
> > programmatically, I'd like to do that. Glad to take cues here, or
> > if you wanted to jump on a call or chat, can do that too.
> >
> > Sent from my iPhone
> >
> > > On Aug 26, 2021, at 10:11 PM, Robin Lee Powell <
> robinleepowell@gmail.com> wrote:
> > >
> > > =EF=BB=BF
> > > In service to making certain parts of the lojban.org infra a bit
> > > more resilient, I'm updating some stuff that uses
> > > https://github.com/lojban/python-camxes .  This relies on java and
> > > the camxes jar, which, whatever, but it's also built on LEPL, which
> > > no longer works (see for example
> > > https://github.com/modoboa/modoboa/issues/1780 ).
> > >
> > > https://github.com/teleological/camxes-py is a pure Python
> > > replacement, but is a CLI program rather than a library; it's really
> > > not designed to be used as a library.  I'd love it if someone
> > > updated and fixed that.
> > >
> > > Unless there's another option?  What's the state of the art in this
> > > space?
> > >
>

--=20
You received this message because you are subscribed to the Google Groups "=
lojban" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to lojban+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/=
lojban/CAB-HkawLxFuQZT8_vuZf8wmudq%3DkBkY%3Dy5yC%3DOWom2fNyvQANw%40mail.gma=
il.com.

--000000000000d7920905cab35044
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>I&#39;m going to be spending most of the day driving,=
 but before I do that, I&#39;ll try to address a few questions here, and th=
en can follow up by mail or IRC.</div><br><div>(1) I want to confirm that c=
amxes-py is the preferred Python option<br>these days<br></div><div><br></d=
iv><div>I&#39;m not aware of other parsers in python. I specifically develo=
ped the parser because I wanted a python implementation to complement your =
java implementation and Masato and Ilmen&#39;s javascript parsers.</div><di=
v><br></div><div>I notice now that Randall Holmes has developed a Python PE=
G parser for Loglan.</div><div><br></div><div>(2) I want to be able to run =
=E2=80=A6=C2=A0<span class=3D"gmail-Apple-converted-space">=C2=A0</span>it =
in a direct, straightforward way =E2=80=A6 and tree should contain an obvio=
us python representation of the<br></div>parse tree<div>(5) Make a mode tha=
t collapses productions with only one child<br><div><br></div><div>Running =
will be the easy part. The representation of the parse tree raises some int=
eresting questions.</div><div><br></div><div>For camxes-py, I created a tra=
nsformation of the parse tree which replicated the output of Ilmentufa. I d=
id this so that I could run against the test corpus=C2=A0that you set up fo=
r java camxes=C2=A0and verify not only that the python parser could accept =
the same corpus as the java and javascript parsers, but that it was compreh=
ending the same structures.</div><div><br></div><div>That said, the output =
exposes a lot of the mechanics of the parser specification and obscures the=
 semantics. Ideally, I&#39;d like for the test suites to target compatibili=
ty with a semantically-structured representation of the parse. There&#39;s =
been some work on Ilmentufa to post-process the parse tree into something m=
ore palatable. Have you taken a look at that?</div><div><br></div><div>(3) =
Update to most-recent parsimonious; it currently breaks on=C2=A00.8.1, but =
works on 0.6.2<br></div><div><br></div><div>I wrote against the most recent=
 version of parsimonious at that time. Glad to see work has continued. I re=
member the author was working on some performance enhancements, and one pro=
blem with camxes-py in its current form is that it is slow.</div><div><br><=
/div><div>(4) Update to Python 3<br></div><div><br></div><div>I agree that =
this should be done.</div><div><br></div></div></div><br><div class=3D"gmai=
l_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Aug 28, 2021 at 12:0=
2 AM Robin Lee Powell &lt;<a href=3D"mailto:robinleepowell@gmail.com">robin=
leepowell@gmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-styl=
e:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Feel free to c=
ome find me on Libera IRC, or suggest a preferred chat<br>
option for you.<br>
<br>
The stuff I want is actually quite simple, though:<br>
<br>
(1) I want to confirm that camxes-py is the preferred Python option<br>
these days<br>
<br>
(2) I want to be able to run &quot;run&quot; (see<br>
<a href=3D"https://github.com/teleological/camxes-py/blob/master/camxes.py#=
L89" rel=3D"noreferrer" target=3D"_blank">https://github.com/teleological/c=
amxes-py/blob/master/camxes.py#L89</a><br>
) or something like it in a direct, straightforward way, i.e.:<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 import camxespy<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 tree =3D camxespy.run(&quot;mi klama&quo=
t;, transformer=3D&#39;camxes-morphology&#39;)<br>
<br>
, and tree should contain an obvious python representation of the<br>
parse tree.<br>
<br>
This requires, AFAICT (I don&#39;t actually know Python very well) that<br>
camxes-py have a library struture to it that it doesn&#39;t currently<br>
have and that the options be configurable in some way other than<br>
OptionParser.<br>
<br>
I can actually do all that myself, but I&#39;m not really a pythonista<br>
and what I do won&#39;t be idiomatic at all.<br>
<br>
Stretch goals:<br>
<br>
(3) Update to most-recent parsimonious; it currently breaks on<br>
0.8.1, but works on 0.6.2<br>
<br>
(4) Update to Python 3, but I&#39;m perfectly capable of making a PR for<br=
>
this myself.<br>
<br>
(5) Make a mode that collapses productions with only one child, i.e.<br>
make the output look like this (in terms of productions not syntax):<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 rlpowell@stodi&gt; echo &quot;mi klama&quot; | =
camxes -f<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Flat layout requested.<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0text=3D(=C2=A0 sentence=3D(=C2=A0 CMAVO=
=3D(=C2=A0 KOhA=3D( mi )=C2=A0 )=C2=A0 BRIVLA=3D(=C2=A0 gismu=3D( klama )=
=C2=A0 )=C2=A0 )=C2=A0 )<br>
<br>
Instead of this:<br>
<br>
root@66324b4aed4b:/src# python camxes.py &quot;mi klama&quot;<br>
[&quot;text&quot;,[&quot;text_1&quot;,[&quot;paragraphs&quot;,[&quot;paragr=
aph&quot;,[&quot;statement&quot;,[&quot;statement_1&quot;,[&quot;statement_=
2&quot;,[&quot;statement_3&quot;,[&quot;sentence&quot;,[[&quot;terms&quot;,=
[&quot;terms_1&quot;,[&quot;terms_2&quot;,[&quot;abs_term&quot;,[&quot;abs_=
term_1&quot;,[&quot;sumti&quot;,[&quot;sumti_1&quot;,[&quot;sumti_2&quot;,[=
&quot;sumti_3&quot;,[&quot;sumti_4&quot;,[&quot;sumti_5&quot;,[&quot;sumti_=
6&quot;,[&quot;KOhA_clause&quot;,[[&quot;KOhA&quot;,&quot;mi&quot;]]]]]]]]]=
]]]]]],[&quot;CU&quot;]],[&quot;bridi_tail&quot;,[&quot;bridi_tail_1&quot;,=
[&quot;bridi_tail_2&quot;,[&quot;bridi_tail_3&quot;,[&quot;selbri&quot;,[&q=
uot;selbri_1&quot;,[&quot;selbri_2&quot;,[&quot;selbri_3&quot;,[&quot;selbr=
i_4&quot;,[&quot;selbri_5&quot;,[&quot;selbri_6&quot;,[&quot;tanru_unit&quo=
t;,[&quot;tanru_unit_1&quot;,[&quot;tanru_unit_2&quot;,[&quot;BRIVLA_clause=
&quot;,[[&quot;BRIVLA&quot;,[&quot;gismu&quot;,&quot;klama&quot;]]]]]]]]]]]=
]]],[&quot;tail_terms&quot;,[&quot;VAU&quot;]]]]]]]]]]]]]]]<br>
<br>
, but as I said before this is not hard to do after the fact once<br>
you have the parse tree.<br>
<br>
<br>
On Fri, Aug 27, 2021 at 11:15:54PM -0400, Riley Martinez-Lynch<br>
wrote:<br>
&gt; Robin, I&#39;d be happy to make whatever changes are needed to make it=
<br>
&gt; work. I don&#39;t see the CLI interface as an essential part of the<br=
>
&gt; interface, and if I can do something to make it easier to access<br>
&gt; programmatically, I&#39;d like to do that. Glad to take cues here, or<=
br>
&gt; if you wanted to jump on a call or chat, can do that too.<br>
&gt; <br>
&gt; Sent from my iPhone<br>
&gt; <br>
&gt; &gt; On Aug 26, 2021, at 10:11 PM, Robin Lee Powell &lt;<a href=3D"mai=
lto:robinleepowell@gmail.com" target=3D"_blank">robinleepowell@gmail.com</a=
>&gt; wrote:<br>
&gt; &gt; <br>
&gt; &gt; =EF=BB=BF<br>
&gt; &gt; In service to making certain parts of the <a href=3D"http://lojba=
n.org" rel=3D"noreferrer" target=3D"_blank">lojban.org</a> infra a bit<br>
&gt; &gt; more resilient, I&#39;m updating some stuff that uses<br>
&gt; &gt; <a href=3D"https://github.com/lojban/python-camxes" rel=3D"norefe=
rrer" target=3D"_blank">https://github.com/lojban/python-camxes</a> .=C2=A0=
 This relies on java and<br>
&gt; &gt; the camxes jar, which, whatever, but it&#39;s also built on LEPL,=
 which<br>
&gt; &gt; no longer works (see for example<br>
&gt; &gt; <a href=3D"https://github.com/modoboa/modoboa/issues/1780" rel=3D=
"noreferrer" target=3D"_blank">https://github.com/modoboa/modoboa/issues/17=
80</a> ).<br>
&gt; &gt; <br>
&gt; &gt; <a href=3D"https://github.com/teleological/camxes-py" rel=3D"nore=
ferrer" target=3D"_blank">https://github.com/teleological/camxes-py</a> is =
a pure Python<br>
&gt; &gt; replacement, but is a CLI program rather than a library; it&#39;s=
 really<br>
&gt; &gt; not designed to be used as a library.=C2=A0 I&#39;d love it if so=
meone<br>
&gt; &gt; updated and fixed that.<br>
&gt; &gt; <br>
&gt; &gt; Unless there&#39;s another option?=C2=A0 What&#39;s the state of =
the art in this<br>
&gt; &gt; space?<br>
&gt; &gt; <br>
</blockquote></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban+unsubscribe@googlegroups.com">lojban+unsub=
scribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/lojban/CAB-HkawLxFuQZT8_vuZf8wmudq%3DkBkY%3Dy5yC%3DOWom2fNyvQANw=
%40mail.gmail.com?utm_medium=3Demail&utm_source=3Dfooter">https://groups.go=
ogle.com/d/msgid/lojban/CAB-HkawLxFuQZT8_vuZf8wmudq%3DkBkY%3Dy5yC%3DOWom2fN=
yvQANw%40mail.gmail.com</a>.<br />

--000000000000d7920905cab35044--