Received-SPF: pass (google.com: best guess record for domain of rlpowell@digitalkingdom.org designates 173.13.139.236 as permitted sender) client-ip=173.13.139.236;
Date: Mon, 25 Mar 2013 11:08:21 -0700
From: Robin Lee Powell <rlpowell@digitalkingdom.org>
To: lojban-list@lojban.org, jbovlaste@lojban.org
Subject: [lojban] Need some jbovlaste programming help.
Message-ID: <20130325180820.GU6328@stodi.digitalkingdom.org>
Mail-Followup-To: lojban-list@lojban.org, jbovlaste@lojban.org
MIME-Version: 1.0
User-Agent: Mutt/1.5.21 (2010-09-15)
Reply-To: lojban@googlegroups.com
Precedence: list
Mailing-list: list lojban@googlegroups.com; contact lojban+owners@googlegroups.com
Sender: lojban@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Disposition: inline
X-Spam_score: 0.1
X-Spam_score_int: 1
X-Spam_bar: /
X-Spam-Report: Spam detection software, running on the system "stodi.digitalkingdom.org", has
 identified this incoming email as possible spam.  The original message
 has been attached to this so you can view it (if it isn't spam) or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  Some bad data has snuck its way in to jbovlaste (a good chunk
    from an import script I screwed up that we can't just re-run, but some of
    it isn't from that, so not sure what's going on) and it needs cleaning. [...]
    
 
 Content analysis details:   (0.1 points, 5.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  0.0 URIBL_BLOCKED          ADMINISTRATOR NOTICE: The query to URIBL was blocked.
                             See
                             http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
                              for more information.
                             [URIs: googlegroups.com]
  0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily valid
  0.0 T_DKIM_INVALID         DKIM-Signature header exists but is not valid


Some bad data has snuck its way in to jbovlaste (a good chunk from
an import script I screwed up that we can't just re-run, but some of
it isn't from that, so not sure what's going on) and it needs
cleaning.

I have neither the time nor inclination.

I don't much care what it's written in as long as it's UTF-8 safe
(i.e. bash isn't going to cut it), but we need something that does
the following:

For every natlang word:

  if a duplicate (same word, meaning, and langid) exists,
  consolidate them.  This means deleting the duplicate, combining
  the "notes" field for the two of them, and updating all instances
  of the id you just deleted to point to the one that still exists
  in the tables threads, keywordmapping, natlangwordbestguesses, and
  natlangwordvotes.  natlangwordbestguesses has to be handled
  specially there, as it shouldn't end up with two identical rows
  (identical across all 3 fields); that shouldn't be possible given
  that manipulation, but check anyway.

  if the word is unused, delete it; unused means that its id does
  not occur in the appropriate column in threads, keywordmapping,
  natlangwordbestguesses, and natlangwordvotes. 

For context, here's the code: https://github.com/lojban/jbovlaste ,
here's a script that works
https://github.com/lojban/jbovlaste/blob/master/bin/snarfgismu_tabs
(the script in question, in fact, but fixed) in case you want to
keep to the same code style, and here's the schema:
https://github.com/lojban/jbovlaste/blob/master/help/schema.txt

Looking forward to some help.

-Robin

-- 
http://intelligence.org/ :  Our last, best hope for a fantastic future.
.i ko na cpedu lo nu stidi vau loi jbopre .i danfu lu na go'i li'u .e
lu go'i li'u .i ji'a go'i lu na'e go'i li'u .e lu go'i na'i li'u .e
lu no'e go'i li'u .e lu to'e go'i li'u .e lu lo mamta be do cu sofybakni li'u

-- 
You received this message because you are subscribed to the Google Groups "lojban" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lojban+unsubscribe@googlegroups.com.
To post to this group, send email to lojban@googlegroups.com.
Visit this group at http://groups.google.com/group/lojban?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.