Sender: lojban@googlegroups.com
Received-SPF: pass (google.com: domain of maikxlx@gmail.com designates 2607:f8b0:4864:20::335 as permitted sender) client-ip=2607:f8b0:4864:20::335;
MIME-Version: 1.0
References: <20200206062919.GU26741@stodi.digitalkingdom.org>
In-Reply-To: <20200206062919.GU26741@stodi.digitalkingdom.org>
From: "Mike S." <maikxlx@gmail.com>
Date: Fri, 7 Feb 2020 07:58:58 -0500
Message-ID: <CAOYwWb18QpfALOwJoSc-JxbiePDj1e1J35ePeHCVQVMUpi1=Hw@mail.gmail.com>
Subject: Re: [lojban] What's going with CloudFlare and caching and such.
To: lojban@googlegroups.com
Content-Type: multipart/alternative; boundary="00000000000034704a059dfbf5db"
Reply-To: lojban@googlegroups.com
Precedence: list
Mailing-list: list lojban@googlegroups.com; contact lojban+owners@googlegroups.com
X-Spam_score: -2.6
X-Spam_score_int: -25
X-Spam_bar: --

--00000000000034704a059dfbf5db
Content-Type: text/plain; charset="UTF-8"

On Thu, Feb 6, 2020 at 1:29 AM Robin Lee Powell <rlpowell@digitalkingdom.org>
wrote:

> Some of you have noticed problems with dynamic content on lojban.org
> now that we've started using CloudFlare.  I haven't been able to
> figure out how to fix this, so here's what's going on; maybe
> somebody else will have ideas.
>
> So the goal of CloudFlare, primarily, was to make it so that if my
> server went down, the site would be basically fine.  This we have
> acheived.
>

Judging from what I saw during the last outage, I believe Cloudflare is
caching only pages that have been requested at least once. Assuming I am
right about that, I don't know if there is a way to make Cloudflare cache
the whole site without clicking through every page on the Wiki.  By the
way, I don't care about this issue since the site is up most of the time
anyway, but since the whole point of the exercise is to cache pages and
that doesn't necessarily happen, I thought I'd mention it!

As far as the main issue, I can only guess.  It might have something to do
with an extension hijacking the Cache-Control header, but I don't know.

At any rate, your bypass rule for talk pages seems to be working okay.
Since we don't know what the problem is, I suggest the following for a temp
fix --  Please add the following rules in the proper location:

*lojban.org/*LFK*          Cache Level: Bypass
*lojban.org/*[bypasscache]*          Cache Level: Bypass

The first line will help the new committee work on their pages.  The second
line will allow a person to work on an arbitrary page without the cache.
When the page is finished, it can then be moved to the correct location.

I *believe* these rules will be sufficient for the time being and get you
off the hook.  Your help is very much appreciated!.

-Mike


>
> However, to do so, I had to use brute force.  Here's our CloudFlare
> page rules:
>
> *lojban.org/*&*          Cache Level: Bypass
> *lojban.org/*edit*       Cache Level: Bypass
> *lojban.org/*Special:*  Cache Level: Bypass
> *lojban.org/*Talk:*     Cache Level: Bypass
> *lojban.org/*           Browser Cache TTL: 30 minutes, Always Online: On,
> Cache Level: Cache Everything, Edge Cache TTL: 2 hours, Origin Cache
> Control: Off
>
> That last one is a very large hammer that says "just cache the hell
> out of everything".
>
> The reason I need that hammer is that mediawiki is returning
> absurdly wrong caching headers.  Here's an example that entirely
> bypasses CloudFlare:
>
> $ curl -k -v -H 'Host: mw.lojban.org' -L
> https://jukni.lojban.org/papri/pronunciation 2>&1 | less
> [snip]
> < HTTP/1.1 200 OK
> < Date: Thu, 06 Feb 2020 06:26:31 GMT
> < Server: Apache/2.4.38 (Debian)
> < X-Powered-By: PHP/7.3.14
> < X-Content-Type-Options: nosniff
> < Content-language: en
> < Vary: Accept-Encoding,Cookie
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < Cache-Control: private, must-revalidate, max-age=0
>
> ^^ That.   That Cache-Control line is absurd, and effectively
> completely disableds CloudFlare.
>
> I have tested this by talking *directly* to the mediawiki server,
> no SSL, no proxies, no nothing; it's the same.
>
> Our LocalSettings.php file is attached.
>
> I don't care if the solution is on the mediawiki side or the
> CloudFlare side, but at this point I've sunk more time into this
> than I can afford and I've come up empty, so I'd appreciate any
> ideas you might have.
>
> Having said that, if you're going to point me at
> https://www.mediawiki.org/wiki/Manual:CloudFlare#Integration_with_MediaWiki
> , you'll need to tell me exactly which part you think is relevant,
> and why.  Most of that page is about making X-Forwarded-For: do the
> right thing, which is totally irrelevant to this problem.
>
> Thanks for any help.
>
> --
> You received this message because you are subscribed to the Google Groups
> "lojban" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lojban+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/lojban/20200206062919.GU26741%40stodi.digitalkingdom.org
> .
>

-- 
You received this message because you are subscribed to the Google Groups "lojban" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lojban+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lojban/CAOYwWb18QpfALOwJoSc-JxbiePDj1e1J35ePeHCVQVMUpi1%3DHw%40mail.gmail.com.

--00000000000034704a059dfbf5db
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div di=
r=3D"ltr"><br clear=3D"all"></div><br><div class=3D"gmail_quote"><div dir=
=3D"ltr" class=3D"gmail_attr">On Thu, Feb 6, 2020 at 1:29 AM Robin Lee Powe=
ll &lt;<a href=3D"mailto:rlpowell@digitalkingdom.org">rlpowell@digitalkingd=
om.org</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex">Some of you have noticed problems with dynamic content on <a href=3D"=
http://lojban.org" rel=3D"noreferrer" target=3D"_blank">lojban.org</a><br>
now that we&#39;ve started using CloudFlare.=C2=A0 I haven&#39;t been able =
to<br>
figure out how to fix this, so here&#39;s what&#39;s going on; maybe<br>
somebody else will have ideas.<br>
<br>
So the goal of CloudFlare, primarily, was to make it so that if my<br>
server went down, the site would be basically fine.=C2=A0 This we have<br>
acheived.<br></blockquote><div><br></div><div>Judging from what I saw durin=
g the last outage, I believe Cloudflare is caching only pages that have bee=
n requested at least once. Assuming I am right about that, I don&#39;t know=
 if there is a way to make Cloudflare cache the whole site without clicking=
 through every page on the Wiki.=C2=A0 By the way, I don&#39;t care about t=
his issue since the site is up most of the time anyway, but since the whole=
 point of the exercise is to cache pages and that doesn&#39;t necessarily h=
appen, I thought I&#39;d mention it!=C2=A0 <br><br></div><div>As far as the=
 main issue, I can only guess.=C2=A0 It might have something to do with an =
extension hijacking the Cache-Control header, but I don&#39;t know.<br><br>=
At any rate, your bypass rule for talk pages seems to be working okay.=C2=
=A0 Since we don&#39;t know what the problem is, I suggest the following fo=
r a temp fix --=C2=A0 Please add the following rules in the proper location=
:<br></div><br>*<a href=3D"http://lojban.org/*LFK*">lojban.org/*LFK*</a> =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Cache Level: Bypass<br>*<a href=3D"http:/=
/lojban.org/*[bypasscache]*">lojban.org/*[bypasscache]*</a> =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0Cache Level: Bypass<br><br></div><div class=3D"gmail_qu=
ote">The first line will help the new committee work on their pages.=C2=A0 =
The second line will allow a person to work on an arbitrary page without th=
e cache.=C2=A0 When the page is finished, it can then be moved to the corre=
ct location.<br><br></div><div class=3D"gmail_quote">I *believe* these rule=
s will be sufficient for the time being and get you off the hook.=C2=A0 You=
r help is very much appreciated!.<br></div><div class=3D"gmail_quote"><br><=
/div><div class=3D"gmail_quote">-Mike<br></div><div class=3D"gmail_quote"><=
br><br></div><div class=3D"gmail_quote"><div>=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">
<br>
However, to do so, I had to use brute force.=C2=A0 Here&#39;s our CloudFlar=
e<br>
page rules:<br>
<br>
*<a href=3D"http://lojban.org/*&amp;*" rel=3D"noreferrer" target=3D"_blank"=
>lojban.org/*&amp;*</a>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Cache Level: Bypa=
ss<br>
*<a href=3D"http://lojban.org/*edit*" rel=3D"noreferrer" target=3D"_blank">=
lojban.org/*edit*</a>=C2=A0 =C2=A0 =C2=A0 =C2=A0Cache Level: Bypass<br>
*<a href=3D"http://lojban.org/*Special:*" rel=3D"noreferrer" target=3D"_bla=
nk">lojban.org/*Special:*</a>=C2=A0 Cache Level: Bypass<br>
*<a href=3D"http://lojban.org/*Talk:*" rel=3D"noreferrer" target=3D"_blank"=
>lojban.org/*Talk:*</a>=C2=A0 =C2=A0 =C2=A0Cache Level: Bypass<br>
*<a href=3D"http://lojban.org/*" rel=3D"noreferrer" target=3D"_blank">lojba=
n.org/*</a>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Browser Cache TTL: 30 m=
inutes, Always Online: On, Cache Level: Cache Everything, Edge Cache TTL: 2=
 hours, Origin Cache Control: Off<br>
<br>
That last one is a very large hammer that says &quot;just cache the hell<br=
>
out of everything&quot;.<br>
<br>
The reason I need that hammer is that mediawiki is returning<br>
absurdly wrong caching headers.=C2=A0 Here&#39;s an example that entirely<b=
r>
bypasses CloudFlare:<br>
<br>
$ curl -k -v -H &#39;Host: <a href=3D"http://mw.lojban.org" rel=3D"noreferr=
er" target=3D"_blank">mw.lojban.org</a>&#39; -L <a href=3D"https://jukni.lo=
jban.org/papri/pronunciation" rel=3D"noreferrer" target=3D"_blank">https://=
jukni.lojban.org/papri/pronunciation</a> 2&gt;&amp;1 | less <br>
[snip]<br>
&lt; HTTP/1.1 200 OK<br>
&lt; Date: Thu, 06 Feb 2020 06:26:31 GMT<br>
&lt; Server: Apache/2.4.38 (Debian)<br>
&lt; X-Powered-By: PHP/7.3.14<br>
&lt; X-Content-Type-Options: nosniff<br>
&lt; Content-language: en<br>
&lt; Vary: Accept-Encoding,Cookie<br>
&lt; Expires: Thu, 01 Jan 1970 00:00:00 GMT<br>
&lt; Cache-Control: private, must-revalidate, max-age=3D0<br>
<br>
^^ That.=C2=A0 =C2=A0That Cache-Control line is absurd, and effectively<br>
completely disableds CloudFlare.<br>
<br>
I have tested this by talking *directly* to the mediawiki server,<br>
no SSL, no proxies, no nothing; it&#39;s the same.<br>
<br>
Our LocalSettings.php file is attached.<br>
<br>
I don&#39;t care if the solution is on the mediawiki side or the<br>
CloudFlare side, but at this point I&#39;ve sunk more time into this<br>
than I can afford and I&#39;ve come up empty, so I&#39;d appreciate any<br>
ideas you might have.<br>
<br>
Having said that, if you&#39;re going to point me at<br>
<a href=3D"https://www.mediawiki.org/wiki/Manual:CloudFlare#Integration_wit=
h_MediaWiki" rel=3D"noreferrer" target=3D"_blank">https://www.mediawiki.org=
/wiki/Manual:CloudFlare#Integration_with_MediaWiki</a><br>
, you&#39;ll need to tell me exactly which part you think is relevant,<br>
and why.=C2=A0 Most of that page is about making X-Forwarded-For: do the<br=
>
right thing, which is totally irrelevant to this problem.<br>
<br>
Thanks for any help.<br>
<br>
-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban%2Bunsubscribe@googlegroups.com" target=3D"=
_blank">lojban+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/lojban/20200206062919.GU26741%40stodi.digitalkingdom.org" rel=3D=
"noreferrer" target=3D"_blank">https://groups.google.com/d/msgid/lojban/202=
00206062919.GU26741%40stodi.digitalkingdom.org</a>.<br>
</blockquote></div></div></div></div></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;lojban&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:lojban+unsubscribe@googlegroups.com">lojban+unsub=
scribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/lojban/CAOYwWb18QpfALOwJoSc-JxbiePDj1e1J35ePeHCVQVMUpi1%3DHw%40m=
ail.gmail.com?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google=
.com/d/msgid/lojban/CAOYwWb18QpfALOwJoSc-JxbiePDj1e1J35ePeHCVQVMUpi1%3DHw%4=
0mail.gmail.com</a>.<br />

--00000000000034704a059dfbf5db--