Before, both request and response bodies were included as base64-encoded
blobs within the JSON that client and server exchange. This was workable
because we elsewhere limited the size of bodies to 64 KB. Now the
restriction on body length is gone, and we want to forward the response
body as it is received, not all at once after having read it all.

How it works now is we send a sequence of chunked streams. Each is a
sequence of length-preceded byte chunks. The first stream contains the
JSON blob (everything except the body), and the next contains the body.

In preparation for changing the helper protocol.

In support of #12857.

https://gitweb.torproject.org/pluggable-transports/goptlib.git/commit/?id=65fc8d72eccd82237562744ddc85760d426561d8

https://trac.torproject.org/projects/tor/ticket/14135
https://gitweb.torproject.org/pluggable-transports/goptlib.git/commit/?id=50b39b746c6ff34bf31977b658848d876ee84fbf

These were the cause of the profile error that you saw when restarting
Tor Browser after an update.
https://trac.torproject.org/projects/tor/ticket/13247#comment:17

Ensure that update-related browser preferences are written to
disk by flushing them during startup.

It started being closed in 1fa2da7c,
which added compliance with the TOR_PT_EXIT_ON_STDIN_CLOSE environment
variable. Closing stdin of the subprocess doesn't make sense, because
the purpose of terminateprocess-buffer is to keep the chile stdin open
after its own stdin has closed.

This fixes a regression in Tor Browser 4.5 on Windows 7:
https://trac.torproject.org/projects/tor/ticket/15872#comment:5
However, the same code, even when it closes stdin, works on Windows 8,
and I don't know why.

This improves performance quite a lot--previously we were doing a
complete TCP and TLS handshake to meek-server for every single request,
which, apart from increasing latency, also caused a lot of CPU usage on
meek-server. it was up above 80% when I checked it.

Now we reuse connections until they error out, making a new connection
if all others are currently busy.

It was reading unlimitedly in this case.

There's now an official way for tor to request this behavior.

Since #15435, we can treat terminateprocess-buffer as a compatibility
shim that adds support for TOR_PT_EXIT_ON_STDIN_CLOSE to versions of tor
that don't have it.

Stripping Host made it not work to use a url= without a front=. When you
omit front=, it's supposed to take the Host from the url= argument. This
is when you connect directly to the bridge, like
	bridge meek 0.0.2.0:3 url=https://meek.bamsoftware.com/

No reason for it to be a member function.

https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/NsIStreamListener#onDataAvailable%28%29
says "Your implementation of this method must read exactly aCount bytes
of data before returning."

It's already downloaded, so it doesn't cost much to read it out of the
buffer. Without this, I got "http channel Listener OnDataAvailable
contract violation" in the console. (I had to purposely induce an
overlong download to get it to happen.)

This enables us to send a meaningful {"error": "..."} blob back to the
requestor.

Use exceptions instead of dumping to console.

All our deployed backends are using HTTPS in forwarding. Might as well
make it the default for DIY deployers. I seem to remember that we once
had trouble on some platform (maybe some PHP host) because the installed
PHP library didn't have SSL support, but whatever.

let is block scope, which is what I always want.

05f244a5 removed logging of a couple of
errors that could contain client IP addresses. It turns out that these
errors were of type *net.OpError, the address field of which you can
overwrite from the outside.

https://github.com/golang/go/wiki/CodeReviewComments#initialisms
"This rule also applies to 'ID' when it is short for 'identifier,' so
write 'appID' instead of 'appId'."

These include User-Agent, Accept, Accept-Language, and Accept-Encoding,
and don't do anything for us but make the requests larger.

With this removal, the header drops to about 162 bytes, saving 219 bytes
on top of the 32 in the previous commit, or about 50%.

https://trac.torproject.org/projects/tor/ticket/12778

This reduces the encoded length of a session ID from 44 bytes down to
12. If we have 1000 simultaneous users, and they're all idle and polling
at 5 requests/s, then this change saves 1000*(44-12)/5 = 6.4 KB/s.
According to https://trac.torproject.org/projects/tor/ticket/12778, the
header length is about 413 bytes, so the shorter session ID saves about
7% in the header.

Relevant to #12778, we can reduce the size of requests by reducing the
length of the Session-Id. Note that minSessionIdLength in meek-server is
the encoded length. In base64, 8 characters represents 6 bytes or 48
bits. There would have to be about 2^24 simultaneous sessions to have a
probability of collision more than half, so it still seems safe.

This makes console.log work. You still have to disable the headlessness
in order to get the Browser Console to appear.

https://trac.torproject.org/projects/tor/ticket/13335