Measuring Tor's performance as users experience it ================================================== The performance (latency and throughput) that Tor users experience depends on numerous factors and is the subject of current research. In order to evaluate progress in improving Tor's performance, we need to continuously measure how fast Tor really is for our users. The primary purpose of these measurements is to compare how performance evolves over time. This howto uses a trivial SOCKS client to download files of various sizes over the Tor network and write down how long substeps take. This setup can be extended by Tor controllers that log which circuits were used and controllers that influence guard node selection. The following configuration parameters are used: - 50 KB file, downloaded every 5 minutes, timeout of 4:55 minutes, MaxCircuitDirtiness of 1 minute, SOCKS port 9020, no entry guards - 1 MB file, downloaded every 30 minutes, timeout of 29:55 minutes, default MaxCircuitDirtiness of 10 minutes, SOCKS port 9021, no entry guards - 5 MB file, downloaded every 60 minutes, timeout of 59:55 minutes, default MaxCircuitDirtiness of 10 minutes, SOCKS port 9022, no entry guards This document explains: 1) how to setup Torperf to measure download times and a Tor controller to log which circuits were used and 2) how to attach another Tor controller to influence guard node selection. 1 Setting up Torperf to log download times and circuits -------------------------------------------------------- Create sample files of correct sizes and make them available somewhere via http: $ dd if=/dev/urandom of=.50kbfile count=1 bs=50K $ dd if=/dev/urandom of=.1mbfile count=1 bs=1M $ dd if=/dev/urandom of=.5mbfile count=1 bs=5M These files are available at http://torperf.torproject.org/.50kbfile (and the other two file names). You may use these files, or you can use your own files instead. Check out and build torperf: $ git clone git://git.torproject.org/torperf ~/torperf $ cd ~/torperf $ make Create data directories for the Tor clients (every download uses its own Tor client) and write torrc files: $ mkdir torclient50kb $ mkdir torclient1mb $ mkdir torclient5mb $ cat <> torclient50kb/torrc DataDirectory . SocksPort 9020 MaxCircuitDirtiness 1 minute UseEntryGuards 0 RunAsDaemon 1 Log notice file log ControlPort 10020 CookieAuthentication 1 EOF $ cat <> torclient1mb/torrc DataDirectory . SocksPort 9021 UseEntryGuards 0 RunAsDaemon 1 Log notice file log ControlPort 10021 CookieAuthentication 1 EOF $ cat <> torclient5mb/torrc DataDirectory . SocksPort 9022 UseEntryGuards 0 RunAsDaemon 1 Log notice file log ControlPort 10022 CookieAuthentication 1 EOF ControlPort and CookieAuthentication are configured so that we can attach controllers to write additional statistics about the chosen paths (and possibly influence guard node selection as described below). Now we need to get TorCtl: $ git clone git://git.torproject.org/pytorctl TorCtl Write a start script to start the Tor clients and the controllers, and execute it (line breaks are only for formatting purposes here): $ cat <> start-tors #!/bin/bash cd ~/torperf/torclient50kb && tor -f ~/torperf/torclient50kb/torrc cd ~/torperf/torclient1mb && tor -f ~/torperf/torclient1mb/torrc cd ~/torperf/torclient5mb && tor -f ~/torperf/torclient5mb/torrc sleep 5 cd ~/torperf/torclient50kb && python ../extra_stats.py --truncate 10020 ../50kb.extradata & cd ~/torperf/torclient1mb && python ../extra_stats.py --truncate 10021 ../1mb.extradata & cd ~/torperf/torclient5mb && python ../extra_stats.py --truncate 10022 ../5mb.extradata & EOF (Omit the --truncate switch if you don't want .extradata files to be truncated once per week to contain only the last 4 days of data.) $ chmod a+x start-tors $ ./start-tors Install the timeout tool: $ sudo apt-get install coreutils Add crontab entries to start the regular requests (line breaks are only for formatting purposes here): $ crontab -e */5 * * * * timeout -s2 295 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9020 /.50kbfile 51200 >> ~/torperf/50kb.data 2>/dev/null; python ~/torperf/truncate-data.py ~/torperf/50kb.data 2,32 * * * * timeout -s2 1795 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9021 /.1mbfile 1048576 >> ~/torperf/1mb.data 2>/dev/null; python ~/torperf/truncate-data.py ~/torperf/1mb.data 8 * * * * timeout -s2 3595 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9022 /.5mbfile 5242880 >> ~/torperf/5mb.data 2>/dev/null; python ~/torperf/truncate-data.py ~/torperf/5mb.data (Omit the truncate-data.py command if you don't want .data files to be truncated once per week to contain only the last 4 days of data.) Note that Python 2.6 or higher is required for the truncate-data.py script. From now on, the three files 50kb.data, 1mb.data, and 5mb.data should accumulate lines like this (50kb.data shown here; line breaks are only for formatting purposes): 1244638801 612981 1244638801 612991 1244638801 613051 1244638801 613126 1244638801 613134 1244638814 282441 1244638814 282466 1244638817 164531 1244638826 335235 84 51466 0 The column headers might be: startsec startusec socketsec socketusec connectsec connectusec negotiatesec negotiateusec requestsec requestusec responsesec responseusec datarequestsec datarequestusec dataresponsesec dataresponseusec datacompletesec datacompleteusec writebytes readbytes didtimeout The 50kb.extradata, 1mb.extradata, and 5mb.extradata files should accumulate lines with KEY=value pairs describing the following properties: CIRC_ID: The circuit id from the control port LAUNCH: The launch time of the circuit PATH: The identity fingerprints of the members of the path BUILDTIMES: Cumulative timer of how long before each hop was completed USED_AT: Closing time for the Torperf stream (should match datacomplete*sec) USED_BY: Control port stream id for this Torperf stream TIMEOUT: The current CircuitBuildTimeout (in ms, pre-rounding) QUANTILE: The current CircuitBuildTimeout quantile cutoff In the event of error, the following fields may be present in lieu of completion data: FAIL_REASONS: Reasons for circuit failure STREAM_FAIL_REASONS: Reasons for stream failure The files will also contain direct copies of the control port event BUILDTIMEOUT_SET. See control-spec.txt for details of this. The USED_AT timestamp will NOT necessarily match the timestamps in the .data file exactly, because additional processing is necessary. The timestamp from the .data files that is closest to USED_AT is "datacompletesec datacompleteusec". If the times are more than a second apart, something is really wrong. You may want to consolidate a .data file and an .extradata file together by calling the consolidate-stats script like this: $ ~/torperf/consolidate-stats DATAFILE EXTRADATAFILE OUTFILE If everything works, you might want to let your system start these Tor clients on system startup. On Debian, this can be done using a crontab entry, too: $ crontab -e @reboot ~/torperf/start-tors 2>/dev/null 2 Setting up Torperf with modified guard node selection -------------------------------------------------------- Instead of using a vanilla Tor client, it is possible to attach a Tor controller that makes the Tor client select its guards in specific ways. This can help answer questions about the influence of guard nodes choices on the overall performance of a Tor client. The entrycons.py script provides that functionality. It will make Tor choose guards from sets of the a) absolute fastest, b) absolute slowest, c) best rated vs. advertised ratio or d) worst rated vs. advertised ratio nodes. The ratio mechanisms provide a way to select the nodes that the bandwidth authorities think stand out in their measurement. Put together with a Tor that picks guards randomly, this allows five different modes of operation for comparison. To use this, you will want to run 15 Tor instances: 50kb, 1mb, 5mb for the slow, fast, slowratio, fastratio, regular cases. You can add a line like this to your start-tors script for each tor instance (linebreak for formatting reasons): cd ~/torperf/DATADIR && python ../entrycons.py CONTROLPORT SPEED >> LOGFILE & Remember that for the regular case, you don't need extra entrycons.py calls. Check out the example_start-tors_fastslow.txt file for how that setup might look. Don't forget to add twelve entries to your crontab in addition to the three entries above (linebreaks for formatting reasons): 1,6,11,16,21,26,31,36,41,46,51,56 * * * * timeout -2 295 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9023 /.50kbfile 51200 >> ~/torperf/slow50kb.data 2>/dev/null 2,7,12,17,22,27,32,37,42,47,52,57 * * * * timeout -2 295 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9026 /.50kbfile 51200 >> ~/torperf/slowratio50kb.data 2>/dev/null 3,8,13,18,23,28,33,38,43,48,53,58 * * * * timeout -2 295 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9029 /.50kbfile 51200 >> ~/torperf/fast50kb.data 2>/dev/null 4,9,14,19,24,29,34,39,44,49,54,59 * * * * timeout -2 295 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9032 /.50kbfile 51200 >> ~/torperf/fastratio50kb.data 2>/dev/null 8,38 * * * * timeout -2 1795 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9024 /.1mbfile 1048576 >> ~/torperf/slow1mb.data 2>/dev/null 14,44 * * * * timeout -2 1795 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9027 /.1mbfile 1048576 >> ~/torperf/slowratio1mb.data 2>/dev/null 20,50 * * * * timeout -2 1795 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9030 /.1mbfile 1048576 >> ~/torperf/fast1mb.data 2>/dev/null 26,56 * * * * timeout -2 1795 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9033 /.1mbfile 1048576 >> ~/torperf/fastratio1mb.data 2>/dev/null 20 * * * * timeout -2 3595 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9025 /.5mbfile 5242880 >> ~/torperf/slow5mb.data 2>/dev/null 32 * * * * timeout -2 3595 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9028 /.5mbfile 5242880 >> ~/torperf/slowratio5mb.data 2>/dev/null 44 * * * * timeout -2 3595 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9031 /.5mbfile 5242880 >> ~/torperf/fast5mb.data 2>/dev/null 56 * * * * timeout -2 3595 ~/torperf/trivsocks-client torperf.torproject.org 127.0.0.1:9034 /.5mbfile 5242880 >> ~/torperf/fastratio5mb.data 2>/dev/null