summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Updates to exit list spectask-29624Iain R. Learmonth2019-04-09
|
* Temp commit: tweak exit list spec based on discussion.Karsten Loesing2019-03-08
|
* Temp commit: add first draft of exit list spec.Karsten Loesing2019-03-04
|
* Add three new @type annotations.Karsten Loesing2019-01-26
| | | | Resolves #28615.
* Include op-ab in OnionPerf graphs.Karsten Loesing2019-01-24
| | | | | | | | Turns out that op-ab's domain name matches our '%.onion%' regex, that we're using to distinguish public from onion server requests. Trying a bit harder to distinguish the two. Related to #29107.
* Update news.json to version 308 of doc/MetricsTimeline.Karsten Loesing2019-01-11
|
* Simplify Rserve setup.Karsten Loesing2019-01-11
|
* Leave gaps for missing data.Karsten Loesing2019-01-11
|
* Add levels = NULL to col_factor().Karsten Loesing2019-01-11
| | | | | This is not required for readr_1.3.0 which runs locally here but required for readr_1.1.1 which runs on the server.
* Switch to readr's read_csv() everywhere.Karsten Loesing2019-01-10
|
* Make write_* functions obsolete.Karsten Loesing2019-01-10
| | | | | | | | | | | | In most cases these functions would call their prepare_* equivalents, possibly tweak the result, and write it to a .csv file. This patch moves all those tweaks to the prepare_* functions, possibly reverts them in the plot_* functions, and makes the write_* functions obsolete. The result is not only less code. We're also going to find bugs in written .csv files sooner, because the same code is now run for writing graph files, and the latter happens much more often.
* Simplify plot_webstats_tb_locale function.Karsten Loesing2019-01-10
|
* Split up huge plot_userstats function.Karsten Loesing2019-01-10
| | | | | | | The mere size of this function made it hard to impossible to refactor things to using more recent R packages dplyr and tidyr. Now there are four plot_userstats_* functions with accompanying prepare_userstats_* that make the corresponding write_userstats_* functions really small.
* Update news.json to version 307 of doc/MetricsTimeline.Karsten Loesing2019-01-07
|
* Stop calling censorship detector BETA.Karsten Loesing2019-01-07
|
* Properly skip previously imported webstats files.Karsten Loesing2019-01-07
| | | | | | | | | | | | | | | | | Turns out we never skipped previously imported webstats files due to two bugs: 1. While building a list of previously imported webstats files we reassembled their file names as ${server}_${site}_* rather than ${site}_${server}_* which was the file name format we chose in an earlier version of the CollecTor module. 2. When checking whether a given webstats file already exists in the database we compared the full file name to the reassembled file name from the database with ${server} being truncated to 32 characters. This commit fixes both bugs.
* Stop hard-coding versions.Karsten Loesing2019-01-07
|
* Fix by-country CSV files.Karsten Loesing2018-12-29
| | | | Fixes #28945.
* Fix Traffic link on start page.Karsten Loesing2018-12-29
|
* Rewrite advbwdist's aggregate.R in Java.Karsten Loesing2018-12-23
| | | | | | This is yet another step torwards making the daily update Java-only. Implements #28801.
* Remove two unused R files from censorship detector.Karsten Loesing2018-12-20
| | | | Still part of #21588.
* Rewrite censorship detector in Java.Karsten Loesing2018-12-20
| | | | | | | This allows us to remove the last remaining Python parts from the daily updater. Implements #21588.
* Access userstats database from Java only.Karsten Loesing2018-12-20
| | | | | | | Previously, we used Java to write .sql files, imported them using psql, and afterwards made queries via psql. Now we're using Java to interact with the database directly. This is another step towards making the daily updater Java-only.
* Re-add missing COMMIT commands to bwhist module.Karsten Loesing2018-12-20
| | | | | | | | | | | | | | | | | | | | | | Last month, in commit f8fa108 where we modernized the legacy module and renamed it to bwhist, we split up the closeConnection() into one method commit() to commit changes and another method closeConnection() to close the connection. However, we somehow forgot to invoke the commit() method. This had two effects: 1. Newly added data was not made persistent in the database. This lead to a moving window of roughly one week for new data and an increasing gap between the last committed data and this 1-week window. 2. The result of aggregating newly added data was not made persistent. So, even after fixing the first issue above, we accumulated newly added data, rather than only keeping the most recent two weeks. This made the database slower over time. This change adds two commit() calls at the right places.
* Remove bandwidth and bwhist-flags graphs.Karsten Loesing2018-12-20
| | | | | | | | | | | | The "Advertised and consumed bandwidth by relay flags" graph now contains everything that's contained in the "Total relay bandwidth" and the "Consumed bandwidth by Exit/Guard flag combination" graphs. Removing these two graphs as obsolete. Also update documentation for the newly deployed "Advertised and consumed bandwidth by relay flags" graph. Part of #28353.
* Tweak Advertised and consumed bandwidth by relay flag graph.Karsten Loesing2018-12-20
| | | | | | | | | | This graph now contains everything that's contained in the Total relay bandwidth and the Consumed bandwidth by Exit/Guard flag combination graph. Removing those graphs will be done in a separate commit. Part of #28353.
* Document another change to OnionPerf graphs.Karsten Loesing2018-12-20
| | | | Still related to #28603.
* Document changes to OnionPerf graphs.Karsten Loesing2018-12-20
| | | | Still related to #28603.
* Remove Torperf/OnionPerf plots with all sources.Karsten Loesing2018-12-20
| | | | | | | | | | | | | | OnionPerf results look to be comparable over time, but between vantage points there are systematic deltas between the results. The "all" plots show rises and falls where they actually don't exist, it's just that a particular vantage point was offline so the average of the two remaining moves noticeably. In this commit we remove the source parameter from these graphs and always include all sources separately in the graph, but not a combination of all measurements together. Implements #28603.
* Take out "We're hiring" link from start page.Karsten Loesing2018-12-19
|
* Use readr to speed up drawing graphs.Karsten Loesing2018-12-19
| | | | | | | | | | | | | | | | | | | | | | | | Over two years ago, in commit 1f90b72 from October 2016, we made our user graphs faster by avoiding to read the large .csv file on demand. Instead we read it once as part of the daily update, saved it to disk as .RData file using R's save() function, and loaded it back to memory using R's load() function when drawing a graph. This approach worked okay. It just had two disadvantages: 1. We had to write a small amount of R code for each graph type, which is why we only did it for graphs with large .csv files. 2. Running these small R script as part of the daily update made it harder to move away from Ant towards a Java-only execution model. The new approach implemented in this commit uses read_csv() fromt the readr package which reads CSV files several times faster than read.csv(). Requires installing the readr package from CRAN, which is available on Debian in stretch-backports and later as r-cran-readr. Implements #28799.
* Document changes to the totalcw graph.Karsten Loesing2018-12-15
| | | | Still part of #28137, #28328, and #28352.
* Add consensuses to totalcw graph.Karsten Loesing2018-12-15
| | | | Implements #28352.
* Break down totalcw numbers by Guard/(Bad)Exit flags.Karsten Loesing2018-12-15
| | | | | | Requires updating the vote table and the totalcw view in the database. Implements #28328.
* Only include Running relays in totalcw graph.Karsten Loesing2018-12-15
| | | | | | | | Previously we included measured bandwidths of all relays in a vote in the totalcw graph. Now we only include relays with the Running flag in the vote. Implements #28137.
* Remove unused parts of totalcw module.Karsten Loesing2018-12-15
| | | | | | Requires updating the vote table of the database. Part of #28137, #28328, and #28352.
* Schedule changes related to #28353.Karsten Loesing2018-12-05
| | | | | | | | In roughly two weeks from now we're going to update one bandwidth graph and remove two obsolete ones. Similar to commit b14dbc3, announce this change now, so that any folks using our CSV files get the chance to update their tools.
* Schedule changes related to #28603.Karsten Loesing2018-12-05
| | | | | | | | | In roughly two weeks from now we're going to remove source parameters and output rows with aggregates over all sources from all Torperf/OnionPerf related graphs. Announce this change now, so that any folks using our CSV files get the chance to update their tools.
* Update Reproducible Metrics document.Karsten Loesing2018-11-30
| | | | Reflects code changes made in #28116 and #28305.
* Include client numbers with fractions up to 110%.Karsten Loesing2018-11-30
| | | | | | | | | Turns out that almost all relays report directory-request statistics these days, including a small number of relays that temporarily drop out of the consensus. We're now accepting up to 10% of those additional statistics. See #28305 for more details.
* Stop signing jars.Karsten Loesing2018-11-28
| | | | Implements #28584.
* Also update module name for init-ipv6servers.sql.Karsten Loesing2018-11-22
| | | | Part of #28116.
* Update module name for tordir.sql.Karsten Loesing2018-11-22
| | | | Part of #28116.
* Document versions graph better.Karsten Loesing2018-11-22
| | | | Fixes #28462.
* Rename ipv6servers module to servers.Karsten Loesing2018-11-14
| | | | Part of #28116.
* Modernize legacy module and rename it to bwhist.Karsten Loesing2018-11-14
| | | | | | | | | | | | | | | | | | Changes include using similar mechanisms for configuration, calling the database aggregation function, querying the database, and writing results as we're using in the ipv6servers and other modules. Configuration options can now be changed via the following Java properties: bwhist.descriptors bwhist.database bwhist.history bwhist.output The legacy.config file, if one exists, will be ignored. Part of #28116.
* Remove long unused code from legacy module.Karsten Loesing2018-11-14
| | | | | | | | | | | | | | | | This includes the lock file, the option to write raw output files for importing into the database, and a couple boolean config options that have always been true. Required changes to existing legacy.config (removals): ImportDirectoryArchives KeepDirectoryArchiveImportHistory WriteRelayDescriptorDatabase WriteRelayDescriptorsRawFiles RelayDescriptorRawFilesDirectory Part of #28116.
* Remove advbw column from bandwidth.csv.Karsten Loesing2018-11-14
| | | | | | | | | | | | | | | | | | | | | | Instead use advbw data from ipv6servers module. As a result, we can stop aggregating advertised bandwidths in the legacy module. Required schema changes to live tordir databases: DROP VIEW stats_bandwidth; CREATE VIEW stats_bandwidth [...] CREATE OR REPLACE FUNCTION refresh_all() [...] DROP FUNCTION refresh_bandwidth_flags(); DROP FUNCTION refresh_relay_statuses_per_day(); DROP TABLE relay_statuses_per_day; DROP TABLE bandwidth_flags; DROP TABLE consensus; DROP FUNCTION delete_old_descriptor(); DROP TABLE descriptor; Part of #28116.
* Stop generating servers.csv in legacy module.Karsten Loesing2018-11-14
| | | | | | | | | | | | | | | | | | | | | | | | | Also stop importing bridge network size statistics into the database. Required changes to existing legacy.config (removals): ImportSanitizedBridges SanitizedBridgesDirectory KeepSanitizedBridgesImportHistory WriteBridgeStats Required schema changes to live tordir databases: DROP VIEW stats_servers; CREATE OR REPLACE FUNCTION refresh_all() [...] DROP TABLE bridge_network_size; DROP FUNCTION refresh_relay_versions(); DROP FUNCTION refresh_relay_platforms(); DROP FUNCTION refresh_network_size(); DROP TABLE relay_versions; DROP TABLE relay_platforms; DROP TABLE relay_countries; DROP TABLE network_size; Part of #28116.
* Fix ipv6servers unit tests.Karsten Loesing2018-11-14
|
generated by cgit v1.2.1 (git 2.18.0) at 2024-02-02 09:33:04 +0000