<feed xmlns='http://www.w3.org/2005/Atom'>
<title>user/karsten/collector, branch task-32890</title>
<subtitle>Karsten's collector repository</subtitle>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/'/>
<entry>
<title>Remember processed files between module runs.</title>
<updated>2020-01-07T14:44:26+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2020-01-07T12:30:28+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=6b54e950c15c0674e4e131de2f65840e0e9344c2'/>
<id>6b54e950c15c0674e4e131de2f65840e0e9344c2</id>
<content type='text'>
The three recently added modules to archive Snowflake statistics,
bridge pool assignments, and BridgeDB metrics have in common that they
process any input files regardless of whether they already processed
them before.

The problem is that the input files processed by these modules are
either never removed (Snowflake statistics) or only removed manually
by the operator (bridge pool assignments and BridgeDB statistics).

The effect is that non-recent BridgeDB metrics and bridge pool
assignments are being placed in the indexed/recent/ directory in the
next execution after they are deleted for being older than 72 hours.
The same would happen with Snowflake statistics after the operator
removes them from the out/ directory.

The fix is to use a state file containing file names of previously
processed files and only process a file not found in there. This is
the same approach as taken for bridge descriptor tarballs.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The three recently added modules to archive Snowflake statistics,
bridge pool assignments, and BridgeDB metrics have in common that they
process any input files regardless of whether they already processed
them before.

The problem is that the input files processed by these modules are
either never removed (Snowflake statistics) or only removed manually
by the operator (bridge pool assignments and BridgeDB statistics).

The effect is that non-recent BridgeDB metrics and bridge pool
assignments are being placed in the indexed/recent/ directory in the
next execution after they are deleted for being older than 72 hours.
The same would happen with Snowflake statistics after the operator
removes them from the out/ directory.

The fix is to use a state file containing file names of previously
processed files and only process a file not found in there. This is
the same approach as taken for bridge descriptor tarballs.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on metrics-lib's log package (4/4).</title>
<updated>2019-11-25T16:02:07+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-23T17:07:41+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=8263cc7bdbb0a632f12a84fb2051dd9a25c28142'/>
<id>8263cc7bdbb0a632f12a84fb2051dd9a25c28142</id>
<content type='text'>
 - Remove package-internal abstract class.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
 - Remove package-internal abstract class.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on metrics-lib's log package (3/4).</title>
<updated>2019-11-25T16:01:09+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-23T16:55:17+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=c11b61465a644940559b97c93a769fda84287970'/>
<id>c11b61465a644940559b97c93a769fda84287970</id>
<content type='text'>
 - Remove package-internal interfaces InternalLogDescriptor and
   InternalWebServerAccessLog.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
 - Remove package-internal interfaces InternalLogDescriptor and
   InternalWebServerAccessLog.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on metrics-lib's log package (2/4).</title>
<updated>2019-11-25T16:01:00+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-23T16:43:45+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=ea1b1b4f6ab11e7ac933b0e00d5f8c040e4cc11e'/>
<id>ea1b1b4f6ab11e7ac933b0e00d5f8c040e4cc11e</id>
<content type='text'>
 - Remove unused code.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
 - Remove unused code.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on metrics-lib's log package (1/4).</title>
<updated>2019-11-25T16:00:44+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-23T16:13:22+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=859476ecaec2164e0d84bbba4377da11c90034b2'/>
<id>859476ecaec2164e0d84bbba4377da11c90034b2</id>
<content type='text'>
 - Copy types from metrics-lib to this code base.
 - Update package and import statements.
 - Copy remaining parts of metrics-lib's FileType.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
 - Copy types from metrics-lib to this code base.
 - Update package and import statements.
 - Copy remaining parts of metrics-lib's FileType.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on DescriptorIndexCollector.</title>
<updated>2019-11-22T17:49:46+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-22T17:49:46+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=cc3aa57e57cdcd632a6a8a360f08880f0ec57242'/>
<id>cc3aa57e57cdcd632a6a8a360f08880f0ec57242</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove dependency on metrics-lib's internal package.</title>
<updated>2019-11-22T17:01:11+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-22T16:54:26+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=5a0e6be21c2de4b35e4111364b3ecd7caf424164'/>
<id>5a0e6be21c2de4b35e4111364b3ecd7caf424164</id>
<content type='text'>
The only functionality contained in metrics-lib's internal package is
file (de-)compression, which in turn uses a third-party library that
we're using anyway. This is a weak reason for depending on our own
library for this functionality. Removing this dependency will make it
easier to make changes to our library in the future.

The new FileType class is based on a copy of the same enum type in
metrics-lib without @since tags and without methods that we don't use.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The only functionality contained in metrics-lib's internal package is
file (de-)compression, which in turn uses a third-party library that
we're using anyway. This is a weak reason for depending on our own
library for this functionality. Removing this dependency will make it
easier to make changes to our library in the future.

The new FileType class is based on a copy of the same enum type in
metrics-lib without @since tags and without methods that we don't use.
</pre>
</div>
</content>
</entry>
<entry>
<title>Stop checking and reloading configuration file.</title>
<updated>2019-11-22T11:03:57+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-20T12:03:25+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=de10fcd5b3bc9d29c0e2a011728e34fbe9b0fd5d'/>
<id>de10fcd5b3bc9d29c0e2a011728e34fbe9b0fd5d</id>
<content type='text'>
Removes a deprecation warning and simplifies code.

Implements #32554.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Removes a deprecation warning and simplifies code.

Implements #32554.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove String[][] as configuration value type.</title>
<updated>2019-11-21T08:49:49+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-21T08:49:49+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=edf505ae06479b86a87e81ea0a2266a1cad2f7fc'/>
<id>edf505ae06479b86a87e81ea0a2266a1cad2f7fc</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove final modifier from private method.</title>
<updated>2019-11-20T15:44:25+00:00</updated>
<author>
<name>Karsten Loesing</name>
<email>karsten.loesing@gmx.net</email>
</author>
<published>2019-11-20T15:44:25+00:00</published>
<link rel='alternate' type='text/html' href='https://gitweb.torproject.org/user/karsten/collector.git/commit/?id=ccdad654f27aa90cd9c86273c5a7f03871e2c3ec'/>
<id>ccdad654f27aa90cd9c86273c5a7f03871e2c3ec</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
