user/karsten/collector, branch task-19828 Karsten's collector repository Extend descriptorCutOff by 6 hours. 2020-03-09T11:17:39+00:00 Karsten Loesing karsten.loesing@gmx.net 2020-03-09T11:17:39+00:00 555ad1358b1d475a2b362524ad579886152234de Fixes #19828. 
Fixes #19828.
Bump version to 1.14.1-dev. 2020-01-16T11:16:30+00:00 Karsten Loesing karsten.loesing@gmx.net 2020-01-16T11:16:30+00:00 145045478f3c7aa9286b1244e28a5ab4e5728460 






Prepare for 1.14.1 release.
2020-01-16T11:10:12+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-16T11:10:12+00:00

b5ac823aec79bc95303043657e06a84831d9fdab












Fix smoke test for recent scheduler changes.
2020-01-16T11:10:05+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-16T11:09:36+00:00

0e9aedef745e8ed3ccbe7daffb6892197873ae9d












Fix non-RunOnce mode.
2020-01-16T10:40:35+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-16T10:40:35+00:00

a6359c607438d05e66fa1aa0126af9a44a9bf22e

This was accidentally broken in #32554. The RunOnce mode worked just
fine, but the non-RunOnce mode terminated immediately after scheduling
tasks.





This was accidentally broken in #32554. The RunOnce mode worked just
fine, but the non-RunOnce mode terminated immediately after scheduling
tasks.







Bump version to 1.14.0-dev.
2020-01-15T22:20:01+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-15T22:20:01+00:00

c75f0c781ec45b6b56aedea5738692d0404bdc8a












Prepare for 1.14.0 release.
2020-01-15T22:07:02+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-15T22:07:02+00:00

3a9f05e01f1abe8315d25c780b3aab72376412f5












Update to metrics-lib 2.10.0.
2020-01-15T21:59:26+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-15T21:58:17+00:00

27e41ea7393c24bc72429915dd74c309c2583fa2












Remember processed files between module runs.
2020-01-15T21:56:30+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-07T12:30:28+00:00

741401a0daffda52fd1de81b29b276ed9e939ba5

The three recently added modules to archive Snowflake statistics,
bridge pool assignments, and BridgeDB metrics have in common that they
process any input files regardless of whether they already processed
them before.

The problem is that the input files processed by these modules are
either never removed (Snowflake statistics) or only removed manually
by the operator (bridge pool assignments and BridgeDB statistics).

The effect is that non-recent BridgeDB metrics and bridge pool
assignments are being placed in the indexed/recent/ directory in the
next execution after they are deleted for being older than 72 hours.
The same would happen with Snowflake statistics after the operator
removes them from the out/ directory.

The fix is to use a state file containing file names of previously
processed files and only process a file not found in there. This is
the same approach as taken for bridge descriptor tarballs.





The three recently added modules to archive Snowflake statistics,
bridge pool assignments, and BridgeDB metrics have in common that they
process any input files regardless of whether they already processed
them before.

The problem is that the input files processed by these modules are
either never removed (Snowflake statistics) or only removed manually
by the operator (bridge pool assignments and BridgeDB statistics).

The effect is that non-recent BridgeDB metrics and bridge pool
assignments are being placed in the indexed/recent/ directory in the
next execution after they are deleted for being older than 72 hours.
The same would happen with Snowflake statistics after the operator
removes them from the out/ directory.

The fix is to use a state file containing file names of previously
processed files and only process a file not found in there. This is
the same approach as taken for bridge descriptor tarballs.







Update copyright to 2020.
2020-01-15T20:36:34+00:00

Karsten Loesing
karsten.loesing@gmx.net

2020-01-15T20:36:34+00:00

d2a74b676a0d1c8563638ca3607a866b95877949