Empty tor data directory between integ tests

When authoring our integ tests years ago I had a decision to make: retain tor's data directory between runs or start fresh. From the standpoint of testing best practices this should be a no brainer: start fresh. However, I decided against this because at the time we ran our 'ONLINE' target by default, and losing our cache added a frustrating amount of runtime. We no longer run tests that require network activity by default, and keeping our data directory around adds up over time... % du -h test/data/tor_log 5.3M test/data/tor_log % ls test/data/torrc.orig.* | wc -l 98 I'm about to change our logging runlevel which will raise our log size by an order of magnitude. This is negligible, but if we don't first change our cleaning behavior it'll add up.

Empty tor data directory between integ tests
8b07bca6 · Damian Johnson · a69ebcae · 8b07bca6 · 8b07bca6
Commit 8b07bca6 authored May 27, 2019 by Damian Johnson
--- a/run_tests.py
+++ b/run_tests.py
@@ -223,6 +223,7 @@ def main():
    test.task.CLEAN_PYC,
    test.task.UNUSED_TESTS,
    test.task.IMPORT_TESTS,
+    test.task.REMOVE_TOR_DATA_DIR if args.run_integ else None,
    test.task.PYFLAKES_TASK if not args.specific_test else None,
    test.task.PYCODESTYLE_TASK if not args.specific_test else None,
  )

--- a/test/task.py
+++ b/test/task.py
@@ -18,6 +18,7 @@
  |- PYFLAKES_VERSION - checks our version of pyflakes
  |- PYCODESTYLE_VERSION - checks our version of pycodestyle
  |- CLEAN_PYC - removes any *.pyc without a corresponding *.py
+  |- REMOVE_TOR_DATA_DIR - removes our tor data directory
  |- IMPORT_TESTS - ensure all test modules have been imported
  |- UNUSED_TESTS - checks to see if any tests are missing from our settings
  |- PYFLAKES_TASK - static checks
@@ -27,6 +28,7 @@
 import os
 import platform
 import re
+import shutil
 import sys
 import time
 import traceback
@@ -136,6 +138,20 @@ def _clean_orphaned_pyc(paths):
  return ['removed %s' % path for path in stem.util.test_tools.clean_orphaned_pyc(paths)]


+def _remove_tor_data_dir():
+  """
+  Empties tor's data directory.
+  """
+
+  config_test_dir = CONFIG['integ.test_directory']
+
+  if config_test_dir and os.path.exists(config_test_dir):
+    shutil.rmtree(config_test_dir, ignore_errors = True)
+    return 'done'
+  else:
+    return 'skipped'
+
+
 def _import_tests():
  """
  Ensure all tests have been imported. This is important so tests can
@@ -316,6 +332,7 @@ MOCK_VERSION = ModuleVersion('mock version', ['unittest.mock', 'mock'], stem.pre
 PYFLAKES_VERSION = ModuleVersion('pyflakes version', 'pyflakes')
 PYCODESTYLE_VERSION = ModuleVersion('pycodestyle version', ['pycodestyle', 'pep8'])
 CLEAN_PYC = Task('checking for orphaned .pyc files', _clean_orphaned_pyc, (SRC_PATHS,), print_runtime = True)
+REMOVE_TOR_DATA_DIR = Task('emptying our tor data directory', _remove_tor_data_dir)
 IMPORT_TESTS = Task('importing test modules', _import_tests, print_runtime = True)

 UNUSED_TESTS = Task('checking for unused tests', _check_for_unused_tests, [(