summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* test the exit-policy getinfo commandexit-policySebastian Hahn2016-03-29
|
* Update test_get_exit_policy slightlySebastian Hahn2016-03-28
|
* Inverted skip checksDamian Johnson2016-03-27
| | | | Baka, skipped on non-osx rather than osx. Oops.
* Skip couple manual unit tests on OSXDamian Johnson2016-03-27
| | | | | | | OSX is still proving an issue for some of these assertions. Skipping the tests for now... https://trac.torproject.org/projects/tor/ticket/18660
* Expand IPv6 addresses from the proc moduleDamian Johnson2016-03-27
| | | | | | | | | Sebastian's running into unit test failures because the socket module gives him a different but equivalent address.. https://trac.torproject.org/projects/tor/ticket/18659 Normalizing addresses the module gives us.
* Use a real random generator instead of time.time()8865Sebastian Hahn2016-03-27
| | | | Fixes spurious issues in the integration tests, ticket #8865.
* Python3 regression when reading descriptorsDamian Johnson2016-03-27
| | | | | | | Oops! Recent commit broke all descriptor parsing for python3, caught by toralf... https://trac.torproject.org/projects/tor/ticket/18658
* Don't provide man's '--encoding' argument on OSXDamian Johnson2016-03-27
| | | | | | Turns out the argument isn't available on OSX. Caught by Sebastian... https://trac.torproject.org/projects/tor/ticket/18660
* Explicit non-ascii content validationDamian Johnson2016-03-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Few hours ago a relay started publishing a malformed extrainfo descriptor... https://trac.torproject.org/projects/tor/ticket/18656 This is fine, bugs happen. This is why we check for malformed content. But this one cost me a few hours since the non-ascii content then caused DocTor to choke, providing me a useless stacktrace... Traceback (most recent call last): File "./descriptor_checker.py", line 99, in <module> main() File "./descriptor_checker.py", line 57, in main log.warn("Unable to retrieve the %s: %s" % (descriptor_type, query.error)) UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 76: ordinal not in range(128) Instead now it provides... source: http://86.59.21.38:80/tor/extra/all.z time: 03/26/2016 16:30 error: 'dirreq-v3-reqs' line had non-ascii content: S?=4026597208,S?=4026597208,S?=4026597208,S?=4026597208,S?=4026597208,S?=4026597208,??=4026591624,6?=4026537520,6?=4026537520,6?=4026537520,us=8 Non-ascii strings are toxic for systems they're in. They break printing, logging, exception handling, and anything else that touches them unless they're handled specially. Changing Stem to explicitly validate that content is ascii, and provide the user with escaped strings in those exceptions.
* Adding the NoEdConsensus flagDamian Johnson2016-03-25
| | | | Dirspec recently got a new flag. Reflecting that.
* Don't mention Vidalia on FAQDamian Johnson2016-03-12
| | | | | Link is dead and Vidalia's no longer something most of our visitors would be familiar with.
* Fix misspelled terminal text attributeDamian Johnson2016-03-09
| | | | | Misspelled 'HIGHLIGHT' as 'HILIGHT' for an enum. Old name will still work until our 2.0.x release.
* test_shorthand_aliases integ test brokenDamian Johnson2016-03-07
| | | | | | | | | | | | | | | | | | Our unit tests caused us to cache a Mock object rather than a DescriptorDownloader for our singleton. This in turn broke our later integ tests... ====================================================================== ERROR: test_shorthand_aliases ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/runner.py", line 161, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/runner.py", line 178, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/integ/descriptor/remote.py", line 31, in test_shorthand_aliases desc = list(stem.descriptor.remote.get_server_descriptors('9695DFC35FFEB861329B9F1AB04C46397020CE31').run())[0] IndexError: list index out of range
* Deprecated the DescriptorDownloader's get_microdescriptors()Damian Johnson2016-03-07
| | | | | | This methed never worked since it was never actually implemented in tor... https://trac.torproject.org/projects/tor/ticket/9271
* Don't use fallback directories when fetching descriptorsDamian Johnson2016-03-06
| | | | | | | | | Bah. Added fallback directories into our default pool to lessen load on dirauths, but they can't serve extrainfo descriptors... https://trac.torproject.org/projects/tor/ticket/18489 This is usually fine for tor clients, but not for us.
* Use dirauth for last requestDamian Johnson2016-03-06
| | | | | | When request descriptors we default to using a directory authority for our last attempt if earlier tries failed. Mistakenly we were including fallback directories in this final selection.
* Key fallback directories on fingerprintDamian Johnson2016-03-06
| | | | | | | | | | Internal detail, but in our config file we keyed fallback directories on their nickname. This made the config more readable but is a no-go since nicknames are non-unique. At present tor has two Unnamed fallback directories and as a result we could only have information about one. This is all internal details. Only impact for users is that we're no longer missing one of the fallbacks.
* Shorthand functions for stem.descriptor.remoteDamian Johnson2016-03-06
| | | | | | | | | | | | | | | | | | | | | | Damn stem.descriptor.remote is useful! This has become our best method of fetching descriptors but its usage is a bit formulaic... import stem.descriptor.remote downloader = stem.descriptor.remote.DescriptorDownloader() for desc in downloader.get_server_descriptors(): ... do stuff... Why make users get a downloader object? It's useful for power users that need to customize how descriptors are downloaded, but 95% of users don't care. They simply want the bloody descriptors, so providing shorthand methods that do just that! import stem.descriptor.remote for desc in stem.descriptor.remote.get_server_descriptors(): ... do stuff...
* Normalize accept6/reject6 into normal exit policy rulesDamian Johnson2016-03-05
| | | | | On reflection accept6/reject6 is just syntatic sugar (and pretty worthless syntatic sugar at that). Normalizing these rules into normal exit policies.
* Support accept/reject6, *4, and *6 in exit policiesDamian Johnson2016-03-04
|\ | | | | | | | | | | | | | | Tor has expanded its exit policies quite a bit to provide more flexability in how IPv4 and IPv6 policies can be specified. Thanks to teor for the explanation! https://trac.torproject.org/projects/tor/ticket/16103#comment:5
| * Expand *4 and *6 exit poicy wildcardsDamian Johnson2016-03-04
| | | | | | | | | | | | Like accept6 and reject6, we have a couple new wildcards to specify all IPv4 or IPv6 addresses. This is akin to being a '/0' so simply translating 'em into that. :P
| * Skip accept/reject6 rules with IPv4 addressesDamian Johnson2016-03-04
| | | | | | | | | | | | | | | | | | Damn. These are invalid and tor should outright reject them but according to the manual they're just skipped... Using an IPv4 address with accept6 or reject6 is ignored and generates a warning. That was a mistake but oh well. It's minor.
| * ExitPolicy support for accept6 and reject6 rulesDamian Johnson2016-03-04
|/ | | | | | | Thanks to teor for explaining these! Adding support for these exit policy rule types... https://trac.torproject.org/projects/tor/ticket/16103#comment:5
* Tor manual now has TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTriesDamian Johnson2016-02-28
| | | | | Resyncing cached data. We can drop a hack from our tests now that https://trac.torproject.org/projects/tor/ticket/17913 is merged.
* Fix broken get_hidden_service_descriptor actionDonncha O'Cearbhaill2016-02-27
| | | | | | | | | The `get_hidden_service_descriptor` function takes an onion address and a list of hidden service descriptors to fetch the hidden service descriptor from. This commit fixes a bug where a space was missing between the onion address and the first SERVER= argument.
* Minor rewording for fallback directory additionDamian Johnson2016-02-24
|
* Integ test failed when tor versions didn't include a spaceDamian Johnson2016-02-24
| | | | | | | Great catch from toralf. When using a tor release rather than the git version test_getinfo_version_parsing fails with... ValueError: '0.2.7.' isn't a properly formatted tor version
* FallbackDirectory unicode normalization issue for python3Damian Johnson2016-02-23
| | | | | | | | | | | | | | | | | | | Need to normalize so both python2 and python3 are happy... ====================================================================== ERROR: test_fallback_directories_from_remote ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.2/dist-packages/mock/mock.py", line 1305, in patched return func(*args, **keywargs) File "/home/atagar/Desktop/stem/test/unit/descriptor/remote.py", line 193, in test_fallback_directories_from_remote fallback_directories = stem.descriptor.remote.FallbackDirectory.from_remote() File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 949, in from_remote elif line.startswith('"'): TypeError: startswith first arg must be bytes or a tuple of bytes, not str ---------------------------------------------------------------------- Ran 6 tests in 0.018s
* DescriptorDownloader TypeError under python3Damian Johnson2016-02-23
| | | | | | | | | | | | | | | | | | | | Oops, python3 collections issue from recent changes... ====================================================================== ERROR: test_persisting_a_consensus ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.2/dist-packages/mock/mock.py", line 1305, in patched return func(*args, **keywargs) File "/home/atagar/Desktop/stem/test/unit/tutorial_examples.py", line 343, in test_persisting_a_consensus exec_documentation_example('persisting_a_consensus.py') File "/home/atagar/Desktop/stem/test/unit/__init__.py", line 24, in exec_documentation_example exec(code) File "/home/atagar/Desktop/stem/docs/_static/example/persisting_a_consensus.py", line 4, in <module> downloader = DescriptorDownloader() File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 404, in __init__ directories = get_authorities().values() + FallbackDirectory.from_cache().values() TypeError: unsupported operand type(s) for +: 'dict_values' and 'dict_values'
* Ignore missing TestingClientBootstrapConsensusAuthorityOnlyMaxDownloadTriesDamian Johnson2016-02-22
| | | | | | | We've had a ticket about this for a couple month now. Lets stop failing our integ tests over it... https://trac.torproject.org/projects/tor/ticket/17913
* Fix couple online integ testsDamian Johnson2016-02-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't run tests requiring our ONLINE target by default so they get a little less love. Fixing the following couple... ====================================================================== FAIL: test_authorities_are_up_to_date ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/runner.py", line 161, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/runner.py", line 178, in wrapped return func(self, *args, **kwargs) File "/home/atagar/Desktop/stem/test/integ/descriptor/remote.py", line 38, in test_authorities_are_up_to_date self.fail("%s isn't a recognized directory authority in stem" % auth.nickname) AssertionError: dannenberg-legacy isn't a recognized directory authority in stem ---------------------------------------------------------------------- ====================================================================== FAIL: test_attributes ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/integ/manual.py", line 198, in test_attributes assert_equal('commandline options', EXPECTED_CLI_OPTIONS, set(manual.commandline_options.keys())) File "/home/atagar/Desktop/stem/test/integ/manual.py", line 190, in assert_equal self.fail("Changed tor's man page? The %s changed as follows...\n\nexpected: %s\n\nactual: %s" % (category, expected, actual)) AssertionError: Changed tor's man page? The commandline options changed as follows... expected: set(['-f FILE', '--ignore-missing-torrc', '--list-fingerprint', '--defaults-torrc FILE', '--allow-missing-torrc', '--nt-service', '--verify-config', '--service remove|start|stop', '--quiet|--hush', '--list-torrc-options', '--service install [--options command-line options]', '--version', '--hash-password PASSWORD', '-h, -help']) actual: set([u'-f FILE', u'--hash-password PASSWORD', u'--ignore-missing-torrc', u'--list-fingerprint', u'--defaults-torrc FILE', u'--allow-missing-torrc', u'--nt-service', u'--verify-config', u'--service remove|start|stop', u'--keygen [--newpass]', u'--list-torrc-options', u'--service install [--options command-line options]', u'--quiet|--hush', u'--version', u'-h, -help']) I'm still getting a couple other failures but one is from #17913 and the rest are spurious.
* Example for using the FallbackDirectoriesDamian Johnson2016-02-21
| | | | | On reflection might as well include the example I just gave teor on #18177 in our pydocs as well.
* Script for caching fallback directoriesDamian Johnson2016-02-21
| | | | Oops! Forgot to add a couple new files to support fallback directories.
* Fallback directory informationDamian Johnson2016-02-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently tor added fallback directory authorities to help clients bootstrap when the dirauths are unavailable... https://trac.torproject.org/projects/tor/ticket/17158 To help support this effort teor asked for a DocTor check to notify us when the directories become unavailable... https://trac.torproject.org/projects/tor/ticket/18177 Great idea but I'm expanding this. Stem now has a FallbackDirectory class with two methods for getting this information... * FallbackDirectory.from_remote() - Reads the latest fallback_dirs.inc from gitweb, providing the latest fallback directories in tor's master branch. * FallbackDirectory.from_cache() - Provides the latest fallback directories Stem has cached. This is only as up-to-date as the Stem release you're using but is quicker and avoids relying on gitweb. Advantages are... * Stem's descriptor.remote module now puts less load on the directory authorities since it uses fallback directories as well. * Running Stem's integ tests with the ONLINE target includes a test that exercises all the fallback directories, notifying us if any are down.
* Updating with new config optionsDamian Johnson2016-02-13
| | | | | | | Just pulling in the latest torrc changes, most notably the addition of ClientUseIPv4 and ClientPreferIPv6DirPort... https://gitweb.torproject.org/tor.git/commit/?id=2d33d19
* GETCONF provides an empty string for CommaList and RouterListDamian Johnson2016-02-06
| | | | | | Workaround for a tor bug... https://trac.torproject.org/projects/tor/ticket/18263
* Don't add 'settings_loaded' to configsDamian Johnson2016-02-06
| | | | | | Our @uses_settings annotation stored and loaded a 'settings_loaded' value to track if it had already lazy loaded content. This is inappropriate - we shouldn't muck with the content of our user's configs.
* Proc unit testing failure under python3Damian Johnson2016-02-02
| | | | | | | | | | | | | | | | | | | Simple bytes/unicode error in our tests. This caused a couple proc tests to fail under python3... ====================================================================== ERROR: test_connections_ipv6 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/stem/util/proc.py", line 370, in connections with open(proc_file_path, 'rb') as proc_file: File "/usr/local/lib/python3.2/dist-packages/mock/mock.py", line 1062, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/usr/local/lib/python3.2/dist-packages/mock/mock.py", line 1128, in _mock_call ret_val = effect(*args, **kwargs) File "/home/atagar/Desktop/stem/test/unit/util/proc.py", line 262, in <lambda> '/proc/net/tcp': io.BytesIO(''), TypeError: 'str' does not support the buffer interface
* Python3 regression in socket's recv_message()Damian Johnson2016-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Oops, shame on me for not testing with python3. Recent log truncation change introduced a regression... ====================================================================== ERROR: test_hs_desc_content_event ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/atagar/Desktop/stem/test/unit/response/events.py", line 877, in test_hs_desc_content_event event = _get_event(HS_DESC_CONTENT_EVENT) File "/home/atagar/Desktop/stem/test/unit/response/events.py", line 458, in _get_event controller_event = mocking.get_message(content) File "/home/atagar/Desktop/stem/test/mocking.py", line 259, in get_message return stem.response.ControlMessage.from_str(content) File "/home/atagar/Desktop/stem/stem/response/__init__.py", line 160, in from_str msg = stem.socket.recv_message(StringIO(content)) File "/home/atagar/Desktop/stem/stem/socket.py", line 601, in recv_message print([b'... %i more lines...' % (len(log_message_lines) - TRUNCATE_LOGS)]) TypeError: unsupported operand type(s) for %: 'bytes' and 'int' ---------------------------------------------------------------------- Thanks to toralf for pointing this out. Trouble is that under python3 bytes doesn't support % formatting at all... >>> b'hi %i' % 5 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for %: 'bytes' and 'int' This is pretty pesky, and something they plan to change... http://legacy.python.org/dev/peps/pep-0461/
* Hook to disable trace logging for cache fetchesDamian Johnson2016-01-31
| | | | | Cache hits generally aren't very interesting and can generate a lot of noise. Providing an option so callers can disable them.
* Truncate trace level loggingDamian Johnson2016-01-31
| | | | | | | | | At the trace level we have request/reply logging. Very useful, but some tor requests like 'GETINFO ns/all' are huge. Truncating responses to ten lines by default. This can be customized via a constant. This reduces the debug log when nyx is run for a few seconds from 1.6M to 80K, mostly because it no longer causes us to dump the full consensus into it.
* Don't log debug messages for multi-line config optionsDamian Johnson2016-01-31
| | | | | | | | | | | | Our config module supports multi-line strings but these cause us to log debug messages like the following... 01/31/2016 12:15:22 [DEBUG] Config entry 'config_options.Bridge.description' is expected to be of the format 'Key Value', defaulting to 'config_options.Bridge.description' -> '' Just a bug where we logged this warning before we had the handling for them. On reflection though this message doesn't have any value so simply dropping it.
* Cache get_hidden_service_conf() responsesDamian Johnson2016-01-31
| | | | | | | | | Nyx calls this quite a bit which is fine, but it bypasses our GETCONF cache. Adding a cache for this similar to our exit policy cache. Even after just a few seconds this caused thousands of calls so pretty important to nyx... % grep 'GETCONF HiddenServiceOptions (runtime' /tmp/dump | wc -l 6943
* Expand caching of GETINFO resultsDamian Johnson2016-01-30
| | | | | Caching several GETINFO queries that can only be changed with a SETCONF. These are queries nyx cares about, but we can expand the list as others crop up.
* Skip udp addresses with a blank destinationDamian Johnson2016-01-30
| | | | | | | | | | | We can filter tcp connections by if the connection is established, but for udp we need to pick something else. Omitting addresses with a blank destination... * 127.0.0.1:53 => 0.0.0.0:0 (udp) * 0.0.0.0:45768 => 0.0.0.0:0 (udp) * 0.0.0.0:5353 => 0.0.0.0:0 (udp) * :::5353 => :::0 (udp) * :::36611 => :::0 (udp)
* Making the resolver argument of get_connections() optionalDamian Johnson2016-01-30
| | | | | | | | | Callers of this function just about always call us with... get_connections(system_resolvers()[0]) No need to mandate this argument. We can make a smart guess at a connection resolver if they don't want to specify one in particular.
* Add function pids_by_user()Damian Johnson2016-01-30
| | | | Simple utility for the system module to get pids for a user.
* Allow stem.util.proc.connection() to query by userDamian Johnson2016-01-30
| | | | | | | | | | Our proc contents provide connection metadata with only two things we can filter by: inode and uid. Thus far we've used inodes to allow lookups by pid (thing we usually want to resolve connections by), but soon I'm gonna have a need for resolution by uid. This also allows the funciton to provide all system connections if no pid or user is provided.
* Add TorTP to examples pageDamian Johnson2016-01-28
| | | | | TorTP's new version uses stem to determine the iptables rules to make. Adding it to our examples page.
* Have get_connections() resolve the pid if not provided with oneDamian Johnson2016-01-28
| | | | | | | Three of our connection resolvers require a pid to function. When provided with a process name instead we raised a ValueError, but we can at least *try* to resolve the name to a pid. If unsuccessful we now raise an IOError. Thanks to toralf for suggesting this change.
generated by cgit v1.2.1 (git 2.18.0) at 2024-02-02 09:55:57 +0000