Skip to content
Commit c23cffc5 authored by Damian Johnson's avatar Damian Johnson
Browse files

Concatenate strings with bytearray rather than BytesIO

While skimming through the python docs I came across a note suggesting
bytearray for string concatenation...

  https://docs.python.org/3/faq/programming.html#what-is-the-most-efficient-way-to-concatenate-many-strings-together

Gave this a whirl and it's not only more readable, but about 10% faster
too...

============================================================
concat_with_bytesio.py
============================================================

import io
import time

start = time.time()
full_bytes = io.BytesIO()

for i in range(10000000):
  full_bytes.write(b'hiiiiiiiiiiiiiiiiiiiii')

result = full_bytes.getvalue()
print('total length: %i, took %0.2f seconds' % (len(result), time.time() - start))

============================================================
concat_with_bytearray.py
============================================================

import time

start = time.time()
full_bytes = bytearray()

for i in range(10000000):
  full_bytes += b'hiiiiiiiiiiiiiiiiiiiii'

result = bytes(full_bytes)
print('total length: %i, took %0.2f seconds' % (len(result), time.time() - start))

============================================================
Results
============================================================

% python2 --version
Python 2.7.12

% python3 --version
Python 3.5.2

% for ((n=0;n<3;n++)); do python2 concat_with_bytearray.py; done
total length: 220000000, took 1.18 seconds
total length: 220000000, took 1.19 seconds
total length: 220000000, took 1.17 seconds

% for ((n=0;n<3;n++)); do python3 concat_with_bytearray.py; done
total length: 220000000, took 1.02 seconds
total length: 220000000, took 0.99 seconds
total length: 220000000, took 1.03 seconds

% for ((n=0;n<3;n++)); do python2 concat_with_bytesio.py; done
total length: 220000000, took 1.40 seconds
total length: 220000000, took 1.38 seconds
total length: 220000000, took 1.38 seconds

% for ((n=0;n<3;n++)); do python3 concat_with_bytesio.py; done
total length: 220000000, took 1.18 seconds
total length: 220000000, took 1.19 seconds
total length: 220000000, took 1.20 seconds
parent 72ba79d8
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment