- Adds tests for correctness of tokenizer (which also would have shown shlex
wouldn't have worked).
- Should conform with the original's behavior, though we won't be able to match
the error messages without a lot of extra work as a non-regexp version is
likely a no go on python due to speed.
