tests: tokenize: Move syntax tests to new file

tokenize: Move syntax to its own module

tests: parse: Move syntax tests to tokenize ...

This helps with mutmut.

tests: tokenize: Remove reference to NEWLINE

tests: tokenize: Draw classified tokens for testing locations ...

This way things are actually likely to draw newlines.

parse: Move Syntax to tokenize and replace Token ...

These are basically the same classes so merge them

tokenize: Remove TokenType enum ...

In practice this wasn't used: The parse code always checks the value AND
the type. So just check the value only.

tests: tokenize: Fix typo

testss: tokenize: Add documentation on the expected behaviours

tests: tokenize: Give garbled tokens to the classifier ...

This helps prove that the classifier only classifies full tokens, not
substrings.

tokenize: Add shebang types

tokenize: Classify language keywords

tokenize: Add booleans

test: tokenize: Move draw_tokens_list to draw_tokens_to_split ...

While we're at it include the source flattening code.

tests: tokenize: Add a draw_token_classified strategy

tests: tokenize: Use random tokens again for locations ...

By using random tokens and passing that set to the test we now have both
random values, random locations and random types to help prove that the
location pass only uses types and length for locating.

tests: tokenize: Test location using types only

tokenize: Use type for locating newlines ...

This allows us to test locating with a broader variety of inputs, not
just what we pass to split_tokens.

tokenize: Add NEWLINE token type ...

This isn't used yet but is intended to make location handling easier.

tests: tokenize: Add more accurate comments

tests: tokenize: Have draw_token_* use draw_token_random

tests: tokenize: Put generic draws at top of file

tokenize: Add tokenize wrapper function

tests: tokenize: Add comments

tests: tokenize: Run classification with random input

tokenize: Add classify_tokens to set token types

tests: tokenize: Rename draw_token to draw_token_random

tokenize: Add WHITESPACE TokenType ...

This will be used later.

tests: Use random token types for tokens

tokenize: Add type value for token type