parse: Move Syntax to tokenize and replace Token ...

These are basically the same classes so merge them

tokenize: Remove TokenType enum ...

In practice this wasn't used: The parse code always checks the value AND
the type. So just check the value only.

tokenize: Add shebang types

tokenize: Classify language keywords

tokenize: Add booleans

tokenize: Add is_newline and is_whitespace functions

tokenize: Move classify_tokens to before locate_tokens

tokenize: Use type for locating newlines ...

This allows us to test locating with a broader variety of inputs, not
just what we pass to split_tokens.

tokenize: Add NEWLINE token type ...

This isn't used yet but is intended to make location handling easier.

tokenize: Add tokenize wrapper function

tokenize: Add classify_tokens to set token types

tokenize: Add WHITESPACE TokenType ...

This will be used later.

tokenize: Add type value for token type

src: tokenize: Rename symbol to token

tokenize: Add locate_symbols function to add symbol locations ...

This isn't the best soltuion yet, but it's a start.

tokenize: Always specify a location for symbols ...

This gets rid of the default symbol location. The 'default' used now is
just the result of split_symbols.

tokenize: Fix no mutate pragmas

tokenize: Add location to symbols ...

This isn't used right now but will be soon.

src: tokenize: Use symbol objects for symbols ...

This will be useful for when we add line numbers.

src: tokenize: Don't join whitespaces when splitting ...

This will be useful later when we need to track line numbers.

src: tokenize: Add some function comments

src: tokenize: Always append current buffer ...

Current will always have data in its buffer, so append it.

Bug found by mutmut.

src: tokenizer: Add beginning of new tokenizer