src: Rename syntax to token

parse: Remove SyntaxType ...

Once again I find myself in the pattern of having a useless 'type' value
that just complicates my life. In this case the correct appproach is to
output the AST directly.

tokenize: Support CR LF new line sequences

tokenize: Add more new line symbols

parse: Ignore whitespace in text, instead join words with spaces ...

This cuts down on complexity as now only the tokenizer deals with
whitespace and makes whitespace no longer a factor in the language.

docs: Clarify that we mean code points, not characters

syntax: Rename SyntaxLocation's column to offset ...

We don't know anything about columns or displaying the code points in
our source file, so be clear it's an offset and not a column.

tokenize: Move syntax to its own module

tokenize: Remove unused list of keywords

parse: Move Syntax to tokenize and replace Token ...

These are basically the same classes so merge them

tokenize: Remove TokenType enum ...

In practice this wasn't used: The parse code always checks the value AND
the type. So just check the value only.

tokenize: Add shebang types

tokenize: Classify language keywords

tokenize: Add booleans

tokenize: Add is_newline and is_whitespace functions

tokenize: Move classify_tokens to before locate_tokens

tokenize: Use type for locating newlines ...

This allows us to test locating with a broader variety of inputs, not
just what we pass to split_tokens.

tokenize: Add NEWLINE token type ...

This isn't used yet but is intended to make location handling easier.

tokenize: Add tokenize wrapper function

tokenize: Add classify_tokens to set token types

tokenize: Add WHITESPACE TokenType ...

This will be used later.

tokenize: Add type value for token type

src: tokenize: Rename symbol to token

tokenize: Add locate_symbols function to add symbol locations ...

This isn't the best soltuion yet, but it's a start.

tokenize: Always specify a location for symbols ...

This gets rid of the default symbol location. The 'default' used now is
just the result of split_symbols.

tokenize: Fix no mutate pragmas

tokenize: Add location to symbols ...

This isn't used right now but will be soon.

src: tokenize: Use symbol objects for symbols ...

This will be useful for when we add line numbers.

src: tokenize: Don't join whitespaces when splitting ...

This will be useful later when we need to track line numbers.

src: tokenize: Add some function comments