src: tokenize: Add some function comments

src: tokenize: Always append current buffer ...

Current will always have data in its buffer, so append it.

Bug found by mutmut.

src: tokenizer: Add beginning of new tokenizer