Correct handing of whitespace

[matklad]

Lexer should use the right unicode property for whicespace. We need rust-unic crate for this. This also needs all sorts of funky tests. Vertical tab, anyone? :)

[matklad]

There's a new version of rust-unic published, it might contain all the necessary stuff:
https://github.com/behnam/rust-unic

[theotherphil]

Is this still an issue? It looks like the following definition is currently being used:

https://docs.rs/ra_rustc_lexer/0.1.0-pre.2/src/ra_rustc_lexer/lib.rs.html#666-689

[matklad]

Indeed this is fixed now, thanks!