[Snek] [keith-packard/snek] e85084: Return invalid token on lex error

[Keith Packard]

  Branch: refs/heads/master
  Home:   https://github.com/keith-packard/snek
  Commit: e85084dffbabca30e83d6913f968f50fa2a2ffce
      https://github.com/keith-packard/snek/commit/e85084dffbabca30e83d6913f968f50fa2a2ffce
  Author: Keith Packard <keithp at keithp.com>
  Date:   2020-02-15 (Sat, 15 Feb 2020)

  Changed paths:
    M snek-lex.c

  Log Message:
  -----------
  Return invalid token on lex error

When the lexer detects an error, such as an invalid character sequence
or out of memory, return an invalid token instead of TOKEN_NONE, which
just causes the parser to lex another token.

We'll use FIRST_NON_TERMINAL as that's never a valid token.

Reported-by: Helmut Grohne <helmut at subdivi.de>
Signed-off-by: Keith Packard <keithp at keithp.com>


  Commit: 2f00cdcd5679cf0d6acf980ff62319bc6ac01fc9
      https://github.com/keith-packard/snek/commit/2f00cdcd5679cf0d6acf980ff62319bc6ac01fc9
  Author: Keith Packard <keithp at keithp.com>
  Date:   2020-02-15 (Sat, 15 Feb 2020)

  Changed paths:
    M test/Makefile
    A test/lex-bang.py

  Log Message:
  -----------
  Add test for lexer returning error on invalid token

'!' is not a valid token, so the lexer should return an error when it
occurs in the input.

Reported-by: Helmut Grohne <helmut at subdivi.de>
Signed-off-by: Keith Packard <keithp at keithp.com>


  Commit: 6fa5b7875fb226f409a9ece16562bf070025d707
      https://github.com/keith-packard/snek/commit/6fa5b7875fb226f409a9ece16562bf070025d707
  Author: Keith Packard <keithp at keithp.com>
  Date:   2020-02-15 (Sat, 15 Feb 2020)

  Changed paths:
    M doc/snek.adoc

  Log Message:
  -----------
  doc: '!' is not the not operator, not is

Python uses words for boolean operators instead of C's punctuation.

Reported-by: Helmut Grohne <helmut at subdivi.de>
Signed-off-by: Keith Packard <keithp at keithp.com>


  Commit: 7f37a5801091cfd722d84c2139d8a11263f5feda
      https://github.com/keith-packard/snek/commit/7f37a5801091cfd722d84c2139d8a11263f5feda
  Author: Keith Packard <keithp at keithp.com>
  Date:   2020-02-15 (Sat, 15 Feb 2020)

  Changed paths:
    M doc/snek.adoc

  Log Message:
  -----------
  doc: Add a bunch of missing operators

and, or, is, is not, in, not in were all missing from the language
reference.

Signed-off-by: Keith Packard <keithp at keithp.com>


Compare: https://github.com/keith-packard/snek/compare/ea1c97a275d4...7f37a5801091