About Keywords and Reserved words
In my previous post I forgot to describe what happened to keywords and reserved words in new version, and how their treatment had changed. These terms are mentioned in the caption of one of the sections but I forgot to actually describe the new functionality. So here it is.
In the previous versions the Grammar class had a Keywords field which was a string list. The programmer had to explicitly add the language keywords to the list, so that they could be highlighted in the code editor, for ex. It was a bit superflous as each of the keywords was already used in the grammar (and therefore Irony could discover them), so what's the point of listing them again? In addition, there was some confusion about what this list should contain - keywords or reserved words only. Now the issue had been clarified. Irony now maintains a clear distinction between the two terms.
A keyword is any word (symbol starting with a letter) used in language/grammar definition. A reserved word is a keyword that is reserved by the language as special symbol and cannot be used as identifier or anything else. New version of Irony does not hold separate lists for these special words: keywords and reserved words are symbol terminals that are marked by special flags: TermOptions.IsKeyword and Term.Options.IsReservedWord. Any token produced by scanner has its Terminal object attached to it, so tokens containing special words would have SymbolTerminal in Terminal field with appropriate flag set.
Now, how these flags are set. Irony sets the IsKeyword flag automatically for any word (symbol starting with a letter) that it finds in grammar rules. The grammar author does not need to add keywords explicitly to any lists - Irony finds them automatically, and highlighter would color them appropriately. On the other hand, the reserved words in the language should be marked explicitly using MarkReservedWords(...) method. This call is important for languages with reserved words, as it tells scanner and parser to always recongize the word as keyword, and never as indentifier or something else. Without this call the grammar may become ambiguous and parser can make wrong decisions.
In conclusion - a little less code to write for you, fellow language creators. Enjoy it!