[xquery-talk] VarRef

[Michael Kay]

> 
> You could have defined the non-terminal 'VarRef ' as below,
> 
> [84] VarRef    ::=    "$" S? VarName
> 
> which will definitely help to avoid similar questions.

The XML grammar is unusual because it does not have a clear boundary between
a lexical analyzer and a syntax analyzer. XQuery has inherited some aspects
of this because part of its syntax tries to mimic XML. In more conventional
languages that do distinguish clearly between the lexical level and the
syntactic level, whitespace between tokens is generally swallowed by the
lexer and is not represented explicitly in the grammar.

Including whitespace in all the grammar rules of XQuery would make the
grammar much more complicated and more difficult to read. In particular, it
would be difficult to make the whitespace explicit in all productions
without introducing technical ambiguity (where a space in the input could
match one of several S tokens in the grammar. For example, if you changed
the rule for FunctionCall from

[91] FunctionCall ::= <QName "("> (ExprSingle ("," ExprSingle)*)? ")"

to

[91] FunctionCall ::= <QName S? "("> S? (ExprSingle S? ("," S?
ExprSingle)*)? S? ")"

then you would not only make the production rule more difficult to read, you
would also introduce an ambiguity as to whether the space in "true( )"
matches the second or the fifth S? in the production. This can be avoided
here by moving the S? tokens around, but the extra complexity that's
introduced delivers no real benefits.

Michael Kay
http://www.saxonica.com/