sijo at atinav.com
Mon May 30 16:18:22 PDT 2005
> In more conventional languages that do distinguish clearly
> between the lexical level and the syntactic level, whitespace
> between tokens is generally swallowed by the lexer and is
> not represented explicitly in the grammar.
Right. I went through the grammar specifications of some of
the programming languages :-)
> then you would not only make the production rule more
> difficult to read, you would also introduce an ambiguity as
> to whether the space in "true( )" matches the second or the
> fifth S? in the production. This can be avoided here by
> moving the S? tokens around, but the extra complexity
> that's introduced delivers no real benefits.
That make real sense. Thank you for a good response.
----- Original Message -----
From: "Michael Kay" <mhk at mhk.me.uk>
To: "'Sijo Mathew'" <sijo at atinav.com>; "'Bob Foster'" <bob at objfac.com>;
<talk at xquery.com>
Sent: Monday, May 30, 2005 1:59 PM
Subject: RE: [xquery-talk] VarRef
> > You could have defined the non-terminal 'VarRef ' as below,
> >  VarRef ::= "$" S? VarName
> > which will definitely help to avoid similar questions.
> The XML grammar is unusual because it does not have a clear boundary
> a lexical analyzer and a syntax analyzer. XQuery has inherited some
> of this because part of its syntax tries to mimic XML. In more
> languages that do distinguish clearly between the lexical level and the
> syntactic level, whitespace between tokens is generally swallowed by the
> lexer and is not represented explicitly in the grammar.
> Including whitespace in all the grammar rules of XQuery would make the
> grammar much more complicated and more difficult to read. In particular,
> would be difficult to make the whitespace explicit in all productions
> without introducing technical ambiguity (where a space in the input could
> match one of several S tokens in the grammar. For example, if you changed
> the rule for FunctionCall from
>  FunctionCall ::= <QName "("> (ExprSingle ("," ExprSingle)*)? ")"
>  FunctionCall ::= <QName S? "("> S? (ExprSingle S? ("," S?
> ExprSingle)*)? S? ")"
> then you would not only make the production rule more difficult to read,
> would also introduce an ambiguity as to whether the space in "true( )"
> matches the second or the fifth S? in the production. This can be avoided
> here by moving the S? tokens around, but the extra complexity that's
> introduced delivers no real benefits.
> Michael Kay
> talk at xquery.com
More information about the talk