[xquery-talk] XML Query Test Suite version 0.8.4

[David Carlisle]

Andrew,

> We encourage implementors to build their test harnesses and to provide 
> feedback to the XML Query Working Group.  We have received the results of 
> running XQTS with three implementations of XQuery: Saxon-SA, xq2xsl, and 
> X-Hive/DB. A report that reflects these results is available from our web 
> page.

I updated mine to 0.8.4 at the usual place
http://monet.nag.co.uk/xq2xml/xq2xslresults.xml
or as HTML at
http://monet.nag.co.uk/xq2xml/XQTSReport.html

If you could update your copy of the xq2xsl test results in the main
test distribution that would be good, thanks: The new test suite did
it's job and turned up a couple of, er, features-that-I-changed in xq2xsl
(and a couple of features in the w3c test parser applet that I've reported.)

I claim 143 "fail" tests in the above report, although several of those
are on tests for which there are open bug reports in bugzilla (I haven't
changed any of the tests even where bugs are reported).

As before, some tests are marked as "not applicable" rather than "fail" as
they implictly depend on schema validation, similarly the new explictly
optional tests on schema import and schema validation are skipped.

I don't test error codes, so any error is counted as pass if an error is
raised when one is expected. This is noted in the comment on each test
result. I could change these to mark as not tested rather than pass,
which probably I should do if you get to a test suite 1.0 but I'm hoping
to improve error code handling before then in the convertor. The main
problem is not the error raised or the error message given to the user,
just the pesky error codes which are different (some of the time) between
XSLT and XQuery, and xq2xsl raises the XSLT ones, since it's running on
an XSLT engine... I think probably in a "pure" Xquery to xslt
translation that is the best you can do (as you can't catch errors in
xslt) but if I restrict to saxon I could (and probably will) wrap it in
some java to trap the error messages and translate the xslt codes back
to xquery ones. However I've tried to avoid any saxon-specific code (as
making an xquery-to-xslt translator specific to saxon when saxon natively
executes both xquery and xslt seems peverse even to me).

It still seems to be necessary to normalise white space in many of the
comparisons (528 tests). In most cases this is due to extra or missing
white space in the expected result files, although it's possible this
normalization step is masking white space bugs in xq2xsl.


I'm not sure if you intend to use these xq2xsl reports as part of your
CR exit procedures. You are quite welcome to them if they are useful but
probably you need to take a view on whether it is or isn't an
independent implementation. My own view is that when testing any of the
declarations in the Query prolog, or anything to do with node
construction, FLWOR or typeswitch expressions, then it is essentially a
new XQuery implementation (implemented over XSLT, just as some other
implementations are implemented over SQL) however for the tests that
just test a single xpath expression (including the block of tests
testing individual functions from F&O) it is probably stretching things
to claim that it is a different implementation. An XQuery of 1+2 is
translated by xq2xsl to <xsl:sequence select="1+2"/> and quite probably
the run time codes produced by compiling the first with saxon's Xquery
engine and the second with its XSLT one are the same. It may be that
what I am saying is that if the the test suite's catalog marks a test as
is-XPath2="true" then xq2xsl's implementation of the test is (or should
be, at least) trivially the same as that of the underlying XSLT
engine's, and so in the case that this is saxon/XSLT the implementation
of Xpath in xq2xsl is trivially the same as that of saxon/Xquery.


David


________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________