[xquery-talk] BumbleBee XQuery Test Harness

[Sander Bos]

Dear Jason,

> I hope that this mailing list will host some discussion about 
> the tool 
> and what it discovers about engine quality, so if you have 
> any thoughts 
> please write in.

I like it very much. IntelliJ did not have much problems deciphering
what I was supposed to implement (a possible DMCA violation in your
country?).
I could write and test an adapter for our product (X-Hive/DB) in about
1.5 hours, including a little 'hacking' (turns out we read over the
document -> doc function renaming in the spec), and query adjusting (we
implement the later 22 August version of the spec). So congratulations
on such an easy to use product! I will post my adapter class on this
list once I can run it against a non-obfuscated version of Adapter.

With respect to our success percentage, well we have a little work to
do.... I did look into detail into the usecases tests (also to tune the
adapter against the test-framework). What I saw is that we had some
problems with doc(...) as mentioned, maybe we include incorrectly at
some points (not sure whether that's a bug), so 800.0 instead of 800, we
do not implement import schema yet, we include default attributes when
importing into another constructed element (very probably a bug), and
then there were a few test failures I could not directly explain (I am
not one of the XQuery programmers here).

I think there is also a bug in the test-framework that causes hundreds
of tests to fail (596 mentions in the log), we have a lot of results
like:
Query: 
<results>
       {fn:boolean(xs:integer("-99999999999999999999"))}
      </results>
Actual Result: 
<results>true</results>
Expected Possible Result 1: 
true
Failure Message 1: 
Expected node type '3' but was '1' - comparing <BumbleBee_Result ...>
true </BumbleBee_Result> at /BumbleBee_Result[1]/text()[1] to
<results...> at /BumbleBee_Result[1]/results[1]

I think we produce the result in the right way, so surrounded by a
results element? But BumbleBee does not like it.

Kind regards,

--Sander.