[xquery-talk] BumbleBee XQuery Test Harness

[Jason Hunter]

Hi everyone,

I'm proud to announce here a product I've helped develop: the BumbleBee 
XQuery test harness.  It's an automated test harness that you can use 
both for evaluating XQuery engines (seeing which vendor is more spec 
compliant and scalable) and for validating your own queries (making sure 
your queries continue to work as you upgrade or change engines).

BumbleBee -- named because it buzzes around FLWORs -- has the following 
features:

* Includes over 70 automated tests representative of the W3C XQuery
   Use Cases
* Includes over 1000 automated tests representative of the NIST
   XQuery Test Suite
* Includes adapters to 6 different free and commercial XQuery engines
* Includes the runtime libraries for 3 free XQuery engines
* Allows you to write and run your own custom XQuery tests
* Tests are automated, self checking, and report an unambiguous
   pass or fail status
* Tests can define multiple, possible expected results
* Can run arbitrary suites of standard and/or custom tests
* Can run tests against one or more XQuery engines
* Generates a comprehensive report of passed and failed tests
   per XQuery engine being tested
* Supports the addition of new XQuery engines through a pluggable
   adapter design
* Allows disabling and enabling tests, globally or on a per-engine
   basis
* Allows customization of whitespace comparison behavior
* Supports negative testing, where the engine must report an error
   to pass
* Written in Java for maximum portability

A free, no-login-required, fully-functioning 30-day evaluation copy of 
BumbleBee and the "Getting Started With BumbleBee" quick-start guide is 
available at:

     http://xquery.com/bumblebee

Licensed copies of BumbleBee are available free of charge for developers 
of open source XQuery implementations, and for qualified non-profit and 
educational use.  Licenses of BumbleBee are available for commercial use 
by contacting "buzz" at xquery.com with your usage requirements.

I hope that this mailing list will host some discussion about the tool 
and what it discovers about engine quality, so if you have any thoughts 
please write in.

-jh-