[xquery-talk] Bumblebee: closed source

[Jason Hunter]

Michael Kay wrote:

 > I've been taking a look at Bumblebee, hoping I might be able at the same
 > time to get value from it and to add value, e.g. by supplying updated
 > tests or drivers, but I've come to the conclusion that it's not viable
 > so long as it remains a closed-source product. To test software you've
 > got to be able to find out what's going wrong when things fail, and you
 > can't do any effective debugging when the source code is all hidden.

We understand the value of source when doing advanced integration work, 
such as writing new adapters, and for people/companies who purchase 
BumbleBee source code is absolutely negotiable.  We believe and have 
found that when engaged in typical BumbleBee use (i.e. using it with the 
established adapters) the source is not particularly useful.  The logs 
tell you exactly what was produced, what was expected, and where the 
mismatch occurred.

 > In writing a new Adapter, the big stumbling block has been that there's
 > no documentation about the format required for the query output. For
 > example, it seems (empirically) that bumblebee falls over if the query
 > results contain an XML declaration. But without source code, I'm groping
 > in the dark, and it's just not fun.

I apologize that you're having a hard time.  You're right that 
declarations aren't permitted in output at this time.  The SaxonAdapter 
we provide turns off the declaration that Saxon by default includes.  I 
believe Saxon is the only engine we've found that default serializes 
with declarations.  If you're writing a new adapter, this is something 
to know, and we'll add mention in the docs.

 > My other gripe about it is the non-XML format used for data. I
 > appreciate it's difficult to use XML when XQuery itself is a non-XML
 > format (and that's something I've been struggling with in converting my
 > XSLT test harness to handle XQuery), but I'm really not prepared to move
 > to something that uses a home-grown proprietary markup language.

The file format to use is a matter of taste.  I've stated earlier on the 
mailing list our reasons for avoiding XML:

-- 
Side note: It's especially difficult to author tests in XML not just 
because XML is more verbose, but because the query and result contents 
themselves use XML.  Thus the tests would have to be escaped or in CDATA 
sections, and neither of those solutions is human friendly.  What's 
more, when the query or result use CDATA sections themselves, then it 
gets extremely difficult to decipher what's what.
-- 

We believe users don't want to read and write tests with XML embedded in 
XML, especially when you'd need special CDATA escapes to keep the file 
format legal.  If we're wrong, we welcome hearing from users.

I also can't resist pointing out that just because something's in XML 
doesn't make it open while something in quite readable ASCII is 
"home-grown proprietary".  You would write a converter between a .bee 
format and a Bee-styled XML in short order.  Of course, the .bee file 
would be much more readable and amenable to new test authoring.

 > The odd thing is that the real IPR in a package like this is in the test
 > data, not in the code, and the test data is there for the taking
 > (although I appreciate that the only value you've added so far is in
 > supplying test results.)

To an XQuery vendor, yes, it probably looks like the IPR's in the test 
data -- because BumbleBee's tests are the first practical and public 
attempt to provide a set of tests that can run across vendors and help 
them establish a higher level of compabibility.  However, to someone 
programming in XQuery itself, there's a different view.  To them, the 
set of test data we provide probably will be important in choosing a 
vendor and in testing upgrades, but more so it'll act as a demo for how 
they can write their own tests, to test their XQuery application's 
behavior.  I already use BumbleBee for the XQuery training class I 
teach.  I now know before walking into the classroom whether the engine 
I'm teaching against works for the labs or not.  This is very valuable.

To the XQuery developer, BumbleBee provides a great amount of time 
savings for "grunt work" work they don't want to do, and work that's not 
interesting to them.  They just unpack BumbleBee, use its provided tests 
to find the right vendor for them, then use its easy infrastructure to 
write tests for their own XQuery code.  We've spent time doing the grunt 
work so others don't have to.  We think it's fair to recover some of 
that cost.

 > For the moment: thanks, but no thanks. I'll revert to plan A and
 > maintain my own test harness.

As a vendor of an open source XQuery engine, you're entitled to a free 
license as mentioned on the BumbleBee web page.  As part of that we 
could provide you with source if you think it would help your work.

-jh-