From yoshiokamoto at yahoo.com Sun Mar 10 07:31:52 2013 From: yoshiokamoto at yahoo.com (Yoshi Okamoto) Date: Sun, 10 Mar 2013 15:31:52 +0000 (GMT) Subject: [xquery-talk] XQuery 3.0: formatting integers Message-ID: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> Dear readers, I have a question regarding the new fn:format-integer() function of XQuery. I noticed that the implementations I tried give me different results for various queries. This is one example: ? format-integer(-1, '#####12.3') The results are: * Saxon: -0.0.1 * BaseX, XMLPrime: -00.1 * Zorba: -1 Does someone know what is the correct result? I spent about one hour in the Functions and Operators 3.0 specification to finally confess that I did not manage to sort out the relevant details. This is the text I read: ? http://www.w3.org/TR/xpath-functions-30/#func-format-integer Maybe someone in the community is faster than me? Maybe someone even know the answer by heart? Maybe I should try more implementations? I assume that Michael Kay's Saxon processor is probably the implementation to trust most? Thank you for reading this, and thank your for sharing your time, Best regards Yoshi OKAMOTO From yoshiokamoto at yahoo.com Sun Mar 10 08:43:51 2013 From: yoshiokamoto at yahoo.com (Yoshi Okamoto) Date: Sun, 10 Mar 2013 16:43:51 +0000 (GMT) Subject: [xquery-talk] XQuery 3.0: formatting integers (2) In-Reply-To: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> References: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> Message-ID: <1362933831.87592.YahooMailNeo@web172101.mail.ir2.yahoo.com> Dear readers (2), my impression is that it may be too early to test fn:format-integer(), because the returned values of all implementations I have tested so far seem to remain unpredictable. Indeed, I believe I have managed to bring the tested implementations together in all possible combinations! Two other examples that I want to present: Query 1: ? format-integer(-1, '1;o') The results are: * xMLPrime: -1st * Saxon, BaseX, Zorba: error I believe that XMLPrime's result is correct (?). Query 2: ? format-integer(-1, '1o') The results are: * xMLPrime, Saxon: -1 * BaseX, Zorba: -1st Here, I would have expected an error, because, following the specification, the string contains no semicolon, and '1o' does not match the pattern '^((\p{Nd}|#|[^\p{N}\p{L}])+?)$'. But I am afraid I may be so wrong? Yoshi OKAMOTO From liam at w3.org Sun Mar 10 09:11:19 2013 From: liam at w3.org (Liam R E Quin) Date: Sun, 10 Mar 2013 13:11:19 -0400 Subject: [xquery-talk] XQuery 3.0: formatting integers (2) In-Reply-To: <1362933831.87592.YahooMailNeo@web172101.mail.ir2.yahoo.com> References: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> <1362933831.87592.YahooMailNeo@web172101.mail.ir2.yahoo.com> Message-ID: <1362935479.25191.74.camel@localhost.localdomain> On Sun, 2013-03-10 at 16:43 +0000, Yoshi Okamoto wrote: > Dear readers (2), > > my impression is that it may be too early to test fn:format-integer(), > because the returned values of all implementations I have tested so > far seem to remain unpredictable. It might mean that our test suite needs to be improved in this area, and we'd really welcome contributions,,, For moving XQuery 3.0 to a W3C Recommendation, we need to show good interoperability. There's a test suite at http://dev.w3.org/2011/QT3-test-suite/ which includes a link to a page about submitting test cases. Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org freenode/#xml From liam at w3.org Sun Mar 10 09:23:42 2013 From: liam at w3.org (Liam R E Quin) Date: Sun, 10 Mar 2013 13:23:42 -0400 Subject: [xquery-talk] XQuery 3.0 test suite - Call for implementations of XQuery 3 Message-ID: <1362936222.25191.87.camel@localhost.localdomain> The XQuery 3.0 specification is currently a W3C "Candidate Recommendation", meaning we (W3C) are waiting to know that it can be implemented by multiple people such that people can move queries from one implementation to another reasonably easily. If you have made an XQuery 3 implementation, please run the test suite at [1] and send us the results. We can publish them anonymously if you prefer (email me directly for instructions). Thank you for helping XQuery 3.0 move forward, and helping with this work! [1] http://dev.w3.org/2011/QT3-test-suite/ -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org freenode/#xml From mike at saxonica.com Sun Mar 10 10:08:22 2013 From: mike at saxonica.com (Michael Kay) Date: Sun, 10 Mar 2013 18:08:22 +0000 Subject: [xquery-talk] XQuery 3.0: formatting integers In-Reply-To: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> References: <1362929512.13675.YahooMailNeo@web172103.mail.ir2.yahoo.com> Message-ID: <513CCC16.8020408@saxonica.com> There has been quite a bit of tightening up of the format-integer specification over successive drafts. The Candidate Recommendation says this: If the primary format token contains at least one Unicode digit then it is taken as a decimal digit pattern, and in this case it must match the regular expression ^((\p{Nd}|#|[^\p{N}\p{L}])+?)$. If it contains a digit but does not match this pattern, a dynamic error is raised [err:FODF1310]. which makes it pretty clear that in your example, the picture is an error (because it contains a "."). >I assume that Michael Kay's Saxon processor is probably the implementation to trust most? No, because the specification has been changed since Saxon 9.4 was released. At this stage of the game, while the specification is still a draft, differences between implementations - especially in edge cases - are to be expected. Michael Kay Saxonica From christian.gruen at gmail.com Sun Mar 10 11:01:57 2013 From: christian.gruen at gmail.com (=?ISO-8859-1?Q?Christian_Gr=FCn?=) Date: Sun, 10 Mar 2013 20:01:57 +0100 Subject: [xquery-talk] XQuery 3.0 test suite - Call for implementations of XQuery 3 In-Reply-To: <1362936222.25191.87.camel@localhost.localdomain> References: <1362936222.25191.87.camel@localhost.localdomain> Message-ID: Liam, thanks for the reminder! We are working on the finalization of XQuery 3.0, and we will submit our results as soon as we are approximating 100%. Yoshi OKAMOTO san: our implementation of format-integer() still needs to be adjusted to the latest version of the specification. I can just confirm that many results of this function will be somewhat unexpected. Best, Christian ___________________________ > The XQuery 3.0 specification is currently a W3C "Candidate > Recommendation", meaning we (W3C) are waiting to know that it can be > implemented by multiple people such that people can move queries from > one implementation to another reasonably easily. > > If you have made an XQuery 3 implementation, please run the test suite > at [1] and send us the results. We can publish them anonymously if you > prefer (email me directly for instructions). > > Thank you for helping XQuery 3.0 move forward, and helping with this > work! > > > [1] http://dev.w3.org/2011/QT3-test-suite/ > > -- > Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ > Pictures from old books: http://fromoldbooks.org/ > Ankh: irc.sorcery.net irc.gnome.org freenode/#xml > > _______________________________________________ > talk at x-query.com > http://x-query.com/mailman/listinfo/talk From msokolov at safaribooksonline.com Wed Mar 27 04:31:40 2013 From: msokolov at safaribooksonline.com (Michael Sokolov) Date: Wed, 27 Mar 2013 08:31:40 -0400 Subject: [xquery-talk] where clause Message-ID: <5152E6AC.3040203@safaribooksonline.com> I'm trying to come up with examples in which "where" clauses cannot be rewritten as XPath predicates. So far, the ones I have all involve an "at" counter. Are there others? -- Michael Sokolov Senior Architect Safari Books Online From gkholman at CraneSoftwrights.com Wed Mar 27 05:18:41 2013 From: gkholman at CraneSoftwrights.com (G. Ken Holman) Date: Wed, 27 Mar 2013 09:18:41 -0400 Subject: [xquery-talk] where clause In-Reply-To: <5152E6AC.3040203@safaribooksonline.com> References: <5152E6AC.3040203@safaribooksonline.com> Message-ID: <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> At 2013-03-27 08:31 -0400, Michael Sokolov wrote: >I'm trying to come up with examples in which "where" clauses cannot >be rewritten as XPath predicates. So far, the ones I have all >involve an "at" counter. Are there others? I can see no difference off hand when creating 1-tuples, as a 1-tuple is essentially just a sequence. However, when creating n-tuples (either with the "at" keyword as you say or with other bound variables in your expression), the where clause acts on the tuple, not just on a set of values. Here is a 3-tuple expression example: for $a in expr1, $b in expr2, $c in expr3 where $a=$b+$c return $a I hope this is helpful. . . . . . . . . Ken -- Public XSLT, XSL-FO, UBL and code list classes in Europe -- Apr 2013 | Contact us for world-wide XML consulting and instructor-led training | Free 5-hour lecture: http://www.CraneSoftwrights.com/links/udemy.htm | Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ | G. Ken Holman mailto:gkholman at CraneSoftwrights.com | Google+ profile: https://plus.google.com/116832879756988317389/about | Legal business disclaimers: http://www.CraneSoftwrights.com/legal | From bas.debakker at emc.com Wed Mar 27 05:26:52 2013 From: bas.debakker at emc.com (deBakker, Bas) Date: Wed, 27 Mar 2013 13:26:52 +0000 Subject: [xquery-talk] where clause In-Reply-To: <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> References: <5152E6AC.3040203@safaribooksonline.com> <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> Message-ID: Wouldn't that be equivalent to for $a in expr1, $b in expr2, $c in expr3[$a = $b + .] return $a Bas -----Original Message----- From: talk-bounces at x-query.com [mailto:talk-bounces at x-query.com] On Behalf Of G. Ken Holman Sent: Wednesday, March 27, 2013 14:19 To: talk at x-query.com Subject: Re: [xquery-talk] where clause At 2013-03-27 08:31 -0400, Michael Sokolov wrote: >I'm trying to come up with examples in which "where" clauses cannot be >rewritten as XPath predicates. So far, the ones I have all involve an >"at" counter. Are there others? I can see no difference off hand when creating 1-tuples, as a 1-tuple is essentially just a sequence. However, when creating n-tuples (either with the "at" keyword as you say or with other bound variables in your expression), the where clause acts on the tuple, not just on a set of values. Here is a 3-tuple expression example: for $a in expr1, $b in expr2, $c in expr3 where $a=$b+$c return $a I hope this is helpful. . . . . . . . . Ken -- Public XSLT, XSL-FO, UBL and code list classes in Europe -- Apr 2013 | Contact us for world-wide XML consulting and instructor-led training | Free 5-hour lecture: http://www.CraneSoftwrights.com/links/udemy.htm | Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ | G. Ken Holman mailto:gkholman at CraneSoftwrights.com | Google+ profile: https://plus.google.com/116832879756988317389/about | Legal business disclaimers: http://www.CraneSoftwrights.com/legal | _______________________________________________ talk at x-query.com http://x-query.com/mailman/listinfo/talk From gkholman at CraneSoftwrights.com Wed Mar 27 07:01:31 2013 From: gkholman at CraneSoftwrights.com (G. Ken Holman) Date: Wed, 27 Mar 2013 11:01:31 -0400 Subject: [xquery-talk] where clause In-Reply-To: References: <5152E6AC.3040203@safaribooksonline.com> <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> Message-ID: <7.0.1.0.2.20130327101850.023ba720@wheresmymailserver.com> At 2013-03-27 13:26 +0000, deBakker, Bas wrote: >Wouldn't that be equivalent to > > for $a in expr1, $b in expr2, $c in expr3[$a = $b + .] > return $a Thank you, Bas, yes I grant that your expression is equivalent to mine. I failed to acknowledge that tuples are only created when there is a binding value for every member of the tuple. I appreciate the clarification. . . . . . . . . Ken >Bas > >-----Original Message----- >From: talk-bounces at x-query.com [mailto:talk-bounces at x-query.com] On >Behalf Of G. Ken Holman >Sent: Wednesday, March 27, 2013 14:19 >To: talk at x-query.com >Subject: Re: [xquery-talk] where clause > >At 2013-03-27 08:31 -0400, Michael Sokolov wrote: > >I'm trying to come up with examples in which "where" clauses cannot be > >rewritten as XPath predicates. So far, the ones I have all involve an > >"at" counter. Are there others? > >I can see no difference off hand when creating 1-tuples, as a >1-tuple is essentially just a sequence. > >However, when creating n-tuples (either with the "at" keyword as you >say or with other bound variables in your expression), the where >clause acts on the tuple, not just on a set of values. Here is a >3-tuple expression example: > > for $a in expr1, $b in expr2, $c in expr3 > where $a=$b+$c > return $a > >I hope this is helpful. > >. . . . . . . . Ken -- Public XSLT, XSL-FO, UBL and code list classes in Europe -- Apr 2013 | Contact us for world-wide XML consulting and instructor-led training | Free 5-hour lecture: http://www.CraneSoftwrights.com/links/udemy.htm | Crane Softwrights Ltd. http://www.CraneSoftwrights.com/m/ | G. Ken Holman mailto:gkholman at CraneSoftwrights.com | Google+ profile: https://plus.google.com/116832879756988317389/about | Legal business disclaimers: http://www.CraneSoftwrights.com/legal | From mike at saxonica.com Wed Mar 27 14:43:28 2013 From: mike at saxonica.com (Michael Kay) Date: Wed, 27 Mar 2013 22:43:28 +0000 Subject: [xquery-talk] where clause In-Reply-To: <5152E6AC.3040203@safaribooksonline.com> References: <5152E6AC.3040203@safaribooksonline.com> Message-ID: <51537610.6000603@saxonica.com> On 27/03/2013 12:31, Michael Sokolov wrote: > I'm trying to come up with examples in which "where" clauses cannot be > rewritten as XPath predicates. So far, the ones I have all involve an > "at" counter. Are there others? > The ones that are trickiest to rewrite are those that reference the context. For example $y/(for $x in //item where $x/@status = . return $x) To translate this into a predicate you need to introduce a variable: $y/(let $z := . return //item[@status = $z]) Michael Kay Saxonica From msokolov at safaribooksonline.com Wed Mar 27 17:09:58 2013 From: msokolov at safaribooksonline.com (Michael Sokolov) Date: Wed, 27 Mar 2013 21:09:58 -0400 Subject: [xquery-talk] where clause In-Reply-To: <5152E6AC.3040203@safaribooksonline.com> References: <5152E6AC.3040203@safaribooksonline.com> Message-ID: <51539866.8020806@safaribooksonline.com> On 3/27/13 8:31 AM, Michael Sokolov wrote: > I'm trying to come up with examples in which "where" clauses cannot be > rewritten as XPath predicates. So far, the ones I have all involve an > "at" counter. Are there others? > OK - thanks, Ken, Michael, Daniela - I'll just summarize: in XQuery 1.0 (I didn't say, but that's my current target) the only examples we have are those with "at" - Bas and Michael demonstrated that most expressions can be rewritten, including Ken's, even if with some difficulty. Daniela's examples (group by, count) are from XQuery 3.0 - they look irreducible. Thanks again -Mike Sokolov From msokolov at safaribooksonline.com Wed Mar 27 18:10:35 2013 From: msokolov at safaribooksonline.com (Michael Sokolov) Date: Wed, 27 Mar 2013 22:10:35 -0400 Subject: [xquery-talk] Fwd: Re: where clause In-Reply-To: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> References: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> Message-ID: <5153A69B.30206@safaribooksonline.com> (forwarding Daniela's message) Michael, I think the ones with grouby and a where after the groupby are impossible to rewrite in general (remember this from my optimizing SQL days). In this case the WHERE is the good old HAVING clause of SQL. Something along the lines of for $x in blah for $y in blah group by $z:= $x/a, $t:= $y/b where $z+$t >25 return $z I am putting an informal syntax, obviously. Another case is when the COUNT clause exists. You cannot move the where up and down in the tuple stream in such a case. for $x in Blah1 for $y in Blah2 count $i where $x/a eq $y/b return something using $i HTH Dana On Mar 27, 2013, at 5:31 AM, Michael Sokolov wrote: > I'm trying to come up with examples in which "where" clauses cannot be rewritten as XPath predicates. So far, the ones I have all involve an "at" counter. Are there others? > > -- > Michael Sokolov > Senior Architect > Safari Books Online > > _______________________________________________ > talk at x-query.com > http://x-query.com/mailman/listinfo/talk From benito at benibela.de Thu Mar 28 04:15:50 2013 From: benito at benibela.de (Benito van der Zander) Date: Thu, 28 Mar 2013 13:15:50 +0100 Subject: [xquery-talk] [ANN] Pascal XQuery 1 engine now with JSONiq Message-ID: <51543476.7000904@benibela.de> Hi, there is now a new release of my Pascal-based XQuery engine which adds JSONiq support. (not JSONiq Update through, but there is an old global variable extension, which can be used to update some (but not all) object properties without involving snapshots ) The XQuery part supports everything of XQuery 1 except xml schemas, and static type checking, with some extensions like pattern matching, global variables or a XPath 2-restricted mode. Main library page:http://www.benibela.de/sources_en.html#internettools Online XQuery tester: http://videlibri.sourceforge.net/cgi-bin/xidelcgi?extract-kind=xquery Command Line Tool: http://videlibri.sourceforge.net/xidel.html XQTS Results: http://www.benibela.de/documentation/internettools/internettoolsxqts.html (skipping invalid inputs) Benito From dflorescu at me.com Thu Mar 28 08:23:02 2013 From: dflorescu at me.com (daniela florescu) Date: Thu, 28 Mar 2013 09:23:02 -0700 Subject: [xquery-talk] where clause In-Reply-To: <5153A69B.30206@safaribooksonline.com> References: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> <5153A69B.30206@safaribooksonline.com> Message-ID: Michael, in addition to the queries I already gave you with count and groupby, here is another one in pure XQuery 1.0 that cannot be reduced declare function foo($x) { if ($x/@a) then $x else }; for $x in Blah1 for $y in Blah2 let $z := foo($y) where $z is $z return $x (variable $z cannot be unfolded because of the generation of new nodes, and then the where cannot be move up because of the let) Sorry, I felt like being twisted minded this morning :-) XQuery is wicked :-) Best Dana On Mar 27, 2013, at 7:10 PM, Michael Sokolov wrote: > (forwarding Daniela's message) > > Michael, > > I think the ones with grouby and a where after the groupby are impossible to rewrite > in general (remember this from my optimizing SQL days). In this case the WHERE is the good old > HAVING clause of SQL. > > Something along the lines of > > for $x in blah > for $y in blah > group by $z:= $x/a, $t:= $y/b > where $z+$t >25 > return $z > > I am putting an informal syntax, obviously. > > Another case is when the COUNT clause exists. You cannot move the where up and down in the tuple stream in > such a case. > > for $x in Blah1 > for $y in Blah2 > count $i > where $x/a eq $y/b > return something using $i > > > HTH > Dana > > > On Mar 27, 2013, at 5:31 AM, Michael Sokolov wrote: > >> I'm trying to come up with examples in which "where" clauses cannot be rewritten as XPath predicates. So far, the ones I have all involve an "at" counter. Are there others? >> >> -- >> Michael Sokolov >> Senior Architect >> Safari Books Online >> >> _______________________________________________ >> talk at x-query.com >> http://x-query.com/mailman/listinfo/talk > > > > _______________________________________________ > talk at x-query.com > http://x-query.com/mailman/listinfo/talk From msokolov at safaribooksonline.com Thu Mar 28 10:15:20 2013 From: msokolov at safaribooksonline.com (Michael Sokolov) Date: Thu, 28 Mar 2013 14:15:20 -0400 Subject: [xquery-talk] where clause In-Reply-To: References: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> <5153A69B.30206@safaribooksonline.com> Message-ID: <515488B8.1020601@safaribooksonline.com> On 03/28/2013 12:23 PM, daniela florescu wrote: > Michael, > > in addition to the queries I already gave you with count and groupby, here is another one in > pure XQuery 1.0 that cannot be reduced > > declare function foo($x) > { > if ($x/@a) then $x else > }; > > > for $x in Blah1 > for $y in Blah2 > let $z := foo($y) > where $z is $z > return $x > > > (variable $z cannot be unfolded because of the generation of new nodes, and then the where cannot be move up because > of the let) > > > Sorry, I felt like being twisted minded this morning :-) > > XQuery is wicked :-) > Yes, that is twisted for sure. But isn't it also equivalent to: for $x in Blah1 for $y in Blah2 for $z in foo($y)[. is .] return $x ? not that that really makes any difference, in some sense, but given the problem as stated, I don't think a where clause is required -- Michael Sokolov Senior Architect Safari Books Online From dflorescu at me.com Thu Mar 28 10:22:08 2013 From: dflorescu at me.com (daniela florescu) Date: Thu, 28 Mar 2013 11:22:08 -0700 Subject: [xquery-talk] where clause In-Reply-To: <515488B8.1020601@safaribooksonline.com> References: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> <5153A69B.30206@safaribooksonline.com> <515488B8.1020601@safaribooksonline.com> Message-ID: <8758B461-CD2D-4EA4-8C3E-9621B1943413@me.com> Not in the general case of such pattern -- because of cardinality. (I told you XQuery is twisted :-) Imagine that the function would have looked like this instead of the one I wrote: declare function foo($x) { if ($x/@a) then ($x, $x) else }; then count(...) of the total result is different ! :-) Dana >> >> > Yes, that is twisted for sure. But isn't it also equivalent to: > > for $x in Blah1 > for $y in Blah2 > for $z in foo($y)[. is .] > return $x > > ? > > not that that really makes any difference, in some sense, but given the problem as stated, I don't think a where clause is required > > -- > Michael Sokolov > Senior Architect > Safari Books Online > > _______________________________________________ > talk at x-query.com > http://x-query.com/mailman/listinfo/talk From msokolov at safaribooksonline.com Thu Mar 28 11:08:13 2013 From: msokolov at safaribooksonline.com (Michael Sokolov) Date: Thu, 28 Mar 2013 15:08:13 -0400 Subject: [xquery-talk] where clause In-Reply-To: <8758B461-CD2D-4EA4-8C3E-9621B1943413@me.com> References: <8B07C097-332D-4CA1-BAA9-54EDBAAC6FD2@me.com> <5153A69B.30206@safaribooksonline.com> <515488B8.1020601@safaribooksonline.com> <8758B461-CD2D-4EA4-8C3E-9621B1943413@me.com> Message-ID: <5154951D.4010200@safaribooksonline.com> I see - nice! Still trying to wrap my head around it to generalize, but I see your point. -Mike On 03/28/2013 02:22 PM, daniela florescu wrote: > Not in the general case of such pattern -- because of cardinality. > (I told you XQuery is twisted :-) > > Imagine that the function would have looked like this instead of the one I wrote: > > > declare function foo($x) > { > if ($x/@a) then ($x, $x) else > }; > > then count(...) of the total result is different ! > > :-) > > Dana > > > > >>> >>> >> Yes, that is twisted for sure. But isn't it also equivalent to: >> >> for $x in Blah1 >> for $y in Blah2 >> for $z in foo($y)[. is .] >> return $x >> >> ? >> >> not that that really makes any difference, in some sense, but given the problem as stated, I don't think a where clause is required >> >> -- >> Michael Sokolov >> Senior Architect >> Safari Books Online >> >> _______________________________________________ >> talk at x-query.com >> http://x-query.com/mailman/listinfo/talk >> > -- Michael Sokolov Senior Architect Safari Books Online From liam at w3.org Thu Mar 28 23:35:38 2013 From: liam at w3.org (Liam R E Quin) Date: Fri, 29 Mar 2013 03:35:38 -0400 Subject: [xquery-talk] where clause In-Reply-To: References: <5152E6AC.3040203@safaribooksonline.com> <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> Message-ID: <1364542538.17360.192.camel@localhost.localdomain> On Wed, 2013-03-27 at 13:26 +0000, deBakker, Bas wrote: > Wouldn't that be equivalent to > > for $a in expr1, $b in expr2, $c in expr3[$a = $b + .] > return $a I notice that BaseX does exactly that rewrite. In SQL with a WHERE clause some of the values may be NULL, but that can't occur in XQuery today; if it could, there might be tuples without a value for $c, in which case the rewrite wouldn't work. A smart optimizer with knowledge of the input could rewrite at runtime to put the [predicate] on the expression likely to have the fewest nodes, or could rewrite to say, for $a in expr1[. ge min($b) + min($c)]; this sort of rewrite can turn a theoretically O(n^3) operation into O(n) in practice. I think the answer is, use "where" when it makes the query more readable, or if there are positional or grouping clauses. Some implementatins do more optimizatoins than others, though, so sometimes readability ends up second to speed. Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org freenode/#xml From mike at saxonica.com Thu Mar 28 23:54:06 2013 From: mike at saxonica.com (Michael Kay) Date: Fri, 29 Mar 2013 07:54:06 +0000 Subject: [xquery-talk] where clause In-Reply-To: <1364542538.17360.192.camel@localhost.localdomain> References: <5152E6AC.3040203@safaribooksonline.com> <7.0.1.0.2.20130327091238.023baa10@wheresmymailserver.com> <1364542538.17360.192.camel@localhost.localdomain> Message-ID: <1CBCC50E-96DF-4A8A-8520-15D054D38640@saxonica.com> It's not just the WHERE clause, of course: ORDER BY and GROUP BY also affect the strategy. In XQuery 1.0 nearly every FLWOR expression could be rewritten as an expression on sequences of items (rather than tuples). The only exception was a rather weird and rare case involving ORDER BY where the order of tuples couldn't be reduced to an ordering of items in the input sequences. Saxon reduced everything to sequence operations, handling this special case with a bit of custom code that essentially wrapped the tuple as a map-like item. In XQuery 3.0 there are many more FLWOR expressions that can't be rewritten in terms of operations on item sequences, and native support for tuple streams in the run-time engine becomes almost unavoidable. Once you have that support, there's not necessarily any benefit in eliminating the WHERE clauses (though Saxon still attempts to do so, mainly to take advantage of the other optimisations available for filter expressions). The important thing is to promote each conjunctive term of the WHERE clause up the pipeline to the earliest place where it can be evaluated; whether it is then turned into a filter expression is relatively unimportant. Michael Kay Saxonica On 29 Mar 2013, at 07:35, Liam R E Quin wrote: > > On Wed, 2013-03-27 at 13:26 +0000, deBakker, Bas wrote: >> Wouldn't that be equivalent to >> >> for $a in expr1, $b in expr2, $c in expr3[$a = $b + .] >> return $a > > I notice that BaseX does exactly that rewrite. > > In SQL with a WHERE clause some of the values may be NULL, but that > can't occur in XQuery today; if it could, there might be tuples without > a value for $c, in which case the rewrite wouldn't work. > > A smart optimizer with knowledge of the input could rewrite at runtime > to put the [predicate] on the expression likely to have the fewest > nodes, or could rewrite to say, for $a in expr1[. ge min($b) + min($c)]; > this sort of rewrite can turn a theoretically O(n^3) operation into O(n) > in practice. > > I think the answer is, use "where" when it makes the query more > readable, or if there are positional or grouping clauses. Some > implementatins do more optimizatoins than others, though, so sometimes > readability ends up second to speed. > > Liam > > -- > Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ > Pictures from old books: http://fromoldbooks.org/ > Ankh: irc.sorcery.net irc.gnome.org freenode/#xml > > _______________________________________________ > talk at x-query.com > http://x-query.com/mailman/listinfo/talk