From andrew.j.welch at gmail.com Tue Sep 1 10:50:48 2009 From: andrew.j.welch at gmail.com (Andrew Welch) Date: Tue Sep 1 01:23:58 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> Message-ID: <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> 2009/8/31 Daniela Florescu : >> why do developers and architects so stubbornly refuse just to _listen_ ? > > Because we don't talk to them (a) enough and (b) with a message that they > understand. What is the number one "killer app" for XQuery, shining example that can be given to people when talking about XML databases - one that is high profile enough that they may have heard of it? -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/ From andrew.j.welch at gmail.com Tue Sep 1 10:52:33 2009 From: andrew.j.welch at gmail.com (Andrew Welch) Date: Tue Sep 1 01:25:40 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <6dca08520908290800w704178acw4adb46816bceef6e@mail.gmail.com> References: <92F57853-837A-4860-9F78-A337E039D74C@mac.com> <7870f82e0908272018y59e2bc5dy4e462bc30037f104@mail.gmail.com> <7870f82e0908280239t1b19fc4at2c6187063955a406@mail.gmail.com> <6dca08520908280505p1f588b4at5c1f8f4e42eec7b0@mail.gmail.com> <7870f82e0908281000w3f2575g2c86bdf78ccd6ba7@mail.gmail.com> <899731D1-8EDD-432E-86F8-0D9A720D6162@gmail.com> <6dca08520908290800w704178acw4adb46816bceef6e@mail.gmail.com> Message-ID: <74a894af0909010152k3108cedemd28f7cbec0985885@mail.gmail.com> 2009/8/29 Martin Probst : > JSON sucks for mixed content, while XML/XQuery is pretty bad at > supporting key/value maps. Both is possible in each of them, but > painful. Sequences of sequences would help here... -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/ From mlu at as-guides.com Tue Sep 1 12:21:06 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Tue Sep 1 01:53:48 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: <4A9CE782.90200@as-guides.com> Andrew Welch schrieb: > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? Maybe Markmail. But being a commercial endeavour, it does not attract a so-called "community". I haven't used eXist yet, but it looks very appealing. Build a new open source bugtracker using XQuery on an app server like eXist. If it works and looks nice and shiny, it'll catch the attention of some developers. That way they will become interested in XQuery. -- Michael Ludwig From mail at martin-probst.com Tue Sep 1 12:30:54 2009 From: mail at martin-probst.com (Martin Probst) Date: Tue Sep 1 02:03:17 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: References: <200908311900.n7VJ03Cq010468@jhunter.x-query.com> <972615.14216.qm@web27108.mail.ukl.yahoo.com> Message-ID: <6dca08520909010230u23ba901ay5bb923b0fbcc941@mail.gmail.com> > With all due respect, one major reason why XML/XQuery/XSLT isn't as popular > as many other programming languages has less to do with ignorance of it's > capabilities and more to do with it's 1) lack of performance, 2) overall > information bloat, and 3) unintuitive language constructs. Regarding performance, we're certainly not at the performance levels of relational databases, but I think for many cases we're "good enough" (even though faster is always better/cheaper). About language constructs: yes, there are some things which could have been better in my opinion, but probably everyone has its own opinion about that. My pet peeves are xmlns="" declarations in literals that change namespace scope, "return" that does not return, and the general wordiness of the language. For my taste in languages, XQuery has much too much syntax. This is one of the reasons why I wouldn't recommend it as a general purpose programming language. > Sure disk space is cheap and bandwidth is getting more *broad* every day, > and "many elegant designs have been sacrificed for the sake of > performance...yada...yada...yada...", but at the end of the day, the same > data that can be represented in XML can be represented a lot more concise in > other platform-independent formats. If you look at XML databases, you might find that they do not go around and store angle brackets. In xDB's case, what ends up on the hard drive is usually quite a bit smaller than the original XML (you know, the one with angle brackets). So if you need to store and process large amounts of XML, just pick the right tools - this is not an inherent problem with XML. What I find much more problematic is the amount of cruft XML by now has, and which greatly complicates anything built on it. We have processing instructions, DTDs, notations, entity references, internal DTD subsets, document fragments, and so on. All of these need to be supported by XML processors at least to some degree, and they greatly increase the conceptual surface of the language. I'd be willing to bet that you can find some XML construct that behaves at least unexpectedly in any larger XML processing tool, just because it's getting increasingly difficult to cover all cases. This is where simplified schemes such as JSON shine - they have a much smaller "surface". Martin From mlu at as-guides.com Tue Sep 1 13:13:18 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Tue Sep 1 02:45:11 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <972615.14216.qm@web27108.mail.ukl.yahoo.com> References: <200908311900.n7VJ03Cq010468@jhunter.x-query.com> <972615.14216.qm@web27108.mail.ukl.yahoo.com> Message-ID: <4A9CF3BE.7080406@as-guides.com> Hans-Juergen Rennau schrieb: > > of course you are right in that XSLT and XQuery are built on the XDM, > not on XML as a serialization format, and you are right in that the > XDM can be completely separated from the serialization format. But I > prefer to view XML and the XDM as one organic whole, and exactly this > is what I meant by the term "deep understanding": an understanding > covering three _levels_ of XML: level #1: the lexical; level #2: the > inforset (= structural model totally unconcerned with aspects of > processing); level #3: the XDM extending the infoset in order to > embrace the reality of processing = a global information model which > preserves the structural achievements of the infoset and embeds it > into a stunningly flexible definition of information value. Each level > evolved from the previous one, I think, was it not so, historically? Hallo Hans-J?rgen, historically, I'm a newbie; but the other day on XML-Dev, one SGML/XML veteran said that it was rather the work of "ladling committees" (had to look up the word - seems similar to "anflanschen" in what it conveys) ladling on patch after patch to XML. RE: [xml-dev] Xml Revisited - Len Bullard http://markmail.org/message/n5zscmypan4rzmaj I really like this realistic view of things, but I also have a lot of sympathy for idealistic interpretations. As for the three levels you're referring to, I don't know enough about level 2 - the XML InfoSet - in order to locate it in the scheme of things. What I've learnt (from Ken Holman) is that with XML, syntax came first, and data models (plural) are an afterthought. Maybe a simple and obvious thing to many, but - at least to me - crucial to understanding this whole XML business. Like a lot of things, it only makes sense seen with history in mind. > This evolution is a wonderful achievement Why not call it an Evolution? The process that has brought forth XML is too visible in order for us to call it a Creation. And it works too well to call it a Fabrication. (Although unfortunately, some fabrications work pretty well, too.) > and I am emphatically against "marketing" XQuery and XSLT as tree > processing languages, as if you could come along with some tree and > get it processed. Well, you can. You just have to make sure your tree is represented in a format amenable to an XML processing chain. Quite often, I start out with documents like , parse those into a DOM or XOM, and then append data retrieved from various sources (HTTP, SQL, XQuery) to then finally hand off the XML tree to XSLT and generate an HTML tree. > What should be done is rather: encourage people to get familiar with > the new reality which XML (via XDM) constitutes: a global information > model of stunning power and flexibility, enabling XPath, XSLT, XQuery > and what is to come. > > A poet (Joseph Brodsky) once wrote: it is not so that the poets should > adopt the language of the masses; common people should adopt the > language of literature. But common people have, to my knowledge, never done so; which is fine. And if you take a look around today, I fear you'll have to concede that there is movement, but in the opposite sense of what Brodsky deemed desirable. > In the realm of IT, XDM/XPath/XSLT/XQuery are like literature, and I > would not sell them for something else. I agree there is beauty in it: the power and flexibility of the tree navigation model. Just consider that the computer has to perform some work to set up this nice machinery; and while I find it often worth while doing so and pretty convenient for me, I have to concede that it is not always necessary. -- Michael Ludwig From newmanjw at upmc.edu Tue Sep 1 16:35:13 2009 From: newmanjw at upmc.edu (Newman, John W) Date: Tue Sep 1 14:10:21 2009 Subject: [xquery-talk] Help with multiple deletes Message-ID: Thanks Martin, that works. Except the 3rd delete doesn't run on the updated tree from the 2nd delete, so 2 copy statements are necessary. Is there a different way to do this, so the delete statements operate on the updated tree w/o doing a copy each time? i.e. some sort of piping construct ? copy $x := doc("tree.xml") modify ( delete node $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] ) return copy $y := $x modify ( delete node $y//tree//branch[count(.//leaf) = 0] ) return copy $z := $y modify ( delete node $z//tree[count(.//branch) = 0] ) return $z Thanks -----Original Message----- From: Martin Probst [mailto:mail@martin-probst.com] Sent: Saturday, August 29, 2009 10:47 AM To: Newman, John W Cc: Michael Kay; talk@x-query.com Subject: Re: [xquery-talk] Help with multiple deletes > After I spent a nice chunk of time testing this in stylus studio, I took the > concept and ported it into our application.? Only to discover that the > implementation we are using, MXQuery 0.6.0, does not support ancestor! > Argh..? =)? So if anyone thinks it can possibly be rewritten without using > ancestor (or any other axes I'm guessing), please share your thoughts.? But > I think we are going to end up using an XSLT for this operation. You could simply transform twice, which might be less efficient, but its certainly simpler to write: copy $x := doc("tree.xml") modify ( delete node $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] ) return copy $y := $x modify ( delete node $x//tree//branch[count(.//leaf) = 0] , delete node $x//tree[count(.//branch) = 0] ) return $y Martin From dflorescu at mac.com Tue Sep 1 09:55:54 2009 From: dflorescu at mac.com (Daniela Florescu) Date: Tue Sep 1 14:17:03 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: > > > I honestly think that all the talk about XDM as a data model and XQ > as a turing complete declarative language, etc., is sort of moot > unless they do somehow enable something that RDBMS/BI/Data > warehouses/search vendors can't readily do with existing approaches. Well, that's relatively simple to answer, I would say. Here is are some examples of what you could do with XML/XQuery that you cannot do with the traditional three tier stack (communication + application logic written in an imperative language + declarative query layer) 1. Have schema flexibility. Start writing code without a fixed idea of your schema, and/or change your mind later. (one schema is evil enough, three layers of superimposed schemas, in three different models, are exponentially evil for flexibility and customizability. Almost no evolution of schema is possible.) 2. Integrate text and documents into your information processing flow without adding software license, yet another layer, more mappings, and different programming languages. 3. Optimize the performance of your information processing program globally, not locally, layer by layer, by adding more and more intermediate caches and shortcuts, which in turn will turn your code maintenance a nightmare. 4. Be done in 2 weeks instead of 10 months. Time to market, by writing significantly less amount of code, layer to layer connectors, plus data mappers from one model to another, etc, etc. Less useless and redundant code to write and later deal with. 5. Spend less time maintaining the code. If the XQuery optimizer is doing a good job, you don't need to constantly rewrite your code to fix performance problems, as you do in java. Plus less number of lines of code usually means less bugs. So smaller total cost of ownership of the code. That would be my quick personal answer. But that of course works if you know what you are doing with XQuery .... Best Dana > If XQ is just an incremental improvement, it will always be a > marginal language (for example, that's how I see the XQ scripting in > the browser situation; it's simply not THAT much better than > JavaScript, and has the disadvantage of not having a million > libraries and code snippets and programmers). I actually think that > it is much more than incremental in search, and it may be elsewhere > as well. > > -Rob > > > On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch > wrote: > 2009/8/31 Daniela Florescu : > >> why do developers and architects so stubbornly refuse just to > _listen_ ? > > > > Because we don't talk to them (a) enough and (b) with a message > that they > > understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? > > > > > -- > Andrew Welch > http://andrewjwelch.com > Kernow: http://kernowforsaxon.sf.net/ > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/8519c99c/attachment.htm From dflorescu at mac.com Tue Sep 1 10:09:05 2009 From: dflorescu at mac.com (Daniela Florescu) Date: Tue Sep 1 14:30:53 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: <4202DE8D-7411-412D-AFD3-8A56611759C1@mac.com> And let me add two more things that you could do with XQuery that you can hardy do otherwise -- because there are more cool :-) 6) virtualize it for the cloud execution. Scale it up and down for cloud execution. The more layers, and less declarative the layers, the harder is this to do. 7) automatically translate Xquery programs into parallel Hadoop programs. This doesn't work with imperative programs. Best regards Dana On Sep 1, 2009, at 8:55 AM, Daniela Florescu wrote: >> >> >> I honestly think that all the talk about XDM as a data model and XQ >> as a turing complete declarative language, etc., is sort of moot >> unless they do somehow enable something that RDBMS/BI/Data >> warehouses/search vendors can't readily do with existing approaches. > > Well, that's relatively simple to answer, I would say. > > Here is are some examples of what you could do with XML/XQuery that > you cannot do with the traditional three > tier stack (communication + application logic written in an > imperative language + declarative query layer) > > 1. Have schema flexibility. Start writing code without a fixed idea > of your schema, and/or change > your mind later. (one schema is evil enough, three layers of > superimposed schemas, in three different models, > are exponentially evil for flexibility and customizability. Almost > no evolution of schema is possible.) > > 2. Integrate text and documents into your information processing > flow without adding software license, yet another layer, > more mappings, and different programming languages. > > 3. Optimize the performance of your information processing program > globally, not locally, layer by layer, > by adding more and more intermediate caches and shortcuts, which in > turn will turn your code maintenance a nightmare. > > 4. Be done in 2 weeks instead of 10 months. Time to market, by > writing significantly less amount > of code, layer to layer connectors, plus data mappers from one model > to another, etc, etc. Less > useless and redundant code to write and later deal with. > > 5. Spend less time maintaining the code. If the XQuery optimizer is > doing a good job, you don't need > to constantly rewrite your code to fix performance problems, as you > do in java. Plus less number of lines of > code usually means less bugs. So smaller total cost of ownership of > the code. > > That would be my quick personal answer. > > But that of course works if you know what you are doing with > XQuery .... > > Best > Dana > > > > > > > > >> If XQ is just an incremental improvement, it will always be a >> marginal language (for example, that's how I see the XQ scripting >> in the browser situation; it's simply not THAT much better than >> JavaScript, and has the disadvantage of not having a million >> libraries and code snippets and programmers). I actually think >> that it is much more than incremental in search, and it may be >> elsewhere as well. >> >> -Rob >> >> >> On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch > > wrote: >> 2009/8/31 Daniela Florescu : >> >> why do developers and architects so stubbornly refuse just to >> _listen_ ? >> > >> > Because we don't talk to them (a) enough and (b) with a message >> that they >> > understand. >> >> What is the number one "killer app" for XQuery, shining example that >> can be given to people when talking about XML databases - one that is >> high profile enough that they may have heard of it? >> >> >> >> >> -- >> Andrew Welch >> http://andrewjwelch.com >> Kernow: http://kernowforsaxon.sf.net/ >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/d6058752/attachment.htm From dflorescu at mac.com Tue Sep 1 10:09:08 2009 From: dflorescu at mac.com (Daniela Florescu) Date: Tue Sep 1 14:35:41 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: > it will always be a marginal language (for example, that's how I > see the XQ scripting in the browser situation; it's simply not THAT > much better than JavaScript, and has the disadvantage of not having > a million libraries and code snippets and programmers). XQuery in the broswer's advantage is not the fact that you write better code then in Javascript (I agree, good enough is good enough, and javascript is good enough). The major advantage of XQuery in the browser is the fact that you can do data and query shipping from the server to the browser and back. The fact that you don't need to write the processing twice : once in Java when you want it to run it on the App server, and once in Javascript when you want to run it in the client. Same XML data can be used on both sides, and same XQuery code can run on server and client, and you can decide what to run where LATER. That's a major problem with existing applications servers today: they cannot run in disconnected mode. You cannot take your relational data and SQL server with you, plus the app server and download them on your laptop before you take a plane. You CAN do that with XML and XQuery. Best regards Dana From int19h at gmail.com Tue Sep 1 09:21:28 2009 From: int19h at gmail.com (Pavel Minaev) Date: Tue Sep 1 14:45:02 2009 Subject: [xquery-talk] Tuple types In-Reply-To: <4A9D2544.8000207@oracle.com> References: <4A9D2544.8000207@oracle.com> Message-ID: On Tue, Sep 1, 2009 at 6:44 AM, John Snelson wrote: > Tuple types won't make it into XQuery 1.1, but the closures support in > higher order functions allows you to effectively return sequences of > sequences: > > (: Wrap a sequence as a function item :) > declare function local:ref($arg as item()*) as function() as item()* > { > ? ?function() { $arg } > }; > > (: Unwrap a sequence from a function item :) > declare function local:deref($arg as function() as item()*) as item()* > { > ? ?$arg() > }; This makes sense, though it would be a nuisance to have to define ref() and deref() every time. Is there any consideration to make them standard library functions? From rob.gonzalez at gmail.com Tue Sep 1 12:33:51 2009 From: rob.gonzalez at gmail.com (Rob Gonzalez) Date: Tue Sep 1 15:53:58 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: This really is the interesting question. My bet is something in search. Given that all Excel/Word/Outlook/etc. documents are or can be represented in XML, and that standard text search doesn't do a great job generally in the space (top vendors in the enterprise use facets, entity extraction, and other tricks to help the problem), XQuery has a chance to be the syntax that enables custom search solutions. Someone on this list brought up MarkMail; that's really all that is. Banging out a search solution in a standard language like XQ that's really good at dealing with semi-structured data is simply easier than using other tools. I honestly think that all the talk about XDM as a data model and XQ as a turing complete declarative language, etc., is sort of moot unless they do somehow enable something that RDBMS/BI/Data warehouses/search vendors can't readily do with existing approaches. If XQ is just an incremental improvement, it will always be a marginal language (for example, that's how I see the XQ scripting in the browser situation; it's simply not THAT much better than JavaScript, and has the disadvantage of not having a million libraries and code snippets and programmers). I actually think that it is much more than incremental in search, and it may be elsewhere as well. -Rob On Tue, Sep 1, 2009 at 4:50 AM, Andrew Welch wrote: > 2009/8/31 Daniela Florescu : > >> why do developers and architects so stubbornly refuse just to _listen_ ? > > > > Because we don't talk to them (a) enough and (b) with a message that they > > understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? > > > > > -- > Andrew Welch > http://andrewjwelch.com > Kernow: http://kernowforsaxon.sf.net/ > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/9f688093/attachment.htm From john.snelson at oracle.com Tue Sep 1 17:38:41 2009 From: john.snelson at oracle.com (John Snelson) Date: Tue Sep 1 16:10:36 2009 Subject: [xquery-talk] Tuple types In-Reply-To: References: <4A9D2544.8000207@oracle.com> Message-ID: <4A9D4001.2080208@oracle.com> Pavel Minaev wrote: > On Tue, Sep 1, 2009 at 6:44 AM, John Snelson wrote: >> Tuple types won't make it into XQuery 1.1, but the closures support in >> higher order functions allows you to effectively return sequences of >> sequences: >> >> (: Wrap a sequence as a function item :) >> declare function local:ref($arg as item()*) as function() as item()* >> { >> function() { $arg } >> }; >> >> (: Unwrap a sequence from a function item :) >> declare function local:deref($arg as function() as item()*) as item()* >> { >> $arg() >> }; > > This makes sense, though it would be a nuisance to have to define > ref() and deref() every time. Is there any consideration to make them > standard library functions? My recollection is that the WG considered adding them and rejected the idea on the grounds that functions that can be defined in XQuery itself better belong in something like EXPath function modules. It's highly likely that I'll submit such a module to EXPath in the future. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From michael.reiche at oracle.com Tue Sep 1 13:34:02 2009 From: michael.reiche at oracle.com (Michael Reiche) Date: Tue Sep 1 16:15:51 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com 3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> Message-ID: > why do developers and architects so stubbornly refuse just to _listen_ ? Maybe we should try listening to them? To?find out why they are not including xquery products in their architectures. I can make a few guesses - ? This are somethings that customers might tell you: ? 1) until recently, no update was available - making it simply not a choice. 2) there are some queries in my?xquery project?that don't?perform well.?Is this problem intrinsic to Xquery? The implementation? My xquery code? ?How do I fix it?(This is usually related to a join or predicate not pushed to a database because of the way the xquery was written.? Often in a function that calls several other functions, that call other functions... eventually involving a thousand lines of xquery or so - making the exact problem difficult to locate) 3) I found the Books/Authors xquery examples helpful.??Now, I have 5,000 database tables, and I need to join up to 50 of them at a time. Do you have any samples like that? 4) I want to include the output of a stored procedure that retuns an Object type in my xquery. How do I do that? 5) How can I use Xquery for ETL? (it needs to be better than my existing ETL tool). ? - Mike ? ? ? ? _____ ? From: Daniela Florescu [mailto:dflorescu@mac.com] Sent: Monday, August 31, 2009 3:39 PM To: Hans-Juergen Rennau Cc: talk@x-query.com Subject: Re: [xquery-talk] size of XQuery developer community Only a few days ago it occurred to me that "information processing language" may be better than "data processing language", Thanks for saying this. As soon as I hit the send button I remembered that I promised myself never use "data" in connection to XML, but "information". (I am a database person, certain habits are hard to break....) But yes, I would think too that XQuery is a declarative information processing language. why do developers and architects so stubbornly refuse just to _listen_ ? Because we don't talk to them (a) enough and (b) with a message that they understand. I remember years ago, when I was really frustrated that architects would not _listen_ that Xquery was this wonderful and useful thing, I asked advice from Jim Gray. He was always ready to listen and help, and, even though he was too much of a believer in SQL and schemas to start embracing XQuery, he gave me? the following advice. If you want people to listen then (a) write articles and give talks, (b) write software that works,? (c) make interesting demos. Then repeat, and don't loose patience. I think that at this point we have around XQuery lots of software that does work: databases, in memory processors, etc. What?we?can?do?is?write?more?articles,?give?more?talks (Web 2.0, eGov 2.0 anyone ?),?and?make more? interesting?demos. About demos: I am still surprised that nobody is trying XQuery's mashup strength on the examples on?http://www.programmableweb.com/. ?Wonderful site, BTW. Or that noone is using XQuery on the http://www.data.gov/?site and win the price of the best eGovernment application.... Best regards Dana With kind regards, Hans-Juergen ----- Urspr?ngliche Mail ---- Von: Daniela Florescu An: Hans-Juergen Rennau CC: HYPERLINK "mailto:talk@x-query.com"talk@x-query.com Gesendet: Montag, den 31. August 2009, 08:58:34 Uhr Betreff: Re: [xquery-talk] size of XQuery developer community a widespread lack of a _deep_ understanding of XML. Hans-Juergen, Yes, you are right. ?There is a deep misunderstanding in IT of our core message about the role of XML in the new architectures. XML is not syntax, it can/should be the primary way of modeling data. XQuery is not a query language (oh well), but a full blown (declarative) data processing language. However, if the world misunderstands it, it is hardly the world's fault. It is that we do not express this message in a clear, concise and convincing enough manner -- and probably not in the right places, either. I hope we don't have to wait 20 years for the message to be heard.... Best regards Dana _______________________________________________ HYPERLINK "mailto:talk@x-query.com"talk@x-query.com http://x-query.com/mailman/listinfo/talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/3ff1d2a6/attachment.htm From john.snelson at oracle.com Tue Sep 1 15:44:36 2009 From: john.snelson at oracle.com (John Snelson) Date: Tue Sep 1 16:31:01 2009 Subject: [xquery-talk] Tuple types In-Reply-To: References: Message-ID: <4A9D2544.8000207@oracle.com> Hi Pavel, Tuple types won't make it into XQuery 1.1, but the closures support in higher order functions allows you to effectively return sequences of sequences: (: Wrap a sequence as a function item :) declare function local:ref($arg as item()*) as function() as item()* { function() { $arg } }; (: Unwrap a sequence from a function item :) declare function local:deref($arg as function() as item()*) as item()* { $arg() }; John Pavel Minaev wrote: > I can't find the link now, but I recall seeing some mentions of a > possibility of introducing some form of tuple type (in a sense of an > aggregate of arbitrary-typed values - including sequences) in XQuery > 1.1, specifically to tackle the multiple-return-values and > sequences-of-sequences problems. Are there any recent developments on > the subject? I don't see anything like this in the most recent draft, > but then higher-order functions aren't there either, and yet, judging > by their presence in Saxon 9.2, they are well-shaped by now. > > If this isn't going to make it into 1.1, are there any other new > features that I might have missed that would make it possible, for > example, for a function to return two sequences? > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From rpbourret at rpbourret.com Tue Sep 1 20:36:42 2009 From: rpbourret at rpbourret.com (Ronald Bourret) Date: Tue Sep 1 19:21:14 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: <4A9DDA3A.9070809@rpbourret.com> I agree with Daniela. For some real-world examples of this kind of thing, see: http://www.rpbourret.com/xml/UseCases.htm It's worth noting that people tried to build many of these applications on relational databases and failed. The article explains some of the reasons why. This article is based on a non-scientific survey of XML database companies. It was done in 2005 and included about half of the XML database companies at the time, including most of the major ones. -- Ron Daniela Florescu wrote: >> >> >> I honestly think that all the talk about XDM as a data model and XQ as >> a turing complete declarative language, etc., is sort of moot unless >> they do somehow enable something that RDBMS/BI/Data warehouses/search >> vendors can't readily do with existing approaches. > > Well, that's relatively simple to answer, I would say. > > Here is are some examples of what you could do with XML/XQuery that you > cannot do with the traditional three > tier stack (communication + application logic written in an imperative > language + declarative query layer) > > 1. Have schema flexibility. Start writing code without a fixed idea of > your schema, and/or change > your mind later. (one schema is evil enough, three layers of > superimposed schemas, in three different models, > are exponentially evil for flexibility and customizability. Almost no > evolution of schema is possible.) > > 2. Integrate text and documents into your information processing flow > without adding software license, yet another layer, > more mappings, and different programming languages. > > 3. Optimize the performance of your information processing program > globally, not locally, layer by layer, > by adding more and more intermediate caches and shortcuts, which in turn > will turn your code maintenance a nightmare. > > 4. Be done in 2 weeks instead of 10 months. Time to market, by writing > significantly less amount > of code, layer to layer connectors, plus data mappers from one model to > another, etc, etc. Less > useless and redundant code to write and later deal with. > > 5. Spend less time maintaining the code. If the XQuery optimizer is > doing a good job, you don't need > to constantly rewrite your code to fix performance problems, as you do > in java. Plus less number of lines of > code usually means less bugs. So smaller total cost of ownership of the > code. > > That would be my quick personal answer. > > But that of course works if you know what you are doing with XQuery .... > > Best > Dana From rpbourret at rpbourret.com Tue Sep 1 20:41:00 2009 From: rpbourret at rpbourret.com (Ronald Bourret) Date: Tue Sep 1 19:25:28 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> Message-ID: <4A9DDB3C.7070904@rpbourret.com> Hmmmmm. What's the single killer app for SQL? In both cases, I don't think there is one, in the sense that the Web is the killer app for the Internet or spreadsheets were the killer app for PCs. I think that the combination of XQuery and XML databases simply make it possible to build applications that weren't feasible before because of limitations of the relational model to represent certain kinds of data, such as documents and semi-structured data. Whether any of these is killer really depends on what business you're in. If you're an accountant, I doubt you'll be interested. If you're a publisher, you probably are. -- Ron Andrew Welch wrote: > 2009/8/31 Daniela Florescu : >>> why do developers and architects so stubbornly refuse just to _listen_ ? >> Because we don't talk to them (a) enough and (b) with a message that they >> understand. > > What is the number one "killer app" for XQuery, shining example that > can be given to people when talking about XML databases - one that is > high profile enough that they may have heard of it? From gary.m.lewis at gmail.com Tue Sep 1 12:40:42 2009 From: gary.m.lewis at gmail.com (Gary Lewis) Date: Tue Sep 1 19:33:02 2009 Subject: [xquery-talk] size of XQuery developer community Message-ID: <1be6002b0909010840g21ea64f3r25b52b351a965fc0@mail.gmail.com> On Wed Aug 26 17:39:05 PDT 2009, Daniela Florescu wrote: > ... XQuery has tremendous potential for adding > value to customers, but the proof isn't there yet, and the path isn't > clear either. > > There has to be a larger XQuery pool of expertise ... Hi Daniela - I finally got around to reading this entire thread thru to 9/1. It's truly humbling to see how much about XQuery, XML, etc I do not know. But perhaps my newcomer status will provide a useful perspective. First some quick background. I definitely don't yet qualify as an XQuery developer. But I've been working pretty conscientiously over the past 4 or 5 months. I come to XQuery with decades of SQL, data warehouse design, and policy analysis background (ie, I see databases as a means to analysis, not an end in themselves). I got interested in XQuery because I needed a tool that would let me query the Web in a similar fashion to querying relational databases. The Web and web query tools are still primitive by comparison, but I've been quite pleased with what's possible. For example, my latest XQuery demonstration project mashed up US Dept of Education data with Federal Reserve data to examine the question of whether higher education in the US is countercyclical. See: http://garymlewis.com/instchg/2009/08/10/another-xquery-use-case-is-higher-education-countercyclical/ Given a choice, I'll almost always choose tools with power and capability versus those with ease-of-use but limited application. So, for example, I'm willing to curse and beat my head against a wall every time I use the R stat programs because I know that somehow there's a solution in R and that someone in the R community will know what the solution is. I've probably written several thousand XQuery programs now. But I only feel comfortable in a very narrow niche. There is just oodles about XML and all the other X standards and tools that I do not know. Some of this is surely just beginner's lament (ie, the notion that anything you can't learn instantly is way too complex). But some too is a steep learning curve in the absence of adequate support for learning. You ask why there is not more XQuery expertise. The XQuery community might benefit from looking at the R community [see: http://www.r-project.org/ ] and the considerable help resources available there. It's easier for a developer to get untracked with any new tool if there is visible help available. With regard to R, I'm talking here about specialized search sites, repositories of learning materials, open source and free tools, dozens upon dozens of forums, examples as code fragments, online books and manuals, and a community of very active R developers/enhancers. I like XQuery a lot. I will continue to use it. And no doubt in the course of using it, I'll fill in the many holes of what I don't know. But I could sure benefit from something like an XQuery learning resource site. Gary From steve at asanderson.com Wed Sep 2 00:31:57 2009 From: steve at asanderson.com (A. Steven Anderson) Date: Tue Sep 1 20:15:27 2009 Subject: [xquery-talk] How XML Threatens Big Data Message-ID: How timely....;-) http://dataspora.com/blog/xml-and-big-data/ -- A. Steven Anderson Independent Consultant -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090901/ddaefb71/attachment.htm From kennorth at sbcglobal.net Tue Sep 1 21:37:30 2009 From: kennorth at sbcglobal.net (Ken North) Date: Tue Sep 1 20:20:51 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <4A9DDB3C.7070904@rpbourret.com> References: <200908261828.n7QISsCq032486@jhunter.x-query.com> <238672.26663.qm@web27104.mail.ukl.yahoo.com> <7E20E567-FFAA-4C9E-81C1-59808F294260@mac.com> <751372.77346.qm@web27104.mail.ukl.yahoo.com> <3F5D272C-12EB-43D4-8D43-0AEC4A0A6643@mac.com> <74a894af0909010150n3c24ade1l4bacaecf09b213be@mail.gmail.com> <4A9DDB3C.7070904@rpbourret.com> Message-ID: <380785.26893.qm@web81304.mail.mud.yahoo.com> Ron Bourret wrote: >>?What's the single killer app for SQL? Ad hoc queries. For the most part, the?DBMS platforms that?preceded the arrival of SQL, the hierarchical and CODASYL models, implemented queries with?traversal logic in application programs. There were exceptions, including IBM GIS and TRW GIM, that actually used English-like commands for searching, deleting, inserting and updating. But IMS and CODASYL were the defacto and de jure standard and?writing a program?using a known schema and sub-schema was how you implemented a query. From mail at martin-probst.com Wed Sep 2 10:33:10 2009 From: mail at martin-probst.com (Martin Probst) Date: Wed Sep 2 00:12:08 2009 Subject: [xquery-talk] Help with multiple deletes In-Reply-To: References: Message-ID: <6dca08520909020033s550ede56y4cdb58d428fb8603@mail.gmail.com> I guess your approach is a bit different than what the spec planned - their idea was that you'd just want to perform the updates, but not "witness" the effects. A different approach would be this: copy $x := doc("tree.xml") modify ( let $node := $x//tree[@ID = 1] /branch[@ID = 2] /branch[@ID = 13] /branch[@ID = 65] /branch[@ID = 15] /leaf[@ID = 25] let $empty-branches := $x//branch[count(.//leaf except $node) = 0] let $empty-trees := $x//tree[count(.//branch except $empty-branches) = 0] return delete nodes ($empty-trees, $empty-branches, $node) ) return $x There might be more efficient ways to find the nodes that need pruning, in particular if your implementation would support the ancestor:: axis, you could walk upwards from $node instead of scanning the whole tree for the $empty-* variables. Martin From mlu at as-guides.com Wed Sep 2 11:49:57 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Wed Sep 2 01:27:47 2009 Subject: [xquery-talk] How XML Threatens Big Data In-Reply-To: References: Message-ID: <4A9E31B5.10408@as-guides.com> A. Steven Anderson schrieb: > How timely....;-) > http://dataspora.com/blog/xml-and-big-data/ Well, if data is tabular, what benefits can be obtained by storing it as XML? None that I could see. If you later need XML, it is really easy to generate. So why would you store it as XML in the first place? Because you've fallen victim to some hype? That does not mean that "XML fails". (See Nestor's comment on the blog.) If your data is full of optional stuff and pretty irregular in general, XML can mean a significant improvement. A twenty-table-join is not easy to understand. I've never set up Hibernate, but I think that isn't easy either - a handy 700 page manual is required to bridge the OO/RDBMS/OO mismatch. For those situations, objects or document databases may be a much better match. There's more than one tool in the box. Also, it never fails to amaze me how people can think that documents exceeding a certain size - 1 MB, okay: 5 MB - are a good idea. Just imagine the amount of processing that has to be done before discovering on parse completion that something's wrong. -- Michael Ludwig From dengpingwei at gmail.com Wed Sep 2 13:36:50 2009 From: dengpingwei at gmail.com (dengping wei) Date: Wed Sep 2 03:13:10 2009 Subject: [xquery-talk] request: get the attribute value Message-ID: Dear all, I have a question when I using xquery the get the attribute value of an element. when my query is for $x in doc(\"aa.wsdl\")/wsdl: definitions/wsdl:types/xsd:schema/xsd:complexType return {$x} It returns the results as follows: Then I want to get the value of attribute "name", I use the query for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType return {$x/@name} It works and return the results: But when I want to get value of attribute "sawsdl:modelReference", I use the query for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType return {$x/@sawsdl:modelReference} It returns nothing. How could I get the value of the attribute " sawsdl:modelReference"? Thanks very much. -- Dengping WEI School of Computer Science, National University of Defense Technology, Changsha, Hunan 410073 P.R. China -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090902/5f143868/attachment-0001.htm From lists at fgeorges.org Wed Sep 2 11:44:02 2009 From: lists at fgeorges.org (Florent Georges) Date: Wed Sep 2 03:20:14 2009 Subject: [xquery-talk] Tuple types In-Reply-To: <4A9D4001.2080208@oracle.com> Message-ID: <354837.7241.qm@web23001.mail.ird.yahoo.com> John Snelson wrote: Hi, > > This makes sense, though it would be a nuisance to have to > > define ref() and deref() every time. Is there any > > consideration to make them standard library functions? > My recollection is that the WG considered adding them and > rejected the idea on the grounds that functions that can be > defined in XQuery itself better belong in something like EXPath > function modules. It's highly likely that I'll submit such a > module to EXPath in the future. My first reaction was: "Why? Just use the HOF feature directly." But defining ref() and deref() would actually have the double advantage to 1/ make the intent clearer and 2/ enable one to provide them on a processor that does not support HOFs (yet.) Good idea. Regards, -- Florent Georges http://www.fgeorges.org/ From lists at fgeorges.org Wed Sep 2 11:57:41 2009 From: lists at fgeorges.org (Florent Georges) Date: Wed Sep 2 03:33:39 2009 Subject: [xquery-talk] request: get the attribute value In-Reply-To: Message-ID: <648596.77747.qm@web23003.mail.ird.yahoo.com> dengping wei wrote: Hi, > Then I want to get the value of attribute "name", > I use the query [...] return {$x/@name} BTW, that does not give you the value of the attribute @name but that gives you the attribute node itself (that's why you end up with an attribute on your element 'node'. > [...] > {$x/@sawsdl:modelReference} > It returns nothing. How could I get the value of the > attribute " sawsdl:modelReference"? Because you wrote doc(\"aa.wsdl\") (with the backslashes) I guess you evaluate your queries directly from Java or another similar language. How do you bind the prefix 'sawsdl' to the correct URI on the static context? Regards, -- Florent Georges http://www.fgeorges.org/ From mpilman at student.ethz.ch Wed Sep 2 14:08:46 2009 From: mpilman at student.ethz.ch (Markus Pilman) Date: Wed Sep 2 03:44:55 2009 Subject: [xquery-talk] request: get the attribute value In-Reply-To: References: Message-ID: <60f8b5ce0909020408r6f124842l58367e12c59cfd55@mail.gmail.com> > But when I want to get value of attribute "sawsdl:modelReference", I use the > query > > for $x in > doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType > return {$x/@sawsdl:modelReference} > > It returns nothing. How could I get the value of the attribute " > sawsdl:modelReference"? Only to be sure about that: did you declare the namespace? You would do that by adding the line declare namespace sawsdl="http://www.w3.org/ns/sawsdl"; As an alternative you could use the query for $x in doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType return {$x/@*:modelReference} which looks for attributes "modelReference" in all namespaces. Markus On Wed, Sep 2, 2009 at 12:36 PM, dengping wei wrote: > Dear all, > > I have a question when I using xquery the get the attribute value of an > element. > > when my query is > for $x in doc(\"aa.wsdl\")/wsdl: > definitions/wsdl:types/xsd:schema/xsd:complexType > return {$x} > It returns the results as follows: > > ?? ??????????????????? xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" > > xmlns:impl="http://dmas.dfki.de/axis/services/CarPrice-impl" > ??????????????????? xmlns:sawsdl="http://www.w3.org/ns/sawsdl" > > xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" > ??????????????????? xmlns:intf="http://dmas.dfki.de/axis/services/CarPrice" > ??????????????????? xmlns:apachesoap="http://xml.apache.org/xml-soap" > ??????????????????? xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" > ??????????????????? xmlns:xsd="http://www.w3.org/2001/XMLSchema" > ??????????????????? xmlns="http://dmas.dfki.de/axis/services/CarPrice" > ??????????????????? name="PriceType" > > sawsdl:modelReference="http://127.0.0.1/ontology/concept.owl#Price"> > ??????? > ????????? > ????????? > ??????? > ????? > > > Then I want to get the value of attribute "name", I use the query > for $x in > doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType > return {$x/@name} > > It works and return the results: > > But when I want to get value of attribute "sawsdl:modelReference", I use the > query > > for $x in > doc(\"aa.wsdl\")/wsdl:definitions/wsdl:types/xsd:schema/xsd:complexType > return {$x/@sawsdl:modelReference} > > It returns nothing. How could I get the value of the attribute " > sawsdl:modelReference"? > > Thanks very much. > > > > -- > Dengping WEI > School of Computer Science, > National University of Defense Technology, > Changsha, Hunan ?410073 > P.R. China > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From lists at fgeorges.org Wed Sep 2 17:15:33 2009 From: lists at fgeorges.org (Florent Georges) Date: Wed Sep 2 09:09:22 2009 Subject: [xquery-talk] request: get the attribute value In-Reply-To: Message-ID: <952079.26067.qm@web23002.mail.ird.yahoo.com> dengping wei wrote: > staticContext.declareNamespace("sawsdl", > "??? http://www.w3.org/ns/sawsdl"); > xmlns:sawsdl="http://www.w3.org/ns/sawsdl" Both strings do not match (see the extra whitespaces in Java.) Regards, -- Florent Georges http://www.fgeorges.org/ From int19h at gmail.com Wed Sep 2 10:36:59 2009 From: int19h at gmail.com (Pavel Minaev) Date: Wed Sep 2 09:30:40 2009 Subject: [xquery-talk] Tuple types In-Reply-To: <354837.7241.qm@web23001.mail.ird.yahoo.com> References: <4A9D4001.2080208@oracle.com> <354837.7241.qm@web23001.mail.ird.yahoo.com> Message-ID: On Wed, Sep 2, 2009 at 3:44 AM, Florent Georges wrote: >> > This makes sense, though it would be a nuisance to have to >> > define ref() and deref() every time. ?Is there any >> > consideration to make them standard library functions? > >> My recollection is that the WG considered adding them and >> rejected the idea on the grounds that functions that can be >> defined in XQuery itself better belong in something like EXPath >> function modules. ?It's highly likely that I'll submit such a >> module to EXPath in the future. > > ?My first reaction was: "Why? Just use the HOF feature directly." It's an obvious solution in case of deref(), but having to type out identity lambdas in full every time will get repetitive fast, which is why I think ref() is still handy regardless. From rasmussen.bryan at gmail.com Thu Sep 3 11:21:39 2009 From: rasmussen.bryan at gmail.com (bryan rasmussen) Date: Thu Sep 3 00:58:58 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <1be6002b0909010840g21ea64f3r25b52b351a965fc0@mail.gmail.com> References: <1be6002b0909010840g21ea64f3r25b52b351a965fc0@mail.gmail.com> Message-ID: <3bb44c6e0909030121n7d3920fag7fbe2cfd48b2b7fc@mail.gmail.com> The difference between R and XQuery however is that R is a language with one implementation. Thus the site for downloading R can also provide all this information to users, for XQuery it is too fragmented. Best Regards, Bryan Rasmussen On Tue, Sep 1, 2009 at 5:40 PM, Gary Lewis wrote: > On Wed Aug 26 17:39:05 PDT 2009, Daniela Florescu wrote: > >> ... XQuery has tremendous potential for adding >> value to customers, but the proof isn't there yet, and the path isn't >> clear either. >> >> There has to be a larger XQuery pool of expertise ... > > Hi Daniela - I finally got around to reading this entire thread thru > to 9/1. It's truly humbling to see how much about XQuery, XML, etc I > do not know. > > But perhaps my newcomer status will provide a useful perspective. > > First some quick background. I definitely don't yet qualify as an > XQuery developer. But I've been working pretty conscientiously over > the past 4 or 5 months. I come to XQuery with decades of SQL, data > warehouse design, and policy analysis background (ie, I see databases > as a means to analysis, not an end in themselves). I got interested in > XQuery because I needed a tool that would let me query the Web in a > similar fashion to querying relational databases. The Web and web > query tools are still primitive by comparison, but I've been quite > pleased with what's possible. For example, my latest XQuery > demonstration project mashed up US Dept of Education data with Federal > Reserve data to examine the question of whether higher education in > the US is countercyclical. See: > http://garymlewis.com/instchg/2009/08/10/another-xquery-use-case-is-higher-education-countercyclical/ > > Given a choice, I'll almost always choose tools with power and > capability versus those with ease-of-use but limited application. So, > for example, I'm willing to curse and beat my head against a wall > every time I use the R stat programs because I know that somehow > there's a solution in R and that someone in the R community will know > what the solution is. > > I've probably written several thousand XQuery programs now. But I only > feel comfortable in a very narrow niche. There is just oodles about > XML and all the other X standards and tools that I do not know. Some > of this is surely just beginner's lament (ie, the notion that anything > you can't learn instantly is way too complex). But some too is a steep > learning curve in the absence of adequate support for learning. > > You ask why there is not more XQuery expertise. The XQuery community > might benefit from looking at the R community [see: > http://www.r-project.org/ ] and the considerable help resources > available there. It's easier for a developer to get untracked with any > new tool if there is visible help available. With regard to R, I'm > talking here about specialized search sites, repositories of learning > materials, open source and free tools, dozens upon dozens of forums, > examples as code fragments, online books and manuals, and a community > of very active R developers/enhancers. > > I like XQuery a lot. I will continue to use it. And no doubt in the > course of using it, I'll fill in the many holes of what I don't know. > But I could sure benefit from something like an XQuery learning > resource site. > > Gary > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From mlu at as-guides.com Thu Sep 3 12:35:18 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Thu Sep 3 02:11:47 2009 Subject: [xquery-talk] How XML Threatens Big Data In-Reply-To: <6ce57d220909021549y7893c879o3cec7623da8b3ed2@mail.gmail.com> References: <4A9E31B5.10408@as-guides.com> <6ce57d220909021549y7893c879o3cec7623da8b3ed2@mail.gmail.com> Message-ID: <4A9F8DD6.7010806@as-guides.com> Ivan Pedruzzi schrieb: > On Wed, Sep 2, 2009 at 4:49 AM, Michael Ludwig > wrote: >> Also, it never fails to amaze me how people can think that documents >> exceeding a certain size - 1 MB, okay: 5 MB - are a good idea. Just >> imagine the amount of processing that has to be done before >> discovering on parse completion that something's wrong. > > If you consider 5 MB documents a bad idea may be you are not using the > appropriate tools :) > > Ivan Pedruzzi > Progress DataDirect Technologies If I control the whole processing pipeline I can use the tools I want. But if I publish documents over the wire I may not be in a position to tell consumers what tools to use. -- Michael Ludwig From gary.m.lewis at gmail.com Thu Sep 3 09:21:52 2009 From: gary.m.lewis at gmail.com (Gary Lewis) Date: Thu Sep 3 05:21:48 2009 Subject: [xquery-talk] size of XQuery developer community Message-ID: <1be6002b0909030521m71894878k5e7042cfa98b10f1@mail.gmail.com> > The difference between R and XQuery however is that R is a language > with one implementation. > Thus the site for downloading R can also provide all this information > to users, for XQuery it is too fragmented. > > Best Regards, > Bryan Rasmussen Hi Bryan - Absolutely true. And that's part of the point I was trying to make. If the community of xquery developers is small, part of the reason is the multitude of xquery implementations. That's not going to change, but the various implementors might want to consider what they have in common and then do something (eg, create an xquery learning resource site) that would benefit all xquery developers regardless of the flavor of xquery they use. Yes, very definitely I am saying that part of the cause for Daniela's lament originates in the balkanization of the xquery community. Gary From oliver at xqsharp.com Thu Sep 3 14:57:01 2009 From: oliver at xqsharp.com (Oliver Hallam) Date: Thu Sep 3 06:01:59 2009 Subject: [xquery-talk] [ANN] XQSharp 1.0 (XQuery for the .NET Framework) Message-ID: <4A9FBD1D.8030808@xqsharp.com> The XQSharp team are pleased to announce the long-awaited release of XQSharp 1.0 XQSharp is an XQuery processor for the .NET Framework. It is developed by Clinical & Biomedical Computing Limited, a member of the W3C. XQSharp 1.0 is the first complete release of XQSharp containing our .NET API, which has been designed to work in conjunction with the classes in the System.Xml namespace. The full API Reference can be viewed at http://www.xqsharp.com/xqsharp/doc/. Key Features: * Up-to-date and Standards Compliant (exemplary performance in the XQuery Test Suite) * Schema-Aware * High performance * Comprehensive and well documented API, with full IntelliSense documentation. * All optional XQuery features implemented * Full implementation of the XDM Serialization specification. * XPath 2.0 support sharing the same optimization and runtime engine as XQuery. * Can be used in Low and Medium Trust environments. It is completely free for non-commercial use; commercial licenses are also available. For more information please visit http://www.xqsharp.com/ or contact us at info@xqsharp.com Oliver Hallam http://www.xqsharp.com From joewiz at gmail.com Thu Sep 3 17:00:32 2009 From: joewiz at gmail.com (Joe Wicentowski) Date: Thu Sep 3 12:49:27 2009 Subject: [xquery-talk] size of XQuery developer community In-Reply-To: <1be6002b0909030521m71894878k5e7042cfa98b10f1@mail.gmail.com> References: <1be6002b0909030521m71894878k5e7042cfa98b10f1@mail.gmail.com> Message-ID: <2e0234190909031300kb1fc933q78c8ff0b6ecd26b4@mail.gmail.com> On Thu, Sep 3, 2009 at 8:21 AM, Gary Lewis wrote: > ... but the various implementors might want to consider what they > have in common and then do something (eg, create an xquery learning > resource site) that would benefit all xquery developers regardless of > the flavor of xquery they use. I think the XQuery Wikibook http://en.wikibooks.org/wiki/XQuery is an ideal starting point. It's got a great base of material already. It's currently very eXist-centric, but there's no reason that there couldn't be articles about other implementations. - Joe From hrennau at yahoo.de Fri Sep 4 00:08:09 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Thu Sep 3 22:43:46 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery Message-ID: <879025.97180.qm@web27105.mail.ukl.yahoo.com> As Daniela Florescu emphasized, XQuery is an information processing language, rather than a query language, and this makes it a great implementation language for tool development. But two problems discourage the use of "raw queries" - without frontent or wrapping of any sort - as tools: a) no default parameters b) no standard-based access to system commands, namely no file discovery a) is in conflict with the requirement of powerful tools to offer a great number of options; and b) forces the caller to specifiy all input files - possibly hundreds or more - explicitly. (Or to provide secondary input files containing the file names, making things complicated..) So David A. Lee recently wrote: "I had to write a bunch of "framework" code in Java just to arrange to *call* xquery ( you know, find the input files, get the xquery files, setup the parameters, call it, serialize the results etc)." David's contribution drew my attention to xmlsh which seems to me a fascinating approach. But I feel that (at least presently) tool development per XQuery needs a lightweight approach to wrapping. (No?) I have taken to an extremely simple technique of wrapping queries in Perl, thus overcoming the problems mentioned above: a) get the powerful command line interface of Perl scripts (options and arguments, defaulting) b) leave the preliminary file discovery to Perl (e.g. per glob) The queries themselves I insert as here documents right into the Perl code, one function per query, like this: sub xq_foo { return << 'EOF'; (: here comes query 'foo', possibly thousands of lines ... :) EOF } This is significant for keeping things simple: one Perl script can infact be a library with dozens of queries of unlimited complexity. (This way the tool software can be structured into a small number of deliverables each of which is a whole application, rather than a script in the conventional sense.) The approach is rounded off by letting Perl select the query, construct the command line invocation of your XQuery processor (e.g. Saxon), calling the processor and returning the output on stdout. Having written a couple of such Perl-XQuery-hybrids you recognize a uniform pattern of how to organize this little translation of Perl input into XQuery invocation, including input checks, so that from the third specimen onwards you have, in terms of effort, virtually no overhead, just write query after query INTO your Perl frame, and add invocation branch after invocation branch. Now my questions: - would you recommend alternatives for rapid tool development? - if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect? Any feedback would be highly welcome! With kind regards, Hans-Juergen Rennau From john.snelson at oracle.com Fri Sep 4 11:39:10 2009 From: john.snelson at oracle.com (John Snelson) Date: Fri Sep 4 02:10:54 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <879025.97180.qm@web27105.mail.ukl.yahoo.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> Message-ID: <4AA0E03E.9090608@oracle.com> Hans-Juergen Rennau wrote: > As Daniela Florescu emphasized, XQuery is an information processing language, rather than a query language, and this makes it a great implementation language for tool development. But two problems discourage the use of "raw queries" - without frontent or wrapping of any sort - as tools: > > a) no default parameters > b) no standard-based access to system commands, namely no file discovery I think that these are very good examples of functions that need to be added to EXPath. > a) is in conflict with the requirement of powerful tools to offer a great number of options; and b) forces the caller to specifiy all input files - possibly hundreds or more - explicitly. (Or to provide secondary input files containing the file names, making things complicated..) > > So David A. Lee recently wrote: > "I had to write a bunch of "framework" code in Java just to arrange to *call* xquery ( you know, find the input files, get the xquery files, setup the parameters, call it, serialize the results etc)." > > David's contribution drew my attention to xmlsh which seems to me a fascinating approach. But I feel that (at least presently) tool development per XQuery needs a lightweight approach to wrapping. (No?) I have taken to an extremely simple technique of wrapping queries in Perl, thus overcoming the problems mentioned above: > > a) get the powerful command line interface of Perl scripts (options and arguments, defaulting) > b) leave the preliminary file discovery to Perl (e.g. per glob) > > The queries themselves I insert as here documents right into the Perl code, one function per query, like this: > > sub xq_foo { > return << 'EOF'; > (: here comes query 'foo', possibly thousands of lines ... :) > EOF > } > > This is significant for keeping things simple: one Perl script can infact be a library with dozens of queries of unlimited complexity. (This way the tool software can be structured into a small number of deliverables each of which is a whole application, rather than a script in the conventional sense.) The approach is rounded off by letting Perl select the query, construct the command line invocation of your XQuery processor (e.g. Saxon), calling the processor and returning the output on stdout. Having written a couple of such Perl-XQuery-hybrids you recognize a uniform pattern of how to organize this little translation of Perl input into XQuery invocation, including input checks, so that from the third specimen onwards you have, in terms of effort, virtually no overhead, just write query after query INTO your Perl frame, and add invocation branch after invocation branch. > > Now my questions: > - would you recommend alternatives for rapid tool development? I can't fault what you're doing right now - but it would be nice to be able to handle this in pure XQuery. > - if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect? I think that pure functional XQuery is always going to have issues with tool writing, since it believes that the world doesn't change during the time a query is executed (and goes to great lengths to make this seem so). This is clearly a good use case for XQuery Scripting Extensions, which introduces a notion of time passing and the world changing whilst still remaining largely functional. http://www.w3.org/TR/xquery-sx-10/ John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From dlee at calldei.com Fri Sep 4 08:09:21 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 03:39:46 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <879025.97180.qm@web27105.mail.ukl.yahoo.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> Message-ID: <4AA0F561.5040605@calldei.com> > > Now my questions: > - would you recommend alternatives for rapid tool development? > - if taking a similar approach - would you like to recommend special details, perhaps the script language actually used, or other aspect? > In my opinion the problem with doing this in perl is that unless the xquery implementation itself is in perl or atleast runs within the same process you will run into horrible performance problems. See my (with Norm Walsh's) paper : http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html What we found is that for our test cases there is a 100-200x (yes 10000 % - 20000 % ) performance penalty of using a scripting language to call xml processing programs. This *can* be optimized but the exact use cases of using a off-the-shelf scripting language to do this kind of thing is typically by the audience of people who do not want to spend the extra effort to optimize it, or who are not experts in the type of software development/languages required to do it, or both. i.e its exactly why they are using scripting - so they don't have to do all that extra work. This is the primary reason xmlsh was invented instead of re-using an existing scripting language. I took a "toy" program in a scripting language it worked great. But when I loaded up all the files I needed it to run it died a horrible death. This is what I call "The Brick Wall" and why scripting XML processes fail so many of us. The presentation cited above has some good charts and figures as well as the full test case code. This is why I suggest either (both) * Use a scripting language that already is 'in process' with all the XML core languages you want to use (xquery, xslt etc) -> examples XProc, xmlsh * Encourage scripting languages developers to embed these XML languages directly into the scripting languages (say perl). -> This is hard work and may in fact involve re-implementing many of the core tools from scratch. -> Some of the work is done but is incomplete ... I've seen references to XSLT implementions native in perl where the author quoted something like "This isnt a complete implementation of XSLT 1.0 but it works pretty good for me". -- David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From mail at martin-probst.com Fri Sep 4 14:29:03 2009 From: mail at martin-probst.com (Martin Probst) Date: Fri Sep 4 03:59:03 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA0F561.5040605@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: <6dca08520909040429k7dbb9ea5oba2723c7edcf2596@mail.gmail.com> > This is why I suggest either (both) > > * Use a scripting language that already is 'in process' with all the XML > core languages you want to use (xquery, xslt etc) > -> examples XProc, xmlsh > > * Encourage scripting languages developers to embed these XML languages > directly into the scripting languages (say perl). While XProc is probably the cleanest solution, another approach in particular for ad-hoc scripting might be to use an existing scripting language that can run on top of the JVM (1), e.g. jython or jruby. This gives you the flexibility of handy scripting but doesn't incur the overhead of re-parsing huge documents. And of course on the JVM you'll find implementations for everything-XML, from XSLT via XQuery and XML databases to various validation tools etc., all nicely co-existing based on the DOM and SAX (2). Regards Martin (1) this might also be possible on other platforms, e.g. the CLR, but I have no clue about those (2) while I'm the first to bash the DOM for its horrible programming interface, it's role as a common ground for all XML tools is wonderful. From jakob.fix at gmail.com Fri Sep 4 14:36:10 2009 From: jakob.fix at gmail.com (Jakob Fix) Date: Fri Sep 4 04:06:01 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: Talking of tools, what is the experience of people on this list with xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a close contender to xmlsh (but haven't looked closely)? cheers, Jakob. On Fri, Sep 4, 2009 at 13:09, David A. Lee wrote: > > > >> >> Now my questions: - would you recommend alternatives for rapid tool >> development? - if taking a similar approach - would you like to recommend >> special details, perhaps the script language actually used, or other aspect? >> > > In my opinion the problem with doing this in perl is that unless the xquery > implementation itself is in perl or atleast runs within the same process you > will run into horrible performance problems. ? See my (with Norm Walsh's) > paper : > > http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html > > What we found is that for our test cases there is a 100-200x (yes 10000 % - > 20000 % ) performance penalty of using a scripting language to call xml > processing programs. ? This *can* be optimized but the exact use cases of > using a off-the-shelf scripting language to do this kind of thing is > typically by the audience of people who do not want to spend the extra > effort to optimize it, or who are not experts in the type of software > development/languages required to ?do it, or both. i.e its exactly why they > are using scripting - so they don't have to do all that extra work. > > This is the primary reason xmlsh was invented instead of re-using an > existing scripting language. ? I took a "toy" program in a scripting > language it worked great. > But when I loaded up all the files I needed it to run it died a horrible > death. ? This is what I call "The Brick Wall" and why scripting XML > processes fail so many of us. ? The presentation cited above has some good > charts and figures as well as the full test case code. > > This is why I suggest either (both) > > * Use a scripting language that already is 'in process' with all the XML > core languages you want to use (xquery, xslt etc) > -> examples XProc, xmlsh > > * Encourage scripting languages developers to embed these XML languages > directly into the scripting languages (say perl). > -> This is hard work and may in fact involve re-implementing many of the > core tools from scratch. > -> Some of the work is done but is incomplete ... I've seen references to > XSLT implementions native in perl where the author quoted ?something like > ?"This isnt a complete implementation of XSLT 1.0 but it works pretty good > for me". > > > -- > David A. Lee > dlee@calldei.com ?http://www.calldei.com > http://www.xmlsh.org > 812-482-5224 > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From dlee at calldei.com Fri Sep 4 08:46:52 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 04:16:38 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <6dca08520909040429k7dbb9ea5oba2723c7edcf2596@mail.gmail.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <6dca08520909040429k7dbb9ea5oba2723c7edcf2596@mail.gmail.com> Message-ID: <4AA0FE2C.5010009@calldei.com> > While XProc is probably the cleanest solution, another approach in > particular for ad-hoc scripting might be to use an existing scripting > language that can run on top of the JVM (1), e.g. jython or jruby. > This gives you the flexibility of handy scripting but doesn't incur > the overhead of re-parsing huge documents. And of course on the JVM > you'll find implementations for everything-XML, from XSLT via XQuery > and XML databases to various validation tools etc., all nicely > co-existing based on the DOM and SAX (2). > > Regards > Martin > jypthon or jruby solve the performance problem but they dont solve the complexity problem. The fact is these XML API technologies are a royal pain to call and worse to glue to together. If you can find one vendor/ implementation that has all the pices its a tad easier (document types match !) but its still a bit of hell. Then add technologies that VendorA doesnt do and try to wrap VendorB's data type and push it through VendorA's API. I think you'll find it about as painful to to use jruby or jython to say call xinclude , then rng validate then xquery then pass its results to xslt as writing it in pure java. In xproc this is very clean, but admittedly verbose. (guess about 40 lines of xproc code). In xmlsh its as simple as xinclude file | rngvalidate schema | xquery -f schema | xslt -f stylesheet To get these kind of languages (jruby etc) into shape where XML is actually *easy to use* is going to require work. Possibly at the language level to make it really easy. If this is done to more scripting languages (making it both efficient and simple to use) rather then just "possible" I suspect adoption of the core technologies would skyrocket. Thats what we need, IMHO, is more *simplicity* of using and intemixing the core xml tools in a way that doesn't incur huge performance costs. But if they dont include the core technologies such as XQuery , XSLT (2.0 ideally) , xinclude , validation ... in such a way that they can be easily intermixed accessed then people wont use them. The reason people are using scripting languages is for ease of use. I talked to a Ruby programmer at work after a demonstration on XQuery and his answer was "I can do that in ruby ... why would I want to learn xquery ? And ruby can do a LOT more then just xml processing " ... After seeing some of his code I had to admit he had a point. I still have been unable to successfully convince him the value of xquery over Ruby's XML API's. Although he's never had to do anything *complicated* ... but because of that he cant envision the need for a different language. He's at the stage I was when I first started XML Programming in java ... I would look at XSLT programs "Why would I want to write something that complicated when I can just use the easy DOM calls in java. It wasn't until several years of having to do *difficult* transformations that I finally realized how big my java programs were getting. David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From mlu at as-guides.com Fri Sep 4 14:50:13 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Fri Sep 4 04:19:56 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA0F561.5040605@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: <4AA0FEF5.5060709@as-guides.com> David A. Lee schrieb: >> Now my questions: >> - would you recommend alternatives for rapid tool development? >> - if taking a similar approach - would you like to recommend special >> details, perhaps the script language actually used, or other aspect? >> > In my opinion the problem with doing this in perl is that unless the > xquery implementation itself is in perl or atleast runs within the > same process you will run into horrible performance problems. Of course. JVM invocation overhead, IPC overhead. No persistent process supported by indexes to speed up queries, all possible optimizations forfeited. But Hans-J?rgen's approach is convenient and sufficient where performance is not paramount. In Perl, reasonable XQuery integration can currently be achieved by using Oracle Berkeley DB XML. Of course, while the interface is in Perl, the implementation is in C++. If there were Perl bindings for XQilla XQuery in Perl would be possible within the process but without the Berkeley database. But the main speed factor for XQuery is simply the indexes available in the database. How could it be otherwise? So you need a server process built on indexes. That's usually C/C++ or Java, like most servers; scripting APIs are a nice extra. An alternative is to have some persistent XQuery process running and having Perl/PHP/Python processes interface to it via HTTP/REST/WHATEVER. > * Use a scripting language that already is 'in process' with all the > XML core languages you want to use (xquery, xslt etc) > -> examples XProc, xmlsh > > * Encourage scripting languages developers to embed these XML > languages directly into the scripting languages (say perl). More precisely, write bindings to implementations in C/C++. -- Michael Ludwig From dlee at calldei.com Fri Sep 4 08:51:15 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 04:20:53 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: <4AA0FF33.4090504@calldei.com> xmlstar looks very useful !!! Taking a quick look though it seems it will suffer exactly the same performance problems as most other scripting languages because it has no way to "glue" the commands together in process. So you end up having a process execution for each command and full text serialization in and out. ( also missing the one key thing this list is about aka "xquery" ). David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Jakob Fix wrote: > Talking of tools, what is the experience of people on this list with > xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a > close contender to xmlsh (but haven't looked closely)? > > cheers, > Jakob. > > > > On Fri, Sep 4, 2009 at 13:09, David A. Lee wrote: > >> >> >>> Now my questions: - would you recommend alternatives for rapid tool >>> development? - if taking a similar approach - would you like to recommend >>> special details, perhaps the script language actually used, or other aspect? >>> >>> >> In my opinion the problem with doing this in perl is that unless the xquery >> implementation itself is in perl or atleast runs within the same process you >> will run into horrible performance problems. See my (with Norm Walsh's) >> paper : >> >> http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html >> >> What we found is that for our test cases there is a 100-200x (yes 10000 % - >> 20000 % ) performance penalty of using a scripting language to call xml >> processing programs. This *can* be optimized but the exact use cases of >> using a off-the-shelf scripting language to do this kind of thing is >> typically by the audience of people who do not want to spend the extra >> effort to optimize it, or who are not experts in the type of software >> development/languages required to do it, or both. i.e its exactly why they >> are using scripting - so they don't have to do all that extra work. >> >> This is the primary reason xmlsh was invented instead of re-using an >> existing scripting language. I took a "toy" program in a scripting >> language it worked great. >> But when I loaded up all the files I needed it to run it died a horrible >> death. This is what I call "The Brick Wall" and why scripting XML >> processes fail so many of us. The presentation cited above has some good >> charts and figures as well as the full test case code. >> >> This is why I suggest either (both) >> >> * Use a scripting language that already is 'in process' with all the XML >> core languages you want to use (xquery, xslt etc) >> -> examples XProc, xmlsh >> >> * Encourage scripting languages developers to embed these XML languages >> directly into the scripting languages (say perl). >> -> This is hard work and may in fact involve re-implementing many of the >> core tools from scratch. >> -> Some of the work is done but is incomplete ... I've seen references to >> XSLT implementions native in perl where the author quoted something like >> "This isnt a complete implementation of XSLT 1.0 but it works pretty good >> for me". >> >> >> -- >> David A. Lee >> dlee@calldei.com http://www.calldei.com >> http://www.xmlsh.org >> 812-482-5224 >> >> >> >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk >> >> > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090904/c0df45fc/attachment.htm From mlu at as-guides.com Fri Sep 4 14:59:53 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Fri Sep 4 04:29:23 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: <4AA10139.2050801@as-guides.com> Jakob Fix schrieb: > Talking of tools, what is the experience of people on this list with > xmlstarlet (http://xmlstar.sourceforge.net/) which seems to me like a > close contender to xmlsh (but haven't looked closely)? It is built on LibXML2 and LibXSLT by Daniel Veillard. It is like a bag of tricks. It allows you to code shortcuts for XSLT on the command line. But you're limited to a subset of XSLT and the quirks of the command line (notably, quotes). I still use "xmlstarlet list" when I need a directory listing in XML because it's convenient and I happen to know about it. -- Michael Ludwig From dlee at calldei.com Fri Sep 4 09:03:21 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 04:32:50 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA0FEF5.5060709@as-guides.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <4AA0FEF5.5060709@as-guides.com> Message-ID: <4AA10209.2010809@calldei.com> > >> * Use a scripting language that already is 'in process' with all the >> XML core languages you want to use (xquery, xslt etc) >> -> examples XProc, xmlsh >> >> * Encourage scripting languages developers to embed these XML >> languages directly into the scripting languages (say perl). > > More precisely, write bindings to implementations in C/C++. > Only if the scripting language itself is written in C/C++ or has easy bindings to it. If its in another language then bindings in that language are useful. But equally important, IMHO, is to be able to "glue" the data from one module to another efficiently. That needs common in-memory formats. And a way in the language to make calling these modules and passing data to/from them easy in a way thats 'natural' for that language. That may mean, for example, that the language runtime define a standard XML in-memory representation and every module can use it natively. So that you can have say variables of XML Documents and not have to transform them to each module's internal representation (rebuild the document). To do this well I suggest is more then just writing bindings. Its a language level problem as well as a binding problem. -D David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From mail at martin-probst.com Fri Sep 4 15:12:03 2009 From: mail at martin-probst.com (Martin Probst) Date: Fri Sep 4 04:41:30 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA0FE2C.5010009@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <6dca08520909040429k7dbb9ea5oba2723c7edcf2596@mail.gmail.com> <4AA0FE2C.5010009@calldei.com> Message-ID: <6dca08520909040512x60572920jfe984457fe93122b@mail.gmail.com> > If you can find one vendor/ implementation that has all the pices its a tad > easier (document types match !) but its still a bit of hell. That is where I'd say at least in Java land, you can get everything to accept SAX or DOM. It might not be the absolute best choice (e.g. Saxon and its TinyTree), but it will work. I actually can't come up with a tool I needed to use in Java that didn't support DOM. Can you? > I think you'll find it about as painful to to use jruby or jython to say > call ?xinclude , then rng validate then xquery then pass its results to xslt > as writing it in pure java. Indeed, the problem is that these APIs are extremely verbose and have an erratic programming model. Parsing a document in Java is a 20 line+ endeavor, and if you want to get it right you'll have to set magic parser features using undocumented strings no one knows where to find. But this is the kind of thing you can do right if you don't need the flexibility to supply your own DocumentBuilderFactory (or whatever), just by mandating a default. > In xmlsh its as simple as ? xinclude file | rngvalidate schema | xquery -f > schema | xslt -f stylesheet > > To get these kind of languages (jruby etc) into shape where XML is actually > *easy to use* is going to require work. ?Possibly at the language level to > make it really easy. Yes, but only a reasonable amount of work. The above code would actually be valid JRuby syntax, so you wouldn't need any language extensions, just some library routines. > I talked to a Ruby programmer at work after a demonstration on XQuery and > his answer was "I can do that in ruby ... why would I want to learn xquery ? > ?And ruby can do a LOT more then just xml processing " ... After seeing some > of his code I had to admit he had a point. ?I still have been unable to > successfully convince him the value of xquery over Ruby's XML API's. Ruby has this nice feature of being very amenable to domain specific languages, and some of the XML APIs use this nicely. However the built-in XML support in Ruby is plain horrible, REXML is a huge can of bugs. And of course with imperative code in Ruby you loose all the declarative/optimizable/... niceness of XQuery and XSLT. Martin From mlu at as-guides.com Fri Sep 4 15:24:47 2009 From: mlu at as-guides.com (Michael Ludwig) Date: Fri Sep 4 05:24:41 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA10209.2010809@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <4AA0FEF5.5060709@as-guides.com> <4AA10209.2010809@calldei.com> Message-ID: <4AA1070F.2040505@as-guides.com> David A. Lee schrieb: >>> * Encourage scripting languages developers to embed these XML >>> languages directly into the scripting languages (say perl). >> More precisely, write bindings to implementations in C/C++. >> > Only if the scripting language itself is written in C/C++ or has easy > bindings to it. If its in another language then bindings in that > language are useful. True. I only thought of Perl/PHP/Python/Ruby - all in C. But Jython and JRuby were mentioned, and I could mention Quercus - all in Java. > But equally important, IMHO, is to be able to "glue" the data from > one module to another efficiently. That needs common in-memory > formats. That's the downside of issuing specifications and waiting for the implementations to happen. Thank God XML does have a standard serialization, so any document can migrate from one implementation to another :-) > That may mean, for example, that the language runtime define a > standard XML in-memory representation and every module can use it > natively. So that you can have say variables of XML Documents and not > have to transform them to each module's internal representation > (rebuild the document). > > To do this well I suggest is more then just writing bindings. Its a > language level problem as well as a binding problem. Isn't .NET a solution to that problem? -- Michael Ludwig From dlee at calldei.com Fri Sep 4 09:39:50 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 05:39:33 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA1070F.2040505@as-guides.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <4AA0FEF5.5060709@as-guides.com> <4AA10209.2010809@calldei.com> <4AA1070F.2040505@as-guides.com> Message-ID: <4AA10A96.6070502@calldei.com> > > That's the downside of issuing specifications and waiting for the > implementations to happen. Thank God XML does have a standard > serialization, so any document can migrate from one implementation > to another :-) > I wish that were true. Its true for "Documents" but not for everything XML tools want to process and consume. My current peeve is sequences. There is *NO* standard representation of sequences period. (please please tell me I'm wrong !) A real example: toolA : produces a sequence of nodes toolB : accepts a sequence of nodes There is no standard way of glueing these together in any language or serialization format that I know of. Its all vendor specific. And you cant even rely on the worse case of text serialization . A typical part-solution/hack is to wrap the sequences in a document, then unwrap them on the target. This works sometimes. It fails miserably for things like sequences of attributes, or even sequences of integers. > > Isn't .NET a solution to that problem? > It is a nice framework. But someone still has to write all the pieces and agree to as common representation for all the data interfaces. And the common .NET languages are about at the same level as Java ... although there is a windows .net scripting language I wouldn't call it "simple". So I dont agree that .NET is a "solution". Its a technology framework not a solution in itself. Saxon has a great .NET binding (runs the JVM directly within .NET but exposes pure .NET interfaces). But its at the same level of complexity to use as java and doesnt have *all* the pieces. Its got lots, but not all. I'd love to see a comprehensive XML tools framework in .NET which is actually as simple to use as scripting. I think it could be done. But to be 'really simple' , IMHO, would require a .NET implementation of a scripting syntax. ruby.net anyone :) ? -David -- David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From int19h at gmail.com Fri Sep 4 10:11:56 2009 From: int19h at gmail.com (Pavel Minaev) Date: Fri Sep 4 09:07:05 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA10209.2010809@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <4AA0FEF5.5060709@as-guides.com> <4AA10209.2010809@calldei.com> Message-ID: On Fri, Sep 4, 2009 at 5:03 AM, David A. Lee wrote: > But equally important, IMHO, ?is to be able to "glue" the data from one > module to another efficiently. ?That needs common in-memory formats. > And a way in the language to make calling these modules and passing data > to/from them easy in a way thats 'natural' for that language. > That may mean, for example, that the language runtime define a standard XML > in-memory representation and every module can use it natively. > So that you can have say variables of XML Documents and not have to > transform them to each module's internal representation (rebuild the > document). This is a very important point. I am mostly dealing with XSLT, XQuery, and the rest of the stack from .NET perspective. There, we actually do have a standard representation of XML - IXPathNavigable factory interface and the associated XPathNavigator cursor class. In fact, it is specifically designed in a way that makes a straightforward mapping to XDM possible (to quote MSDN: "The XPathNavigator class in the System.Xml.XPath namespace is an abstract class which defines a cursor model for navigating and editing XML information items as instances of the XQuery 1.0 and XPath 2.0 Data Model."). All .NET XML APIs provide support for those interfaces one way or another - all in-memory XML tree representations (and there are three of them in the standard library) can be treated as IXPathNavigable, .NET stock XSLT 1.0 implementation takes an IXPathNavigable as a source, stock XML Schema validator also works with it, and so on. The interface is also simple enough that I can take virtually any data source (e.g., an object graph), easily expose it as an XDM instance, and feed that to any XML-processing pipeline (usually XSLT) as an input, with no need to copy data back and forth. So, all stock components integrate really well together, with no representation mismatch (and yet representation itself being nicely abstracted away), but they are dated (XSLT 1.0... 'nuff said). And third-party components often don't integrate that well. For example, there's no XSLT 2.0 implementation for .NET that works directly over IXPathNavigable input trees (without making an extra copy), or that provides output in a form for which an XPathNavigator can be immediately provided without copies. And the first and only XQuery implementation which supports all of the above that I'm aware of is the recently released XQSharp. From hrennau at yahoo.de Fri Sep 4 10:59:19 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Fri Sep 4 09:53:15 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA0F561.5040605@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> Message-ID: <286275.71013.qm@web27104.mail.ukl.yahoo.com> Hello David, scripting and tool devlopment using script-wrapping are akin, but not quite the same. It seems to me important to distinguish clearly between two usecases: 1. the tool's functionality is provided by executing XQuery once (or only a few times) 2. the tool's functionality is provided by executing XQuery very often (as in your tests, hundreds of times) Almost all tools I have developed so far fall into category 1. Obviously there is NO PERFORMANCE PENALTY worth mentioning in this case: I believe the difference between calling the query via its Perl script or alternatively calling it directly from the command line would not be perceptible for a human being. It is not coincidence that category 1 can VERY often be used - rather, it is exactly the sequence philosophy which makes it a common case! Example: if you have a query processing a document received as context item, then one can modify the query to process any number of input documents in succession, producing an aggregation of results by just adding 4 (!) lines: declare variable $fileNames external; declare variable $docs := for $f in tokenize(fileNames, "\s*") return doc($f); {for $d in $docs return - now comes the original query with this modification: prepend $d before any path starting with / or // - } All that is left to be done is pass the names of the input documents as a space separated string to the query. Which is a trivial task for the little Perl section preceding the invocation. If one ignores this dependence on usecase, one arrives at the situation described by Michael Kay on http://www.saxonica.com/questions.html, last question: "Is it true that Altova XML is three times as fast as Saxon?" I never cease to be amazed by exactly this quality of XQuery - its ability to integrate vast sets of input resources with almost no increased effort as compared to the processing of a single resource. With kind regards, Hans-Juergen Rennau ----- Urspr?ngliche Mail ---- Von: David A. Lee An: Hans-Juergen Rennau CC: talk@x-query.com; dflorescu@mac.com Gesendet: Freitag, den 4. September 2009, 04:09:21 Uhr Betreff: Re: [xquery-talk] Tool development: by Perl-wrapped XQuery In my opinion the problem with doing this in perl is that unless the xquery implementation itself is in perl > or atleast runs within the same process you will run into horrible performance problems. See my (with Norm Walsh's) paper : http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html What we found is that for our test cases there is a 100-200x (yes 10000 % - 20000 % ) performance penalty of using a scripting language to call xml processing programs. This *can* be optimized but the exact use cases of using a off-the-shelf scripting language to do this kind of thing is typically by the audience of people who do not want to spend the extra effort to optimize it, or who are not experts in the type of software development/languages required to do it, or both. i.e its exactly why they are using scripting - so they don't have to do all that extra work. This is the primary reason xmlsh was invented instead of re-using an existing scripting language. I took a "toy" program in a scripting language it worked great. But when I loaded up all the files I needed it to run it died a horrible death. This is what I call "The Brick Wall" and why scripting XML processes fail so many of us. The presentation cited above has some good charts and figures as well as the full test case code. This is why I suggest either (both) * Use a scripting language that already is 'in process' with all the XML core languages you want to use (xquery, xslt etc) -> examples XProc, xmlsh * Encourage scripting languages developers to embed these XML languages directly into the scripting languages (say perl). -> This is hard work and may in fact involve re-implementing many of the core tools from scratch. -> Some of the work is done but is incomplete ... I've seen references to XSLT implementions native in perl where the author quoted something like "This isnt a complete implementation of XSLT 1.0 but it works pretty good for me". -- David A. Lee From dlee at calldei.com Fri Sep 4 14:10:37 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 4 10:04:30 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <286275.71013.qm@web27104.mail.ukl.yahoo.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <286275.71013.qm@web27104.mail.ukl.yahoo.com> Message-ID: <4AA14A0D.2090704@calldei.com> I agree that if you stick to use case #1 (only call xquery once or few times) and all your input and output are files (or file names) then pretty much any scripting language that lets you launch a subprocess is equivalent and you will have very little performance hit. Historically, I suggest many people use that use case primarily because its horrendously expensive not to (in most languages) so they have instead built 'monolithic' programs that do all the work within 1 program (xquery, xslt etc) and take a list of filenames input and maybe filenames output to process. This does work. My point is that I suggest that programming model is no longer necessary, and was arrived at due to the poor performance of doing it any other way, not because its a desirable way of coding. If you use a language that can call xquery *efficiently* you can design in a more modular way and use 'itty bitty' xquery programs run thousands of times instead of one huge program run once, just as efficiently, and in my opinion much easier to develop, debug, and understand. It frees you to design and develop in modular ways, using languages of your choice for different operations instead of forcing you into a single monolithic application. Of course it is a personal *opinion* that that is a 'better' way of doing things :) I certainly enjoy it more. Others may prefer writing one huge program that does everything. This exact same philosophical (and technical) debate is what distinguished unix from the mainframe model of programming. I find it fascinating that the same fundamental design issues and discussions evolved again 40 years later. David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Hans-Juergen Rennau wrote: > Hello David, > > scripting and tool devlopment using script-wrapping are akin, but not quite the same. It seems to me important to distinguish clearly between two usecases: > > 1. the tool's functionality is provided by executing XQuery once (or only a few times) > 2. the tool's functionality is provided by executing XQuery very often (as in your tests, hundreds of times) > > Almost all tools I have developed so far fall into category 1. Obviously there is NO PERFORMANCE PENALTY worth mentioning in this case: I believe the difference between calling the query via its Perl script or alternatively calling it directly from the command line would not be perceptible for a human being. > > It is not coincidence that category 1 can VERY often be used - rather, it is exactly the sequence philosophy which makes it a common case! Example: if you have a query processing a document received as context item, then one can modify the query to process any number of input documents in succession, producing an aggregation of results by just adding 4 (!) lines: > > declare variable $fileNames external; > declare variable $docs := for $f in tokenize(fileNames, "\s*") return doc($f); > {for $d in $docs return > - now comes the original query with this modification: prepend $d before any path starting with / or // - > } > > All that is left to be done is pass the names of the input documents as a space separated string to the query. Which is a trivial task for the little Perl section preceding the invocation. > > If one ignores this dependence on usecase, one arrives at the situation described by Michael Kay on http://www.saxonica.com/questions.html, last question: "Is it true that Altova XML is three times as fast as Saxon?" > > I never cease to be amazed by exactly this quality of XQuery - its ability to integrate vast sets of input resources with almost no increased effort as compared to the processing of a single resource. > > With kind regards, > Hans-Juergen Rennau > > > > > ----- Urspr?ngliche Mail ---- > Von: David A. Lee > An: Hans-Juergen Rennau > CC: talk@x-query.com; dflorescu@mac.com > Gesendet: Freitag, den 4. September 2009, 04:09:21 Uhr > Betreff: Re: [xquery-talk] Tool development: by Perl-wrapped XQuery > > > In my opinion the problem with doing this in perl is that unless the xquery implementation itself is in perl > or atleast runs within the same process you will run into horrible performance problems. See my (with Norm Walsh's) paper : > > http://www.balisage.net/Proceedings/vol4/author-pkg/Lee01/BalisageVol4-Lee01.html > > What we found is that for our test cases there is a 100-200x (yes 10000 % - 20000 % ) performance penalty of using a scripting language to call xml processing programs. This *can* be optimized but the exact use cases of using a off-the-shelf scripting language to do this kind of thing is typically by the audience of people who do not want to spend the extra effort to optimize it, or who are not experts in the type of software development/languages required to do it, or both. i.e its exactly why they are using scripting - so they don't have to do all that extra work. > > This is the primary reason xmlsh was invented instead of re-using an existing scripting language. I took a "toy" program in a scripting language it worked great. > But when I loaded up all the files I needed it to run it died a horrible death. This is what I call "The Brick Wall" and why scripting XML processes fail so many of us. The presentation cited above has some good charts and figures as well as the full test case code. > > This is why I suggest either (both) > > * Use a scripting language that already is 'in process' with all the XML core languages you want to use (xquery, xslt etc) > -> examples XProc, xmlsh > > * Encourage scripting languages developers to embed these XML languages directly into the scripting languages (say perl). > -> This is hard work and may in fact involve re-implementing many of the core tools from scratch. > -> Some of the work is done but is incomplete ... I've seen references to XSLT implementions native in perl where the author quoted something like > "This isnt a complete implementation of XSLT 1.0 but it works pretty good for me". > > > -- David A. Lee > > > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From hrennau at yahoo.de Fri Sep 4 22:24:34 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Fri Sep 4 21:05:02 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA14A0D.2090704@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <286275.71013.qm@web27104.mail.ukl.yahoo.com> <4AA14A0D.2090704@calldei.com> Message-ID: <40080.12266.qm@web27106.mail.ukl.yahoo.com> Hello David, yes, I agree that it is desirable to be free in one's choice how to shape the units of functionality, large or small. Still, I would like to make a couple of points. You wrote: "... so they have instead built 'monolithic' programs that do all the work within 1 program (xquery, xslt etc) and take a list of filenames input and maybe filenames output to process. ... I suggest that programming model is no longer necessary, and was arrived at due to the poor performance of doing it any other way," Sometimes, perhaps even often, this may be the case. But one should not forget that XQuery's aggregative potential teaches us to think in terms of sequences rather than in terms of items. This sweeping approach has nothing to to with "monolithic", just as nobody would call XPath expressions monolithic. When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one Michael Kay recently recommended in the case of Excel data) are fine examples where dividing is desirable. But if the dividing produces pieces where the second depends on the first but the first serves no other purpose than prepare the second - then the dividing is bad and an unnecessary increase of complexity. Speaking of XQuery - the design of modules and functions is important, not the creation of little programs. XQuery is a powerful language suited for writing highly complex programs. Doing this does not mean following a "monolithic" programming model. This being said, two final remarks. First, xmlsh seems so important to me because it aims to overcome a sheer anachronism of conventional Unix shells - their inability to deliver structured information. Second, I regard XProc as the sorely needed integration technology which may greatly enhance the value of the various languages (XQuery, ...); even more, it may pave the way to a new paradigm of data processing, in which concepts like "step" and "ports" become so natural to us as "documents" and "objects". With kind regards, Hans-Juergen ----- Urspr?ngliche Mail ---- Von: David A. Lee An: Hans-Juergen Rennau CC: mail@martin-probst.com; talk@x-query..com Gesendet: Freitag, den 4. September 2009, 10:10:37 Uhr Betreff: Re: [xquery-talk] Tool development: by Perl-wrapped XQuery I agree that if you stick to use case #1 (only call xquery once or few times) and all your input and output are files (or file names) then pretty much any scripting language that lets you launch a subprocess is equivalent and you will have very little performance hit. Historically, I suggest many people use that use case primarily because its horrendously expensive not to (in most languages) so they have instead built 'monolithic' programs that do all the work within 1 program (xquery, xslt etc) and take a list of filenames input and maybe filenames output to process. This does work. My point is that I suggest that programming model is no longer necessary, and was arrived at due to the poor performance of doing it any other way, not because its a desirable way of coding. If you use a language that can call xquery *efficiently* you can design in a more modular way and use 'itty bitty' xquery programs run thousands of times instead of one huge program run once, just as efficiently, and in my opinion much easier to develop, debug, and understand. It frees you to design and develop in modular ways, using languages of your choice for different operations instead of forcing you into a single monolithic application. Of course it is a personal *opinion* that that is a 'better' way of doing things :) I certainly enjoy it more. Others may prefer writing one huge program that does everything. This exact same philosophical (and technical) debate is what distinguished unix from the mainframe model of programming. I find it fascinating that the same fundamental design issues and discussions evolved again 40 years later. David A. Lee > From dlee at calldei.com Sat Sep 5 08:57:12 2009 From: dlee at calldei.com (David A. Lee) Date: Sat Sep 5 04:24:21 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <40080.12266.qm@web27106.mail.ukl.yahoo.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <286275.71013.qm@web27104.mail.ukl.yahoo.com> <4AA14A0D.2090704@calldei.com> <40080.12266.qm@web27106.mail.ukl.yahoo.com> Message-ID: <4AA25218.9070101@calldei.com> Hello Hans-Juergen I do agree with a lot of your comments ... However to add to this one There's another side to this. > When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one The third aspect you don't mention is ease of development. I personally find its *vastly easier* to author, debug, and later maintain an xml translation that is decomposed into say 5 steps in individual files with concrete results in each "step". I can run each xquery /xslt individually and output the results (either command line or in a GUI) and debug each one separately. Then put them together and debug the results. Even if *none* of the steps are ever "reusable" either in the past or the future to me this is a very powerful advantage of "divide and conquer". If the end result can be run as efficiently as if they were all in one big program I see that as a huge win. And as such I disagree with this comment "Speaking of XQuery - the design of modules and functions is important, not the creation of little programs. " I suggest that is inventing/imposing an unnecessary distinction between "program" and "modules" and presupposing what is "important". It may not have been 'important' in the authors of xquery's mind, but thats the strength of reusable software - to grow beyond the original intent. XQuery is useful and 'important' at the single expression level, all the way up to a full "program" whatever that is. Just because xquery supports a module implementation does not mean thats the only valuable way of composing larger programs that use xquery as s component. This is what Unix shell programming taught us 40 years ago, and what programs like XProc are teaching us today. Its also what frameworks like .NET that support things like inheritance across languages teaches us. That the distinction between "program" and "module" is fuzzy. And that by making 'little programs' that do one thing only, but do it well, and providing an efficient mechanism for interconnecting them, you can produce exponentially complex *behavior* without exponentially complex *code*. The distinction between 'program' and 'module' is subjective and has overlapping meanings in different contexts. Is "ls" a program or a module ? How about "more" ? then what about 'lsmore' implemented as "ls | more" ? It is this artificial distinction between "module" and "program" itself that leads to what I called "monolithic" programs. There's a long ways we could go with this. Imagine a language that could 'inherit' a single template from xslt in the context of another language ? ( such as .NET allows across disparate languages) . And of course taking it too far you can produce exponentially inefficient and complex code that does nothing but blow up the cpu :) Were beginning to see a more of this (xquery in small pieces), such as using tiny bits of XQuery inside other programs like "XQuery in the browser". Little bits of XQuery in XML data bases. XQuery being used as part of the business logic in a CRM system (yes this exists) coexisting with other languages. Are these "programs" or "modules" or even less just single expressions ? It doesn't matter. What matters is can you easily and efficiently hook them up and use them in ways the original authors may have never dreamed. But in my opinion they are definitely 'important' David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From hrennau at yahoo.de Sat Sep 5 23:09:25 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Sat Sep 5 21:50:55 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <4AA25218.9070101@calldei.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <286275.71013.qm@web27104.mail.ukl.yahoo.com> <4AA14A0D.2090704@calldei.com> <40080.12266.qm@web27106.mail.ukl.yahoo.com> <4AA25218.9070101@calldei.com> Message-ID: <120442.31312.qm@web27103.mail.ukl.yahoo.com> Hello David, gradually I start to get a feeling for your perspective. Pulling "an efficient mechanism for interconnecting them" into focus you move across the boundaries of the XQuery spec's field of vision. Your criticism that design is not only function and module design gains power. Vaguely but vividly this reminds me of the SOAP versus REST controversy: with SOAP clinging to an artificial complication ("I am calling something...") and REST arriving at the more fundamental abstractions of resource and address, enabling a new ease of access and of combining existing resources. Modules are a category emerging only *after* I have called a query, reusable inside queries, whereas the outside is, to put it mildly, bigger and more important. The great primitives of Unix - the fundamental tools - target the basic entities of an OS environment: (byte) streams, files, directories, users, ... I wonder what new targets emerge with the transition to xmlsh? This is a very important question, relevant as much to xmlsh as to tool development in general - which tools should occupy the top of the community's agenda? ! It occurs to me that there are (at least) two targets of prime importance: 1. (of course) item streams (or 'sequences', 'infosets', 'XDM instances) => XDM info commands (or 'services', 'tools', ...) 2. (but also) XML Schemas (!) => xsd info commands There were no "schemas" to describe byte streams, but we have them to describe item streams, and I think here is a great demand of standard tools unlocking their wealth of information. For example: get-simple-type myElementName | simple-type-pattern => yielding the pattern constraining an element (e.g. "\D\D\D\d+"), or get-paths myFile | invalid-paths myTargetNamespacePrefix => yielding any paths not compatible with a certain schema. Reading 'XQuery is useful and 'important' at the single expression level, all the way up to a full "program"' and remembering that XQuery is a fully composable language one feels that it is very important to think about the INTEGRATION of XQuery into software and processes in all its aspects. (Especially because XQuery is an Information Processing Language, but not a general purpose programming language!) One could even argue that the still small acceptance of XQuery is related to still insufficient integration. I used to regard XProc as *the* enabler, *the* integrator. Now I get a feeling that one might look at XProc and xmlsh as sister technologies resembling XSLT and XQuery: both built on the same basis, the XDM and the XPath language; the former a little more powerful and feature-rich, with the cost of rather heavy syntax, the latter very slender and flexible. With kind regards, Hans-Juergen ----- Urspr?ngliche Mail ---- Von: David A. Lee An: Hans-Juergen Rennau CC: talk@x-query.com Gesendet: Samstag, den 5. September 2009, 04:57:12 Uhr Betreff: Re: [xquery-talk] Tool development: by Perl-wrapped XQuery Hello Hans-Juergen I do agree with a lot of your comments ... However to add to this one There's another side to this. > When it comes to the other dimension of scaling - the number of processing steps (rather than the number of units processed in a given step) - it is not at all per se better to have many small programs. As a rule of thumb I suggest "divide the processing if and only if reuseable steps result". So, for example, generic preprocessings (as the one The third aspect you don't mention is ease of development. I personally find its *vastly easier* to author, debug, and later maintain an xml translation that is decomposed into say 5 steps in individual files with concrete results in each "step". I can run each xquery /xslt individually and output the results (either command line or in a GUI) and debug each one separately. Then put them together and debug the results. Even if *none* of the steps are ever "reusable" either in the past or the future to me this is a very powerful advantage of "divide and conquer". If the end result can be run as efficiently as if they were all in one big program I see that as a huge win. And as such I disagree with this comment "Speaking of XQuery - the design of modules and functions is important, not the creation of little programs. " I suggest that is inventing/imposing an unnecessary distinction between "program" and "modules" and presupposing what is "important". It may not have been 'important' in the authors of xquery's mind, but thats the strength of reusable software - to grow beyond the original intent. XQuery is useful and 'important' at the single expression level, all the way up to a full "program" whatever that is. Just because xquery supports a module implementation does not mean thats the only valuable way of composing larger programs that use xquery as s component. This is what Unix shell programming taught us 40 years ago, and what programs like XProc are teaching us today.. Its also what frameworks like .NET that support things like inheritance across languages teaches us. That the distinction between "program" and "module" is fuzzy. And that by making 'little programs' that do one thing only, but do it well, and providing an efficient mechanism for interconnecting them, you can produce exponentially complex *behavior* without exponentially complex *code*. The distinction between 'program' and 'module' is subjective and has overlapping meanings in different contexts. Is "ls" a program or a module ? How about "more" ? then what about 'lsmore' implemented as "ls | more" ? It is this artificial distinction between "module" and "program" itself that leads to what I called "monolithic" programs. There's a long ways we could go with this. Imagine a language that could 'inherit' a single template from xslt in the context of another language ? ( such as .NET allows across disparate languages) . And of course taking it too far you can produce exponentially inefficient and complex code that does nothing but blow up the cpu :) Were beginning to see a more of this (xquery in small pieces), such as using tiny bits of XQuery inside other programs like "XQuery in the browser". Little bits of XQuery in XML data bases. XQuery being used as part of the business logic in a CRM system (yes this exists) coexisting with other languages. Are these "programs" or "modules" or even less just single expressions ? It doesn't matter. What matters is can you easily and efficiently hook them up and use them in ways the original authors may have never dreamed. But in my opinion they are definitely 'important' David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From dlee at calldei.com Sun Sep 6 10:19:27 2009 From: dlee at calldei.com (David A. Lee) Date: Sun Sep 6 06:16:26 2009 Subject: [xquery-talk] Tool development: by Perl-wrapped XQuery In-Reply-To: <120442.31312.qm@web27103.mail.ukl.yahoo.com> References: <879025.97180.qm@web27105.mail.ukl.yahoo.com> <4AA0F561.5040605@calldei.com> <286275.71013.qm@web27104.mail.ukl.yahoo.com> <4AA14A0D.2090704@calldei.com> <40080.12266.qm@web27106.mail.ukl.yahoo.com> <4AA25218.9070101@calldei.com> <120442.31312.qm@web27103.mail.ukl.yahoo.com> Message-ID: <4AA3B6DF.6040603@calldei.com> Hello, Hans-Juergen I do believe you are understanding my perspective. It give me hope that perhaps it is not a singular one and I'm not utterly and entirely insane. You've made some interesting comments ! > 2. (but also) XML Schemas (!) > => xsd info commands > I had never thought of this ! > There were no "schemas" to describe byte streams, but we have them to describe item streams, and I think here is a great demand of standard tools unlocking their wealth of information. For example: > > get-simple-type myElementName | simple-type-pattern > => yielding the pattern constraining an element (e.g. "\D\D\D\d+"), or > > get-paths myFile | invalid-paths myTargetNamespacePrefix > => yielding any paths not compatible with a certain schema. > Wow those are great ideas. I need to think about what they mean and how one would make use of them. The last one could be implemented by iteration over "xvalidate" .. .but building this into something like a "find" command or even making type validity and information at a very low level as apposed to just a command level is interesting. Also considering that a document (or an element) can have multiple types simultaneously brings on a level of complexity. What do we do with these commands ? > I used to regard XProc as *the* enabler, *the* integrator. Now I get a feeling that one might look at XProc and xmlsh as sister technologies resembling XSLT and XQuery: both built on the same basis, the XDM and the XPath language; the former a little more powerful and feature-rich, with the cost of rather heavy syntax, the latter very slender and flexible. > > With kind regards, > Hans-Juergen > I disagree that xproc is more "feature-rich" then xmlsh but we can leave that debate for another day. The important thing, IMHO, is that having these multiple languages and tools demonstrate the importance of integration technologies, and that there is more then one way to do it. Once we know there are 2 ways to do something we can discover there are 10 or 1000 ... and as you indicated this opens up a really interesting area that is really at a different level of thought then any one particular technology, but rather how a system based on XML concepts can behave, not limited to one technology. I think this is an exciting phase in XML technologies. -David -- David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From marcus at mercatorit.com Mon Sep 7 10:42:03 2009 From: marcus at mercatorit.com (Marcus Clemens) Date: Mon Sep 7 01:11:39 2009 Subject: [xquery-talk] (no subject) Message-ID: <9BCC2AEA2D8704498C240217CA020E3A3D1952@SERVER1.smallbusiness.local> Hi Marklogic are looking to hire a couple of permanent consultants . you will be based in the UK and there will be some travel to Europe and the USA Please send a cv if this is of any interest Kind regards Marcus Clemens Director & Senior Consultant Mercator IT Ltd Tel: 01892 611161 Fax: 01892 660185 Email: marcus@mercatorit.com Cooks Corner Business Park, The Old Saw Mill, London Road, Crowborough TN6 1TQ Registered in England no: 05755983 Registered office: 117 Dartford Road, Dartford, Kent DA1 3EN This email may contain privileged/confidential information and is for the intended addressee only. If you have received this message in error then you must not use, retain, disseminate or otherwise deal with it. Please notify the sender by return email and destroy. The views of the author may not necessarily constitute the views of Mercator IT Solutions Ltd. Nothing in this email shall bind Mercator IT Solutions Ltd in any contract or obligation. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090907/0a4361eb/attachment.htm From vyacheslav.sedov at gmail.com Mon Sep 7 13:50:07 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Mon Sep 7 01:19:18 2009 Subject: [xquery-talk] (no subject) In-Reply-To: <9BCC2AEA2D8704498C240217CA020E3A3D1952@SERVER1.smallbusiness.local> References: <9BCC2AEA2D8704498C240217CA020E3A3D1952@SERVER1.smallbusiness.local> Message-ID: <7ab7cbac0909070150ra509f57w33b9f4e155ce2456@mail.gmail.com> what about remote job? On Mon, Sep 7, 2009 at 12:42 PM, Marcus Clemens wrote: > Hi > > > > Marklogic are looking to hire a couple of permanent consultants . you will > be based in the UK and there will be some travel to Europe and the USA > > > > Please send a cv if this is of any interest > > > > Kind regards > > > > Marcus Clemens > > Director & Senior Consultant > > Mercator IT Ltd > > > > Tel: 01892 611161 > > Fax: 01892 660185 > > Email: marcus@mercatorit.com > > Cooks Corner Business Park, The Old Saw Mill, London Road, Crowborough TN6 > 1TQ > > > > Registered in England no: 05755983 > > Registered office: 117 Dartford Road, Dartford, Kent DA1 3EN > > > > This email may contain privileged/confidential information and is for the > intended addressee only.? If you have received this message in error then > you must not use, retain, disseminate or otherwise deal with it.? Please > notify the sender by return email and destroy.? The views of the author may > not necessarily constitute the views of Mercator IT Solutions Ltd.? Nothing > in this email shall bind Mercator IT Solutions Ltd in any contract or > obligation. > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From jesse.wu68 at gmail.com Tue Sep 8 16:22:08 2009 From: jesse.wu68 at gmail.com (Jesse Wu) Date: Tue Sep 8 12:07:14 2009 Subject: [xquery-talk] parse form data to XML Message-ID: Sorry for the newbie question, I am still in the process of learning XQuery. What will be the the best way to get HTML form post data into a XML. For example: get-data returns v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3& .... v10=v10&a10=a10 need to get it into XML as v1 v2 v3 .... v10 tried to use tokenize then string-before and after =, but could not figure out how to get the loop to do 2 elements for each row. {for $parsed-query-term in $parsed-data let $parse-query-value := substring-after($parsed-query-term,"=") return {xmldb:decode($parse-query-value)} } Thank you or your help. -Jes From dlee at calldei.com Tue Sep 8 16:42:05 2009 From: dlee at calldei.com (David A. Lee) Date: Tue Sep 8 12:26:37 2009 Subject: [xquery-talk] parse form data to XML In-Reply-To: References: Message-ID: <4AA6B38D.5040601@calldei.com> Probably not the most elegant way ... ========== declare variable $s := "v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3" ; let $a := tokenize($s , "&" ) for $i in 0 to xs:integer(count($a) div 2 - 1) return { tokenize( $a[($i * 2) + 2 ] , "=" )[2] } ========== David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Jesse Wu wrote: > Sorry for the newbie question, I am still in the process of learning XQuery. > What will be the the best way to get HTML form post data into a XML. > For example: > > get-data returns > > v1=v1&a1=a1&v2=v2&a2=a2&v3=v3&a3=a3& .... v10=v10&a10=a10 > > need to get it into XML as > > v1 > v2 > v3 > .... > > v10 > > tried to use tokenize then string-before and after =, but could not > figure out how to get the loop to do 2 elements for each row. > > {for $parsed-query-term in $parsed-data > let $parse-query-value := substring-after($parsed-query-term,"=") > return {xmldb:decode($parse-query-value)} > } > > Thank you or your help. -Jes > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From mail at martin-probst.com Wed Sep 9 02:06:41 2009 From: mail at martin-probst.com (Martin Probst) Date: Tue Sep 8 15:42:33 2009 Subject: [xquery-talk] parse form data to XML In-Reply-To: References: Message-ID: <6dca08520909081606p282ea9e3ya07904146be7a500@mail.gmail.com> > ? ?{for $parsed-query-term in $parsed-data > ? ? ? ? ? ? ? let $parse-query-value := substring-after($parsed-query-term,"=") > ? ? ? ? ? ? ? return {xmldb:decode($parse-query-value)} > ? ? ? ? ? ? ? } What about for $parsed-query-term in $parsed-data let $parse-query-value := substring-after($parsed-query-term,"=") let $parameter-name := substring-before($parsed-query-term, concat("=", $parse-query-value)) return {xmldb:decode($parse-query-value)} Alternatively you can simply split again: for $parsed-query-term in $parsed-data let $kv := tokenize($parsed-query-term, '=') let $key := $kv[1] let $value := string-join($kv[position()>1],'=') return {xmldb:decode($value)} As the value might contain '=' (I think, or does that need to be percent encoded?), $kv might contain multiple value parts, so the string-join is necessary to glue them together again. Martin From vyacheslav.sedov at gmail.com Wed Sep 9 09:30:03 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Tue Sep 8 20:57:45 2009 Subject: [xquery-talk] parse form data to XML In-Reply-To: <6dca08520909081606p282ea9e3ya07904146be7a500@mail.gmail.com> References: <6dca08520909081606p282ea9e3ya07904146be7a500@mail.gmail.com> Message-ID: <7ab7cbac0909082130n6a4bca63x6140579417c2df3f@mail.gmail.com> what about XRX? XForms-REST-XQuery On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote: >> ? ?{for $parsed-query-term in $parsed-data >> ? ? ? ? ? ? ? let $parse-query-value := substring-after($parsed-query-term,"=") >> ? ? ? ? ? ? ? return {xmldb:decode($parse-query-value)} >> ? ? ? ? ? ? ? } > > What about > > for $parsed-query-term in $parsed-data > let $parse-query-value := substring-after($parsed-query-term,"=") > let $parameter-name := substring-before($parsed-query-term, > concat("=", $parse-query-value)) > return {xmldb:decode($parse-query-value)} > > Alternatively you can simply split again: > > for $parsed-query-term in $parsed-data > let $kv := tokenize($parsed-query-term, '=') > let $key := $kv[1] > let $value := string-join($kv[position()>1],'=') > return {xmldb:decode($value)} > > As the value might contain '=' (I think, or does that need to be > percent encoded?), $kv might contain multiple value parts, so the > string-join is necessary to glue them together again. > > Martin > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From hrennau at yahoo.de Wed Sep 9 00:14:37 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Tue Sep 8 22:39:41 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? Message-ID: <605174.95935.qm@web27108.mail.ukl.yahoo.com> Hello People, XQuery is _the_ language for agile and efficient evaluation of XML resources (who would contradict me?). But there is another important technology, of course, XML Schema: though its use is often reduced to document validation or data binding, schemas offer a great wealth of static information about (valid) documents. I am sometimes amazed how merrily this information is ignored, especially in projects that have to deal with very complex schemas. Unfortunately, schemas are difficult to read and, worse, difficult to use as input for further processing, as the schema language allows to express the (essentially) same information in so many ways, what with model groups, attribute sets, type derivations, etc. Example: some configuration contains a data path, and one has the appropriate schema files available - how can one implement an automatic check of the path string against that schema? In consequence, I believe creating "schema reporting" tools may be an important use case for XQuery.Here, schema reporting is understood as transforming schema files into a different serialized representation of all or some information the schema contains. (I am _not_ thinking of graphical representations as offered by XML IDEs unless they are accompanied by serialized versions appropriate to serve as input for further processing.) Some examples were: a tree representation of document structure, a list of valid data paths, a mapping of element and attribute names to the governing type, a mapping of type names to data paths. Well - would anyone like to comment on the statement about such schema reporting being an important use case for XQuery? In this context it is of course important to know what is already available (commercial or open source). Would anyone like to speak about available schema reporting tools (in the sense defined above)? Thank you very much, with kind regards - Hans-Juergen Rennau From ricardo.queiros at gmail.com Wed Sep 9 09:55:02 2009 From: ricardo.queiros at gmail.com (ricardo queiros) Date: Wed Sep 9 00:16:56 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <605174.95935.qm@web27108.mail.ukl.yahoo.com> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> Message-ID: <6cec8a990909090055x2718d509jc99e7b3ee1778569@mail.gmail.com> Hi, I develop a tool for browsing XML Schema files in a easy way. The tool calls Schem@Doc and can be downloaded as open source at http://www.dcc.fc.up.pt/schemaDoc/. It is based on a XSLT file that transforms the schema file in XHTML code to be interpreted in a general browser. So the only requirement is a web browser. At this moment the tool don't support a few elements of the W3C XML Schema specification (e.g. *field, key, keyref, redefine, selector, unique *elements and *abstract, substitutionGroup *attributtes). It supports Schematron.rules embeded in the schema file. Best regards. 2009/9/9 Hans-Juergen Rennau > Hello People, > > XQuery is _the_ language for agile and efficient evaluation of XML > resources (who would contradict me?). But there is another important > technology, of course, XML Schema: though its use is often reduced to > document validation or data binding, schemas offer a great wealth of static > information about (valid) documents. I am sometimes amazed how merrily this > information is ignored, especially in projects that have to deal with very > complex schemas. > > Unfortunately, schemas are difficult to read and, worse, difficult to use > as input for further processing, as the schema language allows to express > the (essentially) same information in so many ways, what with model groups, > attribute sets, type derivations, etc. Example: some configuration contains > a data path, and one has the appropriate schema files available - how can > one implement an automatic check of the path string against that schema? > > In consequence, I believe creating "schema reporting" tools may be an > important use case for XQuery.Here, schema reporting is understood as > transforming schema files into a different serialized representation of all > or some information the schema contains. (I am _not_ thinking of graphical > representations as offered by XML IDEs unless they are accompanied by > serialized versions appropriate to serve as input for further processing.) > Some examples were: a tree representation of document structure, a list of > valid data paths, a mapping of element and attribute names to the governing > type, a mapping of type names to data paths. > > Well - would anyone like to comment on the statement about such schema > reporting being an important use case for XQuery? > > In this context it is of course important to know what is already available > (commercial or open source). Would anyone like to speak about available > schema reporting tools (in the sense defined above)? > > Thank you very much, > with kind regards - > > Hans-Juergen Rennau > > > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -- Ricardo Queir?s -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090909/a74384db/attachment.htm From vyacheslav.sedov at gmail.com Wed Sep 9 13:39:41 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Wed Sep 9 00:59:41 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <6cec8a990909090055x2718d509jc99e7b3ee1778569@mail.gmail.com> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> <6cec8a990909090055x2718d509jc99e7b3ee1778569@mail.gmail.com> Message-ID: <7ab7cbac0909090139g13cb8eb3u55b2aae920d46565@mail.gmail.com> i am very seriously thinking about XSLT-based convertor from XSD to Schematron rules - i have opinion that Schematron power underestimated in XML world On Wed, Sep 9, 2009 at 11:55 AM, ricardo queiros wrote: > Hi, > I develop a tool for browsing XML Schema files in a easy way. > The tool calls Schem@Doc and can be downloaded as open source > at?http://www.dcc.fc.up.pt/schemaDoc/. > It is based on a XSLT file that transforms the schema file in XHTML code to > be interpreted in a general browser. > So the only requirement is a web browser. > At this moment the tool don't support a few elements of the W3C XML Schema > specification (e.g. field, key, keyref, redefine, selector, unique elements > and abstract, substitutionGroup attributtes). It supports Schematron.rules > embeded in the schema file. > Best regards. > > 2009/9/9 Hans-Juergen Rennau >> >> Hello People, >> >> XQuery is _the_ language for agile and efficient evaluation of XML >> resources (who would contradict me?). But there is another important >> technology, of course, XML Schema: though its use is often reduced to >> document validation or data binding, schemas offer a great wealth of static >> information about (valid) documents. I am sometimes amazed how merrily this >> information is ignored, especially in projects that have to deal with very >> complex schemas. >> >> Unfortunately, schemas are difficult to read and, worse, difficult to use >> as input for further processing, as the schema language allows to express >> the (essentially) same information in so many ways, what with model groups, >> attribute sets, type derivations, etc. Example: some configuration contains >> a data path, and one has the appropriate schema files available - how can >> one implement an automatic check of the path string against that schema? >> >> In consequence, I believe creating "schema reporting" tools may be an >> important use case for XQuery.Here, schema reporting is understood as >> transforming schema files into a different serialized representation of all >> or some information the schema contains. (I am _not_ thinking of graphical >> representations as offered by XML IDEs unless they are accompanied by >> serialized versions appropriate to serve as input for further processing.) >> Some examples were: a tree representation of document structure, a list of >> valid data paths, a mapping of element and attribute names to the governing >> type, a mapping of type names to data paths. >> >> Well - would anyone like to comment on the statement about such schema >> reporting being an important use case for XQuery? >> >> In this context it is of course important to know what is already >> available (commercial or open source). Would anyone like to speak about >> available schema reporting tools (in the sense defined above)? >> >> Thank you very much, >> with kind regards - >> >> Hans-Juergen Rennau >> >> >> >> >> >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk > > > > -- > Ricardo Queir?s > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From george at oxygenxml.com Wed Sep 9 13:26:44 2009 From: george at oxygenxml.com (George Cristian Bina) Date: Wed Sep 9 01:45:50 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <605174.95935.qm@web27108.mail.ukl.yahoo.com> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> Message-ID: <4AA774D4.9090807@oxygenxml.com> The commercial Saxon 9.2 EE (previous Saxon-SA) XML Schema processor has a command line switch that outputs an XML file with the schema information in a format suitable for further processing. For details see the -scmout:filename option http://www.saxonica.com/documentation9.2/schema-processing/commandline.html Best Regards, George -- George Cristian Bina XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com Hans-Juergen Rennau wrote: > Hello People, > > XQuery is _the_ language for agile and efficient evaluation of XML resources (who would contradict me?). But there is another important technology, of course, XML Schema: though its use is often reduced to document validation or data binding, schemas offer a great wealth of static information about (valid) documents. I am sometimes amazed how merrily this information is ignored, especially in projects that have to deal with very complex schemas. > > Unfortunately, schemas are difficult to read and, worse, difficult to use as input for further processing, as the schema language allows to express the (essentially) same information in so many ways, what with model groups, attribute sets, type derivations, etc. Example: some configuration contains a data path, and one has the appropriate schema files available - how can one implement an automatic check of the path string against that schema? > > In consequence, I believe creating "schema reporting" tools may be an important use case for XQuery.Here, schema reporting is understood as transforming schema files into a different serialized representation of all or some information the schema contains. (I am _not_ thinking of graphical representations as offered by XML IDEs unless they are accompanied by serialized versions appropriate to serve as input for further processing.) Some examples were: a tree representation of document structure, a list of valid data paths, a mapping of element and attribute names to the governing type, a mapping of type names to data paths. > > Well - would anyone like to comment on the statement about such schema reporting being an important use case for XQuery? > > In this context it is of course important to know what is already available (commercial or open source). Would anyone like to speak about available schema reporting tools (in the sense defined above)? > > Thank you very much, > with kind regards - > > Hans-Juergen Rennau > > > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk From george at oxygenxml.com Wed Sep 9 13:29:58 2009 From: george at oxygenxml.com (George Cristian Bina) Date: Wed Sep 9 01:49:03 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <7ab7cbac0909090139g13cb8eb3u55b2aae920d46565@mail.gmail.com> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> <6cec8a990909090055x2718d509jc99e7b3ee1778569@mail.gmail.com> <7ab7cbac0909090139g13cb8eb3u55b2aae920d46565@mail.gmail.com> Message-ID: <4AA77596.6000704@oxygenxml.com> Rick Jelliffe published a number of articles on this, the last one is http://broadcast.oreilly.com/2009/08/converting-xml-schemas-to-sche-1.html Best Regards, George -- George Cristian Bina XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com Vyacheslav Sedov wrote: > i am very seriously thinking about XSLT-based convertor from XSD to > Schematron rules - i have opinion that Schematron power underestimated > in XML world > > On Wed, Sep 9, 2009 at 11:55 AM, ricardo > queiros wrote: >> Hi, >> I develop a tool for browsing XML Schema files in a easy way. >> The tool calls Schem@Doc and can be downloaded as open source >> at http://www.dcc.fc.up.pt/schemaDoc/. >> It is based on a XSLT file that transforms the schema file in XHTML code to >> be interpreted in a general browser. >> So the only requirement is a web browser. >> At this moment the tool don't support a few elements of the W3C XML Schema >> specification (e.g. field, key, keyref, redefine, selector, unique elements >> and abstract, substitutionGroup attributtes). It supports Schematron.rules >> embeded in the schema file. >> Best regards. >> >> 2009/9/9 Hans-Juergen Rennau >>> Hello People, >>> >>> XQuery is _the_ language for agile and efficient evaluation of XML >>> resources (who would contradict me?). But there is another important >>> technology, of course, XML Schema: though its use is often reduced to >>> document validation or data binding, schemas offer a great wealth of static >>> information about (valid) documents. I am sometimes amazed how merrily this >>> information is ignored, especially in projects that have to deal with very >>> complex schemas. >>> >>> Unfortunately, schemas are difficult to read and, worse, difficult to use >>> as input for further processing, as the schema language allows to express >>> the (essentially) same information in so many ways, what with model groups, >>> attribute sets, type derivations, etc. Example: some configuration contains >>> a data path, and one has the appropriate schema files available - how can >>> one implement an automatic check of the path string against that schema? >>> >>> In consequence, I believe creating "schema reporting" tools may be an >>> important use case for XQuery.Here, schema reporting is understood as >>> transforming schema files into a different serialized representation of all >>> or some information the schema contains. (I am _not_ thinking of graphical >>> representations as offered by XML IDEs unless they are accompanied by >>> serialized versions appropriate to serve as input for further processing.) >>> Some examples were: a tree representation of document structure, a list of >>> valid data paths, a mapping of element and attribute names to the governing >>> type, a mapping of type names to data paths. >>> >>> Well - would anyone like to comment on the statement about such schema >>> reporting being an important use case for XQuery? >>> >>> In this context it is of course important to know what is already >>> available (commercial or open source). Would anyone like to speak about >>> available schema reporting tools (in the sense defined above)? >>> >>> Thank you very much, >>> with kind regards - >>> >>> Hans-Juergen Rennau >>> >>> >>> >>> >>> >>> _______________________________________________ >>> talk@x-query.com >>> http://x-query.com/mailman/listinfo/talk >> >> >> -- >> Ricardo Queir?s >> >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk >> > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk From bourner at upmc.edu Wed Sep 9 09:44:18 2009 From: bourner at upmc.edu (Bourne, Robert) Date: Wed Sep 9 05:41:40 2009 Subject: [xquery-talk] "Pruning" an xml "tree" based on nodes' ids Message-ID: All, I am trying to prune an xml tree based on known ids for the nodes in that tree. For example in the following code: If I know that I am ultimately after leaf with the ID of 15. However, I also need the ancestor nodes that led me to that leaf. I will know those IDs as well. For example if I know that I'm in tree id 1 and the branches I took in order are 1,4,8, how can show a pruned xml tree where the final result is: I have been able to prune in to this: But the problems comes in the fact that I don't know how many levels of branches there will be before I finally reach the leaf that I'm after. It seems like I need to work backwards starting at the leaf and prune based on the ids that I have, but I can really see a way to do that. Any suggestions would be greatly appreciated. Regards, Bob -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090909/0688c0bf/attachment-0001.htm From mail at martin-probst.com Wed Sep 9 16:00:17 2009 From: mail at martin-probst.com (Martin Probst) Date: Wed Sep 9 05:57:20 2009 Subject: [xquery-talk] "Pruning" an xml "tree" based on nodes' ids In-Reply-To: References: Message-ID: <6dca08520909090600x7feaf066p1e158bdbf15e37af@mail.gmail.com> > I am trying to prune an xml tree based on known ids for the nodes in that > tree. > [...] > For example if I know that I?m in tree id 1 and the branches I > took in order are 1,4,8, Are you doing this using copy/transform like the last time? What about deleting those nodes with the wrong ID? copy $x := ... transform ( delete nodes $x//branch[ID != (4, 8)], $x//leaf[ID != 15] ) return $x (Assuming that you'll never need to remove the tree node itself) Martin From jesse.wu68 at gmail.com Wed Sep 9 10:32:05 2009 From: jesse.wu68 at gmail.com (Jesse Wu) Date: Wed Sep 9 06:28:39 2009 Subject: [xquery-talk] parse form data to XML In-Reply-To: <7ab7cbac0909082130n6a4bca63x6140579417c2df3f@mail.gmail.com> References: <6dca08520909081606p282ea9e3ya07904146be7a500@mail.gmail.com> <7ab7cbac0909082130n6a4bca63x6140579417c2df3f@mail.gmail.com> Message-ID: Hi Sedov, I tried XForms but found when the XML model is loaded from a very large file with many attributes (vary for each element) for each of the element, the screen editing is not practical. May be there is a better way to do this using XForms but I could not find a good example. Thanks -Jes On Wed, Sep 9, 2009 at 12:30 AM, Vyacheslav Sedov wrote: > what about XRX? XForms-REST-XQuery > > On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote: >>> ? ?{for $parsed-query-term in $parsed-data >>> ? ? ? ? ? ? ? let $parse-query-value := substring-after($parsed-query-term,"=") >>> ? ? ? ? ? ? ? return {xmldb:decode($parse-query-value)} >>> ? ? ? ? ? ? ? } >> >> What about >> >> for $parsed-query-term in $parsed-data >> let $parse-query-value := substring-after($parsed-query-term,"=") >> let $parameter-name := substring-before($parsed-query-term, >> concat("=", $parse-query-value)) >> return {xmldb:decode($parse-query-value)} >> >> Alternatively you can simply split again: >> >> for $parsed-query-term in $parsed-data >> let $kv := tokenize($parsed-query-term, '=') >> let $key := $kv[1] >> let $value := string-join($kv[position()>1],'=') >> return {xmldb:decode($value)} >> >> As the value might contain '=' (I think, or does that need to be >> percent encoded?), $kv might contain multiple value parts, so the >> string-join is necessary to glue them together again. >> >> Martin >> >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk >> > From mail at martin-probst.com Wed Sep 9 17:50:19 2009 From: mail at martin-probst.com (Martin Probst) Date: Wed Sep 9 07:44:51 2009 Subject: [xquery-talk] "Pruning" an xml "tree" based on nodes' ids In-Reply-To: <6dca08520909090600x7feaf066p1e158bdbf15e37af@mail.gmail.com> References: <6dca08520909090600x7feaf066p1e158bdbf15e37af@mail.gmail.com> Message-ID: <6dca08520909090750p5ea4f421rf65d4e9654ad3939@mail.gmail.com> > copy $x := ... > transform ( > ?delete nodes $x//branch[ID != (4, 8)], $x//leaf[ID != 15] > ) > return $x Whoops, that should have been copy $x := ... transform ( delete nodes ($x//branch[every $x in (4, 8) satisfies ID != $x], $x//leaf[ID != 15]) ) return $x If you compare using != against a list, you need to use every/satisfies, otherwise the expression will be true if only one of the values is not equal to the list. Also, parentheses around the delete nodes part are needed. Martin From xli at galdosinc.com Wed Sep 9 11:54:05 2009 From: xli at galdosinc.com (Xia Li) Date: Wed Sep 9 10:44:03 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <605174.95935.qm@web27108.mail.ukl.yahoo.com> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> Message-ID: <4E1D53230994FD45A119B5090DFBF460020B140E@andalusia.Galdos.local> As a typical use case, schema information is frequently used to process GML(Geography Markup Language) instance data. Mostly, the schema querying part is done with the help of Xerces-J XML schema processing library in Java. But I do think it is quite useful to have the same thing done in XML processing language directly instead of having to call external functions. Ideally, it is best done in XQuery processors and XPath 2 spec does provide some functions to retrieve schema information. However, the problem is it is not widely supported feature, and there barely open source XPath processors that support such functionalities. I developed an XSLT function library to query schema information. It essentially contains functions to retrieve a type definition for a given element. http://www.ibm.com/developerworks/library/x-schemanode/ It is a much more intriguing programming experience to do this in XSLT than in Java and I think it can be equally done in XQuery as well. Lisa -----Original Message----- From: talk-bounces@x-query.com [mailto:talk-bounces@x-query.com] On Behalf Of Hans-Juergen Rennau Sent: Tuesday, September 08, 2009 11:15 PM To: talk@x-query.com Cc: dlee@calldei.com Subject: [xquery-talk] Schema reporting tools - XQuery use case? Hello People, XQuery is _the_ language for agile and efficient evaluation of XML resources (who would contradict me?). But there is another important technology, of course, XML Schema: though its use is often reduced to document validation or data binding, schemas offer a great wealth of static information about (valid) documents. I am sometimes amazed how merrily this information is ignored, especially in projects that have to deal with very complex schemas. Unfortunately, schemas are difficult to read and, worse, difficult to use as input for further processing, as the schema language allows to express the (essentially) same information in so many ways, what with model groups, attribute sets, type derivations, etc. Example: some configuration contains a data path, and one has the appropriate schema files available - how can one implement an automatic check of the path string against that schema? In consequence, I believe creating "schema reporting" tools may be an important use case for XQuery.Here, schema reporting is understood as transforming schema files into a different serialized representation of all or some information the schema contains. (I am _not_ thinking of graphical representations as offered by XML IDEs unless they are accompanied by serialized versions appropriate to serve as input for further processing.) Some examples were: a tree representation of document structure, a list of valid data paths, a mapping of element and attribute names to the governing type, a mapping of type names to data paths. Well - would anyone like to comment on the statement about such schema reporting being an important use case for XQuery? In this context it is of course important to know what is already available (commercial or open source). Would anyone like to speak about available schema reporting tools (in the sense defined above)? Thank you very much, with kind regards - Hans-Juergen Rennau _______________________________________________ talk@x-query.com http://x-query.com/mailman/listinfo/talk From vyacheslav.sedov at gmail.com Wed Sep 9 23:02:50 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Wed Sep 9 10:52:11 2009 Subject: [xquery-talk] parse form data to XML In-Reply-To: References: <6dca08520909081606p282ea9e3ya07904146be7a500@mail.gmail.com> <7ab7cbac0909082130n6a4bca63x6140579417c2df3f@mail.gmail.com> Message-ID: <7ab7cbac0909091102j5488d033r7c47b98c17ac4048@mail.gmail.com> i guess XForms in any case better then ordinar HTML forms - so i can assume that you just underinvestigate this case :) On Wed, Sep 9, 2009 at 5:32 PM, Jesse Wu wrote: > Hi Sedov, > > I tried XForms but found when the XML model is loaded from a very > large file with many attributes (vary for each element) for each of > the element, the screen editing is not practical. May be there is a > better way to do this using XForms but I could not find a good > example. > > Thanks -Jes > > On Wed, Sep 9, 2009 at 12:30 AM, Vyacheslav Sedov > wrote: >> what about XRX? XForms-REST-XQuery >> >> On Wed, Sep 9, 2009 at 3:06 AM, Martin Probst wrote: >>>> ? ?{for $parsed-query-term in $parsed-data >>>> ? ? ? ? ? ? ? let $parse-query-value := substring-after($parsed-query-term,"=") >>>> ? ? ? ? ? ? ? return {xmldb:decode($parse-query-value)} >>>> ? ? ? ? ? ? ? } >>> >>> What about >>> >>> for $parsed-query-term in $parsed-data >>> let $parse-query-value := substring-after($parsed-query-term,"=") >>> let $parameter-name := substring-before($parsed-query-term, >>> concat("=", $parse-query-value)) >>> return {xmldb:decode($parse-query-value)} >>> >>> Alternatively you can simply split again: >>> >>> for $parsed-query-term in $parsed-data >>> let $kv := tokenize($parsed-query-term, '=') >>> let $key := $kv[1] >>> let $value := string-join($kv[position()>1],'=') >>> return {xmldb:decode($value)} >>> >>> As the value might contain '=' (I think, or does that need to be >>> percent encoded?), $kv might contain multiple value parts, so the >>> string-join is necessary to glue them together again. >>> >>> Martin >>> >>> _______________________________________________ >>> talk@x-query.com >>> http://x-query.com/mailman/listinfo/talk >>> >> > From lists at fgeorges.org Thu Sep 10 14:24:52 2009 From: lists at fgeorges.org (Florent Georges) Date: Thu Sep 10 06:23:49 2009 Subject: [xquery-talk] Local namespace Message-ID: <653897.36925.qm@web23007.mail.ird.yahoo.com> Hi, From what I can see in the XQuery 1.0 REC, the "local" namespace is aimed to be used only in a main module, and only to locally define functions (not variables.) This is a bit surprising to me, as I thought we could define both local:* functions & variables, both in main and library modules... But maybe I misread the REC. Any thoughts? Regards, -- Florent Georges http://www.fgeorges.org/ From hrennau at yahoo.de Thu Sep 10 15:09:58 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Thu Sep 10 14:07:44 2009 Subject: [xquery-talk] Schema reporting tools - XQuery use case? In-Reply-To: <4E1D53230994FD45A119B5090DFBF460020B140E@andalusia.Galdos.local> References: <605174.95935.qm@web27108.mail.ukl.yahoo.com> <4E1D53230994FD45A119B5090DFBF460020B140E@andalusia.Galdos.local> Message-ID: <610954.37723.qm@web27106.mail.ukl.yahoo.com> Hello Lisa, the article was very interesting to me, thank you! Whereas hitherto I only thought of "schema reports", your work shows me that there is a parallel approach one might call "schema querying". (In fact, I borrowed this term from the namespace URI you used in your article.) Trying to define the terms: schema reporting transforms schema documents into new documents conveying part of the schema information (e.g. listing valid data paths); the result is a new resource. Schema querying, on the other hand, is a functionality that provides answers for certain kinds of questions referring to the schema. Schema querying tends to provide dynamic information, referring to instance data, whereas schema reporting usually delivers static information. The border is not sharp: the smaller the report's scope, the closer it resembles an answer to a query, and the use of config parameters may link the report to instance data (e.g. by filtering data). Certainly your function 'load-schemas-by-URI' can be regarded as schema reporting (if you add the possibility to collect the retrieved schemas into a single document). Now I think that the "important use case" I spoke of comprises both, schema reporting and schema querying. And one thing for sure, schema reports can be a convenient basis for implementing schema querying. With kind regards, Hans-Juergen ----- Urspr?ngliche Mail ---- Von: Xia Li An: Hans-Juergen Rennau ; talk@x-query.com CC: dlee@calldei.com Gesendet: Mittwoch, den 9. September 2009, 10:54:05 Uhr Betreff: RE: [xquery-talk] Schema reporting tools - XQuery use case? As a typical use case, schema information is frequently used to process GML(Geography Markup Language) instance data. Mostly, the schema querying part is done with the help of Xerces-J XML schema processing library in Java. But I do think it is quite useful to have the same thing done in XML processing language directly instead of having to call external functions. Ideally, it is best done in XQuery processors and XPath 2 spec does provide some functions to retrieve schema information. However, the problem is it is not widely supported feature, and there barely open source XPath processors that support such functionalities. I developed an XSLT function library to query schema information. It essentially contains functions to retrieve a type definition for a given element. http://www.ibm.com/developerworks/library/x-schemanode/ It is a much more intriguing programming experience to do this in XSLT than in Java and I think it can be equally done in XQuery as well. Lisa From sergiomariano at gmail.com Fri Sep 11 09:45:08 2009 From: sergiomariano at gmail.com (=?ISO-8859-1?Q?S=E9rgio_Mariano_Dias?=) Date: Fri Sep 11 04:16:08 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: References: Message-ID: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> Dear all, I have a problem to ignore a xmls attribute in a tag and did not found information how to ignore this attribute. Somebody can help me. I'm using the saxon HE 9.2 to read a xml. My xml have a tag with "xmlns=" http://www. ..."", how ignore the "xmlns" when a read the file? I needed to ignore this attribute because the address return ?page no found? and so my program return nothing. As test, I remove the address from the xml and the program return the correct value. I'm reading the file using this code: Processor proc = new Processor(false); XQueryCompiler comp = proc.newXQueryCompiler(); XdmNode source = proc.newDocumentBuilder().build(new StreamSource(inputFile)); XQueryExecutable exp = comp.compile("/.../... ..."); XQueryEvaluator eval = exp.load(); eval.setContextItem(source); XdmValue v = eval.evaluate(); System.out.println(v.size()); Regards and thank you very much by any help, ---------- Forwarded message ---------- From: Michael Kay Date: 2009/9/3 Subject: RE: xmlns ignore To: sergiomariano@gmail.com Short answer: you can't. This is actually a standard XQuery coding question, nothing specific to Saxon, so I suggest that you ask it on talk@x-query.com. Michael Kay Saxonica > -----Original Message----- > From: sergiomariano@gmail.com [mailto:sergiomariano@gmail.com] > Sent: 02 September 2009 16:04 > To: mike@saxonica.com > Subject: xmlns ignore > > Dear Michael Kay > > I'm using the saxon HE 9.2 to read a xml. My xml have a tag > with "xmlns="http://www. ..."", how ignore the "xmlns" when > a read the file? > > I'm reading the file using this code: > > Processor proc = new Processor(false); > XQueryCompiler comp = proc.newXQueryCompiler(); > XdmNode source = > proc.newDocumentBuilder().build(new StreamSource(inputFile)); > > > XQueryExecutable exp = comp.compile("/.../... ..."); > XQueryEvaluator eval = exp.load(); > eval.setContextItem(source); > > XdmValue v = eval.evaluate(); > System.out.println(v.size()); > > > Regards , > > > > -- S?rgio Mariano Dias http://www.dcc.ufmg.br/~mariano -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090911/c744fc0d/attachment.htm From davidc at nag.co.uk Fri Sep 11 14:01:05 2009 From: davidc at nag.co.uk (David Carlisle) Date: Fri Sep 11 04:31:49 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> (message from =?ISO-8859-1?Q?S=E9rgio_Mariano_Dias?= on Fri, 11 Sep 2009 08:45:08 -0300) References: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> Message-ID: <200909111201.n8BC156W030462@edinburgh.nag.co.uk> > I have a problem to ignore a xmls attribute in a tag and did not found > information how to ignore this attribute. Somebody can help me. A namespace declaration is not an attribute in the XDM model used by XPath and Xquery. The name of an element includes the namespace. Most likely the part of teh code that you don't show = comp.compile("/.../... ..."); is selecting elements in no-namespace, and you should change it to select elements in the namespace used in your xml source. either by using a prefix /p:aa/p:bb and binding p to the namespace used in the source, or by setting the default namespace in the xquery to that used in the source. David ________________________________________________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. ________________________________________________________________________ From mail at martin-probst.com Fri Sep 11 16:02:13 2009 From: mail at martin-probst.com (Martin Probst) Date: Fri Sep 11 06:01:43 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <653897.36925.qm@web23007.mail.ird.yahoo.com> References: <653897.36925.qm@web23007.mail.ird.yahoo.com> Message-ID: <6dca08520909110602n283ee03eo62e01962072ebf7b@mail.gmail.com> With a big caveat as I have not been involved in any of this, just how it seems to me. > ?From what I can see in the XQuery 1.0 REC, the "local" namespace > is aimed to be used only in a main module, and only to locally > define functions (not variables.) I think this simply came from the problem that functions without namespaces, which were allowed in an earlier draft, can collide with the built in functions or type names, so it was decided to require a namespace for all user defined functions. To make that a bit easier, we now have the local: namespace predeclared. This does not apply to variables as they don't share the same symbol space as functions, i.e. their names don't collide. > This is a bit surprising to me, > as I thought we could define both local:* functions & variables, > both in main and library modules... Indeed, you may not define functions or variables in modules that don't have the same namespace URI as the module: "The name of every variable and function declared in a library module must have a namespace URI that is the same as the target namespace of the module; otherwise a static error is raised [err:XQST0048]." Why that must be, I have never understood. It doesn't save the user from hard-to-find typo errors errors as you'd always declare the namespace prefix while declaring the module namespace, and then use that prefix. And it (IMHO) actively acts against modularization, as local variables or functions are highly desirable. In X-Hive / xDB, we have never followed that spec rule and allowed local functions/variables in modules, those are simply not exported to importing modules. I have also never really understood why it must be possible to have one module with one target namespace split over multiple files; I think Java has shown that the forced one-to-one relationship between conceptual name (class) and file (.java) is a good thing. Martin From john.snelson at oracle.com Fri Sep 11 16:05:35 2009 From: john.snelson at oracle.com (John Snelson) Date: Fri Sep 11 07:03:28 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <6dca08520909110602n283ee03eo62e01962072ebf7b@mail.gmail.com> References: <653897.36925.qm@web23007.mail.ird.yahoo.com> <6dca08520909110602n283ee03eo62e01962072ebf7b@mail.gmail.com> Message-ID: <4AA9DA9F.9090305@oracle.com> Martin Probst wrote: > Indeed, you may not define functions or variables in modules that > don't have the same namespace URI as the module: > > "The name of every variable and function declared in a library module > must have a namespace URI that is the same as the target namespace of > the module; otherwise a static error is raised [err:XQST0048]." > > Why that must be, I have never understood. It doesn't save the user > from hard-to-find typo errors errors as you'd always declare the > namespace prefix while declaring the module namespace, and then use > that prefix. And it (IMHO) actively acts against modularization, as > local variables or functions are highly desirable. In X-Hive / xDB, we > have never followed that spec rule and allowed local > functions/variables in modules, those are simply not exported to > importing modules. XQuery 1.1 is introducing the "private" function modifier, which will allow the definition of functions that aren't exported from a module. > I have also never really understood why it must be possible to have > one module with one target namespace split over multiple files; I > think Java has shown that the forced one-to-one relationship between > conceptual name (class) and file (.java) is a good thing. 1) Modules aren't classes, they are more like library APIs. 2) Java's rules are by no means universally considered a good thing. In particular it might be beneficial to split up a 5000 line Java class into multiple files, or to group together implementations of the same method from multiple implementations of a class. 3) Actually the XQuery spec doesn't say anything about whether a module namespace can be split across multiple module files - that's left implementation defined. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From mail at martin-probst.com Fri Sep 11 17:23:17 2009 From: mail at martin-probst.com (Martin Probst) Date: Fri Sep 11 07:21:29 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <4AA9DA9F.9090305@oracle.com> References: <653897.36925.qm@web23007.mail.ird.yahoo.com> <6dca08520909110602n283ee03eo62e01962072ebf7b@mail.gmail.com> <4AA9DA9F.9090305@oracle.com> Message-ID: <6dca08520909110723s312e3bbeqfb7138085ba1474c@mail.gmail.com> > XQuery 1.1 is introducing the "private" function modifier, which will > allow the definition of functions that aren't exported from a module. Simply dropping the namespace requirement and mandating that only the properly namespaced functions are exported might have accomplished the same thing without additional syntax, right? >> I have also never really understood why it must be possible to have >> one module with one target namespace split over multiple files; I >> think Java has shown that the forced one-to-one relationship between >> conceptual name (class) and file (.java) is a good thing. > > 1) Modules aren't classes, they are more like library APIs. Yes. > 2) Java's rules are by no means universally considered a good thing. In > particular it might be beneficial to split up a 5000 line Java class > into multiple files, or to group together implementations of the same > method from multiple implementations of a class. Well, if you have a single class spanning 5000 lines that might be a sign of doing it wrong in the first place ;-) The thing about grouping together tiny, closely related classes works for me in Java using nested classes, YMMV. My point was that I generally think its beneficial to have only exactly one "entity" (i.e. XQuery module file) completely containing everything thats relevant for one particular namespace, that makes many things much easier, in particular when IDE support (go to definition) isn't all that great yet. > 3) Actually the XQuery spec doesn't say anything about whether a module > namespace can be split across multiple module files - that's left > implementation defined. Yes, but this is how I understood the feature. And at least at some point of time the test suite kind of enforced that feature. Martin From john.snelson at oracle.com Fri Sep 11 16:40:39 2009 From: john.snelson at oracle.com (John Snelson) Date: Fri Sep 11 07:37:57 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <6dca08520909110723s312e3bbeqfb7138085ba1474c@mail.gmail.com> References: <653897.36925.qm@web23007.mail.ird.yahoo.com> <6dca08520909110602n283ee03eo62e01962072ebf7b@mail.gmail.com> <4AA9DA9F.9090305@oracle.com> <6dca08520909110723s312e3bbeqfb7138085ba1474c@mail.gmail.com> Message-ID: <4AA9E2D7.5080504@oracle.com> Martin Probst wrote: >> XQuery 1.1 is introducing the "private" function modifier, which will >> allow the definition of functions that aren't exported from a module. > > Simply dropping the namespace requirement and mandating that only the > properly namespaced functions are exported might have accomplished the > same thing without additional syntax, right? Yes - that's what I originally suggested. However the XQuery working group felt it was important to be able change a function to be exported / not-exported without changing every place that function is called. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From mail at martin-probst.com Sat Sep 12 16:27:53 2009 From: mail at martin-probst.com (Martin Probst) Date: Sat Sep 12 06:25:57 2009 Subject: [xquery-talk] Full Text Grammar ambiguous? Message-ID: <6dca08520909120627p6db1c6d9o7723f955b8039267@mail.gmail.com> Hi all, maybe I'm just a bit late to the party, but is it possible that the full text grammar from XQFT makes the XQuery grammar ambiguous? Consider the expression: element ftcontains { 'x' } As far as I can see, it's impossible to tell whether this should be parsed as an element constructor for the element named "ftcontains" with content 'x', or as a full text matching operation on a child axis selection for 'element' against the words 'x'. Is this a known issue? If yes, how should this be resolved? Martin From christian.gruen at gmail.com Sat Sep 12 16:37:28 2009 From: christian.gruen at gmail.com (=?ISO-8859-1?Q?Christian_Gr=FCn?=) Date: Sat Sep 12 06:35:42 2009 Subject: [xquery-talk] Full Text Grammar ambiguous? In-Reply-To: <6dca08520909120627p6db1c6d9o7723f955b8039267@mail.gmail.com> References: <6dca08520909120627p6db1c6d9o7723f955b8039267@mail.gmail.com> Message-ID: Martin, thanks for the observation! This problem was recently posted in the W3C's bugzilla system: http://www.w3.org/Bugs/Public/show_bug.cgi?id=7247 ..and is currently discussed by the XQFT task force. A decision how to solve to this issue will be posted at the same place in near future. Hope this helps, Christian ___________________________ Christian Gruen Universitaet Konstanz Department of Computer & Information Science D-78457 Konstanz, Germany Tel: +49 (0)7531/88-4449, Fax: +49 (0)7531/88-3577 http://www.inf.uni-konstanz.de/~gruen On Sat, Sep 12, 2009 at 3:27 PM, Martin Probst wrote: > Hi all, > > maybe I'm just a bit late to the party, but is it possible that the > full text grammar from XQFT makes the XQuery grammar ambiguous? > > Consider the expression: > ?element ftcontains { 'x' } > > As far as I can see, it's impossible to tell whether this should be > parsed as an element constructor for the element named "ftcontains" > with content 'x', or as a full text matching operation on a child axis > selection for 'element' against the words 'x'. > > Is this a known issue? If yes, how should this be resolved? > > Martin > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk From mail at martin-probst.com Sat Sep 12 18:00:29 2009 From: mail at martin-probst.com (Martin Probst) Date: Sat Sep 12 07:56:20 2009 Subject: [xquery-talk] Full Text Grammar ambiguous? In-Reply-To: References: <6dca08520909120627p6db1c6d9o7723f955b8039267@mail.gmail.com> Message-ID: <6dca08520909120800w447e6025h1e2e0c8ca7d02e66@mail.gmail.com> > thanks for the observation! This problem was recently posted in the > W3C's bugzilla system: Thanks Christian, somehow the W3C Bugzillas search doesn't like me (the feeling is mutual). Pretty annoying thing though, I think there are already quite a lot of (partial) XQFT implementations out there, including our own. Martin From dlee at calldei.com Sun Sep 13 15:06:27 2009 From: dlee at calldei.com (David A. Lee) Date: Sun Sep 13 10:57:31 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com> Message-ID: <4AAD34A3.6040402@calldei.com> I'm dual-posting this because I think it is relevant to XML in general, but perhaps more so to xquery Summary (postulate) There is no standard or even 'community accepted' way of serializing XDM Why is this a problem XML languages which produce XDM (such as XPath and XQuery) and can accept XDM (such as XQuery and XPath through external variables) or produce XML are not 'closed' wrt serialization. For example XQuery can produce things which have no standardized representation. The only way to consume and produce these "things" are with proprietary formats, and even those only exist in 'in memory' language representations in general. The classic example is an XDM Sequence. There is no standard or even common serialization format for Sequences. Thus an XQuery which produces a sequence (say a sequence of xs:integer) cannot have its output fed back into another XQuery in any standard way. It is true that some implementations of XQuery provide in-memory formats for XDM such that the result of that vendor's XQuery can be used as input to that vendor's XQuery in the space of that vendor's implementation language. And Some frameworks (such as .NET) provide in-memory interfaces for XDM. But in general once you cross vendors or cross languages or exit the realm of languages there is no standardized or common way to accomplish interop between XDM producers and consumers. Why is this a BIG problem I believe one of the reasons XML has achieved its great success to date is due to its standardized Text serialization format. If XML resided purely in the abstract space of "INFOSET" I don't believe it would have proliferated to even 1,000,000th as much as it has today. The fact that newer XML processing languages can produce datum which has no text serialization format is a BIG problem, IMHO. It means that there is no way to exchange data produced by one vendor's implementation to another unless it conforms to the subset of XDM which is the XML Document. Even within *the same vendor* it is almost impossible. For a tougher example, take an XQuery which produces an attribute. There is no XQuery vendor that I know of that can serialize an attribute to a 'file' (or network, or string buffer) and read it back even in its own implementation. Take even something as simple as text. Serialize a single atomic text value such as "1" and try to read it back. Even if a vendor supports reading XDM from a file, (which I know of none), what type would it assign to "1" ? xs:string ? xs:integer ? I believe this lack of standardization for serialization of XDM is both a "Problem" and a "BIG Problem" Before I extend the topic to "suggested solutions" I would love some feedback. Q: Does anyone else agree this is a "Problem" or a "Big Problem" ? Am I alone ? Why is this NOT a "Big Problem"? Q: Is there any known work in progress aimed at solving this problem ? Q: Would it be of value to solve this problem ? (and why not ?) Q: Would there be any interest in tackling this problem in a formal way, such as a W3C recommendation ? Q: If there were a recommendation approved would vendors be interested or willing to adopt / implement it ? David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From hrennau at yahoo.de Sun Sep 13 23:53:38 2009 From: hrennau at yahoo.de (Hans-Juergen Rennau) Date: Sun Sep 13 22:23:44 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <200909131900.n8DJ02eo011676@jhunter.x-query.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> Message-ID: <75667.59306.qm@web27104.mail.ukl.yahoo.com> David, in my opinion it is a BIG problem if one feels (like I do) that you are heading for a GREAT idea: advancing from the concept of a byte stream to the concept of an info stream. (info stream = XDM-modelled character stream.) If one remembers what is gained by enabling the interconnection of units emitting/consuming byte streams, one may be surprised to discover that one cannot readily connect by info stream. (That is, unless accepting that all information must be document-wrapped, the XDM subset you spoke of.) Perhaps it is a question of how tightly the units must be coupled. If the work invested in exchange is small compared with internal work (example: consuming/providing web services) it is usually not too much asked to dress up and present oneself as an XML document. XProc accepted this point of view. However, if one aims at a more immediate coupling (shell ...), one arrives at this Big Problem. I think you are right, it *is* a gap, and it *should* be addressed by a W3C recommendation. (Also, this would mean the emancipation of XDM from XML, which is overdue.) One more thought. XQuery is a prince looking like a frog. If the XDM were not what it is and forced XQuery expressions to exchange documents, rather than item sequences, XQuery were a frog looking like a frog. This alone might already make us think... With kind regards, Hans-Juergen ----- Urspr?ngliche Mail ---- From: "David A. Lee" Subject: [xquery-talk] Serialization of XDM I'm dual-posting this because I think it is relevant to XML in general, but perhaps more so to xquery Summary (postulate) There is no standard or even 'community accepted' way of serializing XDM Why is this a problem XML languages which produce XDM (such as XPath and XQuery) and can accept XDM (such as XQuery and XPath through external variables) or produce XML are not 'closed' wrt serialization. For example XQuery can produce things which have no standardized representation. The only way to consume and produce these "things" are with proprietary formats, and even those only exist in 'in memory' language representations in general. The classic example is an XDM Sequence. There is no standard or even common serialization format for Sequences. Thus an XQuery which produces a sequence (say a sequence of xs:integer) cannot have its output fed back into another XQuery in any standard way. It is true that some implementations of XQuery provide in-memory formats for XDM such that the result of that vendor's XQuery can be used as input to that vendor's XQuery in the space of that vendor's implementation language. And Some frameworks (such as .NET) provide in-memory interfaces for XDM. But in general once you cross vendors or cross languages or exit the realm of languages there is no standardized or common way to accomplish interop between XDM producers and consumers. Why is this a BIG problem I believe one of the reasons XML has achieved its great success to date is due to its standardized Text serialization format. If XML resided purely in the abstract space of "INFOSET" I don't believe it would have proliferated to even 1,000,000th as much as it has today. The fact that newer XML processing languages can produce datum which has no text serialization format is a BIG problem, IMHO. It means that there is no way to exchange data produced by one vendor's implementation to another unless it conforms to the subset of XDM which is the XML Document. Even within *the same vendor* it is almost impossible. For a tougher example, take an XQuery which produces an attribute. There is no XQuery vendor that I know of that can serialize an attribute to a 'file' (or network, or string buffer) and read it back even in its own implementation. Take even something as simple as text. Serialize a single atomic text value such as "1" and try to read it back. Even if a vendor supports reading XDM from a file, (which I know of none), what type would it assign to "1" ? xs:string ? xs:integer ? I believe this lack of standardization for serialization of XDM is both a "Problem" and a "BIG Problem" Before I extend the topic to "suggested solutions" I would love some feedback. Q: Does anyone else agree this is a "Problem" or a "Big Problem" ? Am I alone ? Why is this NOT a "Big Problem"? Q: Is there any known work in progress aimed at solving this problem ? Q: Would it be of value to solve this problem ? (and why not ?) Q: Would there be any interest in tackling this problem in a formal way, such as a W3C recommendation ? Q: If there were a recommendation approved would vendors be interested or willing to adopt / implement it ? David A. Lee ------------------------------ _______________________________________________ talk@x-query.com http://x-query.com/mailman/listinfo/talk End of talk Digest, Vol 77, Issue 19 ************************************ From gazdovsky at gmail.com Mon Sep 14 11:56:24 2009 From: gazdovsky at gmail.com (Evgeny Gazdovsky) Date: Sun Sep 13 23:17:57 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <75667.59306.qm@web27104.mail.ukl.yahoo.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> <75667.59306.qm@web27104.mail.ukl.yahoo.com> Message-ID: <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> What about RDF (any not XML notation) and SPARQL/xSPARQL? From kurt.cagle at gmail.com Mon Sep 14 02:13:43 2009 From: kurt.cagle at gmail.com (Kurt Cagle) Date: Mon Sep 14 00:33:18 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> <75667.59306.qm@web27104.mail.ukl.yahoo.com> <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> Message-ID: <6fa681b10909140113h6e51ca4ag3d57a565cdd1ae19@mail.gmail.com> I'm inclined to thing here that there are multiple issues hiding in the question of serialization of XDM. Serialization of turtle node, as Evgeny suggests, points to a non-XML syntax serialization. Similarly, consider a simple sequence of values: for $a in (1 to 5) return $a * 5 This cannot be serialized in XML notation at all in any meaningful way. Personally I think that this is where output to sequence notation makes sense: (5,10,15,20,25) which would also be consistent with an XQuery consumption model. This is probably something worth talking to Dr. Michael Kay on, as he was the editor of the XQuery/XSLT Serialization spec, but it would seem to me that any XDM serialization should either resolve to a specific type'd entity, a document node, a sequence of type'd entities or a sequence of document nodes. This would hold true just as readily for the RDF/Turtle use case; in an XQuery environment each assertion would be given either as a text node() or, if supported, as a distinct item(), and the results of a SPARQL query would then be a sequence of such nodes. Kurt Cagle Managing Editor http://xmlToday.org On Sun, Sep 13, 2009 at 11:56 PM, Evgeny Gazdovsky wrote: > What about RDF (any not XML notation) and SPARQL/xSPARQL? > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090914/521bcd87/attachment.htm From dlee at calldei.com Mon Sep 14 09:32:10 2009 From: dlee at calldei.com (David A. Lee) Date: Mon Sep 14 04:40:33 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <6fa681b10909140113h6e51ca4ag3d57a565cdd1ae19@mail.gmail.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> <75667.59306.qm@web27104.mail.ukl.yahoo.com> <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> <6fa681b10909140113h6e51ca4ag3d57a565cdd1ae19@mail.gmail.com> Message-ID: <4AAE37CA.8040400@calldei.com> A step towards defining an XDM Serialization would be to knock at the edges of the problem. 1) I think people would agree that a Document should be Serialized (and read) in existing standard XML text form. -- This preserves the most common use case 2) I think it is desirable that a single atomic xs:string be serialized as plain string unescaped and unencoded. -- I think this is a common use case, for example generating plain text, or even things like CSV from xquery It MAY be desirable that any atomic type be serialized as if it were first converted to xs:string ... e.g. xs:integer(1) -> "1" Now comes the fun part, Since it is impossible to distinguish a "Sequence of one item" from "One item" then we need to serialize a Sequence of One identically to One. My guess is this is where the committees started having problems and decided to avoid the issue altogether. If, say a sequence of 1 integer is serialized as "1", but as Kurt suggested a sequence notation could be used for sequences, then say a sequence of 2 integers ( for example) could be "(1,2)". this means that sequences of 1 and sequences of N>1 are serialized differently ! I suggest this is undesirable. There is a common practice that avoids this. That is by using a syntax that avoids header and trailing markup so that an single value is serialized the same as a sequence of 1 value. One example is simply using a separator such as LF or space. The terminator being supplied by the wrapper (such as EOF or NUL or end-of-element). This requires (as any markup) a way of escaping or quoting so that the seperatator can be contained within the data. In order to conform to #2 (simple string output as plain text) that leads out quoting, but does require escaping. Depending on the separator used, this could be annoying or inconsequential. It could also be variable (defined by the environmen). This is essentially CSV ... or its variants ... That would buy us sequences. Even sequences of documents ! *IFF* XML parsers would be so nice as to stop reading when they have read the closing tag instead of throw an error (I believe this is a requirement of a conforming parser, not sure why). But this can be hacked. (I'd prefer a hack *inside* the XML parsers but if not then wrappers can be written which reuse existing XML parsers , not pretty though). Certainly a parser would have to know its reading XDM not XML ahead of time. Well thats a start, its an outline for how to serialize sequences which preserves common use cases. Further work is needed to define the individual item serialization forms and decide if preserving type is critical or not. I.e. if you write an xs:integer is it OK to get back an xs:string ? David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Kurt Cagle wrote: > I'm inclined to thing here that there are multiple issues hiding in > the question of serialization of XDM. Serialization of turtle node, as > Evgeny suggests, points to a non-XML syntax serialization. Similarly, > consider a simple sequence of values: > > for $a in (1 to 5) return $a * 5 > > This cannot be serialized in XML notation at all in any meaningful > way. Personally I think that this is where output to sequence notation > makes sense: > > (5,10,15,20,25) > > which would also be consistent with an XQuery consumption model. > > This is probably something worth talking to Dr. Michael Kay on, as he > was the editor of the XQuery/XSLT Serialization spec, but it would > seem to me that any XDM serialization should either resolve to a > specific type'd entity, a document node, a sequence of type'd entities > or a sequence of document nodes. This would hold true just as readily > for the RDF/Turtle use case; in an XQuery environment each assertion > would be given either as a text node() or, if supported, as a distinct > item(), and the results of a SPARQL query would then be a sequence of > such nodes. > > Kurt Cagle > Managing Editor > http://xmlToday.org > > > On Sun, Sep 13, 2009 at 11:56 PM, Evgeny Gazdovsky > > wrote: > > What about RDF (any not XML notation) and SPARQL/xSPARQL? > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > > > ------------------------------------------------------------------------ > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090914/afaad0f8/attachment.htm From gazdovsky at gmail.com Mon Sep 14 18:48:58 2009 From: gazdovsky at gmail.com (Evgeny Gazdovsky) Date: Mon Sep 14 06:46:48 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAE37CA.8040400@calldei.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> <75667.59306.qm@web27104.mail.ukl.yahoo.com> <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> <6fa681b10909140113h6e51ca4ag3d57a565cdd1ae19@mail.gmail.com> <4AAE37CA.8040400@calldei.com> Message-ID: <508bb1470909140648h5ebbcd5ayba28d4b83d947ac6@mail.gmail.com> We had forgot about JSON! :) From lists at fgeorges.org Mon Sep 14 15:36:11 2009 From: lists at fgeorges.org (Florent Georges) Date: Mon Sep 14 07:32:38 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAD34A3.6040402@calldei.com> Message-ID: <624589.19755.qm@web23005.mail.ird.yahoo.com> David A. Lee wrote: Hi, > There is no standard or even 'community accepted' way of > serializing XDM While I used to think this is a problem, more and more I think it is not (or at most a little problem we can live with.) XDM is a convenient (and needed) data structure to work with within a piece of code (like a stylesheet or a query.) (BTW this is a pity that XProc is not based on XDM, IMHO) But if you want to pass information between two different pieces of code (like between a query and a stylesheet) you can just restrict yourself to nodes. If you want to pass a sequence of integers, you can instead pass an element containing elements the content of which are integers... So the simple rule: independent pieces of code set boundaries between which you can only pass nodes, or even document nodes. That's what XML means, after all. If you use compatible processors, you should then be able to configure them to pass those nodes in an efficient way, and if not you can serialize and parse them without loosing any information. Just my two cents.... Regards, -- Florent Georges http://www.fgeorges.org/ From dlee at calldei.com Mon Sep 14 11:48:31 2009 From: dlee at calldei.com (David A. Lee) Date: Mon Sep 14 07:44:50 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <624589.19755.qm@web23005.mail.ird.yahoo.com> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> Message-ID: <4AAE57BF.4090006@calldei.com> I agree with your sentiment *"if not you can serialize and parse them without loosing any information." *but not the implication. This is (partially) true, but the problem with this is there is not *standardized* way to do this. For simple things like say sequences of nodes, you can wrap them in a document. Or for say integers you can wrap them in something like an tag. The problem is there are a *million* ways to do it ... which means no vendor has agreed, which leaves it up to the programmer to do it. Which means you have to cooperate on both sides of the equation, and since there is no standardized way to do it, different toolkits or pieces are almost certain to do it differently, and possibly in ways that are difficult to parse. By "partially" I am refering to the interesting XDM type of attributes, which noone can agree at all on how to serialize. ( although you could stick them in an element ... ) So to make this actually work in practice, you must control both sides of the partnership, and the serialization in between, and also pass "metadata" either by convention or something to make it all work. A programmer, to say take the output of one XQuery, store it in a file, then put it back into an external varialbe of another XQuery invocation has to do quite a lot of work, the sort of work that XML was designed explicitly to *avoid having to do*. Even defining a standardized schema for 'wrapping' XDM content unambiguously in a XML document would be a huge value, even if the result wasn't pretty, at least it has the potential for interchange. But lets not talk about function items ... I have no clue where to go with those .... David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Florent Georges wrote: > David A. Lee wrote: > > Hi, > > >> There is no standard or even 'community accepted' way of >> serializing XDM >> > > While I used to think this is a problem, more and more I think > it is not (or at most a little problem we can live with.) XDM > is a convenient (and needed) data structure to work with within a > piece of code (like a stylesheet or a query.) (BTW this is a > pity that XProc is not based on XDM, IMHO) > > But if you want to pass information between two different > pieces of code (like between a query and a stylesheet) you can > just restrict yourself to nodes. If you want to pass a sequence > of integers, you can instead pass an element containing elements > the content of which are integers... > > So the simple rule: independent pieces of code set boundaries > between which you can only pass nodes, or even document nodes. > That's what XML means, after all. If you use compatible > processors, you should then be able to configure them to pass > those nodes in an efficient way, and if not you can serialize and > parse them without loosing any information. > > Just my two cents.... > > Regards, > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090914/2d37fe78/attachment.htm From gazdovsky at gmail.com Mon Sep 14 20:02:27 2009 From: gazdovsky at gmail.com (Evgeny Gazdovsky) Date: Mon Sep 14 07:58:04 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAE57BF.4090006@calldei.com> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> Message-ID: <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> Still think that JSON may be a variant From vyacheslav.sedov at gmail.com Mon Sep 14 20:08:59 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Mon Sep 14 08:04:24 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <508bb1470909140648h5ebbcd5ayba28d4b83d947ac6@mail.gmail.com> References: <200909131900.n8DJ02eo011676@jhunter.x-query.com> <75667.59306.qm@web27104.mail.ukl.yahoo.com> <508bb1470909132356w19bcec72gc45626adc28e3103@mail.gmail.com> <6fa681b10909140113h6e51ca4ag3d57a565cdd1ae19@mail.gmail.com> <4AAE37CA.8040400@calldei.com> <508bb1470909140648h5ebbcd5ayba28d4b83d947ac6@mail.gmail.com> Message-ID: <7ab7cbac0909140808o37ecf99q297c786ef1e6efae@mail.gmail.com> On Mon, Sep 14, 2009 at 5:48 PM, Evgeny Gazdovsky wrote: > We had forgot about JSON! :) congratulations - right move to forget JSON > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From mike at saxonica.com Mon Sep 14 21:42:36 2009 From: mike at saxonica.com (Michael Kay) Date: Mon Sep 14 12:32:40 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAD34A3.6040402@calldei.com> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com> <4AAD34A3.6040402@calldei.com> Message-ID: > > I believe this lack of standardization for serialization of > XDM is both > a "Problem" and a "BIG Problem" > Before I extend the topic to "suggested solutions" I would love some > feedback. > > Q: Does anyone else agree this is a "Problem" or a "Big > Problem" ? Am I alone ? Why is this NOT a "Big Problem"? Saxon has a serialization format for arbitrary XDM sequences: you can see it by running a query with -wrap option from the command line. Perhaps oddly, Saxon can generate this format, but there are no tools to parse it and reconstruct the original sequence. I seem to remember someone proposing that a format like this should be standardised in the XQuery WG years ago - if only as a standard way of representing the results of queries in the test suite. But the fact that no-one was prepared to do the work suggests that not many people thought it was a big problem. > Q: Is there any known work in progress aimed at solving this problem ? Not that I know of. > Q: Would it be of value to solve this problem ? (and why not ?) Yes. Whether the value would exceed the cost is anyone's guess. > Q: Would there be any interest in tackling this problem in a > formal way, such as a W3C recommendation ? W3C groups are generally having difficulty finding the resources to complete their existing work programmes, without taking on anything new. > Q: If there were a recommendation approved would vendors be > interested or willing to adopt / implement it ? > If there appeared to be sufficient support behind it, then yes. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay From int19h at gmail.com Mon Sep 14 13:55:51 2009 From: int19h at gmail.com (Pavel Minaev) Date: Mon Sep 14 12:45:59 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> Message-ID: In Mon, Sep 14, 2009 at 8:02 AM, Evgeny Gazdovsky wrote: > Still think that JSON may be a variant I don't see how JSON could be any sort of serious contender to serialize XDM data, given that the latter is much richer than JSON primitives. On the other hand, it seems to me that we already have such a serialization format, and it's XQuery itself - or rather the subset that involves literals, sequence "constructor" (i.e. comma operator), and direct element/attribute/namespace/text constructors. It's as portable as it gets - any XQuery processor can immediately parse it. And it doesn't seem to be missing anything from XDM, either. From dlee at calldei.com Mon Sep 14 17:06:13 2009 From: dlee at calldei.com (David A. Lee) Date: Mon Sep 14 12:56:02 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> Message-ID: <4AAEA235.8050801@calldei.com> Somewhat Ironically, this solves the same problem the same way JSON did i.e. make the serialization format the same as the Code (in Jason's case Javascript). It also has the same problem though. It requires an XQuery paser to parse. Suppose you want to send the results to something that isnt XQuery, say as an external variable to an XSLT or XPATH processor. Those would require XQuery to pase it. Its an interesting idea with some merit though ... David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Pavel Minaev wrote: > In Mon, Sep 14, 2009 at 8:02 AM, Evgeny Gazdovsky wrote: > >> Still think that JSON may be a variant >> > > I don't see how JSON could be any sort of serious contender to > serialize XDM data, given that the latter is much richer than JSON > primitives. > > On the other hand, it seems to me that we already have such a > serialization format, and it's XQuery itself - or rather the subset > that involves literals, sequence "constructor" (i.e. comma operator), > and direct element/attribute/namespace/text constructors. It's as > portable as it gets - any XQuery processor can immediately parse it. > And it doesn't seem to be missing anything from XDM, either. > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090914/8f599b4f/attachment.htm From mike at saxonica.com Mon Sep 14 22:16:22 2009 From: mike at saxonica.com (Michael Kay) Date: Mon Sep 14 13:05:56 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> Message-ID: <34BB9115291342B5A8AFC2535FD1EE95@Sealion> > On the other hand, it seems to me that we already have such a > serialization format, and it's XQuery itself - or rather the > subset that involves literals, sequence "constructor" (i.e. > comma operator), and direct element/attribute/namespace/text > constructors. It's as portable as it gets - any XQuery > processor can immediately parse it. > And it doesn't seem to be missing anything from XDM, either. For some use cases it's important that the format be canonical: that is, there should only be one way of representing a given sequence. Also, XQuery doesn't have any direct way of representing the type annotation on a node: the closest you can get is a validate{} instruction, and the effect of this is very context-dependent. Incidentally, one deficiency of the Saxon -wrap format that I mentioned earlier is that it doesn't contain any information about node identity. If you have a sequence of three elements with the same name and content, it won't tell you whether the sequence being serialized contains three distinct nodes, or a single node repeated three times. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay From int19h at gmail.com Mon Sep 14 14:20:02 2009 From: int19h at gmail.com (Pavel Minaev) Date: Mon Sep 14 13:09:50 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAEA235.8050801@calldei.com> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <4AAEA235.8050801@calldei.com> Message-ID: On Mon, Sep 14, 2009 at 1:06 PM, David A. Lee wrote: > Somewhat Ironically, this solves the same problem the same way JSON did i.e. > make the serialization format the same as the Code (in Jason's case > Javascript). > It also has the same problem though. > It requires an XQuery paser to parse.? Suppose you want to send the results > to something that isnt XQuery, say as an external variable to an XSLT or > XPATH processor.?? Those would require XQuery to pase it. Just as with JSON, the ability to parse data by eval'ing it in XQuery is just a side bonus. Of course you'd also want specialized parsers for this when dealing with such serialized data in anything other than XQuery (and you'd want it even in XQuery for security reasons). But there's no shortage of JSON parsers of all kinds, written in different languages and for different platforms, these days. This could be similar. A more major advantage of using XQuery subset as a basis for such a serialization format is that you don't need to create a specification from scratch. For the most part, you can directly re-use XQuery BNF and semantics from the spec. Also, so long as it is a strict subset, any new dedicated parser could reuse existing XQuery lexer/parser code with no changes. You'd pretty only need a new syntax tree rewriter that would output XDM nodes, in whatever in-memory format a particular implementation wants. By the way, much as DOM is often maligned, the idea of defining some common, strictly specified interfaces (in API sense) still has merit, and having something along those lines for XDM would be nice as well (and would mesh up well with a common serialization format). From int19h at gmail.com Mon Sep 14 14:29:30 2009 From: int19h at gmail.com (Pavel Minaev) Date: Mon Sep 14 13:19:08 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <34BB9115291342B5A8AFC2535FD1EE95@Sealion> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> Message-ID: On Mon, Sep 14, 2009 at 1:16 PM, Michael Kay wrote: >> On the other hand, it seems to me that we already have such a >> serialization format, and it's XQuery itself - or rather the >> subset that involves literals, sequence "constructor" (i.e. >> comma operator), and direct element/attribute/namespace/text >> constructors. It's as portable as it gets - any XQuery >> processor can immediately parse it. >> And it doesn't seem to be missing anything from XDM, either. > > For some use cases it's important that the format be canonical: that is, > there should only be one way of representing a given sequence. > > Also, XQuery doesn't have any direct way of representing the type annotation > on a node: the closest you can get is a validate{} instruction, and the > effect of this is very context-dependent. I wonder if "treat as" operator could be abused for this purpose. The deserializer would have to be specified as if it always runs the input sequence through validate{} recursively, naturally... Of course, there's still the problem of no inline schemas for XQuery, so it's still not a fully self-contained format. Then again, in practice, few people use inline schemas for plain XML serialization as well, so it might not be that much of a problem. > Incidentally, one deficiency of the Saxon -wrap format that I mentioned > earlier is that it doesn't contain any information about node identity. If > you have a sequence of three elements with the same name and content, it > won't tell you whether the sequence being serialized contains three distinct > nodes, or a single node repeated three times. Good point. It's fairly obvious how this could be done with plain XQuery in general with let/return, but I also can't think of any good way to define a canonical serialization format for it. One approach would be to require all nodes that occur more than once in the output (and only such nodes) to produce a single let-clause, with unspecified order and unspecified variable names they're bound to, at the beginning of the query, immediately followed by a single "return" clause that references them. But this wouldn't be very human-readable for moderately sized graphs, IMO. From dlee at calldei.com Mon Sep 14 19:12:07 2009 From: dlee at calldei.com (David A. Lee) Date: Mon Sep 14 14:59:58 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <34BB9115291342B5A8AFC2535FD1EE95@Sealion> References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> Message-ID: <4AAEBFB7.30005@calldei.com> Michael Kay wrote: > > Incidentally, one deficiency of the Saxon -wrap format that I mentioned > earlier is that it doesn't contain any information about node identity. If > you have a sequence of three elements with the same name and content, it > won't tell you whether the sequence being serialized contains three distinct > nodes, or a single node repeated three times. > > Interesting point about Node Identity. I hadnt thought of that. Do you have an oppionion of how important it is to preserve node identity ? Would this be important only at the sequence level or in nested elements as well ? e.g let $a := return ( $a , $a , $a ) ---> recognize all 3 are the same let $a := return ( {$a},{$a},{$a}) --> do we need to know that all 3 are the same ? On first glance I can imagine the former being fairly easy, but the later much more complex. David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From lists at fgeorges.org Mon Sep 14 23:31:09 2009 From: lists at fgeorges.org (Florent Georges) Date: Mon Sep 14 15:18:32 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAEBFB7.30005@calldei.com> Message-ID: <506825.76843.qm@web23001.mail.ird.yahoo.com> David A. Lee wrote: Hi David, > let $a := > return > ( $a , $a , $a ) > ---> recognize all 3 are the same > let $a := > return > ( > {$a},{$a},{$a}) > --> do we need to know that all 3 are the same ? They have the same value, but they are not identical (last paragraph of the section "3.7.1 Direct Element Constructors" ): The result of an element constructor is a new element node, with its own node identity. All the attribute and descendant nodes of the new element node are also new nodes with their own identities, even if they are copies of existing nodes. Regards, -- Florent Georges http://www.fgeorges.org/ From davidc at nag.co.uk Tue Sep 15 00:31:13 2009 From: davidc at nag.co.uk (David Carlisle) Date: Mon Sep 14 15:18:39 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAEBFB7.30005@calldei.com> (dlee@calldei.com) References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> <4AAEBFB7.30005@calldei.com> Message-ID: <200909142231.n8EMVDtX018610@edinburgh.nag.co.uk> > On first glance I can imagine the former being fairly easy, but the > later much more complex. The latter isn't a problem as they are different elements in the result, the $a nodes are _copied_ into their parent as {$a} copy here doesn't copy node identity. David ________________________________________________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. ________________________________________________________________________ From dlee at calldei.com Mon Sep 14 19:36:20 2009 From: dlee at calldei.com (David A. Lee) Date: Mon Sep 14 15:23:43 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <200909142231.n8EMVDtX018610@edinburgh.nag.co.uk> References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> <4AAEBFB7.30005@calldei.com> <200909142231.n8EMVDtX018610@edinburgh.nag.co.uk> Message-ID: <4AAEC564.7080600@calldei.com> What about this document ? Input: Query: ( / , /foo/bar ) If serialized then re-parsed into say : declare variable $a as external; -> stuck into here is it important that $a[1]/foo/bar is $a[2] David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 David Carlisle wrote: >> On first glance I can imagine the former being fairly easy, but the >> later much more complex. >> > > The latter isn't a problem as they are different elements in the result, > the $a nodes are _copied_ into their parent as {$a} copy here > doesn't copy node identity. > > David > > ________________________________________________________________________ > The Numerical Algorithms Group Ltd is a company registered in England > and Wales with company number 1249803. The registered office is: > Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. > > This e-mail has been scanned for all viruses by Star. The service is > powered by MessageLabs. > ________________________________________________________________________ > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090914/7f6b436e/attachment.htm From davidc at nag.co.uk Tue Sep 15 03:45:48 2009 From: davidc at nag.co.uk (David Carlisle) Date: Mon Sep 14 18:27:14 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <4AAEC564.7080600@calldei.com> (dlee@calldei.com) References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> <4AAEBFB7.30005@calldei.com> <200909142231.n8EMVDtX018610@edinburgh.nag.co.uk> <4AAEC564.7080600@calldei.com> Message-ID: <200909150145.n8F1jmG1019658@edinburgh.nag.co.uk> > is it important that > $a[1]/foo/bar is $a[2] well it's inportant if the next process in the pipeline needs to do any kind of query processing, which is presumably the point of wanting to pass an xdm instance in the first place. if the sequence appears to the next process as $input then something as simple as count($input//bar) is going to give different answers depending on whether identity is preserved. But in practice it's probably only important to know what the answer is (whether identity is presrved or not) if you pass xml between processes by serialising and reparsing then typically you end up with new identities as far as xdm is concerned but this is rarely a problem in practice. so if the specification of the serialisation were to specify that you were serialising a _copy_ of any XDM instance, so in particular every item of the sequence always gets a new identity I think that would be OK for the vast majority of cases. David ________________________________________________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. ________________________________________________________________________ From mike at saxonica.com Tue Sep 15 08:58:05 2009 From: mike at saxonica.com (Michael Kay) Date: Mon Sep 14 23:34:38 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <200909150145.n8F1jmG1019658@edinburgh.nag.co.uk> References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion><4AAEBFB7.30005@calldei.com><200909142231.n8EMVDtX018610@edinburgh.nag.co.uk><4AAEC564.7080600@calldei.com> <200909150145.n8F1jmG1019658@edinburgh.nag.co.uk> Message-ID: <833C8C966EED491FA8FA28FDE09D7C85@Sealion> > > > is it important that > > $a[1]/foo/bar is $a[2] > Someone needs to sit down and write down the requirements and use cases. One requirement that someone suggested was of the nature of "lossless round-tripping". That would imply that given a sequence (N, M), not only does (N is M) retain its value after serialization and parsing, but so does (N/.. is M/..): which means you can't just serialize the subtrees rooted at the nodes in the sequence, you have to serialize the entire document in which these nodes are contained. Perhaps this explains why there is currently no standard in this area. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay From gkholman at CraneSoftwrights.com Tue Sep 15 01:38:28 2009 From: gkholman at CraneSoftwrights.com (G. Ken Holman) Date: Tue Sep 15 00:45:01 2009 Subject: [xquery-talk] Announce: XQuery, XSLT, XPath and XSL-FO training in US (XQuery talk) Message-ID: <7.0.1.0.2.20090915003757.0266e1d0@CraneSoftwrights.com> Crane Softwrights Ltd. of Ottawa Canada is pleased to announce a US east-coast (Washington DC/Falls Church VA area) training delivery of its XML material in partnership with Mark Logic Corporation of San Carlos California. Through four separately-subscribed classes over 11 consecutive weekdays in October/November 2009, these intensive hands-on classes cover working with XML for the OASIS code list specifications, the OASIS Universal Business Language 2.0 standard, the W3C XQuery, XSLT and XPath 1.0 and 2.0 standards and the W3C XSL-FO 1.0 and 1.1 standards: Practical Code List Implementation (1 day): 2009-10-28 http://www.CraneSoftwrights.com/training/pcli/pclisyl.htm Abbreviated code: PCLI Cost: US$550. Practical Universal Business Language Deployment (2 days): 2009-10-29/30 http://www.CraneSoftwrights.com/training/publd/publdsyl.htm Abbreviated code: PUBLD Cost: US$1100. Practical Transformation Using XSLT, XQuery and XPath (5 days): 2009-11-02/06 http://www.CraneSoftwrights.com/training/ptuxq/ptuxqsyl.htm Abbreviated code: PTUXQ Cost: US$2750. Practical Formatting Using XSL-FO (3 days): 2009-11-09/11 http://www.CraneSoftwrights.com/training/pfux/pfuxsyl.htm Abbreviated code: PFUX Cost: US$1650. Venue: Homewood Suites, Falls Church, VA (Merrifield) - http://tinyurl.com/lj548u 8130 Porter Road, Merrifield, Virginia, United States 22042 (daily lunch is included in the fee) Registration for these classes is through Mark Logic's training division: http://www.MarkLogic.com/services/training.html and all students must register via email: mailto:training@MarkLogic.com?subject=Crane%20XML%20training%20Washington Class size is limited to only 15 students, so please register early to ensure you get your seat. Seats are allocated in the order of paid registrations received, not email requests. Check out all of our training offerings linked from our home page below. We have plans for future classes and you can tell us if your interest in them: http://www.CraneSoftwrights.com/forms/interest.php Thanks! . . . . . . . . . . Ken cc: XQuery-Talk, XSL List, Yahoo XSL-FO, Antenna House list, RenderX list, XML-Doc, XML-L -- Interested in these classes? http://www.CraneSoftwrights.com/q/i/ Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video Video lesson: http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18 Video overview: http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18 G. Ken Holman mailto:gkholman@CraneSoftwrights.com Male Cancer Awareness Nov'07 http://www.CraneSoftwrights.com/q/bc Legal business disclaimers: http://www.CraneSoftwrights.com/legal From vyacheslav.sedov at gmail.com Tue Sep 15 15:10:46 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Tue Sep 15 02:43:51 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <833C8C966EED491FA8FA28FDE09D7C85@Sealion> References: <624589.19755.qm@web23005.mail.ird.yahoo.com> <4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion> <4AAEBFB7.30005@calldei.com> <200909142231.n8EMVDtX018610@edinburgh.nag.co.uk> <4AAEC564.7080600@calldei.com> <200909150145.n8F1jmG1019658@edinburgh.nag.co.uk> <833C8C966EED491FA8FA28FDE09D7C85@Sealion> Message-ID: <7ab7cbac0909150310r29e4d2fs790d4503dbf5edd@mail.gmail.com> but nice idea "keep it XML" http://kixml.blogspot.com/2009/09/serving-xml_04.html it can help to combine XQuery&XSLT into one engine for example how eXist and Saxon perform exchange? xml as string? On Tue, Sep 15, 2009 at 10:58 AM, Michael Kay wrote: >> >> > is it important that >> > ? ? $a[1]/foo/bar ?is ?$a[2] >> > > Someone needs to sit down and write down the requirements and use cases. > > One requirement that someone suggested was of the nature of "lossless > round-tripping". That would imply that given a sequence (N, M), not only > does (N is M) retain its value after serialization and parsing, but so does > (N/.. is M/..): which means you can't just serialize the subtrees rooted at > the nodes in the sequence, you have to serialize the entire document in > which these nodes are contained. > > Perhaps this explains why there is currently no standard in this area. > > Regards, > > Michael Kay > http://www.saxonica.com/ > http://twitter.com/michaelhkay > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From dlee at calldei.com Tue Sep 15 08:34:58 2009 From: dlee at calldei.com (David A. Lee) Date: Tue Sep 15 04:00:31 2009 Subject: [xquery-talk] Serialization of XDM In-Reply-To: <833C8C966EED491FA8FA28FDE09D7C85@Sealion> References: <624589.19755.qm@web23005.mail.ird.yahoo.com><4AAE57BF.4090006@calldei.com> <508bb1470909140802g74cdb052l7461dfd94b03885e@mail.gmail.com> <34BB9115291342B5A8AFC2535FD1EE95@Sealion><4AAEBFB7.30005@calldei.com><200909142231.n8EMVDtX018610@edinburgh.nag.co.uk><4AAEC564.7080600@calldei.com> <200909150145.n8F1jmG1019658@edinburgh.nag.co.uk> <833C8C966EED491FA8FA28FDE09D7C85@Sealion> Message-ID: <4AAF7BE2.6000301@calldei.com> Excellent points. IMHO, asking for 100% lossless is asking too much. So if its not 100% where's the line ? A lossy implementation would still be very useful If one were to start writing down intent, use cases and a proposal, what would be a good forum for this ? A web page ? A document via email via these user groups ? Suggestions welcome. -David David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Michael Kay wrote: >>> is it important that >>> $a[1]/foo/bar is $a[2] >>> > > Someone needs to sit down and write down the requirements and use cases. > > One requirement that someone suggested was of the nature of "lossless > round-tripping". That would imply that given a sequence (N, M), not only > does (N is M) retain its value after serialization and parsing, but so does > (N/.. is M/..): which means you can't just serialize the subtrees rooted at > the nodes in the sequence, you have to serialize the entire document in > which these nodes are contained. > > Perhaps this explains why there is currently no standard in this area. > > Regards, > > Michael Kay > http://www.saxonica.com/ > http://twitter.com/michaelhkay > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090915/b599c444/attachment.htm From sergiomariano at gmail.com Tue Sep 15 11:07:15 2009 From: sergiomariano at gmail.com (=?ISO-8859-1?Q?S=E9rgio_Mariano_Dias?=) Date: Tue Sep 15 06:05:59 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: <3a08809b0909150432s6ce146a9ve0c7ec825c74a6ac@mail.gmail.com> References: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> <200909111201.n8BC156W030462@edinburgh.nag.co.uk> <3a08809b0909150432s6ce146a9ve0c7ec825c74a6ac@mail.gmail.com> Message-ID: <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> Hi, This is a complete code test: //--------------------------------------------------------------------------------------------------------------------------------- Processor proc = new Processor(false); XQueryCompiler comp = proc.newXQueryCompiler(); XdmNode source = proc.newDocumentBuilder().build(new StreamSource(inputFile)); XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp"); XQueryEvaluator eval = exp.load(); eval.setContextItem(source); XdmItem i =eval.evaluateSingle(); System.out.println(i.getStringValue()); XdmValue v = eval.evaluate(); System.out.println(v.size()); //--------------------------------------------------------------------------------------------------------------------------------- that is the xml file: //--------------------------------------------------------------------------------------------------------------------------------- 29 123658 Venda - direta .... //--------------------------------------------------------------------------------------------------------------------------------- The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I remove the addres, the program return the correct value. That is way a fink to ignore the addres. Is that possible? Regards, 2009/9/11 David Carlisle > > > I have a problem to ignore a xmls attribute in a tag and did not found > > information how to ignore this attribute. Somebody can help me. > > A namespace declaration is not an attribute in the XDM model used by > XPath and Xquery. The name of an element includes the namespace. > > Most likely the part of teh code that you don't show > > = comp.compile("/.../... ..."); > > is selecting elements in no-namespace, and you should change it to > select elements in the namespace used in your xml source. > > either by using a prefix > /p:aa/p:bb > > and binding p to the namespace used in the source, or by setting the > default namespace in the xquery to that used in the source. > > > David > > > ________________________________________________________________________ > The Numerical Algorithms Group Ltd is a company registered in England > and Wales with company number 1249803. The registered office is: > Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. > > This e-mail has been scanned for all viruses by Star. The service is > powered by MessageLabs. > ________________________________________________________________________ > -- S?rgio Mariano -- S?rgio Mariano -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090915/5063b3b0/attachment-0001.htm From mike at saxonica.com Tue Sep 15 15:26:43 2009 From: mike at saxonica.com (Michael Kay) Date: Tue Sep 15 06:25:12 2009 Subject: [xquery-talk] RE: [xml-dev] Serialization of XDM In-Reply-To: <4AAF8E48.5050107@allette.com.au> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com> <4AAD34A3.6040402@calldei.com> <4AAF8E48.5050107@allette.com.au> Message-ID: <1B42A885ECC745C7A9CE3345CF55E20D@Sealion> > > There is no standard or even 'community accepted' way of > serializing XDM > What is wrong with JSON for this? Nearly everything, starting with the fact that the target user community is likely to want something that can be manipulated using XML tools. In fact, what is right with JSON for this? > > The XPath2 data model is a system for *converting* XML into > programming language types, or for operating on a kind of XSD > PSVI. That's not how I would define it (and it's not how the spec defines it). In fact, I don't really see where conversion into "programming language types" comes into it at all. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay From andrew.j.welch at gmail.com Tue Sep 15 15:29:00 2009 From: andrew.j.welch at gmail.com (Andrew Welch) Date: Tue Sep 15 06:27:25 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> References: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> <200909111201.n8BC156W030462@edinburgh.nag.co.uk> <3a08809b0909150432s6ce146a9ve0c7ec825c74a6ac@mail.gmail.com> <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> Message-ID: <74a894af0909150629v37b517fck13b7765c82857fec@mail.gmail.com> > ??? ??? ? XQueryCompiler comp = proc.newXQueryCompiler(); > > ??? ??? ? XdmNode source = proc.newDocumentBuilder().build(new > StreamSource(inputFile)); > > > > ??? ??? ? XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp"); [snip] > > > ??????? Either change your query to include the namespace eg: declare namespace f = "http://www.portalfiscal.inf.br/nfe"; with /f:NFe/f:infNFe/f:ide/f:natOp or declare the namespace on the XQueryCompiler: http://www.saxonica.com/documentation/dotnetdoc/Saxon/Api/XQueryCompiler.html -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/ From mike at saxonica.com Tue Sep 15 15:35:25 2009 From: mike at saxonica.com (Michael Kay) Date: Tue Sep 15 06:33:52 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> References: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com><200909111201.n8BC156W030462@edinburgh.nag.co.uk><3a08809b0909150432s6ce146a9ve0c7ec825c74a6ac@mail.gmail.com> <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> Message-ID: <1D184B8E15714A1F8C94B04BB5BAE4FB@Sealion> You clearly didn't understand David's answer. You can no more "ignore" a namespace declaration in your source document than you can ignore the first character of each element name. The namespace is an intrinsic part of the names of your elements, and when selecting elements by name, you need to be aware of it. If you want to use unqualified names in your XQuery expression to refer to names in a namespace, you can do this by means of a default namespace declaration in the XQuery prolog: declare default element namespace " http://www.portalfiscal.inf.br/nfe" However, you need to be aware that this affects the interpretation of all unprefixed names in your query, not only those in a path expression. If you don't want to change the source of the query, you can achieve the same effect using the Saxon API: comp.declareNamespace("", " http://www.portalfiscal.inf.br/nfe"); Your example query will then work as written. Note, however, that this is a very different thing from "ignoring" the namespace. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay _____ From: talk-bounces@x-query.com [mailto:talk-bounces@x-query.com] On Behalf Of S?rgio Mariano Dias Sent: 15 September 2009 14:07 To: talk@x-query.com Subject: [xquery-talk] Fwd: xmlns ignore Hi, This is a complete code test: //-------------------------------------------------------------------------- ------------------------------------------------------- Processor proc = new Processor(false); XQueryCompiler comp = proc.newXQueryCompiler(); XdmNode source = proc.newDocumentBuilder().build(new StreamSource(inputFile)); XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp"); XQueryEvaluator eval = exp.load(); eval.setContextItem(source); XdmItem i =eval.evaluateSingle(); System.out.println(i.getStringValue()); XdmValue v = eval.evaluate(); System.out.println(v.size()); //-------------------------------------------------------------------------- ------------------------------------------------------- that is the xml file: //-------------------------------------------------------------------------- ------------------------------------------------------- 29 123658 Venda - direta .... //-------------------------------------------------------------------------- ------------------------------------------------------- The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I remove the addres, the program return the correct value. That is way a fink to ignore the addres. Is that possible? Regards, 2009/9/11 David Carlisle > I have a problem to ignore a xmls attribute in a tag and did not found > information how to ignore this attribute. Somebody can help me. A namespace declaration is not an attribute in the XDM model used by XPath and Xquery. The name of an element includes the namespace. Most likely the part of teh code that you don't show = comp.compile("/.../... ..."); is selecting elements in no-namespace, and you should change it to select elements in the namespace used in your xml source. either by using a prefix /p:aa/p:bb and binding p to the namespace used in the source, or by setting the default namespace in the xquery to that used in the source. David ________________________________________________________________________ The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is: Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. ________________________________________________________________________ -- S?rgio Mariano -- S?rgio Mariano -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090915/73c0cba1/attachment.htm From sergiomariano at gmail.com Tue Sep 15 11:44:30 2009 From: sergiomariano at gmail.com (=?ISO-8859-1?Q?S=E9rgio_Mariano_Dias?=) Date: Tue Sep 15 06:42:46 2009 Subject: [xquery-talk] Fwd: xmlns ignore In-Reply-To: <1D184B8E15714A1F8C94B04BB5BAE4FB@Sealion> References: <3a08809b0909110445m498abff2wab86d96e302a4a59@mail.gmail.com> <200909111201.n8BC156W030462@edinburgh.nag.co.uk> <3a08809b0909150432s6ce146a9ve0c7ec825c74a6ac@mail.gmail.com> <3a08809b0909150607iece71fodb559392ebf6c95e@mail.gmail.com> <1D184B8E15714A1F8C94B04BB5BAE4FB@Sealion> Message-ID: <3a08809b0909150644x338f90a1teb118459e8569917@mail.gmail.com> Ok now it's correct. thank you very much by the help. 2009/9/15 Michael Kay > You clearly didn't understand David's answer. > > You can no more "ignore" a namespace declaration in your source document > than you can ignore the first character of each element name. The namespace > is an intrinsic part of the names of your elements, and when selecting > elements by name, you need to be aware of it. > > If you want to use unqualified names in your XQuery expression to refer to > names in a namespace, you can do this by means of a default namespace > declaration in the XQuery prolog: > > declare default element namespace "http://www.portalfiscal.inf.br/nfe" > > However, you need to be aware that this affects the interpretation of all > unprefixed names in your query, not only those in a path expression. > > If you don't want to change the source of the query, you can achieve the > same effect using the Saxon API: > > comp.declareNamespace("", "http://www.portalfiscal.inf.br/nfe"); > > Your example query will then work as written. Note, however, that this is a > very different thing from "ignoring" the namespace. > > > Regards, > > Michael Kay > http://www.saxonica.com/ > http://twitter.com/michaelhkay > > ------------------------------ > *From:* talk-bounces@x-query.com [mailto:talk-bounces@x-query.com] *On > Behalf Of *S?rgio Mariano Dias > *Sent:* 15 September 2009 14:07 > *To:* talk@x-query.com > *Subject:* [xquery-talk] Fwd: xmlns ignore > > > Hi, > > This is a complete code test: > > > //--------------------------------------------------------------------------------------------------------------------------------- > > Processor proc = new Processor(false); > > > XQueryCompiler comp = proc.newXQueryCompiler(); > > XdmNode source = proc.newDocumentBuilder().build(new > StreamSource(inputFile)); > > > > XQueryExecutable exp = comp.compile("/NFe/infNFe/ide/natOp"); > XQueryEvaluator eval = exp.load(); > eval.setContextItem(source); > > > > XdmItem i =eval.evaluateSingle(); > System.out.println(i.getStringValue()); > > > XdmValue v = eval.evaluate(); > System.out.println(v.size()); > > > //--------------------------------------------------------------------------------------------------------------------------------- > > that is the xml file: > > > > //--------------------------------------------------------------------------------------------------------------------------------- > > > > > 29 > 123658 > Venda - direta > .... > > > //--------------------------------------------------------------------------------------------------------------------------------- > > The ""http://www.portalfiscal.inf.br/nfe"" return page not found. If I > remove the addres, the program return the correct value. That is way a fink > to ignore the addres. Is that possible? > > Regards, > > > > > 2009/9/11 David Carlisle > > >> >> > I have a problem to ignore a xmls attribute in a tag and did not found >> > information how to ignore this attribute. Somebody can help me. >> >> A namespace declaration is not an attribute in the XDM model used by >> XPath and Xquery. The name of an element includes the namespace. >> >> Most likely the part of teh code that you don't show >> >> = comp.compile("/.../... ..."); >> >> is selecting elements in no-namespace, and you should change it to >> select elements in the namespace used in your xml source. >> >> either by using a prefix >> /p:aa/p:bb >> >> and binding p to the namespace used in the source, or by setting the >> default namespace in the xquery to that used in the source. >> >> >> David >> >> >> ________________________________________________________________________ >> The Numerical Algorithms Group Ltd is a company registered in England >> and Wales with company number 1249803. The registered office is: >> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom. >> >> This e-mail has been scanned for all viruses by Star. The service is >> powered by MessageLabs. >> ________________________________________________________________________ >> > > > > -- > S?rgio Mariano > > > > > > -- > S?rgio Mariano > > > -- S?rgio Mariano Dias http://www.dcc.ufmg.br/~mariano -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090915/1fe901ce/attachment-0001.htm From lists at fgeorges.org Wed Sep 16 12:54:47 2009 From: lists at fgeorges.org (Florent Georges) Date: Wed Sep 16 05:33:26 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <4AA9E2D7.5080504@oracle.com> Message-ID: <922690.48821.qm@web23007.mail.ird.yahoo.com> John Snelson wrote: > Yes - that's what I originally suggested. However the XQuery > working group felt it was important to be able change a function > to be exported / not-exported without changing every place that > function is called. Thanks, John and Martin, for those helpful precisions. BTW, speaking of XQuery 1.1, is there any estimation about when a first draft would be available? Regards, -- Florent Georges http://www.fgeorges.org/ From john.snelson at oracle.com Wed Sep 16 14:55:47 2009 From: john.snelson at oracle.com (John Snelson) Date: Wed Sep 16 05:54:07 2009 Subject: [xquery-talk] Local namespace In-Reply-To: <922690.48821.qm@web23007.mail.ird.yahoo.com> References: <922690.48821.qm@web23007.mail.ird.yahoo.com> Message-ID: <4AB061C3.7020603@oracle.com> Florent Georges wrote: > John Snelson wrote: > >> Yes - that's what I originally suggested. However the XQuery >> working group felt it was important to be able change a function >> to be exported / not-exported without changing every place that >> function is called. > > Thanks, John and Martin, for those helpful precisions. BTW, speaking of XQuery 1.1, is there any estimation about when a first draft would be available? It's available now, although that draft pretty much only has grouping and windowing in it. http://www.w3.org/TR/xquery-11/ The WG is actively working to get a more recent draft out, which should contain a lot of the latest features we've added. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From dlee at calldei.com Fri Sep 18 18:07:21 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 18 13:56:43 2009 Subject: [xquery-talk] Re: [xml-dev] Serialization of XDM - Use cases / Proposal In-Reply-To: <4AAFAF78.6030602@calldei.com> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com> <4AAD34A3.6040402@calldei.com> <4AAF8E48.5050107@allette.com.au> <1B42A885ECC745C7A9CE3345CF55E20D@Sealion> <4AAF9AC9.5070007@allette.com.au> <4AAF9E3C.50407@calldei.com> <4AAFAB5A.4050900@allette.com.au> <4AAFAF78.6030602@calldei.com> Message-ID: <4AB3F689.8000505@calldei.com> I have created a first pass at documenting the problem of XDM Serialization and created some use cases. I would love any feedback or comments. This is on a new wiki I created for this purpose. If you would like to comment directly on the wiki please reply to me and I will give you the invite code (due to the sad state of affairs anonymous comments and editing are disabled due to wiki-spam-bots. I've found spam within 5 minutes of opening a public wiki ... ) http://xml.calldei.com/XDMSerialize I have NOT included a proposal for a format yet, I'd like to discuss the intent and use cases first before putting up a straw-man proposal. Thank you for any contribution ! I've CC'd this to xproc-dev because one of the use cases if for developers and integrators with XML Pipeline processors such as XProc David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 From mike at saxonica.com Sat Sep 19 00:00:53 2009 From: mike at saxonica.com (Michael Kay) Date: Fri Sep 18 14:46:52 2009 Subject: [xquery-talk] Re: [xml-dev] Serialization of XDM - Use cases /Proposal In-Reply-To: <4AB3F689.8000505@calldei.com> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com><4AAD34A3.6040402@calldei.com> <4AAF8E48.5050107@allette.com.au><1B42A885ECC745C7A9CE3345CF55E20D@Sealion><4AAF9AC9.5070007@allette.com.au> <4AAF9E3C.50407@calldei.com><4AAFAB5A.4050900@allette.com.au> <4AAFAF78.6030602@calldei.com> <4AB3F689.8000505@calldei.com> Message-ID: <9915B4B452614D82A6BFF53ABB33B7F2@Sealion> Good start. The next section before you go into specification should perhaps be "Requirements", including assumptions and constraints. This might include statements such as: * Serialized XDM will retain information about the descendants of nodes in the sequence being serialized, but it will not retain information about their ancestors. * Serialized XDM will not retain information about node identity: that is, the recipient of the serialized XDM will not be able to determine whether two serialized elements originated from the same node or merely from two nodes that were deep-equal to each other. * The consumer of the serialized XDM is assumed to have access to the same schema as the producer of the serialized XDM: that is, a QName identifying a type is assumed to have the same meaning to both the producer and consumer. * If A and B are two XDM sequences, then xml-canonicalize(xdm-serialize(A))) is codepoint-equal to xml-canonicalize(xdm-serialize(B))) if and only if fn:deep-equal(A, B). [Actually, that's probably not a good requirement, because fn:deep-equal() discards comments and PIs and you probably don't want to do that]. These are just example statements, you might want to substitute different ones: they are just intended to illustrate the kind of thing that needs to be said. Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay > -----Original Message----- > From: talk-bounces@x-query.com > [mailto:talk-bounces@x-query.com] On Behalf Of David A. Lee > Sent: 18 September 2009 22:07 > Cc: talk@x-query.com; xml-dev@lists.xml.org; XProc Dev > Subject: [xquery-talk] Re: [xml-dev] Serialization of XDM - > Use cases /Proposal > > I have created a first pass at documenting the problem of XDM > Serialization and created some use cases. > I would love any feedback or comments. This is on a new wiki > I created for this purpose. If you would like to comment > directly on the wiki please reply to me and I will give you > the invite code (due to the sad state of affairs anonymous > comments and editing are disabled due to wiki-spam-bots. > I've found spam within 5 minutes of opening a public wiki ... ) > > > http://xml.calldei.com/XDMSerialize > > I have NOT included a proposal for a format yet, I'd like to > discuss the intent and use cases first before putting up a > straw-man proposal. > > Thank you for any contribution ! > > I've CC'd this to xproc-dev because one of the use cases if > for developers and integrators with XML Pipeline processors > such as XProc > > > David A. Lee > dlee@calldei.com > http://www.calldei.com > http://www.xmlsh.org > 812-482-5224 > > > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk From dlee at calldei.com Fri Sep 18 21:06:51 2009 From: dlee at calldei.com (David A. Lee) Date: Fri Sep 18 16:50:44 2009 Subject: [xml-dev] RE: [xquery-talk] Re: [xml-dev] Serialization of XDM - Use cases /Proposal In-Reply-To: <9915B4B452614D82A6BFF53ABB33B7F2@Sealion> References: <6fa681b10908261316q64e8e934ic3615786038f7293@mail.gmail.com><4AAD34A3.6040402@calldei.com> <4AAF8E48.5050107@allette.com.au><1B42A885ECC745C7A9CE3345CF55E20D@Sealion><4AAF9AC9.5070007@allette.com.au> <4AAF9E3C.50407@calldei.com><4AAFAB5A.4050900@allette.com.au> <4AAFAF78.6030602@calldei.com> <4AB3F689.8000505@calldei.com> <9915B4B452614D82A6BFF53ABB33B7F2@Sealion> Message-ID: <4AB4209B.5050704@calldei.com> Thanks Michael. I think my next step will be to document what attributes of XDM should be maintained through serialization and what will be dropped. I agree that Node Identity is not something that can reasonably be maintained. Ancestry is an interesting concept I hadn't considered ... e.g. if you serialize a node $node ... its not expected that a reconstituted XDM $node could evaluate $node/.. Schema equivilence is interesting, I was thinking of addressing (or punting !) that concept with type information. Ignoring (or assuming) common xsd:* types ... If for example a value is of type mytype:abc ... I think agree its a good presumption that the target environment has access to the same types. Otherwise we open the box of having to serialize all the type information along with the XDM values. Good call. David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org 812-482-5224 Michael Kay wrote: > Good start. > > The next section before you go into specification should perhaps be > "Requirements", including assumptions and constraints. This might include > statements such as: > > * Serialized XDM will retain information about the descendants of nodes in > the sequence being serialized, but it will not retain information about > their ancestors. > > * Serialized XDM will not retain information about node identity: that is, > the recipient of the serialized XDM will not be able to determine whether > two serialized elements originated from the same node or merely from two > nodes that were deep-equal to each other. > > * The consumer of the serialized XDM is assumed to have access to the same > schema as the producer of the serialized XDM: that is, a QName identifying a > type is assumed to have the same meaning to both the producer and consumer. > > * If A and B are two XDM sequences, then xml-canonicalize(xdm-serialize(A))) > is codepoint-equal to xml-canonicalize(xdm-serialize(B))) if and only if > fn:deep-equal(A, B). [Actually, that's probably not a good requirement, > because fn:deep-equal() discards comments and PIs and you probably don't > want to do that]. > > These are just example statements, you might want to substitute different > ones: they are just intended to illustrate the kind of thing that needs to > be said. > > Regards, > > Michael Kay > http://www.saxonica.com/ > http://twitter.com/michaelhkay > > >> -----Original Message----- >> From: talk-bounces@x-query.com >> [mailto:talk-bounces@x-query.com] On Behalf Of David A. Lee >> Sent: 18 September 2009 22:07 >> Cc: talk@x-query.com; xml-dev@lists.xml.org; XProc Dev >> Subject: [xquery-talk] Re: [xml-dev] Serialization of XDM - >> Use cases /Proposal >> >> I have created a first pass at documenting the problem of XDM >> Serialization and created some use cases. >> I would love any feedback or comments. This is on a new wiki >> I created for this purpose. If you would like to comment >> directly on the wiki please reply to me and I will give you >> the invite code (due to the sad state of affairs anonymous >> comments and editing are disabled due to wiki-spam-bots. >> I've found spam within 5 minutes of opening a public wiki ... ) >> >> >> http://xml.calldei.com/XDMSerialize >> >> I have NOT included a proposal for a format yet, I'd like to >> discuss the intent and use cases first before putting up a >> straw-man proposal. >> >> Thank you for any contribution ! >> >> I've CC'd this to xproc-dev because one of the use cases if >> for developers and integrators with XML Pipeline processors >> such as XProc >> >> >> David A. Lee >> dlee@calldei.com >> http://www.calldei.com >> http://www.xmlsh.org >> 812-482-5224 >> >> >> >> _______________________________________________ >> talk@x-query.com >> http://x-query.com/mailman/listinfo/talk >> > > > _______________________________________________________________________ > > XML-DEV is a publicly archived, unmoderated list hosted by OASIS > to support XML implementation and development. To minimize > spam in the archives, you must subscribe before posting. > > [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/ > Or unsubscribe: xml-dev-unsubscribe@lists.xml.org > subscribe: xml-dev-subscribe@lists.xml.org > List archive: http://lists.xml.org/archives/xml-dev/ > List Guidelines: http://www.oasis-open.org/maillists/guidelines.php > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090918/f2f7a03b/attachment.htm From christine.schwartz at ptsem.edu Thu Sep 24 12:08:55 2009 From: christine.schwartz at ptsem.edu (Schwartz, Christine) Date: Thu Sep 24 08:06:28 2009 Subject: [xquery-talk] Can FLWOR be FLOWR? Message-ID: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-eden.org> Hi, I'm helping teach some basic XQuery and was asked this question: In a FLWOR expression, does the where clause have to come before an order by clause? Seems like it should, but can't seem to find this written down anywhere. Thanks, Chris Christine Schwartz Metadata Librarian Princeton Theological Seminary Libraries christine.schwartz@ptsem.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090924/84cc4959/attachment.htm From gkholman at CraneSoftwrights.com Thu Sep 24 12:18:33 2009 From: gkholman at CraneSoftwrights.com (G. Ken Holman) Date: Thu Sep 24 08:18:16 2009 Subject: [xquery-talk] Can FLWOR be FLOWR? In-Reply-To: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-e den.org> References: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-eden.org> Message-ID: <7.0.1.0.2.20090924111802.027257e8@wheresmymailserver.com> At 2009-09-24 11:08 -0400, Schwartz, Christine wrote: >I'm helping teach some basic XQuery and was asked this question: In >a FLWOR expression, does the where clause have to come before an >order by clause? Yes, because it qualifies the set that is to be ordered. >Seems like it should, but can't seem to find this written down anywhere. Production [33] indicates that the where clause precedes the order by clause: http://www.w3.org/TR/2007/REC-xquery-20070123/#id-flwor-expressions I hope this helps. . . . . . . Ken -- Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes. Interested in other classes? http://www.CraneSoftwrights.com/q/i/ Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video Video lesson: http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18 Video overview: http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18 G. Ken Holman mailto:gkholman@CraneSoftwrights.com Male Cancer Awareness Nov'07 http://www.CraneSoftwrights.com/q/bc Legal business disclaimers: http://www.CraneSoftwrights.com/legal From john.snelson at oracle.com Thu Sep 24 17:39:11 2009 From: john.snelson at oracle.com (John Snelson) Date: Thu Sep 24 08:35:46 2009 Subject: [xquery-talk] Can FLWOR be FLOWR? In-Reply-To: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-eden.org> References: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-eden.org> Message-ID: <4ABB929F.2070006@oracle.com> Schwartz, Christine wrote: > I?m helping teach some basic XQuery and was asked this question: In a > FLWOR expression, does the where clause have to come before an order by > clause? > > Seems like it should, but can?t seem to find this written down anywhere. The order of the clauses is fixed for XQuery 1.0, with "order by" coming after "where". In XQuery 1.1 this restriction will be relaxed, allowing clauses in almost any order (you have to start with "for", "let" etc.) and multiple "order by" or "where" clauses. John -- John Snelson, Oracle Corporation http://snelson.org.uk/john Berkeley DB XML: http://oracle.com/database/berkeley-db/xml XQilla: http://xqilla.sourceforge.net From christine.schwartz at ptsem.edu Thu Sep 24 17:27:16 2009 From: christine.schwartz at ptsem.edu (Schwartz, Christine) Date: Thu Sep 24 13:16:26 2009 Subject: [xquery-talk] Can FLWOR be FLOWR? In-Reply-To: <4ABB929F.2070006@oracle.com> References: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8546@eden-mail01.pts-eden.org> <4ABB929F.2070006@oracle.com> Message-ID: <590A4530D9C55C4F9E8CE8E5E5BD3FA3052E8662@eden-mail01.pts-eden.org> Ken and John, Thanks for your help with this. It's interesting to know that XQuery 1.1 will be changing this required order. Regards, Chris > Snelson, John wrote: > > Schwartz, Christine wrote: > > I?m helping teach some basic XQuery and was asked this question: In a > > FLWOR expression, does the where clause have to come before an order > by > > clause? > > > > Seems like it should, but can?t seem to find this written down > anywhere. > > The order of the clauses is fixed for XQuery 1.0, with "order by" > coming > after "where". > > In XQuery 1.1 this restriction will be relaxed, allowing clauses in > almost any order (you have to start with "for", "let" etc.) and > multiple > "order by" or "where" clauses. > > John > From andrew.j.welch at gmail.com Mon Sep 28 10:24:02 2009 From: andrew.j.welch at gmail.com (Andrew Welch) Date: Mon Sep 28 00:49:29 2009 Subject: [xquery-talk] Re: Combining XSLT 2.0's Grouping with eXist In-Reply-To: <10dbf0540909260919p2ce38d8ewfe95081d49e751b@mail.gmail.com> References: <10dbf0540909260919p2ce38d8ewfe95081d49e751b@mail.gmail.com> Message-ID: <74a894af0909280124h307d774v6a60908ce2476c42@mail.gmail.com> Hi Bill, The best thing to do is sign up to one of the mailing lists: "xquery talk" http://x-query.com/mailman/listinfo/talk "xml dev" http://www.xml.org/xml-dev/ Ask your question there, hopefully get a helpful response, and then the email chain gets archived so its useful for anyone encountering a similar problem in the future. The short answer is yes - most xquery engines provide a way of transforming a the result of a query through a proprietary extension, the one for eXist is documented here: http://exist.sourceforge.net/xquery.html#N10375 cheers andrew 2009/9/26 Bill Girten : > Andrew, > > Thanks for the piece you did on this area. ?I'm looking for a little > clarity, however. > > I'm an IT consultant that's worked with BEA's Aqualogic Data Services > XQuery engine in the past. ?I've been playing with eXist 1.26 and now > the 1.4RC. ?Here's my question: > > Is it worth looking into the Sitemap stuff (missed that revolution > years ago) since Wolgang Meir is deprecating a lot of the Cocoon gear? > ?Although I love Java, I want to get away from XSL transformation via > calling on the process via a Java class. ?Is there a way to seamlessly > transform the eXist results set without the need of running the XML > through Java to get XHTML? > > Sorry if I sound like a newbie. > > Thanks, > > Bill Girten > -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/ From adam.retter at googlemail.com Mon Sep 28 12:21:10 2009 From: adam.retter at googlemail.com (Adam Retter) Date: Mon Sep 28 02:44:55 2009 Subject: [xquery-talk] Re: Combining XSLT 2.0's Grouping with eXist In-Reply-To: <74a894af0909280124h307d774v6a60908ce2476c42@mail.gmail.com> References: <10dbf0540909260919p2ce38d8ewfe95081d49e751b@mail.gmail.com> <74a894af0909280124h307d774v6a60908ce2476c42@mail.gmail.com> Message-ID: Bill, Yes eXist provides mechanisms for transforming directly using XSLT without the need for Java - 1) XQuery extension function transform:transform(...) which Andrew already mentioned. 2) Some sort of custom pipeline using our Controller XQuery approach 3) XProc pipeline using our XProcXQ 4) XSL-PI Probably best if you join the eXist mailing list for such questions. 2009/9/28 Andrew Welch : > Hi Bill, > > The best thing to do is sign up to one of the mailing lists: > > "xquery talk" http://x-query.com/mailman/listinfo/talk > > "xml dev" http://www.xml.org/xml-dev/ > > Ask your question there, hopefully get a helpful response, and then > the email chain gets archived so its useful for anyone encountering a > similar problem in the future. > > The short answer is yes - most xquery engines provide a way of > transforming a the result of a query through a proprietary extension, > the one for eXist is documented here: > > http://exist.sourceforge.net/xquery.html#N10375 > > cheers > andrew > > 2009/9/26 Bill Girten : >> Andrew, >> >> Thanks for the piece you did on this area. ?I'm looking for a little >> clarity, however. >> >> I'm an IT consultant that's worked with BEA's Aqualogic Data Services >> XQuery engine in the past. ?I've been playing with eXist 1.26 and now >> the 1.4RC. ?Here's my question: >> >> Is it worth looking into the Sitemap stuff (missed that revolution >> years ago) since Wolgang Meir is deprecating a lot of the Cocoon gear? >> ?Although I love Java, I want to get away from XSL transformation via >> calling on the process via a Java class. ?Is there a way to seamlessly >> transform the eXist results set without the need of running the XML >> through Java to get XHTML? >> >> Sorry if I sound like a newbie. >> >> Thanks, >> >> Bill Girten >> > > > > -- > Andrew Welch > http://andrewjwelch.com > Kernow: http://kernowforsaxon.sf.net/ > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > -- Adam Retter skype :adam.retter http://www.adamretter.org.uk From jesusrodriguesv at gmail.com Tue Sep 29 14:14:31 2009 From: jesusrodriguesv at gmail.com (Jesus Rodrigues) Date: Tue Sep 29 09:10:54 2009 Subject: [xquery-talk] UTF-8 Message-ID: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> Dear, I needed read a lot of xml and some of this file haven't the attribute encoding=*"UTF-8"*. And some xml has too a problem with utf-8 (invalid character like this GA??S). I haven't control about the xml I just needed read. Is possible solution this problem using the saxon? For example, setting the encoding dynamically, before the build (newDocumentBuilder().build(stream))? Regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090929/5c8b6218/attachment.htm From vyacheslav.sedov at gmail.com Tue Sep 29 21:23:12 2009 From: vyacheslav.sedov at gmail.com (Vyacheslav Sedov) Date: Tue Sep 29 09:19:27 2009 Subject: [xquery-talk] UTF-8 In-Reply-To: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> References: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> Message-ID: <7ab7cbac0909290923w5e73ee53lf1bd81d979fd9ca8@mail.gmail.com> "UTF-8" is default value for encoding attribute in xml prolog On Tue, Sep 29, 2009 at 8:14 PM, Jesus Rodrigues wrote: > Dear, > > I needed read a lot of xml and some of this file haven't the attribute > encoding="UTF-8". And some xml has too a problem with utf-8 (invalid > character like this GA??S). I haven't control about the xml I just needed > read. > > Is possible solution this problem using the saxon? For example, setting the > encoding dynamically, before the build (newDocumentBuilder().build(stream))? > > Regards, > > _______________________________________________ > talk@x-query.com > http://x-query.com/mailman/listinfo/talk > From jesusrodriguesv at gmail.com Tue Sep 29 14:35:50 2009 From: jesusrodriguesv at gmail.com (Jesus Rodrigues) Date: Tue Sep 29 09:31:55 2009 Subject: [xquery-talk] UTF-8 In-Reply-To: <7ab7cbac0909290923w5e73ee53lf1bd81d979fd9ca8@mail.gmail.com> References: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> <7ab7cbac0909290923w5e73ee53lf1bd81d979fd9ca8@mail.gmail.com> Message-ID: <77e589830909290935o5c5f0480mcf32f1ebf50a1df4@mail.gmail.com> Dear Vyacheslav Sedov However, sometime the file that I'm try to read wasn't a UFT-8. is it possible set the encoding dynamically? Or it's necessary open the file and convert to UFT-8? Regards 2009/9/29 Vyacheslav Sedov > "UTF-8" is default value for encoding attribute in xml prolog > > On Tue, Sep 29, 2009 at 8:14 PM, Jesus Rodrigues > wrote: > > Dear, > > > > I needed read a lot of xml and some of this file haven't the attribute > > encoding="UTF-8". And some xml has too a problem with utf-8 (invalid > > character like this GA??S). I haven't control about the xml I just needed > > read. > > > > Is possible solution this problem using the saxon? For example, setting > the > > encoding dynamically, before the build > (newDocumentBuilder().build(stream))? > > > > Regards, > > > > _______________________________________________ > > talk@x-query.com > > http://x-query.com/mailman/listinfo/talk > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090929/155eccb0/attachment.htm From mike at saxonica.com Tue Sep 29 18:48:42 2009 From: mike at saxonica.com (Michael Kay) Date: Tue Sep 29 09:44:49 2009 Subject: [xquery-talk] UTF-8 In-Reply-To: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> References: <77e589830909290914n2a9c143ane476fddf6858f05e@mail.gmail.com> Message-ID: <6F8FDA16FE1F4B72AB14533680F01606@Sealion> If the XML document has an encoding declaration () then you must ensure that the file is actually encoded using this encoding. If the XML document has no encoding declaration, then it should typically be encoded in UTF-8 (though there are other ways of indicating the encoding, for example using byte-order-marks, or in an HTTP header). The most common cause of encoding problems is when you use an encoding such as iso-8859-1 for your document, but have no encoding declaration on the XML document. If you know the encoding of your document, you can always decode it into a character stream yourself, and then pass the character stream to the XML parser (e.g. in Java in the form of a Reader). Regards, Michael Kay http://www.saxonica.com/ http://twitter.com/michaelhkay _____ From: talk-bounces@x-query.com [mailto:talk-bounces@x-query.com] On Behalf Of Jesus Rodrigues Sent: 29 September 2009 17:15 To: talk@x-query.com Subject: [xquery-talk] UTF-8 Dear, I needed read a lot of xml and some of this file haven't the attribute encoding="UTF-8". And some xml has too a problem with utf-8 (invalid character like this GA??S). I haven't control about the xml I just needed read. Is possible solution this problem using the saxon? For example, setting the encoding dynamically, before the build (newDocumentBuilder().build(stream))? Regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090929/160019a2/attachment.htm From anilshekhar at hotmail.com Wed Sep 30 19:13:51 2009 From: anilshekhar at hotmail.com (Anil Shekhar) Date: Wed Sep 30 18:03:51 2009 Subject: [xquery-talk] Unexpected token syntax error, unexpected DeclareFunction_ Message-ID: Dear Experts, I am new to XQuery and I need your help. I am getting the following error when accessed ... http://127.0.0.1:8010/shakespeare.xqy 500 Internal Server Error XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, unexpected DeclareFunction_ in /shakespeare.xqy, on line 8 [1.0-ml] Line 8 is where I have defined the function. Thanks Anil Shakespeare ...

{ declare function local:handle-doc( $doc as document-node(), $speeches as element(SPEECH)+ ) {

{ document-uri($doc) }

, for $speech in $speeches let $speaker := $speech/SPEAKER let $lines := $speech/LINE return (

{ $speaker/text() }

,

{ $lines/text() }

) }; for $d in doc("file:///C:/Users/ashekhar/Desktop/Mark Logic/Shakespeare/*.xml") order by $d ascending return local:handle-doc($d, $d//SPEECH) }

) _________________________________________________________________ Insert movie times and more without leaving Hotmail?. http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://x-query.com/pipermail/talk/attachments/20090930/246e6c1b/attachment.htm From gkholman at CraneSoftwrights.com Wed Sep 30 22:53:30 2009 From: gkholman at CraneSoftwrights.com (G. Ken Holman) Date: Wed Sep 30 18:40:41 2009 Subject: [xquery-talk] Unexpected token syntax error, unexpected DeclareFunction_ In-Reply-To: References: Message-ID: <7.0.1.0.2.20090930214934.02593dc8@wheresmymailserver.com> At 2009-09-30 18:13 -0700, Anil Shekhar wrote: >XDMP-UNEXPECTED: (err:XPST0003) Unexpected token syntax error, >unexpected DeclareFunction_ The message says that your declaration of a function is unexpected. >in /shakespeare.xqy, on line 8 [1.0-ml]Line 8 is where I have >defined the function. Sure enough, that isn't an expected place for a function definition. Reviewing: http://www.w3.org/TR/xquery/#id-function-calls >3.1.5 Function Calls >[Definition: The built-in functions supported by XQuery are defined >in [XQuery 1.0 and XPath 2.0 Functions and Operators].] Additional >functions may be declared in a Prolog, imported from a library >module, or provided by the external environment as part of the static context. And that is pretty explicit that functions can only be declared in the prolog. You have: > > > Shakespeare ... > > >

> { > declare function local:handle-doc( > $doc as document-node(), > $speeches as element(SPEECH)+ > ) .... which doesn't have the declaration in the prolog, but in the middle of the query. So I think the error message is quite justified. I hope this helps. . . . . . . . . . . Ken -- Upcoming hands-on code list, UBL, XSLT, XQuery and XSL-FO classes. Interested in other classes? http://www.CraneSoftwrights.com/q/i/ Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ Training tools: Comprehensive interactive XSLT/XPath 1.0/2.0 video Video lesson: http://www.youtube.com/watch?v=PrNjJCh7Ppg&fmt=18 Video overview: http://www.youtube.com/watch?v=VTiodiij6gE&fmt=18 G. Ken Holman mailto:gkholman@CraneSoftwrights.com Male Cancer Awareness Nov'07 http://www.CraneSoftwrights.com/q/bc Legal business disclaimers: http://www.CraneSoftwrights.com/legal