[xquery-talk] Multiple output via a stream of filters

[Ihe Onwuka]

I only read the ginormous  XML once... I apply the 7 filters to each
node read and it gets allocated to one of the 7 output buckets (hows
that for a semantically neutral term).

On Tue, Jan 14, 2014 at 3:50 AM, Ihe Onwuka <ihe.onwuka at gmail.com> wrote:
> Input .... ginormous xml document containing data relating to a credit
> card application that I only want to read once.
>
> Filters. - under 25's, college educated, single mother, bankrupts, 6
> figure income, repeat applicants, self-employed.
>
> 1 input - 7 outputs
>
> All applicants that meet the criteria of a given filter are copied to
> the output corresponding to that filter.
>
> We could camouflage the 7 outputs into one by wrapping an element
> around the whole 7.
>
> On Tue, Jan 14, 2014 at 3:42 AM, David Lee <dlee at calldei.com> wrote:
>> A Single XQuery invocation has only one "output stream" ... but when your working in a database often you don't need any, you want to put the data back into the database.
>> Functions don't have "streams" really either although the word is overloaded.
>> One way it to produce a "sequence" of results which you then do something with, either output in the main output or store into the database (as separate documents).
>>
>> Also I believe eXist has functions for writing to files so your question my have more relevance on the exists mailing list.
>>
>> But if what I think you mean by "stream" is a long result ... and your dealing with GB+ data you need to be very careful,
>> True "streaming" is very hard to achieve in XQuery.  Functions don't "stream" they return values.
>> XPath doesn't "stream" ... there is no real "streams" in XQuery ... so if your not careful you end up with having to hold the entire result set in memory.
>> But if your output is to the 'main' output (is this an HTTP call ?) you might be able to stream to it .. all depends on the vendor implementation.
>>
>> I suspect what your really after is to write multiple results to either documents or files.
>> XQuery doesn't support this generically but XML Databases like eXist do so you might want to ask there.
>>
>> Or I may have completely misunderstood your question.
>>
>>
>>
>> ----------------------------------------
>> David A. Lee
>> dlee at calldei.com
>> http://www.xmlsh.org
>>
>> -----Original Message-----
>> From: Ihe Onwuka [mailto:ihe.onwuka at gmail.com]
>> Sent: Monday, January 13, 2014 4:56 PM
>> To: David Lee
>> Cc: talk at x-query.com
>> Subject: Re: [xquery-talk] Multiple output via a stream of filters
>>
>> If it helps think of an entity that entails a bazillion fields like a credit card application and you have multiple ways you wish to classify it. So you have a filter for each classification scheme and the application "filters" through to the output stream of whichever predicates return true.
>>
>>
>>
>> On Tue, Jan 14, 2014 at 12:38 AM, David Lee <dlee at calldei.com> wrote:
>>> Could you define what you mean by "output" ?
>>>
>>> ----------------------------------------
>>> David A. Lee
>>> dlee at calldei.com
>>> http://www.xmlsh.org
>>>
>>>
>>> -----Original Message-----
>>> From: Ihe Onwuka [mailto:ihe.onwuka at gmail.com]
>>> Sent: Monday, January 13, 2014 4:19 PM
>>> To: David Lee
>>> Cc: talk at x-query.com
>>> Subject: Re: [xquery-talk] Multiple output via a stream of filters
>>>
>>> On Mon, Jan 13, 2014 at 11:54 PM, David Lee <dlee at calldei.com> wrote:
>>>> If your running in exist then pure XQuery is probably as good or better then anything else.
>>>> Could you expand on your problem ?
>>>>
>>>
>>> I have a collection. Suppose I have 10 different outputs I want and the decision as to which output a node is routed to (not necessarily mutual exclusive though) can be encapsulated in a predicate (hence amenable to a filter HOF).
>>>
>>> I only want to iterate over the collection once but I wish to apply the 10 predicates to each node in the sequence so as to determine which of the 10 outputs (again not necessarily mutually exclusive) the node will feature in.
>>>
>>> The algorithm is straightforward using structural recursion (or tail recursion if tail call elimination is an issue). Asking whether there is an idiomatic XQuery solution that is different  from this.