[xquery-talk] specification for Microsoft's U-SQL ?

[Michael Rys]

Dear Dana

You have the ability to either flow JSON or XML as (currently) byte arrays or strings and use C# libraries such as NewtonSoft, System.XML or your own ☺ to process them. Or you can use an XML or JSON extractor (we currently have one of each published as samples on the U-SQL github repo), to extract parts of a JSON or XML document and map parts onto rows (some columns in turns can be maps or arrays).

So it is not taking the route of XQuery or JSONiq in terms of processing arbitrary nested trees without the need of transformations (yet).

To answer your questions inline:

U-SQL is a query language more than a scripting language in that it does not provide snapshots or states inside a U-SQL “script”. However, the syntax of step-wise refinement and composition is closer to a scripting language experience than for example the equivalent common-table-expressions that you normally see in ANSI SQL dialects.

And yes you are right regarding the composition. I would recommend the following blog post to give you some more background about the language philosophy: http://blogs.msdn.com/b/visualstudio/archive/2015/09/28/introducing-u-sql.aspx. It is and evolution of SCOPE (see link in blog post).

Cheers
Michael



From: daniela florescu [mailto:dflorescu at me.com]
Sent: Wednesday, November 4, 2015 12:39 AM
To: Michael Rys <mrys at microsoft.com>
Cc: talk <talk at x-query.com>
Subject: Re: [xquery-talk] specification for Microsoft's U-SQL ?
Importance: High

Dear Michael,

I still have a question: can we use U-SQL to process JSON or XML?

I could’t see a way of doing this — as they are not flat rows…

So is it possible ?

Thanks, regards
Dana


On Nov 3, 2015, at 10:23 PM, daniela florescu <dflorescu at me.com<mailto:dflorescu at me.com>> wrote:

Dear Michael,

Thanks for the pointers.

I was looking at https://azure.microsoft.com/en-us/documentation/articles/data-lake-analytics-u-sql-get-started/<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fazure.microsoft.com%2fen-us%2fdocumentation%2farticles%2fdata-lake-analytics-u-sql-get-started%2f&data=01%7c01%7cmrys%40microsoft.com%7ca3ac8bc2c0994a17685008d2e4f374fe%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=XEqoDPUPCck9Jhw5nz1ZVeQMsCLYX3X4iu8t92z%2fLYQ%3d>
and trying to understand the big picture from examples.

So U-SQL is a dataflow scripting language, where the basic data model is a set of rows with C# types, and each
step in the data flow is an expression that can mix (at many levels) SQL’s Select-from-where constructs with C# code.

That’s what I understand at least. Let me know if I got it wrong.

Seems very useful for expressing complex data transformations of all kinds, and it is still optimizable.

Much more useful for “data science” (whatever this big word means..) then the series of SQL-wannabes languages
 from the NoSQL vendors.

Thanks, best regards
Dana



On Nov 2, 2015, at 4:56 PM, Michael Rys <mrys at microsoft.com<mailto:mrys at microsoft.com>> wrote:

No formal specification. Just a language reference that is still under development :). See http://aka.ms/usql_reference<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2faka.ms%2fusql_reference&data=01%7c01%7cmrys%40microsoft.com%7ca3ac8bc2c0994a17685008d2e4f374fe%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=2XNRKam46acNQGL4gcPr2JEnrmsJUPVB7eigzd61XMQ%3d>  (and pardon the format, I am writing it and our doc people "convert" it).

Let me know if you want a personal demo :).

Cheers
Michael

-----Original Message-----
From: talk-bounces at x-query.com<mailto:talk-bounces at x-query.com> [mailto:talk-bounces at x-query.com] On Behalf Of daniela florescu
Sent: Saturday, October 31, 2015 2:05 PM
To: talk <talk at x-query.com<mailto:talk at x-query.com>>
Subject: [xquery-talk] specification for Microsoft's U-SQL ?

Seems interesting in principle.
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fthenewstack.io%2fmeet-u-sql-microsofts-new-language-big-data%2f&data=01%7c01%7cmrys%40microsoft.com%7caef4ceeec92543784b5608d2e23771f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iZjmF47tPZ316GAyjeXegMSLOq%2bLFAbo3zvRYOwMPy0%3d

But anyone in a luck to see a specification for it ?

Thanks for any pointer, best regards
Dana
_______________________________________________
talk at x-query.com<mailto:talk at x-query.com>
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fx-query.com%2fmailman%2flistinfo%2ftalk&data=01%7c01%7cmrys%40microsoft.com%7caef4ceeec92543784b5608d2e23771f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=dJTn2RgYiSUgZH5GU3e3CsLqpCj9Gvd6uI%2fGS7JomP8%3d

_______________________________________________
talk at x-query.com<mailto:talk at x-query.com>
http://x-query.com/mailman/listinfo/talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://x-query.com/pipermail/talk/attachments/20151104/490cdbf3/attachment.html>