<div dir="ltr">Below is an blog post from Norm Matloff the author of the Art of R - a statistician that lives in a CS department at UC Irvine, the book I referred to in another post, <div><br></div><div><a href="http://blog.revolutionanalytics.com/2014/08/statistics-losing-ground-to-cs-losing-image-among-students.html" style="font-size:12.8000001907349px" target="_blank">http://blog.revolutionanalytics.com/2014/08/statistics-losing-ground-to-cs-losing-image-among-students.html</a><br></div><div><br></div><div>The article is interesting for a number of reasons not the least the parallel of it's core theme, the image problem of a discipline that is perceived to be unfashionable. The reference to a CS usurpation problem is ironic because the reverse argument could also be made - really bad CS (e.g data management) being entrenched as standard by statisticians (and the like) in the name of data science - just goes to show that there are probably 2 very valid sides to that coin.</div><div><br></div><div>Enough preamble, I am wondering if the perusal of some of the comments reveal an opportunity. </div><div><br></div><div>Tom 26/8/14 @ 23.25 surmising that CS students are turned off of Stats classes because of the use of R which "<span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px"><i>as a programming language it is horrible and needs to die in a fire" </i>I was hoping to see a reasoned rebuttal of a viewpoint I share but Matloff really didn't deal with it well.</span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px"><br></span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px">There is another comment by Jaipelai 27/8/14 @ 09:42 that I could almost have written myself. </span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px"><br></span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px">Point being Stonebraker identifies that people will want to do analytics with their query languages but all the analytics tools suck at data management. That market is all the rage now, but the R and Python communities are probably lost causes. </span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px"><br></span></div><div><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px">So rather than bring analytics to a query language, suppose instead one looked at baking in a best of breed query capability to a an analytics language that was fashionable, functional and comprehension friendly - </span><span style="color:rgb(51,51,51);font-family:'lucida grande',helvetica,arial,sans-serif;font-size:12px">Julia. </span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 23, 2015 at 12:52 PM, daniela florescu <span dir="ltr"><<a href="mailto:dflorescu@me.com" target="_blank">dflorescu@me.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><span class=""><blockquote type="cite"><div>On Jun 23, 2015, at 9:14 AM, Ihe Onwuka <<a href="mailto:ihe.onwuka@gmail.com" target="_blank">ihe.onwuka@gmail.com</a>> wrote:</div><br><div><div dir="ltr"><div>Well he didn't comment on SQL for JSON per se but saying that RDBMS are sub-optimal for everything is a tacit repudiation of SQL is it not?</div></div></div></blockquote><div><br></div></span>No, because he said exploitively that the *internals* of a database will be different (columnar, main memory, streaming, etc)….. the</div><div>programming language will STILL be SQL. Or at least for all those databases for whom the data model is STILL relational. </div><span class=""><div><br></div><div><blockquote type="cite"><div><div dir="ltr"><div><br></div><div>He buys into the notion that there will be swarms of data scientists doing clever things with data which will need a different language.</div></div></div></blockquote><div><br></div></div></span><div>Yes. SQL clearly doesn’t solve the R use cases. So yes, R is on the “acceptable OTHER languages” list.</div><div><br></div><div>But that’s not clear that what we (aka the XML community see) as “normal” data processing use cases will be considered necessary use cases</div><div>for the JSON/NoSQL community.</div><div><br></div><div>E.g. scanning the data and automatically extracting a schema. Is this an acceptable use case for JSON ? Or not ?</div><div><br></div><div>If yes, then XQuery has a chance, because XQuery can do that and SQL cannot. </div><div><br></div><div>If no, people will stick to what they know : SQL.</div><div><br></div><div><span class=""><blockquote type="cite"><div><div dir="ltr"><div><br></div><div>He is right that statistical packages suck at data management but that won't isn't going to deter the R community. </div></div></div></blockquote><div><br></div></span><div>Yes, the R implementations (I looked at them in details about 2 years ago) have NO IDEA about how to deal with large volumes </div><div>of data, so probably a mix between data technologies and database technologies is necessary.</div><div><br></div><div>However, don’t underestimate companies like Oracle. They are not dummies, and the know what the market wants.</div><div>R is supported natively inside the Oracle database now.</div><div><br></div><div>I think that Stonebreaker exaggerates when he says that relational databases will disappear in 10 years. Well… I don’t think</div><div>this will happen so quickly.</div><span class=""><div><br></div><blockquote type="cite"><div><div dir="ltr"><div><br></div><div>Do you see XQuery fitting anywhere in this vision. It has potential as a pipeling technology as does for that matter SQL. I think it will always be problematic to do analytics on the source data because it is too dirty.</div></div></div></blockquote><div><br></div></span><div>XQuery COULD be a very good “glue” language between data in various formats (CSV, Excel, PDF, HTML, XML, JSON, relational, whatever).</div><div><br></div><div>But I say “COULD” not “CAN”.</div><div><br></div><div>It needs many extensions to be good at that: scripting, support for JSON, modules to support a variety of data formats and data processing services.</div><div><br></div><div><br></div><div>Best regards</div><div>Dana</div><br><div><br></div><div><br></div><div>P.S. </div><div><br></div><div><div><blockquote type="cite"><div dir="ltr"><div>I am continually surprised that people this smart believe that there is such a pool of data scientists people to draw from.<br></div></div></blockquote><div><br></div>Me too. I fell down from my chair when I saw the article saying that US needs 5 million data scientists in the next 2 years, aka, about 5% of the</div><div>US working population. Not sure if this for laughing, or for crying. </div><div><br></div><div>[[ aka, we will not have cashiers at Safeway anymore ‘cause they are all data scientists….]]</div><div><br></div><div>Someone up there doing the math in this article doesn’t understand jack nothing about numbers and statistics …….</div><div><br></div><div>And all this while:</div><div><a href="http://www.nature.com/news/irreproducible-biology-research-costs-put-at-28-billion-per-year-1.17711?utm_content=buffer95bfb&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer" target="_blank">http://www.nature.com/news/irreproducible-biology-research-costs-put-at-28-billion-per-year-1.17711?utm_content=buffer95bfb&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer</a></div><div><br></div><div>God knows how many medicines are wrongly given to sick people, because nobody knows how to do a proper case study … </div><div>REALLY scary … but that’s another discussion.</div><div><br></div></div><div>Again the same discussion comes up: DON”T look for 5 million data scientists. Just do with a smaller number of smart ones, but GIVE</div><div>THEM BETTER TOOLS and AUTOMATIZE THE PROCESS.</div><div><br></div><div>But hey, how can you stop such a wold wide enthusiasm for “data scientists” !?? Logic doesn’t do it….</div><div><div class="h5"><div><br></div><div><br></div><br><blockquote type="cite"><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 23, 2015 at 11:51 AM, daniela florescu <span dir="ltr"><<a href="mailto:dflorescu@me.com" target="_blank">dflorescu@me.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Ihe,<div><br></div><div><br></div><div>I had discussions with Michael Stonebreaker for 20 years about about the fact that</div><div>XML “exists” or not. With Jim Gray too, before he disappeared. They were both extremely</div><div>supportive for me, yet were both thinking that I am crazy to waste my research career on XML.</div><div><br></div><div>Stonebreaker’s opinion: he doesn’t believe that XML “exists” in industry.</div><div><br></div><div>So he will not mention it, because it doesn’t exist :-)</div><div><br></div><div>But you have to remember that Stonebreaker is a database person. Probably he will not</div><div>understand the facet of XML which is “XML as documents”. It took me and the other database </div><div>people involved in XQuery years before we swallowed it. (Don Chamberlin of SQL fame</div><div>famously once said “who in the world would care about such a corner case as mixed content !?").</div><div><br></div><div>Don’t blame the database people that they don’t “get” XML. On one hand, it has never been explained</div><div>to them properly.</div><div><br></div><div>And again, Stonebreaker, being a database person, he will look at “XML as data” aspect of the story.</div><div>And this today is INDEED non-existing in industry, or almost. Or, when t is, it is mostly for log analysis.</div><div><br></div><div>============</div><div><br></div><div>JSON will completely change the landscape, in surprising ways, that none of us can predict.</div><div><br></div><div>And no, I trust that Michael Stonebreaker is too smart to believe that SQL is a solution to process JSON.</div><div><br></div><div>But time will tell.</div><div><br></div><div>Best regards</div><div>Dana</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br><div><blockquote type="cite"><span><div>On Jun 23, 2015, at 12:15 AM, Ihe Onwuka <<a href="mailto:ihe.onwuka@gmail.com" target="_blank">ihe.onwuka@gmail.com</a>> wrote:</div><br></span><div><span><div dir="ltr"><a href="https://www.youtube.com/watch?v=9K0SWs1mOD0" target="_blank">https://www.youtube.com/watch?v=9K0SWs1mOD0</a><br><div><br></div><div>By implication it puts the kibosh on SQL as the basis of a solution for the future.</div></div></span>
_______________________________________________<br><a href="mailto:talk@x-query.com" target="_blank">talk@x-query.com</a><br><a href="http://x-query.com/mailman/listinfo/talk" target="_blank">http://x-query.com/mailman/listinfo/talk</a></div></blockquote></div><br></div></div></blockquote></div><br></div>
</div></blockquote></div></div></div><br></div></blockquote></div><br></div>