<div dir="ltr">Hmmm I wonder whether this would have worked on the scraped ratings data that I had to clean. Well I did that with XPath in XSLT, might take a look and see.<div></div><div><br></div><div>I have 3 different movie data sets from different sources. The one I just posted containing 3m movies, 1 has 180k movies (created by with JSONiq running against freebase) and the other about 50k movies , all of which I have managed to cast in XML. </div><div><br></div><div>So there is plenty of data to experiment with.</div><div><br></div><div>I look forward to your trip, I'll be around for a few months myself.</div><div><br></div><div><br></div><div class="gmail_extra"><div class="gmail_quote">On Wed, Jul 1, 2015 at 2:59 PM, daniela florescu <span dir="ltr"><<a href="mailto:dflorescu@me.com" target="_blank">dflorescu@me.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Ihe,<div><br></div><div>transforming XQuery to be able to do data cleaning has been a LONG desire of mine.</div><div><br></div><div>Helena Galhardas was a PhD student of mine. She is now a professor in Lisbon,</div><div><br></div><div>She and her students wrote the data cleaning package in Zorba — it’s 100% clean XQuery,</div><div>so you can reuse it for other engines.</div><div><br></div><div>Let me know how it goes.</div><div><br></div><div>On the 7th I am leaving to Europe for 3-4 months.</div><div><br></div><div>I will certainly visit London often.</div><div><br></div><div>Hope we can talk, best</div><div>Dana</div><div><div><div><br></div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><div><div><div><blockquote type="cite"><div>On Jul 1, 2015, at 11:54 AM, daniela florescu <<a href="mailto:dflorescu@me.com" target="_blank">dflorescu@me.com</a>> wrote:</div><br><div><div style="word-wrap:break-word">Ihe,<div><br></div><div>before you load anything anywhere, you need to do data cleaning on this data</div><div>if you do integration from the Web and data has no unique ids…..</div><div><br></div><div>In particular entity resolution…</div><div><br></div><div>Literature is full of data cleaning and entity resolution algorithms.</div><div><br></div><div>One that you will find familiar (because it looks very much like XQuery :-) is here:</div><div><br></div><div><a href="http://www.inesc-id.pt/ficheiros/publicacoes/1259.pdf" target="_blank">http://www.inesc-id.pt/ficheiros/publicacoes/1259.pdf</a></div><div><br></div><div>Best regards</div><div>Dana</div><div><br></div><div><br></div><div><br></div><div><br><div><blockquote type="cite"><div>On Jul 1, 2015, at 10:04 AM, Ihe Onwuka <<a href="mailto:ihe.onwuka@gmail.com" target="_blank">ihe.onwuka@gmail.com</a>> wrote:</div><br><div><div dir="ltr">You will note that the data doesn't have a unique id. Title certainly isn't unique, if you consider how many movies there have been called Batman or Treasure Island.<div><br></div><div>Now I may encounter data about this movie from another source that covers different facets , for example it's box office takings or movie reviews. </div><div><br></div><div>So it's a classic semantic web application. I want to amalgamate disparate data about the same fact in one entity. As I said I have a transformation that does this but it doesn't scale very well because I have to search the entire movie base to find the best match. To overcome this I have to adopt a mapReduce-ish approach to solve the problem.</div><div><br></div><div>The thinking is a graphical representation would eliminate that problem because a graph gives me a persistent data structure already indexed for retrieval via several different axes, whereas indexes constructed in the XSLT transformation for the same purpose are ephemeral and would need to be reconstructed every time you ran the transformation.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 1, 2015 at 12:46 PM, Peter Hunsberger <span dir="ltr"><<a href="mailto:peter.hunsberger@gmail.com" target="_blank">peter.hunsberger@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Should be pretty straight forward to import that into Neo4J or Titan. Neo might be simplest, in particular via conversion of the data into JSON. However, Titan might give you other capabilities such as using Hadoop type processing either for import or for subsequent analytics. Without knowing more about the business requirements can't really give you much more than that...</div><div class="gmail_extra"><span><font color="#888888"><br clear="all"><div><div>Peter Hunsberger</div></div></font></span><div><div>
<br><div class="gmail_quote">On Wed, Jul 1, 2015 at 11:32 AM, Ihe Onwuka <span dir="ltr"><<a href="mailto:ihe.onwuka@gmail.com" target="_blank">ihe.onwuka@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I would like to convert the XML snippet below to a multi-relational graph representation. <div>One way is to transform a triple store via RDF. Another which I am less familiar with is to transform to graphML followed by a subsequent import into some graph database tool.</div><div><br></div><div>The graphical representation is desirable for processing rather than visualization reasons. Chiefly I have a matching algorthim implemented in XSLT which works fine but doesn't scale well, a problem that I think can be solved with a graphical representation. </div><div><br></div><div>I am keen to hear from my elders and betters on the subject.</div><div><div><br></div><div><div style="font-family:monospace;font-size:13px"><span><movie<span> <span>title</span>="<span>20000 lieues sous les mers</span>"</span>></span></div><div style="margin-left:1em;font-family:monospace;font-size:13px"><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><actors></span></div><div style="margin-left:1em"><span></span><div><span><person<span> <span>name</span>="<span>Méliès, Georges</span>"</span>/></span></div><span></span></div><div><span></actors></span></div></div></div><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><alias></span></div><div style="margin-left:1em"><span></span><div><span><title<span> <span>title</span>="<span>20,000 Leagues Under the Sea </span>"</span><span> <span>year</span>="<span>1907</span>"</span>/></span></div><span></span><div><span><title<span> <span>title</span>="<span>Amid the Workings of the Deep </span>"</span><span> <span>year</span>="<span>1907</span>"</span>/></span></div><span></span><div><span><title<span> <span>title</span>="<span>Deux cent mille lieues sous les mers </span>"</span><span> <span>year</span>="<span>1907</span>"</span>/></span></div><span></span><div><span><title<span> <span>title</span>="<span>Le cauchemar d'un pêcheur </span>"</span><span> <span>year</span>="<span>1907</span>"</span>/></span></div><span></span><div><span><title<span> <span>title</span>="<span>Under the Seas </span>"</span><span> <span>year</span>="<span>1907</span>"</span>/></span></div><span></span></div><div><span></alias></span></div></div></div><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><directors></span></div><div style="margin-left:1em"><span></span><div><span><person<span> <span>name</span>="<span>Méliès, Georges</span>"</span>/></span></div><span></span></div><div><span></directors></span></div></div></div><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><genres></span></div><div style="margin-left:1em"><span></span><div><span><tag<span> <span>name</span>="<span>adventure</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>fantasy</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>sci-fi</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>short</span>"</span>/></span></div><span></span></div><div><span></genres></span></div></div></div><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><keywords></span></div><div style="margin-left:1em"><span></span><div><span><tag<span> <span>name</span>="<span>based-on-novel</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>dream</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>fish</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>number-in-title</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>submarine</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>undersea-monster</span>"</span>/></span></div><span></span><div><span><tag<span> <span>name</span>="<span>underwater</span>"</span>/></span></div><span></span></div><div><span></keywords></span></div></div></div><span></span><div><div><div><span style="display:inline-block;width:10px;vertical-align:bottom;min-height:10px;background-repeat:no-repeat"></span><span><producers></span></div><div style="margin-left:1em"><span></span><div><span><person<span> <span>name</span>="<span>Méliès, Georges</span>"</span>/></span></div><span></span></div><div><span></producers></span></div></div></div><span></span></div><div style="font-family:monospace;font-size:13px"><span></movie></span></div></div></div></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></blockquote></div><br></div></div></div></div></blockquote></div><br></div></div>