My Cinema Knowledge: "my movies" aka Multi-language reconciliation using Freebase

In this first episode of " My Cinema Knowledge" I will try to describe my film catalog mixing private information (my disk folders) with public ones (Freebase)
I will use Ubuntu, Open Refine, a little python script and a RDF store, Virtuoso.

Step By Step How-to

• Build a csv with all the folders names in my disk using a linux command ( find . -type d > myMovies.csv )
• import in open refine (I used lod refine, a package including open refine and the rdf extension)
• Extracted movie name from folder name taking only the last part of the location
• Added a reconciliation service based on the freebase dump created previously (the making of is described in This post) imported in a Virtuoso triple store
  For this I used the SPARQL based reconciliation service feature of the RDF extension
  Using a custom reconciliation service over freebase I will not be limited to the english languages provided by the Freebase reconciliation service
• After 10 minutes on a 8gb Ram machine, this the results (out of about 310):
• 138 movies automatically recognized 
• 66 movies with multiple choices (semi automatic)
• 109 without a match
• Reason for the missing matches are:
• Missing in Freebase (mostly italina movies)
• Missing italian title in Freebase
• Missing in my Freebase copy
• Some intermediate folder (about 15)
• I also got a severe BUG in selecting new matches: https://github.com/fadmaa/grefine-rdf-extension/issues/82 (grrrr)
• UPDATE!!!
  There is also a cloud based reconciliation service for freebase, now working also with italian language. It should be included in open refine but it does not work for this bug https://github.com/OpenRefine/OpenRefine/issues/805. You can make it work creating a new standard one using this address:
  

  http://reconcile.freebaseapps.com/reconcile 

• Copy reconciled data in a new column 
• Exported csv. on raw for example is:
  ./doppiati/1984 , 1984, http://rdf.freebase.com/ns/m.03kp2l
• Transformed the csv to rdf using a Python script as simple as this  using python-rdflibsudo apt-get install python-pip (ubuntu)
  

  sudo pip install rdflib

  To use in this way:
  python myMoviesToRDF.py myMovies-csv.csv myMovies.ttl
• Upload data into my RDF store
• Enjoy data analysis NOW!
  In the first attemp i used SPARQL queries in order to get the genre ranking, the directors ranking and the director nationality ranking. A first attemp now, some more will come soon!
  
  
  

  
  
  An interactive version here