SWIP at QALD-3 : results, criticisms and lesson learned

Camille Pradel (1) , Guillaume Peyet (2) , Ollivier Haemmerlé (1, 3) , Nathalie Jane Hernandez (1, 3)

1 IRIT-MELODI - MEthodes et ingénierie des Langues, des Ontologies et du DIscours
2 IRIT - Institut de recherche en informatique de Toulouse
3 UT2J - Université Toulouse - Jean Jaurès

This paper presents the results obtained by the SWIP system while participating in the QALD-3 (Question Answering over Linked Data) challenge, co-located with CLEF 2013 (Conference and Labs of the Evaluation Forum). We tackled task 1, multilingual question answering, whose purpose is to interpret natural language questions in order to return the answers contained in a graph knowledge base. We answered queries of both proposed datasets (one concerning DBpedia, the other Musicbrainz) and took into consideration only questions in English. The system SWIP (Semantic Web Interface using Patterns) aims at automatically generating formal queries from user queries expressed in natural language. For this, it relies on the use of query patterns which enable the complex task of interpreting natural language queries. The results obtained on the Musicbrainz dataset (precision = 0,51, recall = 0,51, F-measure = 0,51) are very satisfactory and encouraging. The results on DBpedia (precision = 0,16, recall = 0,15, F-measure = 0,16) are more disappointing. In this paper, we present both the SWIP approach and its implementation. We then present the results of the challenge in more detail and their analysis. Finally we draw some conclusions on the strengths and weaknesses of our approach, and suggest ways to improve its performance.

Mots clés: Semantic Web Interface using Patterns; SWIP

Domaines: Intelligence artificielle [cs.AI]; Apprentissage [cs.LG]; Logique en informatique [cs.LO]; Informatique et langage [cs.CL]

Camille Pradel, Guillaume Peyet, Ollivier Haemmerlé, Nathalie Jane Hernandez. SWIP at QALD-3 : results, criticisms and lesson learned. 3rd open challenge on Question Answering over Linked Data (QALD 2013), Sep 2013, Valencia, Spain. pp. 1-13. ⟨hal-01193095⟩