A Synthesis Survey of Ontology Evaluation Tools, Applications and Methods to Propose a Novel Branch in Evaluating the Structure of Ontologies: Graph-Independent Approach
Keywords:
Ontology Evaluation Method, Ontology Evaluation Tools and Applications, Categorization and Classification, Structural Analysis, structural ontology evaluation, Conceptual Graph (CG), Graph Representation, Graph-dependent Approach, Graph-Independent ApproaAbstract
Diverse tools, application and methods can logically be organized in clear categories (i.e., Gold standard, Application, Data-driven and Human assessment) or their dimensions (i.e., Functionality (task-based), Usability based and Structural evaluation). This paper attempts to propose a novel branch in structural analysis of ontology through analyzing current methods. Structural dimensions can be involved in evaluating ontologies when the research attempts to analyze the graph representation based on Conceptual Graph (CG). Two types of nodes (i.e., concepts and conceptual relations) can be merely linked with one another via logical conjunction. When logical conjunction between concepts and conceptual relations were removed, the remaining components would be independent domains which would no longer bear the meaning of graph. The separate concepts and conceptual relations cannot be involved in the notion of the graph-dependent approach. Thus, there is the lack of a novel branch in structural analysis which is called Graph-independent approach.
References
Y. Ma, J. Beihong and Y. Feng. “Semantic oriented ontology cohesion metrics for ontology-based systems.” The Journal of Systems and Software. Vol. 83, pp. 143–152, 2010.
G. Bilgin, , I. Dikmen, and M.T. Birgonul. (2014). “Ontology Evaluation: An Example of Delay Analysis”, in Proceedings of Creative Construction Conference 2014, 21-24 June, Prague, Czech Republic.
E. Bolotnikova, “Ontology cognitive ergonomics evaluation based on graph topology.” 2009 Internet: http://www.ht2009.org/src_submissions/ht2009_submission_184-ok.pdf [Dec. 2, 2017].
A. Gangemi, C. Catenacci, M. Ciaramita and J. Lehmann. “A theoretical framework for ontology evaluation and validation”, in proceedings of SWAP 2005.
A. Gomez-Perez and M. Fernandez-Lopez. Corcho, Ontological Engineering. London: Springer-Verlag, 2004.
J. Brank, M. Grobelnik, and D. Mladenić. “A survey of ontology evaluation technique”. in Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD), 2005. [On-line] http://kt.ijs.si/dunja/sikdd2005/Papers/BrankEvaluationSiKDD2005.pdf
W. J. Araujo, G. Â. B. O. Lima and I. Pierozzi Junior. “Data-driven ontology evaluation based on competency questions: a study in the agricultural domain,” in 14th International ISKO Conference, 2016, Rio de Janeiro. Knowledge Organization for a Sustainable World: Challenges and Perspectives for Cultural, Scientific, and Technological Sharing in a Connected Society, Würzburg: Ergon Verlag,, 2016. Vol. 15, pp. 326-332.
J. Brank, D. Mladenić and M. Grobelnik. “Gold Standard Based Ontology Evaluation Using Instance Assignment,” in Proceedings of the 4th Workshop on Evaluating Ontologies for the Web EON, 2006 Edinburgh, Scotland.
L. Obrst, B. Ashpole, W. Ceusters, I. Mani, R. Steve and B. Smith. “The evaluation of ontologies: Toward improved semantic interoperability,” in Semantic Web. Berlin: Springer, 2007.
A.Gangemi, C. Catenacci, M. Ciaramita and J. Lehmann. “Qood grid: A metaontology-based framework for ontology evaluation and selection”, 2006.
A. Gomez-Perez. “Ontology evaluation,” Steffen Staab and Rudi Studer, editors, in Handbook on Ontologies in Information Systems, International Handbooks on Information Systems, chapter 13, pp. 251–274, Berlin: Springer, 2004.
L. Zemmouchi-Ghomari and A. R. Ghomari. “Position paper: a new approach for human assessment of ontologies,” in International Conference on Information Systems and Technologies ICIST, 2015.
S. Mishra and S. Jain. “Ontologies as a semantic model in IoT,” in International Journal of Computers and Applications, 2018. [On-line] https://doi.org/10.1080/1206212X.2018.1504461
D. Zeginis, A. Hasnain, N. Loutas, H. F. Deus and R. Fox and K. A. Tarabanis. “A collaborative methodology for developing a semantic model for interlinking cancer chemoprevention linked-data sources.” Semantic Web, Vol. 5, No. 2, pp. 127– 142, 2014.
P. Cimiano, A. Hotho and S. Staab. “Clustering concept hierarchies from text,” in Proceedings of LREC, 2004.
H. S. Pinto and J. P. Martins. “Ontology Integration: How to perform the Process,” in Joint Session with IJCAI-01 Workshop on e-Business and the Intelligent Web, 2001, pp. 71 -80. Seattle, USA.
W. Dahdul. Annotation of phenotypes using ontologies: a Gold Standard for the training and evaluation of natural language processing systems, bioRxiv, 322156. 2018. [On- line]. https://www.biorxiv.org/content/early/2018/05/15/322156
A. Maedche and S. Staab. “Measuring similarity between ontologies,” in Proceedings of the 13th International Conference on Knowledge Engineering and Knowledge Management, LNAI. vol. 2473. Springer, London, UK., 2002, 251–263.
F. Patrick and H. Florian. Ontology Evaluation. Seminar in Applied Ontology Engineering WS 10/11, 2010. [On- line]. http://www.sti-innsbruck.at/sites/default/files/courses/fileadmin/documents/applied_onto_eng201011/ontology_evaluation_methods.pdf
M. Imen Bouaziz and G. Faiez. “A Gold Standard-Based Approach for Arabic Ontology Evaluation,” in ECKM 2017 18th European Conference on Knowledge Management, 2017, pp. 1153-1161.
E. Zavitsanos, G. Paliouras and G. A. Vouros. “A Distributional Approach to Evaluating Ontology Learning Methods Using a Gold Standard,” in Proc. European Conf. Artificial Intelligence (ECAI '08) Ontology Learning and Population (OLP '08) Workshop, 2008.
N. Guarino. “Towards a formal evaluation of ontology quality (in Why evaluate ontology technologies? Because they work!).” IEEE Intelligent Systems, Vol. 19, No. 4, 2004, pp. 74–81.
R. Romano. “The quality of an ontology: the development and demonstration of an instrument for ontology quality assessment,” M. S. thesis, University of Canberra, Canberra, Australia, 2012.
R. Porzel and R. Malaka. “A task-based approach for ontology evaluation,” in ECAI Workshop Ont. Learning and Population, 2004.
D. Knoell, M. Atzmueller, C. Rieder and K. P. Scherer. “A Scalable Framework for Data-Driven Ontology Evaluation,” in Proc. GWEM 2017, co-located with 9th Conference Professional Knowledge Management (WM 2017)', KIT, Karlsruhe, Germany, CEUR-WS.org, 2017, pp. 97-106.
F. Patrick and H. Florian. “Ontology Evaluation. Seminar in Applied Ontology Engineering WS 10/11,” 2010. [On- line]. http://www.sti-innsbruck.at/sites/default/files/courses/fileadmin/documents/applied_onto_eng201011/ontology_evaluation_methods.pdf
C. Brewster, H. Alani, S. Dasmahapatra and Y. Wilks. “Data driven ontology evaluation,” in Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC), Lisbon, Portugal, European Language Resources Association, 2004.
Dr. N. Vetrivelan and C. Senthil Selvi. “A Review of Enhanced and Secure Ontology Learning Approaches.” International Journal of Computer Sciences and Engineering, Vol. 4, No. 4, 2016, pp. 319-326.
A. Lozano-Tello and A. Gómez-Pérez. “ONTOMETRIC: A Method to Choose the Appropriate Ontology.” Journal of Database Management, Vol. 5, No.2, 2004, pp. 1-18.
Y. Netzer, D. Gabay, M. Adler, Y. Goldberg and M. Elhadad. “Ontology evaluation through text classification,” in Advances in Web and Network Technologies, and Information Management. Springer, 2009.
F. Kawsar, T. Nakajima, J.H. Park and S.S. Yeo “Design and implementation of a framework for building distributed smart object systems,” J. Supercomput, Vol. 54, No. 1, (2010) pp. 4–28.
A. Gomez-Perez, M.F. Lopez and O.C. Garcia, “Ontological Engineering: With Examples from the Areas of Knowledge Management,” in E-Commerce and the Semantic Web, Springer, 2001.
P. Martín Chozas. “Towards a Linked Open Data Cloud of language resources in the legal domain,” M. S. Thesis, E.T.S. de Ingenieros Informáticos, Universidad Politécnica de Madrid (UPM), 2018.
S. Lovrencic and M. Cubrilo. “Ontology evaluation-Comprising verification and validation,” in Proceedings of Central European Conference on Information and Intelligent Systems (CECIIS 2008), 2008, Varazdin, Croatia.
E. Bolotnikova, T.A. Gavrilova and V.A. Gorovoy. “To a method of evaluating ontologies.” Journal of Computer and Systems Sciences International, Vol. 50, No. 3, 2011, pp.448-461.
J. Fernández-Breis, M. Egaña Aranguren and R. Stevens. “A quality evaluation framework for bio-ontologies,” in ICBO 2009: proceedings of the 2009 international conference on biomedical ontology. University at Buffalo, NY. Nature Precedings, 2009.
H. Shah, P. Shah and K. Deulkar “A survey of ontology evaluation techniques for data retrieval.” International journal of engineering and computer science, Vol. 4, Issue 11, 2015, pp. 14960-14962.
J. Yu, J. Thom, and A. Tam. “Requirements-oriented methodology for evaluating ontologies.” Information Systems, Sixteenth ACM, Vol.34 No. 8, 2009, pp. 686–711.
J.Hartmann, P. Spyns, A. Giboin, D. Maynard, R. Cuel, M.C. Suárez-Figueroa and Y. Sure. “D1.2.3 Methods for ontology evaluation,” Deliverable D1.2.3 EU-IST Network of Excellence IST-2004-507482 KEWB, 2005.
M. Amirhosseini. “Analysis of concept structure and semantic relations based on graph-independent structural analysis,” Ph. D. Dissertation. Faculty of Information Sciences and Technology, Universiti Kebangsaan Malaysia, 2016, 388 p.
H. Alani and C. Brewster. “Ontology ranking based on the analysis of concept structures.” in Proceedings of the 3rd International Conference on Knowledge Capture (K-Cap), 2005, Banff, Canada.
D. Eynard, M. Matteucci and F. Marfa “A modular framework to learn seed ontologies from test,” in Semi-automatic ontology development: processes and resources. Hershey, PA: Information Science Reference, 2012.
R. Dividino and D. Sonntag “Controlled Ontology Evolution through Semiotic-based Ontology Evaluation,” in Proceedings of International Workshop on Ontology Dynamics (IWOD 2008), ISWC 2008 Conference.
K. Dellschaft and S. Staab “Strategies for the Evaluation of Ontology Learning,” Buitelaar, Paul; Cimiano, Philip, in Bridging the Gap between Text and Knowledge - Selected Contributions to Ontology Learning and Population from Text. Amsterdam, IOS Press, 2008.
S. Tamilselvam, S. Nagar, A. Mishra and D. Kuntal. “Graph Based Sentiment Aggregation using ConceptNet Ontology,” in the eight international joint conferences on natural language processing (IJCNLP), Taipei, Taiwan, 2017.
S. Wang, H. Cho, C. Zhai, B. Berger and J. Peng. “Exploiting ontology graph for predicting sparsely annotated gene function.” Bioinformatics, Vo. 31, 2015, pp. i357–i364.
Y. Kim, P. Zerfos, V. Sheinin and N. Greco. “Ranking the importance of ontology concepts using document summarization techniques,” in Proceedings of IEEE International Conference on BigData, 2017.
J. F. Sowa. “Conceptual Graphs,” in Handbook of Knowledge Representation. edited by F. van Harmelen, V. Lifschitz and B. Porter. London: Elsevier, 2008.
D. Corbett, “Conceptual Graph Theory Applied to Reasoning in Ontologies,” 2002. [On- line]. www.lsi.us.es/iberamia2002/confman/.../132-annuniuett.pdf.
N. Chah. “OK Google, What Is Your Ontology? Or: Exploring Freebase Classification to Understand Google's Knowledge Graph,” 2018. [On- line]. https://arxiv.org/abs/1805.03885.
V. Wiens, S. Lohmann and S. Auer. “Semantic zooming for ontology graph visualizations,” in Proceedings of the Knowledge Capture Conference (K-CAP’17), 2017, pp.4:1–4:8. ACM.
MÁ. Rodríguez-García and R. Hoehndorf. “Inferring ontology graph structures using OWL reasoning.” BMC Bioinformatics, Vol. 19, No. 1, 2018.
C. S. Peirce. “On the algebra of logic American.” Journal of Mathematics, Vol. 3, 1880, pp. 15-57.
C. S. Peirce. “On the algebra of logic.” American Journal of Mathematics, Vol7, 1885, pp.180-202.
C. S. Peirce. “Reasoning and the Logic of Things.” In The Cambridge Conferences Lectures of 1898, ed. by K. L. Ketner, Harvard University Press, Cambridge, MA, 1992.
J.F. Sowa. “Conceptual Structures” in Information Processing in Mind and Machine. Reading, Mass: Addison-Wesley, 1984.
J.F. Sowa. “Conceptual Graphs Summary, in Conceptual Structures: Current Research and Practice,” Ellis Horwood: Chichester, UK, 1992.
M. Willems. “Projection and Unification for Conceptual Graphs,” in Proc. Third International Conference on Conceptual Structures. Santa Cruz, California, USA: Springer-Verlag, 1995.
M. Chein and M.-L. Mugnier “Conceptual Graphs: Fundamental Notions.” Revue d'Intelligence Artificielle, Vol. 6, No. 4, 1992, pp. 365-406.
D.R. Corbett and R.F. Woodbury “Unification over Constraints in Conceptual Graphs,” in Proc. Seventh International Conference on Conceptual Structures. Blacksburg, Virginia, USA: Springer-Verlag, 1999.
D.R. Corbett “Conceptual Graphs with Constrained Reasoning.” Revue d'Intelligence Artificielle, Vol. 15, No. 1, 2001, pp. 87-116.
G. Polcicova and P. Návrat. “Semantic Similarity in Content-Based Filtering,” in Advances in Databases and Information Systems: 6th East European Conference, ADBIS 2002. Bratislava, Slovakia: Proceedings, 2002.
C. V. Buggenhouta and W. Ceustersb. “A novel view on information content of concepts in a large ontology and a view on the structure and the quality of the ontology.” International Journal of Medical Informatics, Vol. 74, No. 2-4, 2005, pp. 125-132.
M. Amirhosseini and J. Salim “Quantitative evaluation of the movement from complexity toward simplicity in the structure of thesdaurus descriptors.” Malaysian Journal of Library & Information Science, Vol. 20, No. 3, 2015, pp. 47-62.
M. Amirhosseini “Qualitative and quantitative evaluation of effective factor in information storage and retrieval in Persian thesauri”. LIS Ph. D. Dissertation. Shiraz University, 2007.
Downloads
Published
How to Cite
Issue
Section
License
Authors who submit papers with this journal agree to the following terms.
