168
A. Gyrard et al.
lawfulness, richness, structure, (2) Semantic: quality, consistency, interpretability,
precision, (3) Pragmatic: quality, accuracy, adaptability, comprehensiveness, ease of
use, relevance, (4) Social: quality, authority, history, recognition, and (5) Overall. We
are expecting more explanations for why and how those metrics have been chosen,
and the meaning of the evaluation numbers provided within the prototype.
However, this excellent survey paper, published in 2018, misses important references of pioneer work to define metrics (e.g., the authors in [29] outline metrics
such as the ontology competency and completeness). Furthermore, compared to
their work, we collect a set of tools to improve ontologies. We took into consideration others metrics such as documentation, visualization, dissemination on ontology
catalogs. Since those tools have most of the time ontology validators integrated, it
enables to evaluate ontology quality at the same time.
Ma et al. [30] propose the Ontology Usability Scale (OUS), a ten-item Likert scale derived from statements prepared according to a semiotic framework and
an online poll in the semantic web community to evaluate ontology usability. ISO
9241-11 defines usability as follows: “the extent to which a product can be used by
specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specific context of use.” The authors estimate the costs of using ontologies
when developing applications to create better ontologies. An ontology may be consistent (i.e., without any contradictory assertion, complete (i.e., without any missing
definition) and concise (i.e., without any unnecessary definition) but still unusable
or very cumbersome to use (e.g., due to lousy documentation). The authors classify
ontologies into three categories: (1) pragmatics, (2) semantics, and (3) syntax.
Raad et al. [10] investigate what makes a good ontology by analyzing ontology
evaluation methods and discuss their advantages. These methods are used to evaluate
the quality of automatically constructed ontologies. A good ontology can contribute
to the success of semantic services and various knowledge management applications.
Four categories have been designed: (1) gold standard-based, (2) corpus-based, (3)
task-based, and (4) criteria-based.
Hlomani et al. design competency questions for ontology evaluation with a focus
on quality and correctness as follows [11]: (1) Does the model cover required context
information? (2) Does the DL language provide the logical constructs required by the
reasoner? (3) Are the adaptation purposes sufficient to describe a WoT app context?,
and (4) Are the adaptation purposes redundant or overlapping?
Corcho et al. survey the existing ontology methodologies [31] and answer the
following questions: (1) which methods and methodologies can I use for building
ontologies?, (2) which tools give support to the ontology development process, and
(3) which languages can I use to implement the ontology? This work can be used as a
guideline for analyzing the existing methodologies and tools for ontology engineering
but need to be updated since it has been published in 2003.
A. Gyrard et al.
lawfulness, richness, structure, (2) Semantic: quality, consistency, interpretability,
precision, (3) Pragmatic: quality, accuracy, adaptability, comprehensiveness, ease of
use, relevance, (4) Social: quality, authority, history, recognition, and (5) Overall. We
are expecting more explanations for why and how those metrics have been chosen,
and the meaning of the evaluation numbers provided within the prototype.
However, this excellent survey paper, published in 2018, misses important references of pioneer work to define metrics (e.g., the authors in [29] outline metrics
such as the ontology competency and completeness). Furthermore, compared to
their work, we collect a set of tools to improve ontologies. We took into consideration others metrics such as documentation, visualization, dissemination on ontology
catalogs. Since those tools have most of the time ontology validators integrated, it
enables to evaluate ontology quality at the same time.
Ma et al. [30] propose the Ontology Usability Scale (OUS), a ten-item Likert scale derived from statements prepared according to a semiotic framework and
an online poll in the semantic web community to evaluate ontology usability. ISO
9241-11 defines usability as follows: “the extent to which a product can be used by
specified users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specific context of use.” The authors estimate the costs of using ontologies
when developing applications to create better ontologies. An ontology may be consistent (i.e., without any contradictory assertion, complete (i.e., without any missing
definition) and concise (i.e., without any unnecessary definition) but still unusable
or very cumbersome to use (e.g., due to lousy documentation). The authors classify
ontologies into three categories: (1) pragmatics, (2) semantics, and (3) syntax.
Raad et al. [10] investigate what makes a good ontology by analyzing ontology
evaluation methods and discuss their advantages. These methods are used to evaluate
the quality of automatically constructed ontologies. A good ontology can contribute
to the success of semantic services and various knowledge management applications.
Four categories have been designed: (1) gold standard-based, (2) corpus-based, (3)
task-based, and (4) criteria-based.
Hlomani et al. design competency questions for ontology evaluation with a focus
on quality and correctness as follows [11]: (1) Does the model cover required context
information? (2) Does the DL language provide the logical constructs required by the
reasoner? (3) Are the adaptation purposes sufficient to describe a WoT app context?,
and (4) Are the adaptation purposes redundant or overlapping?
Corcho et al. survey the existing ontology methodologies [31] and answer the
following questions: (1) which methods and methodologies can I use for building
ontologies?, (2) which tools give support to the ontology development process, and
(3) which languages can I use to implement the ontology? This work can be used as a
guideline for analyzing the existing methodologies and tools for ontology engineering
but need to be updated since it has been published in 2003.