Text representation using canonical data model

Developing digital technology and the World Wide Web has led to the increase of digital documents that are used for various purposes such as publishing, in turn, appears to be connected to raise the awareness for the requirement of effective techniques that
can help during the search and retrieval of text. Text representation plays a crucial role
in representing text in a meaningful way. The clarity of representation depends tightly on the selection of the text representation methods. Traditional methods of text
representation model documents such as term-frequency invers document frequency
(TF-IDF) ignores the relationship and meanings of words in documents. As a result the
sparsity and semantic problem that is predominant in textual document are not resolved. In this research, the problem of sparsity and semantic is reduced by proposing
Canonical Data Model (CDM) for text representation. CDM is constructed through an
accumulation of syntactic and semantic analysis. A number of 20 news group dataset
were used in this research to test CDM validity for text representation. The text documents goes through a number of pre-processing process and syntactic parsing in order to identify the sentence structure. Text documents goes through a number of preprocessing steps and syntactic parsing in order to identify the sentence structure and then TF-IDF method is used to represent the text through CDM. The findings proved that CDM was efficient to represent text, based on the model validation through
language experts‟ review and the percentage of the similarity measurement methods.