Deep learning framework for hierarchical-based object identification and description

Humans have the capability to quickly classify, identify, and describe objects in the surrounding environment. The Deep learning (DL) and Computer Vision (CV) approaches allow computers to gain high-level understanding from images or videos. Although DL-based CV approaches have achieved various successes in object recognition and identification, these approaches cannot adapt like humans. Existing approaches exhibit setbacks due to their inability to identify objects that are beyond training samples when deployed in real-world applications. Hence, DL approaches lack of global generalization, hierarchical learning, and correlation learning. In addition, DL-based CV approaches only depend on extracting high-level features which are not semantic to recognize parts or subsets of an object. In this study, a new DL framework called Human Identification and Description Framework (HIDF) is developed. The HIDF aims to overcome global generalization, hierarchical and correlation learning limitations by describing an object when it cannot be initially identified. The HIDF components include five phases: 1) a feature extraction network which extracts suitable feature levels for object identification and classification tasks, 2) identification network which specifies the identity of the object, 3) a multi-output classification network for hierarchical-based object classification, 4) object description algorithm which generates sentences describing object characteristics, and 5) shunt connections which regulate prediction direction of HIDF according to the desired task (identification or classification). HIDF achieved 99.27% identification accuracy and 98.61% classification accuracy on benchmark datasets for human identification and human head attribute classification. The framework performance was compared with other state-of-the-art networks based on accuracy, precision, recall, and F1-score measures. The experimental results have shown that HIDF is able to overcome global generalization, hierarchical learning and correlation learning limitations by tracing the hierarchy and correlation of the object. Therefore, HIDF can be employed in CV applications, including but not limited to visually impaired aids, and robot guidance.