UUM Electronic Theses and Dissertation
UUM ETD | Universiti Utara Malaysian Electronic Theses and Dissertation
FAQs | Feedback | Search Tips | Sitemap

Multi-modal association learning using spike-timing dependent plasticity (STDP)

Ibrahim, Mohammed Fadhil (2014) Multi-modal association learning using spike-timing dependent plasticity (STDP). Masters thesis, Universiti Utara Malaysia.

[thumbnail of s809894_01.pdf]

Download (3MB) | Preview
[thumbnail of s809894_02.pdf]

Download (726kB) | Preview


We propose an associative learning model that can integrate facial images with speech signals to target a subject in a reinforcement learning (RL) paradigm. Through this approach, the rules of learning will involve associating paired stimuli (stimulus–stimulus, i.e., face–speech), which is also known as predictor-choice pairs.
Prior to a learning simulation, we extract the features of the biometrics used in the study. For facial features, we experiment by using two approaches: principal component analysis (PCA)-based Eigenfaces and singular value decomposition (SVD). For speech features, we use wavelet packet decomposition (WPD). The
experiments show that the PCA-based Eigenfaces feature extraction approach produces better results than SVD. We implement the proposed learning model by using the Spike- Timing-Dependent Plasticity (STDP) algorithm, which depends on the time and rate of pre-post synaptic spikes. The key contribution of our study is the implementation of learning rules via STDP and firing rate in spatiotemporal neural networks based on the Izhikevich spiking model. In our learning, we implement learning for response group association by following the reward-modulated STDP in terms of RL, wherein the firing rate of the response groups determines the reward that will be given. We perform a number of experiments that use existing face samples from the Olivetti Research Laboratory (ORL) dataset, and speech samples from TIDigits. After several experiments and simulations are performed to recognize a subject, the results show that the proposed learning model can associate the
predictor (face) with the choice (speech) at optimum performance rates of 77.26% and 82.66% for training and testing, respectively. We also perform learning by using real data, that is, an experiment is conducted on a sample of face–speech data, which have been collected in a manner similar to that of the initial data. The performance results are 79.11% and 77.33% for training and testing, respectively. Based on these results, the proposed learning model can produce high learning performance in terms
of combining heterogeneous data (face–speech). This finding opens possibilities to expand RL in the field of biometric authentication

Item Type: Thesis (Masters)
Supervisor : Yusoff, Nooraini
Item ID: 5615
Uncontrolled Keywords: spiking neural network, feature extraction, spike-timing-dependent plasticity, association learning, reinforcement learning
Subjects: T Technology > T Technology (General) > T58.5-58.64 Information technology
Divisions: Awang Had Salleh Graduate School of Arts & Sciences
Date Deposited: 16 May 2016 09:51
Last Modified: 23 May 2022 01:47
Department: Awang Had Salleh Graduate School of Arts and Sciences
Name: Yusoff, Nooraini
URI: https://etd.uum.edu.my/id/eprint/5615

Actions (login required)

View Item
View Item