An Automatic Sound ClassifiCation Framework …
433
tinguishes it from many other machine learning frameworks. Here, we discuss the
computational advantages of this framework and key considerations when implement
this framework on the dense crossbar array of NVM devices.
4.1 Computational Benefits of SOM-SNN Framework
The SOM-SNN framework is organized in a modular manner, whereby acoustic
signals are pre-processed using a biologically plausible auditory front-end, the melscaled filter bank, for frequency content analysis. This framework emulates the functionality of the human cochlea and the non-linearity of human perception of sound
[20]. Although it is still not clear how information is represented and processed in
the auditory cortex, it has been shown that certain neural populations in the cochlear
nuclei and primary auditory cortex are organized in a tonotopic fashion [24, 52].
Motivated by this, the biologically plausible SOM is used for the feature extraction
and representation of mel-scaled filter bank outputs. The selectivity of neurons in
the SOM emerges from unsupervised training and organizes in a tonotopic fashion, whereby adjacent neurons share similar weight vectors. The SOM effectively
improves pattern separation, whereby each sound frame originally represented by a
20-dimensional vector (mel-scaled filter bank output coefficients) is translated into
a single output spike. The resulting BMU activation sequences are shown to have
the property of low intra-class variability and high inter-class variability. Consequently, the SOM provides an effective and sparse representation of acoustic signals
as observed in the auditory cortex [53].
Acoustic signals exhibit large variations not only in their frequency contents but
also in temporal structures. State-of-the-art machine learning based ASC systems
model the temporal transition explicitly, using the HMM, RNN or LSTM, while
our work focuses on building a biologically plausible temporal classifier based on
the SNN. For efficient training, we use supervised temporal learning rules, namely
the membrane-potential based Maximum-Margin Tempotron and spike-timing based
ReSuMe. As demonstrated in our experiments, the SOM-SNN framework achieves
comparable classification results on both the RWCP and TIDIGITS datasets against
other deep learning and SNN-based models.
The SNN-based classifier has an early decision making capability: making a
classification decision when only part of the input is presented. In our experiments,
the SNN-based classifier achieves an accuracy of 95.1%, significantly higher than
those of the RNN and LSTM (25.7% and 69.2% respectively) when only 50% of
the input pattern is presented. This early decision making capability can be further
exploited in noisy environments, as exemplified by the cocktail party problem [54].
The SNN-based classifier can potentially identify discriminative temporal features
and classify accordingly from a time snippet of the acoustic signals that are less
distorted, which is desirable for an environment with fluctuating noise.
Environmental noise poses a significant challenge to the robustness of any sound
classification systems: the accuracy of many such systems degrade rapidly with an
433
tinguishes it from many other machine learning frameworks. Here, we discuss the
computational advantages of this framework and key considerations when implement
this framework on the dense crossbar array of NVM devices.
4.1 Computational Benefits of SOM-SNN Framework
The SOM-SNN framework is organized in a modular manner, whereby acoustic
signals are pre-processed using a biologically plausible auditory front-end, the melscaled filter bank, for frequency content analysis. This framework emulates the functionality of the human cochlea and the non-linearity of human perception of sound
[20]. Although it is still not clear how information is represented and processed in
the auditory cortex, it has been shown that certain neural populations in the cochlear
nuclei and primary auditory cortex are organized in a tonotopic fashion [24, 52].
Motivated by this, the biologically plausible SOM is used for the feature extraction
and representation of mel-scaled filter bank outputs. The selectivity of neurons in
the SOM emerges from unsupervised training and organizes in a tonotopic fashion, whereby adjacent neurons share similar weight vectors. The SOM effectively
improves pattern separation, whereby each sound frame originally represented by a
20-dimensional vector (mel-scaled filter bank output coefficients) is translated into
a single output spike. The resulting BMU activation sequences are shown to have
the property of low intra-class variability and high inter-class variability. Consequently, the SOM provides an effective and sparse representation of acoustic signals
as observed in the auditory cortex [53].
Acoustic signals exhibit large variations not only in their frequency contents but
also in temporal structures. State-of-the-art machine learning based ASC systems
model the temporal transition explicitly, using the HMM, RNN or LSTM, while
our work focuses on building a biologically plausible temporal classifier based on
the SNN. For efficient training, we use supervised temporal learning rules, namely
the membrane-potential based Maximum-Margin Tempotron and spike-timing based
ReSuMe. As demonstrated in our experiments, the SOM-SNN framework achieves
comparable classification results on both the RWCP and TIDIGITS datasets against
other deep learning and SNN-based models.
The SNN-based classifier has an early decision making capability: making a
classification decision when only part of the input is presented. In our experiments,
the SNN-based classifier achieves an accuracy of 95.1%, significantly higher than
those of the RNN and LSTM (25.7% and 69.2% respectively) when only 50% of
the input pattern is presented. This early decision making capability can be further
exploited in noisy environments, as exemplified by the cocktail party problem [54].
The SNN-based classifier can potentially identify discriminative temporal features
and classify accordingly from a time snippet of the acoustic signals that are less
distorted, which is desirable for an environment with fluctuating noise.
Environmental noise poses a significant challenge to the robustness of any sound
classification systems: the accuracy of many such systems degrade rapidly with an