![International Journal of Current Research In Science, Engineering and Technology (IJCRSET)
Volume 1, Issue 2, June 2016, PP 13-17
www.ijcrset.com
www.ijcrset.com 13 | Page
Speech Processing Algorithms for Perception Improvement of
Hearing Impaired Patients
B.M.Magdum1
, Prof.P.A.Dhulekar2
1
(E&TC, Sandip Institute of Technology & Research Center/ SPPU, India)
2
(E&TC, Sandip Institute of Technology & Research Center/ SPPU, India)
Abstract :A novel algorithm is projected to improve speech perception of hearing impaired people. The
algorithm is developed to achieve a specific number of bands of the selected speech material for dichotic
presentation.These bands are obtained using a set of dyadic filters which gives sharp transitions between
selected bands. For implementing this algorithm MATLAB software is used .At first by implementing the code
eighteen bands of Constant Bandwidth, nineteen bands of 1/3 Octave bandwidths and eighteen bands of critical
bandwidth using pair of complementary comb filters from the filter sets are obtained. Assessment was done by
conducting listening tests on five subjects with hearing loss. A set fifteen syllables of VCV context is used as the
speech material for conducting the hearing tests.The conducted test results proves that the projected method
improves the recognition score and also reduces the response time.
Keywords : Hearing mechanism, Hearing loss, Hearing implants, Dichotic presentation.
I. INTRODUCTION
Listening by means of two ears is known as binaural hearing. Presenting same signal to both the ears is known
as diotic presentation and presenting two different signals to the two ears is referred to as dichotic presentation.
Researches have reported the benefit of binaural dichotic mode of hearing in improving speech perception by
persons with sensorineural hearing impairment. Sensorineural hearing impairment is considered by increased
threshold of hearing, reduced dynamic hearing range, degraded frequency selectivity and temporal resolution,
and increased spectral and temporal masking. The ability to perceptually combine binaurally received signals
from two ears improves speech perception under adverse listening conditions. Binaural listening offers better
overall sound quality and intelligibility, more relaxed listening, and it also helps in source localization [1]. The
splitting of information in speech signal for presenting signals to the two ears, in some sort of a complementary
fashion, to provide relaxation for sensory cells of the basilar membrane, may help in reducing the effect of
increased masking and thereby improve the speech reception in cases of bilateral sensorineural hearing
impairment with some residual hearing [1]. Spectral masking or simultaneous masking is a frequency domain
version of temporal masking, and tends to occur in sounds with similar frequencies,a powerful spike will tend to
mask out a lower-level tone. It is masking linking two concurrent sounds. Sometimes called frequency masking
since it is often observed when the sounds share a frequency band e.g. two sine tones at 460 and 470Hz can be
perceived clearly when separated. They cannot be perceived clearly when simultaneous. In masking a sound is
made inaudible by a masker, a noise or unwanted sound of the same duration as the original sound [1].The
greatest masking is when the masker and the signal are the same frequency and these decreases as the signal
frequency moves further away from the masker frequency.This phenomenon is called on-frequency masking
and occurs because the masker and signal are within the same auditory filter. The simultaneous masking reduces
the frequency resolution significantly, so it is more severe compared to the non-simultaneous masking. The
auditory masking occurs because the original neural activity caused by the first signal is reduced by the neural
activity of the other sound .The splitting of speech signal into the two channels or bands can be carried out in a
number of ways.The objective of our investigation is to split the speech in two bands with complementary
spectra on the basis of critical band filtering for binaural dichotic presentation . The study was carried out by
processing speech signals of vowel-consonant-vowel syllables for Fifteen English consonants and listening tests
were conducted on five hearing subjects with simulated sensorineural hearing loss.
II. SPEECH MATERIALS AND SUBJECTS
2.1 The Speech Materials and Subjects
Earlier studies have used CV, VC, CVC, and VCV syllables. It has been reported earlier that greater masking
takes place in intervocalic consonants due to the presence of vowels on both sides [5]. Since our primary
objective is to study improvement in consonantal identification due to reduction in the effect of masking, so
VCV syllables are used. For the evaluation of the speech processing strategies,a set of Fifteen syllables of](https://image.slidesharecdn.com/3-160616071430/85/3-speech-processing-algorithms-for-perception-improvement-of-hearing-impaired-patients-1-320.jpg)
![International Journal of Current Research In Science, Engineering and Technology
www.ijcrset.com 14 | Page
consonants / b,d,f,g,k,l,m,n,p,r,s,t,v,y,z / in VCV and vowel /a/ were used in the experiment.Five subjects
participated in the listening test.
III. PROPOSED SYSTEM
The proposed system is designed in order to overcome the masking effects that are caused in human ear due the
malfunctioning of the auditory system.Then after the removal of noise the filtered output speech signal iused for
automatic speech recognition.The Figure 1 describes the proposed system block diagram.
FIGURE 1. PROPOSED SYSTEM
I.ALGORITHM
The proposed algorithm is divided into two parts first Decomposition by Dyadic AnalysisFilter Bank and
secondly Reconstruction by Synthesis of Dyadic Analysis Filter Banks.
3.1 Algorithm For Decomposition by Dyadic Analysis Filter Bank
1]Acquisition of Speech signals (Prerecorded Speech Materials)
2]Decimation by factor D
3]Design of Dyadic Analysis Filter bank with
4]Spectral splitting of recorded speech signal by filter designed in step 3.
5]Obtain a desired number of critical bands with varying bandwidth.
3.2 Algorithm For Reconstruction by Dyadic Synthesis Filter Bank
1]Design of Dyadic Synthesis Filter bank with
2]Reconstruction of obtained bands in Even-Odd Index
3]Interpolation by factor I where I=D
4]Convert reconstructed bands to Right and left speech signal
5]Dichotic presentation of processed right and left speech signal to Right-Left ear
6]Obtain the Hearing test results with Parameters
d)Recognition score
e)Recognition time
IV. RESULTS
Results of listening tests for Response time, comparative drop off in response times, Percentage of correct
recognition scores and Relative growth in recognition scores obtained with the 3 set of filter which are are
presented in the following sections.
4.1 Recognition time
Table 1 represents the response times for the unprocessed signals,processed signals using constant band, critical
band and 1/3 octave bands for the five subjects DMM, PMN,VRM,RMM and NAR .The Table 2 represents the
comparative fall in response time (Processed signal vs. Unprocessed signal).](https://image.slidesharecdn.com/3-160616071430/85/3-speech-processing-algorithms-for-perception-improvement-of-hearing-impaired-patients-2-320.jpg)
![International Journal of Current Research In Science, Engineering and Technology
www.ijcrset.com 16 | Page
Table 3.Percentage of Recognition scores
Table 4 . Comparative growth in recognition scores
V. CONCLUSION
During the test Response times and Recognition scores were compaired and analyzed. It is observed that
there was a drop off in response time for all the processing schemes compared with unprocessed
signal.Statistical and tabular representation show the Response Time,Comparative fall off in Percentage
,Recognition scores and Comparative growth in recognition scores . The degree of improvement was the
greatest for processed signal (1/3 octave band) for all the subjects,which reflects that the 1/3 octave band is
more efficient to improve the perception of the hearing impaired persons. As stated by the improvements in
response time, dichotic presentation also proves better results to improve the perception scores.
References
[1] Moore, B. C. J,An Introduction to Psychology of Hearing,4th Ed. (Academic, London), 1997.
[2] CHABA, Speech-perception aids for hearing-impaired people:Current status and needed research,J. Acoust. Soc. Am. vol. 90, 637–683,
1991.
[3] GL Ward ,MD MacAllister,Apparatus and method for conveying amplified sound to ear,Google Patent,1993.
Subjec
ts
Unprocess
d signal
Processed
signal
(Constant
band)
Process
d signal
(Critica
l band)
Processed
signal(1/3
Octave
band)
DMM 94.3 97.4 100 100
PMN 86.6 88.7 89.6 93.1
VRM 73.3 86.6 88.4 97.2
RMM 80 92.3 94.2 91.8
NAR 78.3 89.2 92.6 90.6
Subjects Processed
signal(Constant
band)
Processed
signal(Critical
band)
Processed
signal(1/3
Octave band)
DMM 3.1 5.7 5.7
PMN 2.1 3.0 6.5
VRM 13.3 15.1 23.9
RMM 12.3 14.2 11.8
NAR 10.9 14.3 12.3](https://image.slidesharecdn.com/3-160616071430/85/3-speech-processing-algorithms-for-perception-improvement-of-hearing-impaired-patients-4-320.jpg)
![International Journal of Current Research In Science, Engineering and Technology
www.ijcrset.com 17 | Page
[4] Loizou,P.C., Mani, Arunvijay, and Dorman, M.F. ,Dichotic speech recognition in noise using reduced spectral cues , J. Acoust. Soc. Am.
vol. 114(1), 475–483, 2003.
[5] Yasu, K., Kobayashi, K., Hishitani, M., Arai, T and Murahara Y.,Critical band based frequency compression for digital hearing aids, J.
Acoust. Sci. and Tech. vol. 25, 61–63, 2004.
[6] Deniz Baskent, Speech recognition in normal hearing and sensorineural hearing loss as a function of the number of spectral channels,J.
Acoust. Soc. Am. vol. 120(5), 2908–2925,2006.
[7] Kulkarni, P.N., and Pandey, P.C., Optimizing the comb Filters for spectral splitting of speech to reduce the Effect of spectral
masking.IEEE-International Conference on Signal processing, Communications and Networking. Madras Institute of Technology, Anna
University Chennai India, Jan 4-6, 69─73, 2008.
[8] Daniel M. Rasetshwane, Member, IEEE, Michael P. Gorga, and Stephen T. Neely, Member, IEEE,Signal-Processing Strategy for
Restoration of Cross-Channel Suppression in Hearing-Impaired Listeners,IEEE Transactions On Biomedical Engineering, vol. 61, No. 1,
January 2014.
[9] Chanwoo Kim,Kean K.Chin,Michiel Bacchiani,Richard M. Stern, Robust speech recognition using temporal masking and thresholding
algorithm, International Science Congress Association ,ISCA, 2014.
[10] Peng Dai , Frank Rudzicz , Ing Yann Soon , Alex Mihailidis, Huijun Ding ,2D Psychoacoustic modeling of equivalent masking for
automatic speech recognition,ELSEVIER,2015.](https://image.slidesharecdn.com/3-160616071430/85/3-speech-processing-algorithms-for-perception-improvement-of-hearing-impaired-patients-5-320.jpg)
3. speech processing algorithms for perception improvement of hearing impaired patients
- 1. International Journal of Current Research In Science, Engineering and Technology (IJCRSET) Volume 1, Issue 2, June 2016, PP 13-17 www.ijcrset.com www.ijcrset.com 13 | Page Speech Processing Algorithms for Perception Improvement of Hearing Impaired Patients B.M.Magdum1 , Prof.P.A.Dhulekar2 1 (E&TC, Sandip Institute of Technology & Research Center/ SPPU, India) 2 (E&TC, Sandip Institute of Technology & Research Center/ SPPU, India) Abstract :A novel algorithm is projected to improve speech perception of hearing impaired people. The algorithm is developed to achieve a specific number of bands of the selected speech material for dichotic presentation.These bands are obtained using a set of dyadic filters which gives sharp transitions between selected bands. For implementing this algorithm MATLAB software is used .At first by implementing the code eighteen bands of Constant Bandwidth, nineteen bands of 1/3 Octave bandwidths and eighteen bands of critical bandwidth using pair of complementary comb filters from the filter sets are obtained. Assessment was done by conducting listening tests on five subjects with hearing loss. A set fifteen syllables of VCV context is used as the speech material for conducting the hearing tests.The conducted test results proves that the projected method improves the recognition score and also reduces the response time. Keywords : Hearing mechanism, Hearing loss, Hearing implants, Dichotic presentation. I. INTRODUCTION Listening by means of two ears is known as binaural hearing. Presenting same signal to both the ears is known as diotic presentation and presenting two different signals to the two ears is referred to as dichotic presentation. Researches have reported the benefit of binaural dichotic mode of hearing in improving speech perception by persons with sensorineural hearing impairment. Sensorineural hearing impairment is considered by increased threshold of hearing, reduced dynamic hearing range, degraded frequency selectivity and temporal resolution, and increased spectral and temporal masking. The ability to perceptually combine binaurally received signals from two ears improves speech perception under adverse listening conditions. Binaural listening offers better overall sound quality and intelligibility, more relaxed listening, and it also helps in source localization [1]. The splitting of information in speech signal for presenting signals to the two ears, in some sort of a complementary fashion, to provide relaxation for sensory cells of the basilar membrane, may help in reducing the effect of increased masking and thereby improve the speech reception in cases of bilateral sensorineural hearing impairment with some residual hearing [1]. Spectral masking or simultaneous masking is a frequency domain version of temporal masking, and tends to occur in sounds with similar frequencies,a powerful spike will tend to mask out a lower-level tone. It is masking linking two concurrent sounds. Sometimes called frequency masking since it is often observed when the sounds share a frequency band e.g. two sine tones at 460 and 470Hz can be perceived clearly when separated. They cannot be perceived clearly when simultaneous. In masking a sound is made inaudible by a masker, a noise or unwanted sound of the same duration as the original sound [1].The greatest masking is when the masker and the signal are the same frequency and these decreases as the signal frequency moves further away from the masker frequency.This phenomenon is called on-frequency masking and occurs because the masker and signal are within the same auditory filter. The simultaneous masking reduces the frequency resolution significantly, so it is more severe compared to the non-simultaneous masking. The auditory masking occurs because the original neural activity caused by the first signal is reduced by the neural activity of the other sound .The splitting of speech signal into the two channels or bands can be carried out in a number of ways.The objective of our investigation is to split the speech in two bands with complementary spectra on the basis of critical band filtering for binaural dichotic presentation . The study was carried out by processing speech signals of vowel-consonant-vowel syllables for Fifteen English consonants and listening tests were conducted on five hearing subjects with simulated sensorineural hearing loss. II. SPEECH MATERIALS AND SUBJECTS 2.1 The Speech Materials and Subjects Earlier studies have used CV, VC, CVC, and VCV syllables. It has been reported earlier that greater masking takes place in intervocalic consonants due to the presence of vowels on both sides [5]. Since our primary objective is to study improvement in consonantal identification due to reduction in the effect of masking, so VCV syllables are used. For the evaluation of the speech processing strategies,a set of Fifteen syllables of
- 2. International Journal of Current Research In Science, Engineering and Technology www.ijcrset.com 14 | Page consonants / b,d,f,g,k,l,m,n,p,r,s,t,v,y,z / in VCV and vowel /a/ were used in the experiment.Five subjects participated in the listening test. III. PROPOSED SYSTEM The proposed system is designed in order to overcome the masking effects that are caused in human ear due the malfunctioning of the auditory system.Then after the removal of noise the filtered output speech signal iused for automatic speech recognition.The Figure 1 describes the proposed system block diagram. FIGURE 1. PROPOSED SYSTEM I.ALGORITHM The proposed algorithm is divided into two parts first Decomposition by Dyadic AnalysisFilter Bank and secondly Reconstruction by Synthesis of Dyadic Analysis Filter Banks. 3.1 Algorithm For Decomposition by Dyadic Analysis Filter Bank 1]Acquisition of Speech signals (Prerecorded Speech Materials) 2]Decimation by factor D 3]Design of Dyadic Analysis Filter bank with 4]Spectral splitting of recorded speech signal by filter designed in step 3. 5]Obtain a desired number of critical bands with varying bandwidth. 3.2 Algorithm For Reconstruction by Dyadic Synthesis Filter Bank 1]Design of Dyadic Synthesis Filter bank with 2]Reconstruction of obtained bands in Even-Odd Index 3]Interpolation by factor I where I=D 4]Convert reconstructed bands to Right and left speech signal 5]Dichotic presentation of processed right and left speech signal to Right-Left ear 6]Obtain the Hearing test results with Parameters d)Recognition score e)Recognition time IV. RESULTS Results of listening tests for Response time, comparative drop off in response times, Percentage of correct recognition scores and Relative growth in recognition scores obtained with the 3 set of filter which are are presented in the following sections. 4.1 Recognition time Table 1 represents the response times for the unprocessed signals,processed signals using constant band, critical band and 1/3 octave bands for the five subjects DMM, PMN,VRM,RMM and NAR .The Table 2 represents the comparative fall in response time (Processed signal vs. Unprocessed signal).
- 3. International Journal of Current Research In Science, Engineering and Technology www.ijcrset.com 15 | Page Table 1.Recognition Time Subjects Unprocessed signal Processed signal(Constant band) Processed signal(Critical band) Processed signal(1/3 Octave band) APS 2.37 1.30 1.10 1.05 PPP 1.72 2.19 1.98 0.98 VPP 2.47 1.53 2.06 0.87 RMM 3.68 1.74 1.27 0.93 NAR 2.53 1.64 1.13 2.12 Table 2. Comparative Fall in Percentage a. Recognition Scores The recognition scores for unprocessed and processed signal and percentage of comparative growth for processed signal for the five subjects are given in Table 3 and 4.The recognition scores for the 1/3 octave bands showed the better results. Subjects Processed signal(Constant band) Processed signal(Critical band) Processed signal(1/3 Octave band) APS 45.14 53.5 55.6 PPP -27.3 -15.11 43.02 VPP 38.05 16.59 64.77 RMM 52.71 65.48 74.72 NAR 35.17 55.33 16.20
- 4. International Journal of Current Research In Science, Engineering and Technology www.ijcrset.com 16 | Page Table 3.Percentage of Recognition scores Table 4 . Comparative growth in recognition scores V. CONCLUSION During the test Response times and Recognition scores were compaired and analyzed. It is observed that there was a drop off in response time for all the processing schemes compared with unprocessed signal.Statistical and tabular representation show the Response Time,Comparative fall off in Percentage ,Recognition scores and Comparative growth in recognition scores . The degree of improvement was the greatest for processed signal (1/3 octave band) for all the subjects,which reflects that the 1/3 octave band is more efficient to improve the perception of the hearing impaired persons. As stated by the improvements in response time, dichotic presentation also proves better results to improve the perception scores. References [1] Moore, B. C. J,An Introduction to Psychology of Hearing,4th Ed. (Academic, London), 1997. [2] CHABA, Speech-perception aids for hearing-impaired people:Current status and needed research,J. Acoust. Soc. Am. vol. 90, 637–683, 1991. [3] GL Ward ,MD MacAllister,Apparatus and method for conveying amplified sound to ear,Google Patent,1993. Subjec ts Unprocess d signal Processed signal (Constant band) Process d signal (Critica l band) Processed signal(1/3 Octave band) DMM 94.3 97.4 100 100 PMN 86.6 88.7 89.6 93.1 VRM 73.3 86.6 88.4 97.2 RMM 80 92.3 94.2 91.8 NAR 78.3 89.2 92.6 90.6 Subjects Processed signal(Constant band) Processed signal(Critical band) Processed signal(1/3 Octave band) DMM 3.1 5.7 5.7 PMN 2.1 3.0 6.5 VRM 13.3 15.1 23.9 RMM 12.3 14.2 11.8 NAR 10.9 14.3 12.3
- 5. International Journal of Current Research In Science, Engineering and Technology www.ijcrset.com 17 | Page [4] Loizou,P.C., Mani, Arunvijay, and Dorman, M.F. ,Dichotic speech recognition in noise using reduced spectral cues , J. Acoust. Soc. Am. vol. 114(1), 475–483, 2003. [5] Yasu, K., Kobayashi, K., Hishitani, M., Arai, T and Murahara Y.,Critical band based frequency compression for digital hearing aids, J. Acoust. Sci. and Tech. vol. 25, 61–63, 2004. [6] Deniz Baskent, Speech recognition in normal hearing and sensorineural hearing loss as a function of the number of spectral channels,J. Acoust. Soc. Am. vol. 120(5), 2908–2925,2006. [7] Kulkarni, P.N., and Pandey, P.C., Optimizing the comb Filters for spectral splitting of speech to reduce the Effect of spectral masking.IEEE-International Conference on Signal processing, Communications and Networking. Madras Institute of Technology, Anna University Chennai India, Jan 4-6, 69─73, 2008. [8] Daniel M. Rasetshwane, Member, IEEE, Michael P. Gorga, and Stephen T. Neely, Member, IEEE,Signal-Processing Strategy for Restoration of Cross-Channel Suppression in Hearing-Impaired Listeners,IEEE Transactions On Biomedical Engineering, vol. 61, No. 1, January 2014. [9] Chanwoo Kim,Kean K.Chin,Michiel Bacchiani,Richard M. Stern, Robust speech recognition using temporal masking and thresholding algorithm, International Science Congress Association ,ISCA, 2014. [10] Peng Dai , Frank Rudzicz , Ing Yann Soon , Alex Mihailidis, Huijun Ding ,2D Psychoacoustic modeling of equivalent masking for automatic speech recognition,ELSEVIER,2015.