分类目录归档：语音识别

cubieboard cubie debian 下用 PocketSphinx 实现智能家居语音控制。

先照着前面几篇文章安装配置好声卡和PocketSphinx ，接着弄好语言模型和词典然后建立下面的03.py文件并执行就可以了。
03.py里用自己路的语音对官方的tdt_sc_8k语音模型做了适应，具体做法可以参考 http://cmusphinx.sourceforge.net/wiki/tutorialadapt 下面附的a.py和b.py是为做适应写的脚本。

实测发现安静环境下语音识别准确率能达到90%以上，但是如果出现其他噪音(其他人的说话声音、电视声音等)识别率就很悲剧。而为了做到全屋的语音识别，却需要把增益开到很大，这就很悲剧了…如果想解决这个问题，只能上麦克风阵列，然后配合傅里叶变换把各个声源的声音独立出来在做语音识别，这样做难度太大，暂时不做这一块。

语言识别相关文章：http://www.chenwang.net/category/%E8%AF%AD%E9%9F%B3%E8%AF%86%E5%88%AB/

03.py
https://gist.github.com/GameXG/647b6c3606a405a47462

#coding:utf-8
import pygtk,gtk,gobject
gobject.threads_init()
import gst
import os
import time
 
 
 
#pipeline=gst.parse_launch('gconfaudiosrc ! audioconvert ! audioresample  ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')
pipeline=gst.parse_launch('alsasrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true   !  pocketsphinx name=asr ! appsink sync=false name=appsink')
#pipeline=gst.parse_launch('pulsesrc ! audioconvert ! audioresample  ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')
#pipeline=gst.parse_launch('pulsesrc  ! tee  name=t ! queue ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink    sync=false name=appsink  t. queue ! audioconvert ! audioresample ! wavenc ! filesink location=o.wav')
 
def result(asr, text, uttid):
    pipeline.set_state(gst.STATE_PAUSED)
    print "================== "+text+" ========================="
    if text == "朵朵 打开 灯":
            os.system("echo 1 > /sys/class/gpio/gpio17_pg9/value")
            os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=已打开灯&tl=zh-CN"')
    elif text == "朵朵 关闭 灯":
            os.system("echo 0 > /sys/class/gpio/gpio17_pg9/value")
            os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=已关闭灯&tl=zh-CN"')
    else:
        #os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=未知命令，'+text+'&tl=zh-CN"')
        pass
    pipeline.set_state(gst.STATE_PLAYING)
 
 
os.system("echo 17 > /sys/class/gpio/export")
time.sleep(1)
os.system("echo out > /sys/class/gpio/gpio17_pg9/direction")
asr=pipeline.get_by_name('asr')
asr.connect('result', result)
asr.set_property('hmm', 'tdt_sc_8kadapt')
asr.set_property('lm', 'TAR3487/3487.lm')
asr.set_property('dict', 'TAR3487/3487.dic')
asr.set_property('configured', True)
pipeline.set_state(gst.STATE_PLAYING)
gtk.main()
 
# cubie@Cubian:~/yysb/yysb/XueXi$ pocketsphinx_continuous -hmm tdt_sc_8k -lm TAR3487/3487.lm -dict TAR3487/3487.dic

#coding:utf-8

import pygtk,gtk,gobject

gobject.threads_init()

import gst

import os

import time

#pipeline=gst.parse_launch('gconfaudiosrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

pipeline=gst.parse_launch('alsasrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

#pipeline=gst.parse_launch('pulsesrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

#pipeline=gst.parse_launch('pulsesrc ! tee name=t ! queue ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink t. queue ! audioconvert ! audioresample ! wavenc ! filesink location=o.wav')

def result(asr, text, uttid):

pipeline.set_state(gst.STATE_PAUSED)

print "================== "+text+" ========================="

if text == "朵朵打开灯":

os.system("echo 1 > /sys/class/gpio/gpio17_pg9/value")

os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=已打开灯&tl=zh-CN"')

elif text == "朵朵关闭灯":

os.system("echo 0 > /sys/class/gpio/gpio17_pg9/value")

os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=已关闭灯&tl=zh-CN"')

else:

#os.system('mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=未知命令，'+text+'&tl=zh-CN"')

pass

pipeline.set_state(gst.STATE_PLAYING)

os.system("echo 17 > /sys/class/gpio/export")

time.sleep(1)

os.system("echo out > /sys/class/gpio/gpio17_pg9/direction")

asr=pipeline.get_by_name('asr')

asr.connect('result', result)

asr.set_property('hmm', 'tdt_sc_8kadapt')

asr.set_property('lm', 'TAR3487/3487.lm')

asr.set_property('dict', 'TAR3487/3487.dic')

asr.set_property('configured', True)

pipeline.set_state(gst.STATE_PLAYING)

gtk.main()

# cubie@Cubian:~/yysb/yysb/XueXi$ pocketsphinx_continuous -hmm tdt_sc_8k -lm TAR3487/3487.lm -dict TAR3487/3487.dic

a.py
https://gist.github.com/GameXG/a7e7d1a426ab60a9d79f

#coding:utf-8
# 根据文本文件生成训练语音模型所需的文件
# gamexg@gmail.com
 
import sys
import codecs
 
file_name=None
file_gname=None
 
if len(sys.argv)==0:
    print u'请提供需要处理的文件'
elif len(sys.argv)==2:
    file_name = sys.argv[1]
else:
    file_name = sys.argv[0]
 
if file_name[-4]=='.':
    file_gname = file_name[:-4]
else:
    file_gname = file_name
 
 
 
#f=codecs.open(file_name,'rb','utf-8')
f=open(file_name,'rb')
s=f.readlines()
f.close()
 
transcription=[]
listoffiles=[]
for i in range(len(s)):
    s[i]=s[i].replace(',',' ').replace('.',' ').replace('?',' ').replace('，',' ').replace('。',' ').replace('？',' ').strip()
    listoffiles.append('arctic_'+str(i+1))
    transcription.append('<s> %s </s> (arctic_%s)'%(s[i].strip(),str(i+1)))
 
listoffiles_file = open(file_gname+'.listoffiles','wb')
listoffiles_file.write('\r\n'.join(listoffiles))  #.encode('utf8'))
listoffiles_file.close()
 
transcription_file = open(file_gname+'.transcription','wb')
transcription_file.write('\r\n'.join(transcription))  #.encode('utf8'))
transcription_file.close()

#coding:utf-8

# 根据文本文件生成训练语音模型所需的文件

# [email protected]

import sys

import codecs

file_name=None

file_gname=None

if len(sys.argv)==0:

print u'请提供需要处理的文件'

elif len(sys.argv)==2:

file_name = sys.argv[1]

else:

file_name = sys.argv[0]

if file_name[-4]=='.':

file_gname = file_name[:-4]

else:

file_gname = file_name

#f=codecs.open(file_name,'rb','utf-8')

f=open(file_name,'rb')

s=f.readlines()

f.close()

transcription=[]

listoffiles=[]

for i in range(len(s)):

s[i]=s[i].replace(',',' ').replace('.',' ').replace('?',' ').replace('，',' ').replace('。',' ').replace('？',' ').strip()

listoffiles.append('arctic_'+str(i+1))

transcription.append('<s> %s </s> (arctic_%s)'%(s[i].strip(),str(i+1)))

listoffiles_file = open(file_gname+'.listoffiles','wb')

listoffiles_file.write('\r\n'.join(listoffiles)) #.encode('utf8'))

listoffiles_file.close()

transcription_file = open(file_gname+'.transcription','wb')

transcription_file.write('\r\n'.join(transcription)) #.encode('utf8'))

transcription_file.close()

b.py
https://gist.github.com/GameXG/397851c979f8dd626edf

#coding:utf-8 
# 根据 zh_broadcastnews_utf8.dic 为当前字典补全音标
# gamexg@gmail.com
 
import sys
import os
 
file_name=None
 
 
if len(sys.argv)==0:
    print '请提供需要处理的文件'
elif len(sys.argv)==2:
    file_name = sys.argv[1]
else: 
    file_name = sys.argv[0]
 
 
f=open(r'D:\Dropbox\yysb\zh_broadcastnews_utf8.dic','rb')
d={}
i=f.readline()
while(i):
	kv=i.split(' ',1)
	if len(kv)==2 :d[kv[0].strip()]=kv[1].strip()
	i=f.readline()
	
os.rename(file_name,file_name+'.bak')
 
def get_dict(name):
    if not name:
        return ''
    for i in range(len(name)):
        if d.has_key(name[:len(name)-i]):
    	    return d[name[:len(name)-i]] + ' ' + get_dict(name[len(name)-i:])
    return '======= ERR ======='
	
	    
 
 
f=open(file_name+'.bak','rb')
wf = open(file_name,'wb')
ws=[]
for i in f.readlines():
    i=i.strip()
    if i:
        kv = i.split(' ',1)
        ws.append('%s %s'%(kv[0],get_dict(kv[0]).strip()))
wf.write('\r\n'.join(ws))
wf.close()

#coding:utf-8

# 根据 zh_broadcastnews_utf8.dic 为当前字典补全音标

# [email protected]

import sys

import os

file_name=None

if len(sys.argv)==0:

print '请提供需要处理的文件'

elif len(sys.argv)==2:

file_name = sys.argv[1]

else:

file_name = sys.argv[0]

f=open(r'D:\Dropbox\yysb\zh_broadcastnews_utf8.dic','rb')

d={}

i=f.readline()

while(i):

kv=i.split(' ',1)

if len(kv)==2 :d[kv[0].strip()]=kv[1].strip()

i=f.readline()

os.rename(file_name,file_name+'.bak')

def get_dict(name):

if not name:

return ''

for i in range(len(name)):

if d.has_key(name[:len(name)-i]):

return d[name[:len(name)-i]] + ' ' + get_dict(name[len(name)-i:])

return '======= ERR ======='

f=open(file_name+'.bak','rb')

wf = open(file_name,'wb')

ws=[]

for i in f.readlines():

i=i.strip()

if i:

kv = i.split(' ',1)

ws.append('%s %s'%(kv[0],get_dict(kv[0]).strip()))

wf.write('\r\n'.join(ws))

wf.close()

cubieboard cubie debian 下安装 PocketSphinx with GStreamer and Python 实现实时语音识别

发表评论

参考之前在ubuntu下安装的记录 http://www.chenwang.net/?p=460 。

sudo apt-get upgrade
sudo aptitude install python-gst0.10

sudo aptitude install libgstreamer0.10-dev
echo sudo aptitude install python2.7-dev
sudo aptitude install python-dev
sudo aptitude  install gstreamer0.10-alsa gstreamer0.10-tools
sudo aptitude install libsamplerate0-dev
sudo aptitude install libsamplerate0-devexit
sudo aptitude install gstreamer0.10-plugins-base  gstreamer0.10-plugins-good
sudo  aptitude install libgstreamer-plugins-base0.10-dev

wget http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F&ts=1385189168&use_mirror=softlayer-dal
mv pocketsphinx-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F  pocketsphinx-0.8.tar.gz
wget http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F&ts=1385189298&use_mirror=softlayer-dal
mv  sphinxbase-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F   sphinxbase-0.8.tar.gz
tar xzf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8/
./configure
apt-cache search python-dev
./configure
make
sudo make install
cd ..
tar xzf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
sudo make install
pocketsphinx_continuous
export  LD_LIBRARY_PATH=/usr/local/lib
export  PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
pocketsphinx_continuous

sudo apt-get upgrade

sudo aptitude install python-gst0.10

sudo aptitude install libgstreamer0.10-dev

echo sudo aptitude install python2.7-dev

sudo aptitude install python-dev

sudo aptitude install gstreamer0.10-alsa gstreamer0.10-tools

sudo aptitude install libsamplerate0-dev

sudo aptitude install libsamplerate0-devexit

sudo aptitude install gstreamer0.10-plugins-base gstreamer0.10-plugins-good

sudo aptitude install libgstreamer-plugins-base0.10-dev

wget http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F&ts=1385189168&use_mirror=softlayer-dal

mv pocketsphinx-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F pocketsphinx-0.8.tar.gz

wget http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F&ts=1385189298&use_mirror=softlayer-dal

mv sphinxbase-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F sphinxbase-0.8.tar.gz

tar xzf sphinxbase-0.8.tar.gz

cd sphinxbase-0.8/

./configure

apt-cache search python-dev

./configure

make

sudo make install

cd ..

tar xzf pocketsphinx-0.8.tar.gz

cd pocketsphinx-0.8

./configure

make

sudo make install

pocketsphinx_continuous

export LD_LIBRARY_PATH=/usr/local/lib

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

pocketsphinx_continuous

Changing the `location’ property on filesink when a file is open is not supported. 错误解决办法

发表评论

程序运行中动态修改filesink.location来修改文件保存位置会提示 Changing the `location’ property on filesink when a file is open is not supported. 错误，需要将filesink的状态设置为NULL，然后才能修改，例子：

        save = pipeline.get_by_name('save')
        save.set_state(gst.STATE_NULL)
        i+=1
        save.set_property('location','save'+str(i)+'.wav')
        save.set_state(gst.STATE_PAUSED)

save = pipeline.get_by_name('save')

save.set_state(gst.STATE_NULL)

i+=1

save.set_property('location','save'+str(i)+'.wav')

save.set_state(gst.STATE_PAUSED)

参考：http://comments.gmane.org/gmane.comp.video.gstreamer.devel/30074

对了，这样修改就像参考里面说的，由于是直接将wav文件截断的，第二个文件没有格式信息，会无法播放。如果想生成的多个wav文件都能正常播放需要将 wavenc 也设置为 gst.STATE_NULL 一次。

linux 下使用 google tts 做语音朗读

发表评论

google tts 文本转语音 python 库：

https://github.com/hungtruong/Google-Translate-TTS

$ python GoogleTTS.py -l zh-CN -f abc中文朗读测试.txt

#生成out.mp3

sudo apt-get install mplayer2
mplayer2 out.mp3

$ python GoogleTTS.py -l zh-CN -f abc中文朗读测试.txt

#生成out.mp3

sudo apt-get install mplayer2

mplayer2 out.mp3

最简单的办法，直接执行

gamexg@ubuntu:~/yysb$ mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=abc中文 朗读测试&tl=zh-CN"
MPlayer2 UNKNOWN (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing http://translate.google.cn/translate_tts?ie=UTF-8&q=abc中文朗读测试&tl=zh-CN.
Resolving translate.google.cn for AF_INET6...
Connecting to server translate.google.cn[2404:6800:4008:c00::a0]: 80...

Failed to connect to server with AF_INET6
Resolving translate.google.cn for AF_INET...
Connecting to server translate.google.cn[203.208.41.148]: 80...

Cache size set to 320 KBytes
Cache fill:  3.76% (12330 bytes)

Detected file format: Audio only
==========================================================================
Requested audio codec family [mpg123] (afm=mpg123) not available.
Enable it at compilation.
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 22050 Hz, 1 ch, floatle, 32.0 kbit/4.54% (ratio: 4000->88200)
Selected audio codec: [ffmp3float] afm: ffmpeg (FFmpeg MPEG layer-3 audio)
==========================================================================
AO: [pulse] 22050Hz 1ch floatle (4 bytes per sample)
Video: no video
Starting playback...
A:   3.0 (03.0) of 3.0 (03.0)  0.1% 0%


Exiting... (End of file)
gamexg@ubuntu:~/yysb$

gamexg@ubuntu:~/yysb$ mplayer "http://translate.google.cn/translate_tts?ie=UTF-8&q=abc中文朗读测试&tl=zh-CN"

mplayer: could not connect to socket

mplayer: No such file or directory

Failed to open LIRC support. You will not be able to use your remote control.

Playing http://translate.google.cn/translate_tts?ie=UTF-8&q=abc中文朗读测试&tl=zh-CN.

Resolving translate.google.cn for AF_INET6...

Connecting to server translate.google.cn[2404:6800:4008:c00::a0]: 80...

Failed to connect to server with AF_INET6

Resolving translate.google.cn for AF_INET...

Connecting to server translate.google.cn[203.208.41.148]: 80...

Cache size set to 320 KBytes

Cache fill: 3.76% (12330 bytes)

Detected file format: Audio only

==========================================================================

Requested audio codec family [mpg123] (afm=mpg123) not available.

Enable it at compilation.

Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders

AUDIO: 22050 Hz, 1 ch, floatle, 32.0 kbit/4.54% (ratio: 4000->88200)

Selected audio codec: [ffmp3float] afm: ffmpeg (FFmpeg MPEG layer-3 audio)

==========================================================================

AO: [pulse] 22050Hz 1ch floatle (4 bytes per sample)

Video: no video

Starting playback...

A: 3.0 (03.0) of 3.0 (03.0) 0.1% 0%

Exiting... (End of file)

gamexg@ubuntu:~/yysb$

cubieboard ubuntu 下安装 PocketSphinx with GStreamer and Python 实现实时语音识别

发表评论

我是在 arm cubieboard 下做的，i386下应该一样。
对了，在实际做之前先看一下 http://www.chenwang.net/2013/11/21/cmu-sphinx-%e8%af%ad%e9%9f%b3%e8%af%86%e5%88%ab%e5%ad%a6%e4%b9%a0%e7%ac%94%e8%ae%b0%ef%bc%881%ef%bc%89/ 文章里面有中文语音识别模型，下面的那个例子使用了中文模型，需要下载下来和 03.py 放到一个目录。

sudo apt-get install  gstreamer0.10-alsa    gstreamer0.10-tools   libgstreamer0.10-0   libgstreamer0.10-dev   python-gst0.10   python-gst0.10-dev   libgstreamermm-0.10-2    libgstreamermm-0.10-2 make gcc  g++  bison  libbison-dev  libasound2-dev libasound2-python python-alsaaudio python-pyalsa  python-gtk2  soundconverter vim

sudo  aptitude install libgstreamer-plugins-base0.10-dev  # 这个事debian上面的，应该一样的。


sudo apt-get install  gstreamer0.10-alsa    gstreamer0.10-tools   libgstreamer0.10-0   libgstreamer0.10-dev   python-gst0.10   python-gst0.10-dev   libgstreamermm-0.10-2    libgstreamermm-0.10-2
sudo apt-get install make gcc  g++
sudo apt-get install  bison
sudo apt-get install  libbison-dev
sudo apt-get install  libasound2-dev libasound2-python python-alsaaudio python-pyalsa
sudo apt-get install  python-gtk2
sudo apt-get install  soundconverter

wget http://downloads.sourceforge.net/project/cmusphinx/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F&ts=1385189168&use_mirror=softlayer-dal
mv pocketsphinx-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F pocketsphinx-0.8.tar.gz
wget http://downloads.sourceforge.net/project/cmusphinx/sphinxbase/0.8/sphinxbase-0.8.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F&ts=1385189298&use_mirror=softlayer-dal
mv  sphinxbase-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F sphinxbase-0.8.tar.gz
tar xzf sphinxbase-0.8.tar.gz
cd sphinxbase-0.8
./configure
make
sudo make install
cd ..
export  LD_LIBRARY_PATH=/usr/local/lib
export  PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
tar xzf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8
./configure
make
sudo make install
pocketsphinx_continuous

echo "export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-0.10/" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=/usr/local/lib/" >> ~/.bashrc

export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-0.10/


linaro@cubieboard:~/yysb$ cat 03.py
import pygtk,gtk,gobject
gobject.threads_init()
import gst

def result(asr, text, uttid): print text


pipeline=gst.parse_launch('alsasrc device=1 ! audioconvert ! audioresample  ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')
#pipeline=gst.parse_launch('alsasrc ! audioconvert ! audioresample  ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')
#pipeline=gst.parse_launch('pulsesrc ! audioconvert ! audioresample  ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')
#pipeline=gst.parse_launch('pulsesrc  ! tee  name=t ! queue ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink    sync=false name=appsink  t. queue ! audioconvert ! audioresample ! wavenc ! filesink location=o.wav')
asr=pipeline.get_by_name('asr')
asr.connect('result', result)
#asr.set_property('hmm', 'zh_broadcastnews_ptm256_8000')
#asr.set_property('lm', '1.lm')
#asr.set_property('dict', '1.dic')
asr.set_property('configured', True)
pipeline.set_state(gst.STATE_PLAYING)
gtk.main()


linaro@cubieboard:~/yysb$ python 03.py

sudo apt-get install gstreamer0.10-alsa gstreamer0.10-tools libgstreamer0.10-0 libgstreamer0.10-dev python-gst0.10 python-gst0.10-dev libgstreamermm-0.10-2 libgstreamermm-0.10-2 make gcc g++ bison libbison-dev libasound2-dev libasound2-python python-alsaaudio python-pyalsa python-gtk2 soundconverter vim

sudo aptitude install libgstreamer-plugins-base0.10-dev # 这个事debian上面的，应该一样的。

sudo apt-get install gstreamer0.10-alsa gstreamer0.10-tools libgstreamer0.10-0 libgstreamer0.10-dev python-gst0.10 python-gst0.10-dev libgstreamermm-0.10-2 libgstreamermm-0.10-2

sudo apt-get install make gcc g++

sudo apt-get install bison

sudo apt-get install libbison-dev

sudo apt-get install libasound2-dev libasound2-python python-alsaaudio python-pyalsa

sudo apt-get install python-gtk2

sudo apt-get install soundconverter

mv pocketsphinx-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fpocketsphinx%2F0.8%2F pocketsphinx-0.8.tar.gz

mv sphinxbase-0.8.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fcmusphinx%2Ffiles%2Fsphinxbase%2F0.8%2F sphinxbase-0.8.tar.gz

tar xzf sphinxbase-0.8.tar.gz

cd sphinxbase-0.8

./configure

make

sudo make install

cd ..

export LD_LIBRARY_PATH=/usr/local/lib

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

tar xzf pocketsphinx-0.8.tar.gz

cd pocketsphinx-0.8

./configure

make

sudo make install

pocketsphinx_continuous

echo "export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-0.10/" >> ~/.bashrc

echo "export LD_LIBRARY_PATH=/usr/local/lib/" >> ~/.bashrc

export GST_PLUGIN_PATH=/usr/local/lib/gstreamer-0.10/

linaro@cubieboard:~/yysb$ cat 03.py

import pygtk,gtk,gobject

gobject.threads_init()

import gst

def result(asr, text, uttid): print text

pipeline=gst.parse_launch('alsasrc device=1 ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

#pipeline=gst.parse_launch('alsasrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

#pipeline=gst.parse_launch('pulsesrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! appsink sync=false name=appsink')

asr=pipeline.get_by_name('asr')

asr.connect('result', result)

#asr.set_property('hmm', 'zh_broadcastnews_ptm256_8000')

#asr.set_property('lm', '1.lm')

#asr.set_property('dict', '1.dic')

asr.set_property('configured', True)

pipeline.set_state(gst.STATE_PLAYING)

gtk.main()

linaro@cubieboard:~/yysb$ python 03.py

好了，现在可以测试了。
识别准确率固定几条命令的话识别率还行，但是会把其他的不相干的话识别成命令。现在正在尝试本地+在线语音识别，但是板子的音源输入只允许单个程序独占很烦人，现在正在考虑解决办法。

配置声卡参考这里：http://www.chenwang.net/2013/11/21/cubieboard-%E5%AE%98%E6%96%B9-ubuntu-%E5%A3%B0%E9%9F%B3%E9%85%8D%E7%BD%AE/

cubieboard 官方 ubuntu 声音配置

发表评论

默认系统将声音输出到HDMI，需要修改 /etc/asound.conf 文件，改成这样

pcm.!default {
  type hw
  card 1
  device 0
}

ctl.!default {
   type hw
   card 1
}

pcm.!default {

type hw

card 1

device 0

}

ctl.!default {

type hw

card 1

}

或者

pcm.!default {
  type hw
  card 0
  device 0
}

ctl.!default {
   type hw
   card 0
}

pcm.!default {

type hw

card 0

device 0

}

ctl.!default {

type hw

card 0

}

这样，才能在 Head Phone & Speaker (正面的音频插口) 输出声音，可以用 speaker-test -twav -c2 命令测试是否有输出。

然后是另一个音源输入的配置，这里有原理图 http://dl.cubieboard.org/hardware/cubieboard_schematic_2012-08-08.pdf 。可以看到音源输入使用了左声道、右声道和地线3根线，在实际电路上可以看到插头一边有三个焊点，一边有两个焊点。在三个焊点旁边有两个贴片电容，那两个电容连接到插头三个焊点靠里面的那两个焊点，这两个焊点就是左声道和右声道输入，剩下的那一个就是地线。需要注意一件事，这个插头是音源输入，而不是麦克风输入，不能直接接麦克风，只能接机顶盒、vcd等设备的音源输出。我这里是直接接到监控用的拾音器上面的。

然后下一步是开启音源输入的静音，输入 $ alsamixer 命令，选到 LineL 和 LineR 分别按一下 M 键，使得上面的 Off 消失。实际测试发现 Mic Inpu 、 MicL 、 MicR 、Master 或者 ADC Inpu 的音量也会影响音源输入，需要用方向键加到最大。下次测试一下到底是哪个对音源输入有影响。实测发现和这些都没有关系，只要确保主声道和输入正常就行，出现几次没有声音的情况是 cubieboard 板子音频插头的问题。。。

然后就可以用

 $ arecord -f CD --device=plug:hw:0 -d 10 test.wav

1	$ arecord -f CD --device=plug:hw:0 -d 10 test.wav

来录一下音，看看是不是正常了。录下来就可以用

  aplay -f CD --device=plug:hw:0 test.wav

1	aplay -f CD --device=plug:hw:0 test.wav

或

  aplay -f CD --device=plug:hw:1 test.wav

1	aplay -f CD --device=plug:hw:1 test.wav

命令来播放一下，看看录音是不是正常，不过由于 cubieboard 的设计问题，弄的音源输入和输出挨得太紧了，很难同时插上，所以我是直接在 windows 下播放的test.wav 文件的。

目前碰到了一个烦人的问题，gst-alsasrc 录音是独占模式，有些坑人了。测试过 ssh ubuntu i386 ，没有发现这个问题，同时开两个语音识别没有问题（必须登登陆图形界面，不然录不到音，可以先启动后登陆）。

现在看看是在gst里面分成两路还是解决这个独占的问题。

附板子的记录：

linaro@cubieboard:~$ lsof | grep pcmC0D0c
python    1115          linaro    8u      CHR     116,24      0t0  34076 /dev/snd/pcmC0D0c
task0     1115 1124     linaro    8u      CHR     116,24      0t0  34076 /dev/snd/pcmC0D0c
audiosrc- 1115 1125     linaro    8u      CHR     116,24      0t0  34076 /dev/snd/pcmC0D0c
alsasrc0: 1115 1126     linaro    8u      CHR     116,24      0t0  34076 /dev/snd/pcmC0D0c

linaro@cubieboard:~$ lsof | grep pcmC0D0c

python 1115 linaro 8u CHR 116,24 0t0 34076 /dev/snd/pcmC0D0c

task0 1115 1124 linaro 8u CHR 116,24 0t0 34076 /dev/snd/pcmC0D0c

audiosrc- 1115 1125 linaro 8u CHR 116,24 0t0 34076 /dev/snd/pcmC0D0c

alsasrc0: 1115 1126 linaro 8u CHR 116,24 0t0 34076 /dev/snd/pcmC0D0c

i386的记录

未登录
gamexg@ubuntu:~/yysb$ lsof | grep pcmC0D0c
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/111/gvfs
      Output information may be incomplete.
已登录
gamexg@ubuntu:~/yysb$ lsof | grep pcmC0D0c
pulseaudi 2293          gamexg  mem       CHR              116,5             9312 /dev/snd/pcmC0D0c
pulseaudi 2293          gamexg   37u      CHR              116,5      0t0    9312 /dev/snd/pcmC0D0c
alsa-sink 2293 2351     gamexg  mem       CHR              116,5             9312 /dev/snd/pcmC0D0c
alsa-sink 2293 2351     gamexg   37u      CHR              116,5      0t0    9312 /dev/snd/pcmC0D0c
alsa-sour 2293 2353     gamexg  mem       CHR              116,5             9312 /dev/snd/pcmC0D0c
alsa-sour 2293 2353     gamexg   37u      CHR              116,5      0t0    9312 /dev/snd/pcmC0D0c

未登录

gamexg@ubuntu:~/yysb$ lsof | grep pcmC0D0c

lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/111/gvfs

Output information may be incomplete.

已登录

gamexg@ubuntu:~/yysb$ lsof | grep pcmC0D0c

pulseaudi 2293 gamexg mem CHR 116,5 9312 /dev/snd/pcmC0D0c

pulseaudi 2293 gamexg 37u CHR 116,5 0t0 9312 /dev/snd/pcmC0D0c

alsa-sink 2293 2351 gamexg mem CHR 116,5 9312 /dev/snd/pcmC0D0c

alsa-sink 2293 2351 gamexg 37u CHR 116,5 0t0 9312 /dev/snd/pcmC0D0c

alsa-sour 2293 2353 gamexg mem CHR 116,5 9312 /dev/snd/pcmC0D0c

alsa-sour 2293 2353 gamexg 37u CHR 116,5 0t0 9312 /dev/snd/pcmC0D0c

CMU Sphinx 语音识别学习笔记（1）

4条回复

为了快速入门，环境先用windows系统。

第一步，到 http://cmusphinx.sourceforge.net/wiki/download/ 直接下载编译好的程序，先把sphinxbase 、pocketsphinx 、sphinxtrain、、beta6 sphinx4 、cmuclmtk 和 https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/ 里面的中文普通话语言模型全下载下来。

在等文件下载完的时候可以先理解一下语音识别的工作方式，对之后自己调整语音识别准确度很有帮助。语音库有3种类型的库，第一个就是 hmm Hidden Markov Model隐马尔可夫模型模型训练算法，可以理解为声音到音标的库；第二个是dict 词典，可以理解为音标到词(文字)的库；最后一个是lm language model 语言模型，可以理解为各个词的使用频率，这个词和另一个词同时出现的几率。语音识别的过程是先把声音转成音标(通过 hmm Hidden Markov Model隐马尔可夫模型)，然后把音标转成词(通过 dict 词典)，由于语音的不确定性(同音字、语音不清晰、语音不标准等原因)会识别出来多重可能结果，然后根据 lm language model 语言模型选择可能性最高的一个作为结果返回，这就完成了语音的识别过程。

第二步，在下载过程中先检查一下麦克风，用windows自带的录音程序录一下音，看看各个距离麦克风的效果，一般的麦克风距离稍微远些声音就很小了，如果想做远距离的声音识别就需要换成监控上面拾音器了。拾音器需要接到音源输入(电脑声卡的蓝色接口)上面，不是接麦克风(声卡的粉色接口)的那个插口。

第三步，把各个文件全部解压出来。为了使用方便，把 pocketsphinx-0.8-win32\pocketsphinx-0.8-win32\bin\Release 路径加到PATH环境变量里面。

第四步，找到之前解压出来的 zh_broadcastnews_ptm256_8000(这是一个目录，里面有 feat.params、 mdef、 means、 mixture_weights 等文件) 、zh_broadcastnews_64000_utf8.DMP 和 zh_broadcastnews_utf8.dic ，把它们放到一个目录里面去，然后执行pocketsphinx_continuous -hmm zh_broadcastnews_ptm256_8000 -lm zh_broadcastnews_64000_utf8.DMP -dict zh_broadcastnews_utf8.dic 就可以测试语音识别效果了。

我没试这个，从网上看其他人的说是效果很不好。我做这个语音识别的目的是为了识别特定命令，所以直接自己生成的语言模型。可以在这里 http://www.speech.cs.cmu.edu/tools/lmtool-new.html 上传文本文件来提供语料资料，然后根据文本内容自动生成词频和用到的词的音标词典。不过这个不支持中文分词和中文音标，需要上传前用空格手工分词，然后打开生成的 *.dic 音标词典文件，自己根据 zh_broadcastnews_utf8.dic 里面的音标手工添加音标。之后执行 pocketsphinx_continuous -hmm zh_broadcastnews_ptm256_8000(或tdt_sc_8k) -lm 你的lm文件 -dict 你的dic文件就可以测试了。

我的语料文件：

打开 浏览器
打开 音乐
打开 记事本
关闭 窗口
关闭 音乐

打开浏览器

打开音乐

打开记事本

关闭窗口

关闭音乐

生成的1675.dic文件内容

窗口 ch uang k ou
打开 d a k ai
关闭 g uan b i
记事本 j i sh ib b en
浏览器 l iu l an q i
音乐 y in uxs uxe

窗口 ch uang k ou

打开 d a k ai

关闭 g uan b i

记事本 j i sh ib b en

浏览器 l iu l an q i

音乐 y in uxs uxe

生成的1675.lm文件内容

Language model created by QuickLM on Wed Nov 20 04:05:27 EST 2013
Copyright (c) 1996-2010 Carnegie Mellon University and Alexander I. Rudnicky

The model is in standard ARPA format, designed by Doug Paul while he was at MITRE.

The code that was used to produce this language model is available in Open Source.
Please visit http://www.speech.cs.cmu.edu/tools/ for more information

The (fixed) discount mass is 0.5. The backoffs are computed using the ratio method.
This model based on a corpus of 5 sentences and 8 words

\data\
ngram 1=8
ngram 2=11
ngram 3=10

\1-grams:
-0.9031 </s> -0.3010
-0.9031 <s> -0.2430
-1.6021 窗口 -0.2430
-1.1249 打开 -0.2553
-1.3010 关闭 -0.2672
-1.6021 记事本 -0.2430
-1.3010 音乐 -0.2430
-1.6021 浏览器 -0.2430

\2-grams:
-0.5229 <s> 打开 0.0000
-0.6990 <s> 关闭 0.0000
-0.3010 窗口 </s> -0.3010
-0.7782 打开 记事本 0.0000
-0.7782 打开 音乐 0.0000
-0.7782 打开 浏览器 0.0000
-0.6021 关闭 窗口 0.0000
-0.6021 关闭 音乐 0.0000
-0.3010 记事本 </s> -0.3010
-0.3010 音乐 </s> -0.3010
-0.3010 浏览器 </s> -0.3010

\3-grams:
-0.7782 <s> 打开 记事本
-0.7782 <s> 打开 音乐
-0.7782 <s> 打开 浏览器
-0.6021 <s> 关闭 窗口
-0.6021 <s> 关闭 音乐
-0.3010 打开 记事本 </s>
-0.3010 打开 音乐 </s>
-0.3010 打开 浏览器 </s>
-0.3010 关闭 窗口 </s>
-0.3010 关闭 音乐 </s>

\end\

Language model created by QuickLM on Wed Nov 20 04:05:27 EST 2013

The model is in standard ARPA format, designed by Doug Paul while he was at MITRE.

The code that was used to produce this language model is available in Open Source.

Please visit http://www.speech.cs.cmu.edu/tools/ for more information

The (fixed) discount mass is 0.5. The backoffs are computed using the ratio method.

This model based on a corpus of 5 sentences and 8 words

\data\

ngram 1=8

ngram 2=11

ngram 3=10

\1-grams:

-0.9031 </s> -0.3010

-0.9031 <s> -0.2430

-1.6021 窗口 -0.2430

-1.1249 打开 -0.2553

-1.3010 关闭 -0.2672

-1.6021 记事本 -0.2430

-1.3010 音乐 -0.2430

-1.6021 浏览器 -0.2430

\2-grams:

-0.5229 <s> 打开 0.0000

-0.6990 <s> 关闭 0.0000

-0.3010 窗口 </s> -0.3010

-0.7782 打开记事本 0.0000

-0.7782 打开音乐 0.0000

-0.7782 打开浏览器 0.0000

-0.6021 关闭窗口 0.0000

-0.6021 关闭音乐 0.0000

-0.3010 记事本 </s> -0.3010

-0.3010 音乐 </s> -0.3010

-0.3010 浏览器 </s> -0.3010

\3-grams:

-0.7782 <s> 打开记事本

-0.7782 <s> 打开音乐

-0.7782 <s> 打开浏览器

-0.6021 <s> 关闭窗口

-0.6021 <s> 关闭音乐

-0.3010 打开记事本 </s>

-0.3010 打开音乐 </s>

-0.3010 打开浏览器 </s>

-0.3010 关闭窗口 </s>

-0.3010 关闭音乐 </s>

\end\

另附一点资料：
hmm Hidden Markov Model隐马尔可夫模型模型训练算法
个人理解为声音和音标的关联
中文的有 tdt_sc_8k 和 zh_broadcastnews_ptm256_8000 ，是一个目录。

lm language model
lm模型是统计语料得到的模型，lm模型里存的是语料的组合概率。个人理解为这个是存放的那个词比较常用，那个词在另一个词后面的几率更高。
zh_broadcastnews_64000_utf8.DMP 是从网下下的语言模型，在线生成的语言模型的扩展名是 lm 。

dict 词典
音标和词之间的关联。
zh_broadcastnews_utf8.dic 是从网上下的词典。在线生成的扩展名也是 dic 。

Pocketsphinx — 用C语言编写的轻量级识别库
Sphinxbase — Pocketsphinx所需要的支持库
Sphinx3 — 为语音识别研究用C语言编写的解码器
CMUclmtk — 语言模型工具
Sphinxtrain — 声学模型训练工具

附一下在线生成语言模型和词典的工具 http://www.speech.cs.cmu.edu/tools/lmtool-new.html

本文参考了 http://www.cnblogs.com/huanghuang/archive/2011/07/14/2106579.html 和 http://www.cnblogs.com/yin52133/archive/2012/06/21/2557219.html 写的，如果有不明白的地方建议翻一下上面两个博文系列。