1.Learning the knowledge of language model
2.Segment the training/test sentences with the 150k lexicon and get the ppl of the
test: /nfs/home/zhangzhiyong/work/train_470h/test/huawei_disanpi.txt, using the following LM:
3.Build the new LM using the lexicon with the keywords involved. Re-segment the test files, and then test the PPL.
1.To extracte the related sentences of this filed from the original corpus