| 韩旭.基于Transformer-CRF 的文言文断句方法研究——以唐代墓志铭为例[J].情报工程,2021,7(5):030-039 |
| 基于Transformer-CRF 的文言文断句方法研究——以唐代墓志铭为例 |
| Research on Sentence Segmentation of Classical Chinese Based on Transformer-CRF: Taking Epitaph of Tang Dynasty as An Example |
| |
| DOI：10.3772/j.issn.2095-915X.2021.05.003 |
| 中文关键词: Transformer-CRF；繁体字向量；句子边界识别；古籍信息处理 |
| 英文关键词: Transformer-CRF; traditional Chinese character vector; sentence boundary recognition; Ancient Chinese information processing |
| 基金项目:中国科学技术信息研究所创新研究基金面上项目（MS2021-04），中国科学技术信息研究所重点工作（ZD2021-09）。 |
| 作者 | 单位 | | 韩旭 | 中国科学技术信息研究所 北京 100038 |
|
| 摘要点击次数: 1689 |
| 全文下载次数: 1575 |
| 中文摘要: |
| [ 目的 / 意义 ] 本文探索了文言文的断句规则，并以唐代墓志铭为例进行基于语义的句子边界识别，降低了文言文因缺少标点造成的阅读障碍，减少了人工标注标点的工作量，为中国古文的资料分析整理提供技术支撑。[ 方法 / 过程 ] 本文首先使用一种基于汉字偏旁的字表示方法，提取汉字本身隐含的语义信息进行表达。将基于偏旁的字表示输入 Transformer-CRF 模型，并对墓志铭中的缺失字进行了滑动窗口填补操作，降低缺失字对整体模型的影响。该模型在提高并行计算效率的基础上对输出结果进行关联，提高了准确率。[ 结果 / 结论 ] 实验表明，使用基于偏旁的字向量表示方式结合Transformer-CRF 能提升唐代墓志铭的断句准确率，且对于缺失字附近的断句准确率有所提升，该方法对数字人文中信息收集和整理工作起到了一定的辅助支撑作用。 |
| 英文摘要: |
| [Objective/ Significance] In this paper, we explore the rules of sentence segmentation in classical Chinese, and take the epitaph of Tang Dynasty as an example to identify the sentence boundary based on semantics. This method reduces the reading obstacles caused by the lack of punctuation in classical Chinese, reduces the workload of manual punctuation, and provides technical support for the collation and analysis of ancient Chinese information. [Methods/Process] Firstly, this study uses a character representation method based on Chinese character radicals to extract the implied semantic information of Chinese characters. The word representation based on radical is input into Transformer-CRF model, which improves the efficiency of parallel computing and correlates the output results to improve the accuracy. In addition, the missing words in the epitaph are filled by the sliding window to reduce the impact of missing words on the overall model. [Results/Conclusions] The experimental results show that character representation based on radicals combined with Transformer-CRF model can improve the accuracy of sentence segmentation of Tang Dynasty Epitaphs, and improve the ability of sentence segmentation near missing characters. This method plays a supporting role in information collection and collation in Digital Humanities. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
