[摘要] 目的 为精准、有效、直观地为不同人群提供营养和饮食建议，构建包含食物、营养、人群、疾病等实体的多模态营养知识图谱。方法 利用爬虫等技术手段获取营养领域数据集，借鉴OneRel模型完成中文实体关系联合抽取，构建文本库。使用RoBERTa和ResNet模型分别学习文本和图像数据特征，实现图像与文本的链接，构建多模态知识图谱。结果 实体关系联合抽取模型的F1值为0.703，构建的多模态知识图谱中包含3312个文本实体、11 259条关系、1000张图像实体。结论 本研究构建的多模态营养知识图谱可达到较好的效果，该图谱不仅能系统整合营养领域多模态知识，实现良好可视化查询，也能完成智能问答、营养推荐系统等下游任务的底层支撑。

[关键词] 多模态知识图谱；知识表达；健康饮食

[中图分类号] R151；TP391.1" """"[文献标识码] A""" [DOI] 10.3969/j.issn.1673-9701.2025.17.004

Construction of multi-modal nutritional knowledge graph

CHE Meiling， NAN Jiale， LIN Jianhai， GAO Dongping

Institute of Medical Information， Chinese Academy of Medical Sciences， Peking Union Medical College， Beijing 100020， China

[Abstract] Objective To provide precise， effective， and intuitive nutritional and dietary recommendations for different population groups， a multi-modal nutritional knowledge graph was constructed， which includes entities such as food， nutrition， population， and diseases. Methods Data sets in the field of nutrition were obtained using web crawling and other technical means. The OneRel model was referenced to complete the joint extraction of Chinese entity relationships and construct a text library. The RoBERTa-ResNet model were used to learn the features of text and image data separately， to align images with text， and to construct a multi-modal knowledge graph. Results The F1 value of the joint entity relationship extraction model was 0.703. The constructed multi-modal knowledge graph contains 3312 textual entities， 11 259 relationships， and 1000 image entities. Conclusion The algorithms used in this study to construct the multi-modal nutritional knowledge graph achieve good results. This knowledge graph not only systematically integrates multi-modal knowledge in the field of nutrition and enables good visual query capabilities， but also serves as the underlying support for downstream tasks such as intelligent question answering and nutritional recommendation systems.

[Key words] Multi-modal knowledge graph; Knowledge representation; Healthy diet

饮食是人们生活中的关键一环，与人体的健康状况息息相关。《中国国民营养与慢性病状况报告（2020年）》^[1]指出中国营养改善和慢性病防控工作虽已取得积极进展和明显成效，但膳食不合理问题仍十分突出，营养不足、营养过剩等问题仍需引起关注。健康饮食已成为广大民众追求的目标，但中国注册营养师的数量仍不容乐观，网络中的营养学知识通常是海量且分散的，高质量的营养学知识较难获取，而构建营养领域知识图谱是解决这一问题的有效途径。

知识图谱使用基于图的数据模型，从多元、多结构、多模态的数据中获取并表达知识，可用于知识解释、判断和推理^[2-3]；……