大数据时代档案信息资源共享平台数据存储系统的设计与实现
2021-04-12卞咸杰
卞咸杰



摘 要:伴随物联网和互联网技术的发展与应用,数字化档案信息资源呈现爆炸式增长,传统的关系型数据库在海量档案信息资源数据读写效率中存在着诸多的性能瓶颈,如何满足不断增长的档案信息资源数据存储需求成为共享平台建设研究的重点。在对档案信息资源共享平台数据存储需求分析的基础上,对档案信息资源存储系统进行了模块设计,提出了采用HDFS结合Redis作为共享平台数据存储系统的总体架构的方法,并对档案信息资源存储的性能进行测试对比。采用HDFS作为共享平台的存储系统架构支撑,结合Redis缓存技术,能够快速提升档案信息资源的存储处理性能,为档案信息资源共享平台的数据服务提供可靠的基础保障。
关键词:大数据;档案信息;资源共享平台;数据存储;HDFS;Redis
分类号:G273
Design and Implementation of Data Storage System for Archive Information Resource Sharing Platform in the Big Data Era
Bian Xiajie(School of History and Public Administration of Yanchen Teachers University, Library of History and Public Administration of Yanchen Teachers University, Archive History and Public Administration of Yanchen Teachers University, Yancheng, Jiangsu, 224007 )
Abstract:With the development and application of Internet of Things and Internet technologies, digital archival information resources have shown explosive growth. Traditional relational databases are reading data from massive archival information resources. There are many performance bottlenecks in writing efficiency. How to meet the growing data storage needs of archive information resources has become the focus of research on the construction of shared platforms. Based on the analysis of the data storage requirements of the archive information resource sharing platform, a module design was carried out for the archive information resource storage system, and a method of using HDFS and Redis as the overall architecture of the shared platform data storage system was proposed Test and compare the performance of archive information resource storage. Using HDFS as the storage system architecture support of the sharing platform, and integrating the Redis cache technology can quickly improve the storage processing performance of the archive information resources, and provide a reliable basic guarantee for the data services of the archive information resource sharing platform.
Keywords:Big Data;Archive Information;Resource Sharing Platform;Data Storage;HDFS; Redis
隨着物联网、互联网、“互联网+”、人工智能和5G技术的发展,档案管理信息化、数字化、智能化建设也得到了前所未有的高度重视。档案部门通过共享平台进行统一存储并对外提供档案信息资源访问数据[1],使迅速发展的档案数据化行为越发呼应当前“让数据多跑路”的政策要求。大数据时代,数据管理已成为现代档案管理的一项重要内容[2],档案数据化也成为档案界新的关注点。存储方案的优化、空间的增容、满足用户不断增长的个性化需求已成为档案信息资源共享研究的重点。为改变传统的档案信息数据存储方式难以承受数据并发访问的压力,在硬件上,使用新硬件或新技术加快数据访问,以更小的硬件设备存储更大容量的档案信息成为趋势,如仅几毫米的空间存储能容纳TB级的数据[3];……
