基于兴趣度关联规则的海洋气象数据质控算法
2018-11-13李涛张灿张帅弛陆正邦
李涛 张灿 张帅弛 陆正邦
摘 要: 为了提高海洋漂流浮标观测数据的质量,提出一种新的基于兴趣度模型的关联规则挖掘算法。通过该关联规则算法挖掘浮标观测数据,提取出所有关联项对形成范例库,以此构建海洋漂流浮标数据质量控制模型,并与传统数据质量控制方法对比,发现基于新的关联规则算法的质量控制模型在检出率和灵敏度以及性能方面有极大提高,非常具有可行性。通过真实数据验证表明,新算法不仅能够挖掘出所有相关性很强的规则,与同类非Apriori类算法相比,在时间性能上更加优越。
关键词: 海洋漂流浮标; 兴趣度; 关联规则; 挖掘算法; 气象数据; 质量控制
中图分类号: TN011?34; TP391 文献标识码: A 文章编号: 1004?373X(2018)22?0138?05
Abstract: A new association rule mining algorithm based on the interestingness model is proposed to improve the observation data quality of the ocean drifting buoy. The association rule algorithm is adopted to mine the buoy observation data, so as to extract all the correlation pairs to form a sample database, based on which the data quality control model of the ocean drifting buoy is constructed. By comparing with the traditional data quality control methods, it is found that the quality control model based on the new association rule algorithm has improved a lot in detection rate, sensitivity and performance, which is of great feasibility. The results of the real data verification show that the new algorithm can mine all rules with strong correlation, and has more superior time performance than other non?Apriori algorithms of the same class.
Keywords: ocean drifting buoy; interestingness; association rule; mining algorithm; meteorological data; quality control
海洋漂流浮标凭借自身体积小、重量轻、便于投放等一系列特点成为海洋水文观测的重要工具之一。为了确保观测数据的准确性和实用性,对获取的海量观测数据进行科学有效的质量控制必不可少。一般而言,漂流浮标数据的质控内容有时间一致性质控、位置质控、梯度质控、极值控制等。其核心内容是对观测数据中异常值的质控。海洋数据中异常值的检测常抽象为离群数据的搜索和聚类[1]。关于时间序列异常值的判定,比较常用且具有共性特征的是基于经典统计理论的离群点检测方法[2]。该检测方法对单个异常点判断比较准确。而异常值常以连续多个的复杂方式出现,甚至在小段时间内可能超过正常数据的量。……
