陈力 费洪晓 李海峰 何嘉宝 谭风云

摘   要：使用深度神经网络处理计算机视觉问题时，在新任务数据量较少情况下，往往会采用已在大数据集上训练好的模型权值作为新任务的初始权值进行训练，这种训练方式最终得到的模型泛化能力更好.对此现象，传统解释大多只是基于直觉分析而缺少合理的数学推导.本文将深度神经网络这种网络结构不变下层间的学习转为深度神经网络内部的迁移能力，并将学习过程变化形式化到数学表达式.考虑数据集对训练过程带来的影响，利用信息几何分析方法，确定不同数据集流形之上的度量和联络，实现不同数据集之间的嵌入映射，同时将参数空间的变化也放入流形空间，探究其对学习过程的共同影响，最终实现对这种内部迁移现象的数学解释.经过分析和实验验证可得内部迁移过程其实是一种能使网络可以在更广空间进行最优搜索的变化，有利于模型可以在学习过程中获得相对的更优解.

关键词：深度学习; 迁移学习; 信息几何

中图分类号：TP183                                  文献标志码：A

Analysis on Information Geometric Measurement

of Internal Transfer of Deep Neural Network

CHEN Li1，2，FEI Hongxiao2，LI Haifeng1？覮，HE Jiabao2，TAN Fengyun2

（1. School of Geosciences and Info-Physics，Central South University，Changsha 410083，China;

2. School of Software Engineering， Central South University， Changsha 410083，China）

Abstract： When deep learning is used to deal with the computer vision tasks， under little number of new task data， the pre-trained model weight based on a very large data is trained as an initial weight to get better generalization ability. At this point， former explanations are based on the intuitive analysis and lack of reasonable mathematical methods. In this paper， deep neural network， which trains on internal layers with fixed structure，changed into internal transfer ability in deep neural network. The changes of learning process are formalized into a mathematical expression. Considering the influence of the data set on the training process， the information geometric analysis method is used to determine the metrics and connections over manifolds of different data sets， which can realize the embedding mapping between different data sets. At the same time， the change of parameter space is also put into a manifold space to explore its common influence on learning process. Finally， a mathematical explanation is provided for the internal transfer phenomenon. Meanwhile， after the analysis and experiments， the process of internal transfer is identified as a change which can make the network search for optimal search in a wider space. Therefore， the model can obtain a relative better solution in learning process.

Key words： deep learning; transfer learning; information geometry

計算机视觉是人工智能非常重要的研究领域，视觉也是人和动物最重要的感觉，至少有80%以上的外界信息经视觉获得.大数据环境下，图像与视频类数据增长速度达到前所未有的高度.然而，Science杂志认为现状可描述为：“data-rich but analysis-poor”[1].如何学习到好的“特征”，一直是计算机视觉中的基础性问题[2].传统图像识别方法中，大多通过设……