فرايادگيري در فضاهاي ريماني

Meta-learning is a new field of machine learning that allows computers to perform the learning task with a small number of observations and take advantage of past experiences gained from solving other learning tasks. Since many machine-learning problems are modeled and solved in Riemannian spaces, it is necessary to provide meta-learning methods in non-Euclidean spaces. In this research, a framework for Riemannian meta-learning is presented, which is based on a bi-level constrained optimization problem. We also investigate the effect of using the orthogonality constraints on the parameters of the artificial neural networks, equivalent to meta-learning on the Stifel manifold. To implement the proposed method, we have developed a Riemannian optimization method entitled cAdam, an extension of the well-known Euclidean optimization method, Adam. The meta-learning problem includes a two-level optimization problem that is heavy in terms of runtime, computation, and video memory consumption, Because it requires solving two nested optimization problems and calculating tensors of second-order derivatives. This challenge will be greater in Riemannian meta-learning because complex nonlinear operators are used for optimization in that space. Therefore, a first-order method is also presented. Our proposed method, FORML, does not require calculating the second-order derivatives. Also, the proposed method suggests that in addition to using the orthogonality constraint on the parameter matrix of the classifier layer, the input data of that layer should also be normalized, leading to a smoother optimization, decreasing the intra-class variance, and preventing the model from overfitting. Another challenge in meta-learning is dealing with tasks that are sampled from a complex and multimodal distribution. To solve this challenge, we present a modulation-based method that uses the Stifel manifold to generate the modulation coefficients. All three proposed methods have been eva‎luated by extensive few-shot learning experiments. The results of the first presented method show that the few-shot classification accuracy has increased up to 9 compared to its Euclidean counterpart, and it has obtained superior results compared to other sota methods. Also, for the first-order method, the results are competitive compared to its first-order Euclidean counterpart and outperforms other competing methods. In terms of video memory consumption and execution time, FORML shows a significant improvement (12x faster) compared to its non-approximated version, RMAML. In addition, although it uses Riemannian operations, which are computationally complex, it is competitive with the first-order and second-order Euclidean methods, in terms of memory consumption and execution time. The third proposed method for multi-modal meta-learning also has better classification accuracy than its competitors.

7

77255