论文丨Xue-Tao XIE, Yi-Fei PU*, Jian WANG：A Fractional Gradient Descent Algorithm Robust to the Initial Weights of Multilayer Perceptron

本文（A Fractional Gradient Descent Algorithm Robust to the Initial Weights of Multilayer Perceptron）原载Neural Networks，由四川大学蒲亦非教授等科研人员创作，系川大学智慧法治超前部署学科系列学术成果。后续会持续分享四川大学智慧法治超前部署学科系列学术成果，欢迎大家阅读。

Abstract: For multilayer perceptron (MLP), the initial weights will significantly influence its performance. Based on the enhanced fractional derivative extend from convex optimization, this paper proposes a fractional gradient descent (RFGD) algorithm robust to the initial weights of MLP. We analyze the effectiveness of the RFGD algorithm. The convergence of the RFGD algorithm is also analyzed. The computational complexity of the RFGD algorithm is generally larger than that of the gradient descent (GD) algorithm but smaller than that of the Adam, Padam, AdaBelief, and AdaDiff algorithms. Numerical experiments show that the RFGD algorithm has strong robustness to the order of fractional calculus which is the only added parameter compared to the GD algorithm. More importantly, compared to the GD, Adam, Padam, AdaBelief, and AdaDiff algorithms, the experimental results show that the RFGD algorithm has the best robust performance for the initial weights of MLP. Meanwhile, the correctness of the theoretical analysis is verified.

Xue-Tao XIE, Yi-Fei PU*, Jian WANG. “A Fractional Gradient Descent Algorithm Robust to the Initial Weights of Multilayer Perceptron,” Neural Networks, vol. 158, pp. 154-170, 2023.（论文下载）