THE Training a Signal-Tracking Neural Network: A Metaheuristic Approach Using Particle Swarm Optimization
DOI:
https://doi.org/10.32870/recibe.v14i3.440Keywords:
Backpropagation, Artificial Neural Networks, Particle Swarm Optimization, Signal Tracking, Function Approximation, Metaheuristic OptimizationAbstract
This paper presents a comparison between two training methods for forward propagation neural networks designed for a time-series signal tracking task. The objective is to adjust the network's synaptic parameters and for the neuron to approximate a reference signal. As an alternative to the traditional backpropagation algorithm, which uses gradient descent to minimize the mean squared error (MSE), we implemented the Particle Swarm Optimization (PSO) algorithm. This is a global search technique that operates without derivatives and performs a simultaneous exploration of the solution space. To evaluate and compare the performance of both methods, 30 runs were conducted for each algorithm. Performance was measured using the final MSE, the success rate (defined as an MSE below 0.005), and the total computational time. Additionally, the solution surface generated by the neural network was analyzed by representing the cost function in terms of two selected weights. This illustrated the presence of multiple local optima and their impact on each algorithm's effectiveness. The results show that while backpropagation tends to converge quickly, it is susceptible to local minima. In contrast, PSO demonstrated greater stability when faced with the non-convex topology of the solution surface, achieving convergence to low-error regions in the majority of evaluated cases. The observations allow us to discuss the advantages and limitations of each approach in contexts where objective functions are non-differentiable.References
Choromanska, A., Bena, M., LeCun, Y., & Arous, G. (2015). The loss surfaces of multilayer networks. Journal of Machine Learning Research, 38, 192–204.
Clerc, M., & Kennedy, J. (2002). The particle swarm – explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58–73.
Eiben, A. E., & Smith, J. E. (2003). Introduction to evolutionary computing. Springer.
Engelbrecht, A. P. (2007). Computational intelligence: An introduction (2nd ed.). John Wiley & Sons.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
Hagan, M. T., Demuth, H. B., Beale, M. H., & De Jesús, O. (2014).Neural network design (2nd ed.). [Disponible en: http://hagan.okstate.edu/NNDesign.pdf]. Martin Hagan.
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366.
Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN’95 – International Conference on Neural Networks, 4, 1942–1948. https://doi.org/10.1109/ICNN.1995.488968
Ljung, L. (1999). System identification: Theory for the user (2nd ed.). Prentice Hall.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536.
Yudong, Z., & Lenan, W. (2015). A survey on swarm intelligence for training artificial neural network. International Journal of Machine Learning and Cybernetics, 6, 257–271.
