Publicado:

2022-06-24

Número:

Vol. 19 Núm. 1 (2022): Revista Tekhnê

Sección:

Artículos

How to choose an activation function for deep learning

Cómo elegir una función de activación para el aprendizaje profundo

Autores/as

  • Albert I. Rodríguez P. Universidad Distrital Francisco José de Caldas
  • Xiomara D. Buitrago R. Universidad Distrital Francisco José de Caldas

Palabras clave:

Activation function, deep learning, neural network, nonlinearity (en).

Palabras clave:

Aprendizaje profundo, función de activación, no linealidad, red neuronal (es).

Resumen (en)

Activation functions are important in each layer of the neural network because they allow the network to learn complex relationships between the input data and the output data. They also introduce nonlinearity into the network, which is essential for learning patterns in data. Activation functions play a critical role in the training and optimization of deep learning models, and choosing the right activation function can significantly impact the model’s performance. This article presents a summary of the features of these functions.

 

Resumen (es)

Las funciones de activación son importantes en cada capa de la red neuronal porque permiten a la red aprender relaciones complejas entre los datos de entrada y los de salida. También introducen la no linealidad en la red, que es esencial para aprender patrones en los datos. Las funciones de activación desempeñan un papel fundamental en el entrenamiento y la optimización de los modelos de aprendizaje profundo, y la elección de la función de activación adecuada puede influir significativamente en el rendimiento del modelo. Este artículo presenta un resumen de las características de estas funciones.

 

Biografía del autor/a

Albert I. Rodríguez P., Universidad Distrital Francisco José de Caldas

 

 

Xiomara D. Buitrago R., Universidad Distrital Francisco José de Caldas

 

 

Referencias

Alonso, A., Peña, A., & Martínez, F. (2021). Autonomous identification of high-contact surfaces from convolutional neural networks. Journal of Physics: Conference Series, 2135(1), 012001.

Brownlee, J. (2021). Machine learning Mastery. https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/

Chang, Y.-W., & Tsai, C.-Y. (2017). Apply deep learning neural network to forecast number of tourists. 2017 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA). https://doi.org/10.1109/waina.2017.125

Deng, L., Hinton, G., & Kingsbury, B. (2013). New types of deep neural network learning for speech recognition and related applications: An overview. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1109/icassp.2013.6639344

Ibrahim, M. F. I., & Al-Jumaily, A. A. (2018). Auto-encoder based deep learning for surface electromyography signal processing. Advances in Science, Technology and Engineering Systems Journal, 3(1), 94–102. https://doi.org/10.25046/aj030111

Jacinto, E., Martínez, F., & Martínez, F. (2022). Performance evaluation of temporal and frequential analysis approaches of electromyographic signals for gestures recognition using neural networks. International Journal of Advanced Computer Science and Applications, 13(3), 1–8.

Kim, J.-Y., & Cho, S.-B. (2019). Evolutionary optimization of hyperparameters in deep learning models. 2019 IEEE Congress on Evolutionary Computation (CEC). https://doi.org/10.1109/cec.2019.8790354

Maguolo, G., Nanni, L., & Ghidoni, S. (2019). Ensemble of convolutional neural networks trained with different activation functions. arXiv, 1–13.

Martínez, F., Hernández, C., & Rendón, A. (2017). A study on machine learning models for convergence time predictions in reactive navigation strategies. Contemporary Engineering Sciences, 10(25), 1223–1232.

Martínez, F., Martínez, F., & Jacinto, E. (2020). Performance evaluation of the nasnet convolutional network in the automatic identification of covid-19. International Journal on Advanced Science, Engineering and Information Technology, 10(2), 662.

Martínez, F., Martínez, F., & Montiel, H. (2020). Low cost, high performance fuel cell energy conditioning system controlled by neural network. TELKOMNIKA (Telecommunication Computing Electronics and Control), 18(6), 3116–3122.

Martínez, F., Montiel, H., & Martínez, F. (2022). A machine learning model for the diagnosis of coffee diseases. International Journal of Advanced Computer Science and Applications, 13(4), 1–8.

Martínez, F., Penagos, C., & Pacheco, L. (2020). Scheme for motion estimation based on adaptive fuzzy neural network. TELKOMNIKA (Telecommunication Computing Electronics and Control), 18(2), 1030–1037.

Montiel, H., Jacinto, E., & Martínez, F. (2021). A double-loop hybrid approach for the recognition of fissures in bone structures. ARPN Journal of Engineering and Applied Sciences, 16(11), 1151–1156.

Montiel, H., Martínez, F., & Jacinto, E. (2017). Visual patterns recognition in robotic platforms through the use of neural networks and image processing. International Journal of Applied Engineering Research, 12(18), 7770–7774.

Pomerat, J., Segev, A., & Datta, R. (2019). On neural network activation functions and optimizers in relation to polynomial regression. 2019 IEEE International Conference on Big Data (Big Data). https://doi.org/10.1109/bigdata47090.2019.9005674

Ramachandran, P., Zoph, B., & Le, Q. V. (2017). Searching for activation functions. arXiv, 1–13.

Rendón, A., & Martínez, F. (2021). Intelligent sensor for thermal process control using convolutional neural network. Journal of Physics: Conference Series, 1993(1), 012027.

Rendón, A., Martínez, F., & Hernández, C. (2017). Deep regression model for predictive control in a vegetable waste carbonization plant. Contemporary Engineering Sciences, 10(21), 1047–1055.

Szandala, T. (2021). Bio-inspired neurocomputing (A. K. Bhoi, P. K. Mallick, C.-M. Liu, & V. E. Balas, Eds.; Vol. 903). Springer Singapore. https://doi.org/10.1007/978-981-15-5495-7

Zhu, D., Wu, X., & Yang, T. (2022). Benchmarking deep auroc optimization: Loss functions and algorithmic choices. arXiv, 1–32.

Zhu, H., Yang, X., & Wang, Y. (2018). Prediction of daily entrance and exit passenger flow of rail transit stations by deep learning method. Journal of Advanced Transportation, 2018(1), 1–11. https://doi.org/10.1155/2018/6142724

Zhu, X., Li, J., Zhu, M., Jiang, Z., & Li, Y. (2018). An evaporation duct height prediction method based on deep learning. IEEE Geoscience and Remote Sensing Letters, 15(9), 1307–1311. https://doi.org/10.1109/lgrs.2018.2842235

Cómo citar

APA

Rodríguez P., A. I., y Buitrago R., X. D. (2022). How to choose an activation function for deep learning. Tekhnê, 19(1), 23–32. https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337

ACM

[1]
Rodríguez P., A.I. y Buitrago R., X.D. 2022. How to choose an activation function for deep learning. Tekhnê. 19, 1 (jun. 2022), 23–32.

ACS

(1)
Rodríguez P., A. I.; Buitrago R., X. D. How to choose an activation function for deep learning. Tekhnê 2022, 19, 23-32.

ABNT

RODRÍGUEZ P., Albert I.; BUITRAGO R., Xiomara D. How to choose an activation function for deep learning. Tekhnê, [S. l.], v. 19, n. 1, p. 23–32, 2022. Disponível em: https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337. Acesso em: 28 mar. 2024.

Chicago

Rodríguez P., Albert I., y Xiomara D. Buitrago R. 2022. «How to choose an activation function for deep learning». Tekhnê 19 (1):23-32. https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337.

Harvard

Rodríguez P., A. I. y Buitrago R., X. D. (2022) «How to choose an activation function for deep learning», Tekhnê, 19(1), pp. 23–32. Disponible en: https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337 (Accedido: 28 marzo 2024).

IEEE

[1]
A. I. Rodríguez P. y X. D. Buitrago R., «How to choose an activation function for deep learning», Tekhnê, vol. 19, n.º 1, pp. 23–32, jun. 2022.

MLA

Rodríguez P., Albert I., y Xiomara D. Buitrago R. «How to choose an activation function for deep learning». Tekhnê, vol. 19, n.º 1, junio de 2022, pp. 23-32, https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337.

Turabian

Rodríguez P., Albert I., y Xiomara D. Buitrago R. «How to choose an activation function for deep learning». Tekhnê 19, no. 1 (junio 24, 2022): 23–32. Accedido marzo 28, 2024. https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337.

Vancouver

1.
Rodríguez P. AI, Buitrago R. XD. How to choose an activation function for deep learning. Tekhnê [Internet]. 24 de junio de 2022 [citado 28 de marzo de 2024];19(1):23-32. Disponible en: https://revistas.udistrital.edu.co/index.php/tekhne/article/view/20337

Descargar cita

Visitas

17

Descargas

Los datos de descargas todavía no están disponibles.
Loading...