🤖 Neural Networks & Deep Learning
Research on neural architecture search, physics-informed neural networks, generative models, semi-supervised learning, and applications to physical systems.
Neural networks and deep learning have transformed the landscape of artificial intelligence. My research in this area covers the design and optimization of neural architectures through evolutionary and gradient-based methods, the application of physics-informed neural networks to scientific computing, the development of generative models, and the use of neural networks for classification and regression tasks across a variety of domains.
Neural Architecture Search (NAS)
Evolutionary Neural Architecture Search
Development of evolutionary algorithms for automatic design of neural network architectures. Research on how to represent, evaluate, and evolve architectures efficiently, including factorized representations that reduce the search space without sacrificing expressiveness. Analysis of how the choice of variation operators and model components affects the efficiency of neural architecture search.
Multi-network Architectures for Multi-task Learning
Research on the automatic construction of multi-network models for heterogeneous multi-task learning. Development of evolutionary NAS methods that discover how different tasks should share neural network components, including the structure of network connections, the sharing of weights, and the architecture of task-specific heads.
Factorized NAS Representations
Development of factorized model representations for neural architecture search that reduce the search space by decomposing the architecture into independent components. Research on how factorization impacts the computational cost of NAS and the quality of the found architectures, including analysis on convolutional and recurrent network architectures.
Generative Models
Generative Adversarial Networks (GANs)
Research on generative adversarial networks, including their training dynamics, transferability, and optimization using gradient-based and evolutionary methods. Investigation of how GANs can be evolved for Pareto set approximations and analysis of the transferability and robustness of evolved GANs across different problem domains.
Variational Autoencoders (VAEs)
Research on variational autoencoders as generative latent-variable models. Development of multi-task learning architectures (VALPs) that share representations across multiple tasks using VAE-based components. Research on the automatic structural design of these architectures through evolutionary NAS methods.
Physics-Informed Neural Networks (PINNs)
PINNs for Partial Differential Equations
Development and analysis of physics-informed neural networks (PINNs) for solving partial differential equations (PDEs) arising in fluid dynamics. PINNs incorporate physical laws directly into the neural network loss function through automatic differentiation, enabling the network to satisfy both the data and the governing equations.
Generalization of PINNs Outside the Training Domain
Research on how well PINNs generalize to regions outside the training domain (out-of-distribution generalization). Analysis of the hyperparameters that influence PINN generalization, including the size and distribution of collocation points, the network architecture, the activation functions, and the optimization algorithm used for training.
Semi-Supervised Learning
Neuroevolution for Semi-Supervised Classification
Development of neuroevolutionary algorithms guided by neuron coverage metrics for semi-supervised classification. The approach combines evolutionary search with coverage-based objectives inspired by software testing, encouraging the search to explore diverse activation patterns in the network. This improves learning with limited labeled data.
Neural Networks for Physical Systems
Phase Transition Detection with NAS
Application of neural networks and neural architecture search to identify phase transitions in physical systems such as the Ising model. Research on how different neural architectures can detect phase transitions from simulated data and how NAS can be used to automatically find the best architectures for this type of task.
Evolved Neural Networks for Energy Prediction
Application of neuroevolution to design neural networks for building energy prediction. Evolutionary algorithms are used to optimize the architecture and training parameters of neural networks that predict energy consumption in buildings, contributing to intelligent energy management systems.
Random Vector Functional Link Networks
RVFL Forests for Target Recognition
Research on random vector functional link (RVFL) forests and extreme learning forests applied to UAV (unmanned aerial vehicle) automatic target recognition. Evaluation of ensemble methods based on RVFL networks for classification tasks under challenging real-world conditions, including varying imaging angles and noise levels.
Probabilistic Self-Explainable Neural Networks
Development of uncertainty-aware explanations through probabilistic self-explainable neural networks. These models provide calibrated confidence estimates for their predictions and their explanations, improving the reliability and trustworthiness of neural network decisions in high-stakes applications.
Selected Publications
- Lima R, Santana R and Pozo A (2019). Automatic design of convolutional neural networks using grammatical evolution. BRACIS 2019.
- Lima R, Santana R and Pozo A (2021). Automatic Design of Deep Neural Networks Applied to Image Segmentation Problems. Applied Soft Computing.
- Lima R, Santana R and Pozo A (2022). A grammar-based GP approach applied to the design of deep neural networks. GECCO 2022.
- Lima R, Santana R and Pozo A (2020). A symmetric grammar approach for designing segmentation models. BRACIS 2020.
- Garciarena U, Santana R and Mendiburu A (2020). EvoFlow: A Python library for evolving deep neural network architectures in tensorflow. IEEE SSCI 2020.
- Garciarena U, Marti L and Santana R (2020). Analysis of the transferability and robustness of GANs evolved for Pareto set approximations. CEC 2020.
- Montenegro M, Santana R, Bolaños M and Cuayahuitl H (2020). Transfer learning in hierarchical dialogue topic classification with neural networks. IberSPEECH 2020.
- Santana R, Bielza C and Larrañaga P (2013). Changing conduction delays to maximize the number of polychronous groups with an estimation of distribution algorithm. CEC 2013.
- Santana R, Bielza C and Larrañaga P (2012). Maximizing the number of polychronous groups in spiking networks. ESANN 2012.
- LaTorre A, Santana R and Peña JM (2011). A differential evolution algorithm for the detection of synaptic vesicles. CEC 2011.