Deep Sturm–Liouville: Learnable orthogonal basis functions parameterized by neural networks

David Vigouroux (1) , Joseba Dalmau (1) , Louis Béthune (2, 3, 4)

1 IRT Saint Exupéry - Institut de Recherche Technologique
2 IRIT - Institut de recherche en informatique de Toulouse
3 IRIT-ADRIA - Argumentation, Décision, Raisonnement, Incertitude et Apprentissage
4 UT3 - Université Toulouse III - Paul Sabatier

We introduce $\textit{Deep Sturm-Liouville}$ (DSL), a novel function approximator obtained by integrating the Sturm-Liouville theorem (SLT) into the deep learning framework. The Sturm-Liouville theorem deals with a class of eigenvalue problems having a wide range of applications in physics, which motivates us to explore its usage on machine learning tasks. The core idea of our work is to learn a vector field, crossing the input space $\Omega \subset \mathbb{R}^n$, such that the ML task along each of its field lines can be solved more easily due to the regularity of the problem on these field lines. A Sturm-Liouville Problem is solved along each field line to obtain orthogonal basis functions that, combined linearly, form the DSL function approximator. The vector field and the functions appearing in the SLT are parameterized by neural networks and they are learnt simultaneously. We also demonstrate that the DSL formulation appears naturally when solving a Rank-1 Parabolic Eigenvalue Problem. DSL is trained by stochastic gradient descent thanks to the implicit differentiation theorem, achieving comparable performances to neural networks on several multivariate datasets and the $\texttt{MNIST}$ dataset.

Mots clés: Sturm-Liouville theory; Deep Learning; function approximation; orthogonal basis functions; eigenvalues problems; Elliptic eigenvalues problem; Parabolic eigenvalues problem

Domaines: Intelligence artificielle [cs.AI]

David Vigouroux, Joseba Dalmau, Louis Béthune. Deep Sturm–Liouville: Learnable orthogonal basis functions parameterized by neural networks. 2024. ⟨hal-04446268⟩