MesoNet: learning as an optimal-control problem
synaptic updates as controls? state / objective / constraints first; random tangent: add a small cost per weight change instead of only caring about the endpoint
reading direction: Pontryagin's maximum principle and Hamiltonian-based methods. Proposed interpretation: treat synaptic updates as controls acting on a dynamical neuron/network state.
related tangent: neuromodulators as reward-like signals, and whether that offers a route to adjusting plasticity. The notes also consider ImageNet structure and rate versus temporal encoding as future tasks.
evidence status: exploratory framing, not an implemented optimal-control solver or demonstrated energy-efficient training method.
Conceptual learning regimes
References saved in this journal section
- https://en.wikipedia.org/wiki/Pontryagin%27s_maximum_principle
- https://www.frontiersin.org/files/Articles/1079357/fnins-16-1079357-HTML-r1/image_m/fnins-16-1079357-t004.jpg
- https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.1079357/full
- https://viso.ai/deep-learning/imagenet/
- https://web.stanford.edu/class/cme305/Files/l10.pdf
- questions to define before deriving anything: which variables are the state and control, what is the objective, what constraints bound updates, and what terminal or long-horizon condition matters? Necessary conditions for an optimal trajectory are not automatically an implementable local learning rule.