Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

1.2.3 Decoding

Authors
Affiliations
Virginia Tech, USA
Technical University of Denmark
Innatera

As described in the preface of What is Encoding & Decoding?, Decoding is about extracting meaningful information from spikes. Depending on the context of the problem statement and the information required, one can employ different decoding methodologies, or even design their own. The two most common decoding approaches in SNN literature are Rate Decoding and Temporal Decoding. Note that when it comes to training SNNs, the choice of decoding methodology is strongly interrelated with the choice of loss function for training SNNs; we expand on this in the sections below. However, SNN training methodologies are covered in greater detail later in the chapters of Topic 2.

1.2.3.1Rate Decoding

Rate Decoding is analogous to Rate Encoding in that information is decoded from the average firing rate of a neuron. Thus, to retrieve information from a spike train generated by a neuron, one can take the average of those spikes over time and assign some meaning to it. For example, a higher spike count / average indicates higher activation of the neuron, i.e., stronger evidence of the meaning assigned to it.

The formula for Rate Decoding is very simple:

ri=t=1TSi[t]Tr_{i} = \frac{\sum\limits_{t=1}^{T} S_{i}[t]}{T}

where ii is the index of the neuron, and TT is the discrete-time duration of the simulation.

Therefore, for classification tasks, if Rate Encoding is used in an SNN to encode continuous-valued input data as binary spikes, each output neuron of the (output layer of an) SNN can be assigned to a class, and the average firing rate rir_{i} of each output neuron can be computed as in Eq (1). The neuron with the highest firing rate can be considered to denote the final / predicted output class.

1.2.3.2Spike Trace

Computing Spike Traces is a popular method for smoothing out the spike trains, thereby obtaining continuous values from discrete (binary) spikes. This method is also called synaptic filtering, as it is conceptually similar to first-order Low-Pass Filter. Synaptic filtering is grounded in neuroscience, where upon arrival of action potentials (a.k.a. spikes) at a synapse, it triggers the release of neurotransmitters, causing a sudden jump in the postsynaptic current, which then decays exponentially over time. This behavior can be modeled by a first-order linear differential equation:

τzdz(t)dt=z(t)+w.S(t)\tau_{z}\frac{dz(t)}{dt} = -z(t) + w.S(t)

where z(t)z(t) is the filtered synaptic output (i.e., trace), τz\tau_{z} is the synaptic time-constant that dictates how fast the trace decays, and ww is the weight (the amplitude of the jump caused by a single spike); note the similarity of Eq. (2) to the Eq. (3) of LIF neuron. Also note that in the context of decoding output layer spikes - by means of computing spike traces - the contribution of the weight ww is typically ignored (i.e., set to 1). Although ww can also be accounted for as a static / tunable / trainable parameter, in this context it is different from a synaptic weight because there are no synapses past the output layer (where decoding happens).

Discretizing Eq. (2) gives the following:

z[t]=αz[t1]+(1α)S[t]z[t] = \alpha z[t-1] + (1-\alpha) S[t]

where α=eΔtτz[0,1)\alpha = e^{-\frac{\Delta t}{\tau_{z}}} \in [0, 1) (ww from Eq. (2) is assumed to be 1).

In the context of Eq. (3), note that when spikes S[t]S[t] arrive at a high frequency, i.e., faster than the time constant τz\tau_{z} can decay the trace z[t]z[t], the incoming spikes accumulate. That is, the high-frequency “switching” noise of the 0-or-1 binary events is filtered out, leaving a smooth analog signal z[t]z[t] that reflects the mean firing rate of the neuron. Conversely, if the spikes arrive far apart, i.e., at a low frequency, the trace z[t]z[t] has the time to decay (possibly to zero) between the events; however, even sporadic incoming spikes can cause a sharp bump in the trace. Since the spike trace z[t]z[t] is representative of the mean firing rate, it can also be used to compute loss and infer meaning, i.e., decode the SNN’s output.