Older notes on code generation

The following is an outline of how the Brian 2 code generation system works, with indicators as to which packages to look at and which bits of code to read for a clearer understanding.

We illustrate the global process with an example, the creation and running of a single NeuronGroup object:

Parse the equations, add refractoriness to them: this isn’t really part of code generation.
Allocate memory for the state variables.
Create Thresholder, Resetter and StateUpdater objects.
- Determine all the variable and function names used in the respective abstract code blocks and templates
- Determine the abstract namespace, i.e. determine a Variable or Function object for each name.
- Create a CodeObject based on the abstract code, template and abstract namespace. This will generate code in the target language and the namespace in which the code will be executed.
At runtime, each object calls CodeObject.__call__ to execute the code.

Stages of code generation

Equations to abstract code

In the case of Equations, the set of equations are combined with a numerical integration method to generate an abstract code block (see below) which represents the integration code for a single time step.

An example of this would be converting the following equations:

eqs = '''
dv/dt = (v0-v)/tau : volt (unless refractory)
v0 : volt
'''
group = NeuronGroup(N, eqs, threshold='v>10*mV',
                    reset='v=0*mV', refractory=5*ms)

into the following abstract code using the exponential_euler method (which is selected automatically):

not_refractory = 1*((t - lastspike) > 0.005000)
_BA_v = -v0
_v = -_BA_v + (_BA_v + v)*exp(-dt*not_refractory/tau)
v = _v

The code for this stage can be seen in NeuronGroup.__init__, StateUpdater.__init__, and StateUpdater.update_abstract_code (in brian2.groups.neurongroup), and the StateUpdateMethod classes defined in the brian2.stateupdaters package.

For more details, see State update.

Abstract code

‘Abstract code’ is just a multi-line string representing a block of code which should be executed for each item (e.g. each neuron, each synapse). Each item is independent of the others in abstract code. This allows us to later generate code either for vectorised languages (like numpy in Python) or using loops (e.g. in C++).

Abstract code is parsed according to Python syntax, with certain language constructs excluded. For example, there cannot be any conditional or looping statements at the moment, although support for this is in principle possible and may be added later. Essentially, all that is allowed at the moment is a sequence of arithmetical a = b*c style statements.

Abstract code is provided directly by the user for threshold and reset statements in NeuronGroup and for pre/post spiking events in Synapses.

Abstract code to snippet

We convert abstract code into a ‘snippet’, which is a small segment of code which is syntactically correct in the target language, although it may not be runnable on its own (that’s handled by insertion into a ‘template’ later). This is handled by the CodeGenerator object in brian2.codegen.generators. In the case of converting into python/numpy code this typically doesn’t involve any changes to the code at all because the original code is in Python syntax. For conversion to C++, we have to do some syntactic transformations (e.g. a**b is converted to pow(a, b)), and add declarations for certain variables (e.g. converting x=y*z into const double x = y*z;).

An example of a snippet in C++ for the equations above:

const double v0 = _ptr_array_neurongroup_v0[_neuron_idx];
const double lastspike = _ptr_array_neurongroup_lastspike[_neuron_idx];
bool not_refractory = _ptr_array_neurongroup_not_refractory[_neuron_idx];
double v = _ptr_array_neurongroup_v[_neuron_idx];
not_refractory = 1 * (t - lastspike > 0.0050000000000000001);
const double _BA_v = -(v0);
const double _v = -(_BA_v) + (_BA_v + v) * exp(-(dt) * not_refractory / tau);
v = _v;
_ptr_array_neurongroup_not_refractory[_neuron_idx] = not_refractory;
_ptr_array_neurongroup_v[_neuron_idx] = v;

The code path that includes snippet generation will be discussed in more detail below, since it involves the concepts of namespaces and variables which we haven’t covered yet.

Snippet to code block

The final stage in the generation of a runnable code block is the insertion of a snippet into a template. These use the Jinja2 template specification language. This is handled in brian2.codegen.templates.

An example of a template for Python thresholding:

# USES_VARIABLES { not_refractory, lastspike, t }
{% for line in code_lines %}
{{line}}
{% endfor %}
_return_values, = _cond.nonzero()
# Set the neuron to refractory
not_refractory[_return_values] = False
lastspike[_return_values] = t

and the output code from the example equations above:

# USES_VARIABLES { not_refractory, lastspike, t }
v = _array_neurongroup_v
_cond = v > 10 * mV
_return_values, = _cond.nonzero()
# Set the neuron to refractory
not_refractory[_return_values] = False
lastspike[_return_values] = t

Code block to executing code

A code block represents runnable code. Brian operates in two different regimes, either in runtime or standalone mode. In runtime mode, memory allocation and overall simulation control is handled by Python and numpy, and code objects operate on this memory when called directly by Brian. This is the typical way that Brian is used, and it allows for a rapid development cycle. However, we also support a standalone mode in which an entire project workspace is generated for a target language or device by Brian, which can then be compiled and run independently of Brian. Each mode has different templates, and does different things with the outputted code blocks. For runtime mode, in Python/numpy code is executed by simply calling the exec statement on the code block in a given namespace. In standalone mode, the templates will typically each be saved into different files.

Key concepts

Namespaces

In general, a namespace is simply a mapping/dict from names to values. In Brian we use the term ‘namespace’ in two ways: the high level “abstract namespace” maps names to objects based on the Variables or Function class. In the above example, v maps to an ArrayVariable object, tau to a Constant object, etc. This namespace has all the information that is needed for checking the consistency of units, to determine which variables are boolean or scalar, etc. During the CodeObject creation, this abstract namespace is converted into the final namespace in which the code will be executed. In this namespace, v maps to the numpy array storing the state variable values (without units) and tau maps to a concrete value (again, without units). See Equations and namespaces for more details.

Variable

Variable objects contain information about the variable they correspond to, including details like the data type, whether it is a single value or an array, etc.

See brian2.core.variables and, e.g. Group._create_variables, NeuronGroup._create_variables.

Templates

Templates are stored in Jinja2 format. They come in one of two forms, either they are a single template if code generation only needs to output a single block of code, or they define multiple Jinja macros, each of which is a separate code block. The CodeObject should define what type of template it wants, and the names of the macros to define. For examples, see the templates in the directories in brian2/codegen/runtime. See brian2.codegen.templates for more details.

Code guide

This section includes a guide to the various relevant packages and subpackages involved in the code generation process.

codegen

Stores the majority of all code generation related code.

codegen.functions: Code related to including functions - built-in and user-defined - in generated code.
codegen.generators: Each CodeGenerator is defined in a module here.
codegen.runtime: Each runtime CodeObject and its templates are defined in a package here.

core

core.variables: The Variable types are defined here.

equations

Everything related to Equations.

groups

All Group related stuff is in here. The Group.resolve methods are responsible for determining the abstract namespace.

parsing

Various tools using Python’s ast module to parse user-specified code. Includes syntax translation to various languages in parsing.rendering.

stateupdaters

Everything related to generating abstract code blocks from integration methods is here.