Introduction to Neural Networks

Programming Lab 1: Setup, Introduction, and Simulation


Installation

Please download the Anaconda distribution for your operating system. We will be using the Python 3.7 version for work in this class. The file size is 654 MB for the graphical installer - this should take approximately 2 minutes on the UWO internet interface. Detailed installation instructions, including troubleshooting, are available at the Anaconda site. We will reserve time in class for setting up the coding environment.

After installation, you will need to ensure your environment is correctly configured for Anaconda. Please check that you can run the conda command and that running python at the command line returns a header that looks like this:

Python 2.7.10 |Anaconda 2.3.0 (64-bit)| (default, May 28 2015, 17:02:03)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Once everything is correctly installed, type ipython at the command line to launch an interactive Python environment.


What are Python and NumPy?

Python is a scripting language that works across all major platforms (Windows, Linux, OSX). NumPy is a Python module for scientific computing whose focus is on arrays - orderly arrangements of elements in 1, 2, 3, or higher dimensions. To explain what this means, let's import NumPy

import numpy as np

and create a variable "a" in the workspace

a = np.array( [3] )

We can see the size of the array using this command:

np.shape( a )

and we can access its value by typing "a" at the command prompt:

a

Next, we can apply functions to "a" in order to make calculations. To do this, we can type something like:

3*a + 5

What output did you get?


Arrays in NumPy

Now, instead of an array with just a single number (of size 1), we can work with a vector, or a one-dimensional array. To create a vector "a" with 5 elements, we can use an initialization command:

a = np.zeros( 5 )

We can access different elements and store values in the array through a special set of symbols:

a[0] = 2; a[1] = 6; a[2] = 4; a[3] = 8; a[4] = 24

What do you get when you type "a" now at the command prompt? What will happen if you type a[1]?

We can also work with 2-dimensional (and higher) arrays in NumPy. Let's create a 5 x 5 array "a" with zeros in two dimensions:

a = np.zeros( (5,5) )

In this line, we have used a tuple in the call to np.zeros to initialize the array. Now, let's take a look at this 2-dimensional array (or matrix) by typing "a" at the command prompt. Can you see how vectors of values would be useful in simulating a dynamical system, or how higher dimensional arrays would be useful in simulating systems with units arranged on a regularly spaced grid?


Logistic Map

One of the simplest mathematical systems that can exhibit chaotic behavior is the "logistic map", which is defined by the equation:

\begin{equation} x_{n+1} = a x_n ( 1 - x_n ) \tag{1}\label{lm} \end{equation}

with \(x_n \in \mathbb{R}\) and \(n \in \mathbb{N}\). Under repeated applications of this mapping, it can be shown that \(x_n \in [0,1]\, \forall n\) given \(a \in (0,4]\) and \(x_0 \in [0,1]\). This is a discrete-time model of the differential growth introduced almost two centuries ago to model the dynamics of biological populations (Verhulst, 1838). This type of system is called an "iterative map", in which the value at a discrete point in time (e.g. n = 1, 2, ..., 99, 100) is mapped to the next point by relation \ref{lm} above. Because of the nonlinearity, this very simple iterative map is capable of complicated behavior. In particular, it can exhibit chaotic behavior above a critical value of the parameter \(a\). For more background, please see the excellent explanation videos from Numberphile and Melanie Mitchell.

While this system leads to very interesting mathematics, simulating its behavior is quite straightforward. We can implement a simulation of the logistic map by using the code below, starting with defined parameters, initializing an array of zeros "x" with the initial value "x0" in x[0], repeatedly applying the logistic map in a for loop, and then plotting the results. For different values in the range \((0,4]\), the behavior will change dramatically. What is this difference, and where does this change occur?

import numpy as np
import matplotlib.pyplot as plt # parameters a = 4; x0 = 0.6; nsteps = 100; # init x = np.zeros( nsteps ); x[0] = x0; # run simulation for ii in np.arange(1,nsteps): x[ii] = a * x[ii-1] * ( 1 - x[ii-1] ) # plot results plt.plot( x ); plt.xlabel( 'Time steps' ); plt.ylabel( 'x' )