PGR Computing Programming Skills

PGR Computing Programming Skills Dr. I. Hawke 2008 1 Introduction The purpose of computing is to do something faster, more efficiently and more reliably than you could as a human do it. One obvious point that is frequently lost is that there is nothing a computer can do that you, as a human cannot; the computer will just do it so much faster. Sooner or later you will find that no standard package will perform a task that you want to do. At which point, you can either fill in the gaps by hand, or you can try to make the computer do what you want. Any attempt at the latter is essentially programming; it is only useful if it saves you time. Because it takes time to learn how to programme and how to make a computer do what you want, nearly all programming tasks worth doing are 1. repetitive 2. time consuming 3. something you want done more than once. The last point is key, as although you may want the computer to do something more than once, it may be days, or months, or longer, between the times when you do the task for the first and second time. Why is this important? Because you will have forgotten how you did it the first time, so unless you have explained your code properly, you will have to start from scratch. This document gives you some ideas of how to think about programming. It will focus on some simple, mathematical examples. Some of the text is specific to MATLAB, but the theory should carry over to any computing task. 1

2 TACTICS 2 Tactics This section will focus on concrete small details that can massively ease the burden of reading and maintaining code. The aim should be that if you follow these guidelines in spirit, if not detail, then a somebody should be able to use and understand your code without any input from you. 2.1 Documentation Let us start with a simple example. The following MATLAB function should be very simple to understand: function c = MyDumbFunction(a, b) c = a / b; end When saved in the file MyDumbFunction.m this function can be called; it will return the result of the first number divided by the second. Example 1. Download this function from the website. Save the function in a place you can access. Launch MATLAB and ensure that the working directory contains this file (to change working directory, click the button with three dots... in the toolbar to the right of Current Directory ). Try calling the function with some numbers. For example, in the Command Window try MyDumbFunction(1, 2) Check that you get the expected answer for each case. 1 You can define vectors in MATLAB in a simple fashion, such as A = [1 2 3] B = [3 4 5] Try calling the function with a vector in an argument, such as MyDumbFunction(A, 2) MyDumbFunction(A, B) MyDumbFunction(1, A) 1 Note that if you get bored of typing MyDumbFunction all the time then MATLAB has a name completion utility. Try typing the first couple of letters of the function and pressing the TAB key. 2

2 TACTICS 2.1 Documentation Can you explain the results? Defining a matrix in MATLAB is also simple: C = [1 2; 3 4] Experiment and see when and how the matrix can be used as an argument to this function. Finally, try MyDumbFunction(1, 0) What do you think the function should do in this case? By the end of the previous example you should be very bored of typing MyDumbFunction. More to the point, you will probably have almost forgotten what this function actually does. The name of the function gives you no idea as to its purpose. The file contains no information, and no useful help statements. The only way of understanding the purpose of this function is to read and understand the code. As the example above showed, there can be pitfalls in calling even the simplest code that you might not immediately think of. If you can start to lose any idea of the purpose and pitfalls of a simple function after just a few minutes, then you can easily imagine the problem with complex functions over days, weeks or months. The answer to this is documentation. Documentation starts with having sensible names for functions and variables, and continues by including easy-to-find help statements and code comments. Example 2. Modify the function MyDumbFunction so that the function and variables have sensible names. Once you have done that we want to include helpful comments. In the MAT- LAB window try typing help plot. This prints to the screen a long comment describing and explaining how to use the inbuilt MATLAB function plot, what the arguments mean, what values it returns, what the options are, and where to get more help. MATLAB allows you to do this for your own code as well. To include a comment in a MATLAB function it just has to start with the symbol. Any text placed after that will have no effect, and is a useful way of documenting the code. The first block of comments that appear at the top of the file, before the function statement, will be printed to the screen when the help mechanism is used. Add documentation to your renamed function. Check that it prints to the screen. See if your neighbour thinks it is sufficient. You can compare against the function MyDivide on the BLACKBOARD website. 3

2 TACTICS 2.2 Type checking 2.2 Type checking In example 1 we saw that the function MyDumbFunction, now renamed to MyDivide, would happily take any input even though the function was originally designed to work only with single real numbers. In some cases this made sense, such as dividing a matrix by a single number. In many it does not try the function with one argument the number 1 and the other the string rabbit. Many program languages are typed; that is, every variable must be declared to be a particular thing, such as an integer or a string of characters. In strongly typed languages then the arguments to all functions also have to be declared in advance. In these cases the problem of passing nonsense to a function is reduced (although, of course, not eliminated). MATLAB does not type its variables, except implicitly. This has its advantages in producing simple, flexible code. However, as we have seen, it has major disadvantages in producing robust code that will either always work, or will fail gracefully with an informative error message. Example 3. Take the function that you have produced in example 2. The aim is to modify this function to check its arguments so that it only works for single real numbers. This is a restriction that is not totally necessary, but making a general function that knows when, for example, a matrix would work is unnecessarily complex for now. The key functions that we want to use here are the functions isscalar and isnumeric or isreal. All of these are logical functions returning either true or false 2, and each checks for the obvious thing. If the input is not what we want then the function should report an error and stop this can be done in a simple way using the input error function. Modify your function so that it checks the input. Make sure that each argument is a single real number. If it is not, report an appropriate error. 2.3 Graceful failure Another point seen in example 1 was that division by 0 returned Inf, the floating point representation of infinity. Whilst this may be correct, it may not necessarily be exactly what we want. We should also note that a similar thing would happen 2 In common with the C programming language, true can alternatively be interpreted numerically as 1 and false as 0. It is wise not to rely on this behaviour across programming languages as it will sometimes work and sometimes not occasionally it will vary depending on the implementation of a programming language. 4

2 TACTICS 2.3 Graceful failure if we tried to divide by any sufficiently small number (that MATLAB could not distinguish from 0 in terms of its floating point representation). In particular, consider what would happen if we wanted to use this function as part of a more complex algorithm. If some general function was expecting a standard real number to be returned by MyDivide and used it directly, then a simple error in the input could render this assumption invalid. It may be the case that we assume that the divisor is never equal to zero; therefore, if a number is passed through which MATLAB believes to be zero, that this is merely a failure of the floating point representation. We would then have to adjust the algorithm in some way. One simple adjustment would be to say that if the absolute value of the divisor were less than some small number, the divisor is taken to be that small number (with the appropriate sign). Example 4. The aim will be to modify the function MyDivide so that it fails gracefully. There will be two options. 1. The function can be called with two arguments, as above, and it will check to see if the divisor is too close to zero. If so it will fail with a meaningful error message. 2. The function will be called with three arguments, with the third giving the small number that will be used if necessary. To check how close a number is to zero (or indeed to any floating point number), MATLAB has the intrinsic function eps. The gives the gap to the next number that MATLAB can tell to be different. So eps(0) gives the smallest number that can be distinguished from zero. For safety, you should probably check that the Divisor is at least some multiple, say 10, of this number. The use of an optional argument in MATLAB is straightforward. You just write the function out with all the possible arguments it might receive (in this case 3: Numerator, Divisor, threshold) and, when the optional argument is not required just call it with two. Inside the function the variable nargin will give the number of arguments that the function was called with. If the optional argument was not provided then it is good programming practise to give the variable a value anyway. Modify your function so that it will now fail gracefully as suggested; an example is given in MyGracefulDivide.m. 5

2 TACTICS 2.4 Too many options 2.4 Too many options In the previous section we ended up adding an argument to our function. This allowed the function to fail gracefully, which is always good, but it does make the logic of the function less intuitive. The function is meant to divide two numbers yet it has three arguments. This is not necessarily a problem the argument is optional and so the function can still be used in an intuitive fashion but it may indicate the thin end of the wedge. What if there are lots of possible ways to modify the behaviour of a function? Do we have to keep expanding the possible arguments that the function might take? It should be clear that this cannot work. Imagine if there are two optional paths that the function might take. If we declare the function as function Ratio = MySillyDivide(Numerator, Divisor,... option1, option2) then we can only distinguish between option1 and option2 by their position in the argument list. That is, if we want to take the second option we always have to pass through a value corresponding to the first, even though we will never use it! This sort of programming is non-intuitive and very cumbersome as the number of arguments to the function increases, and should be avoided wherever possible. One alternative 3 is to pass only one additional variable which, as above, will be optional. This variable will however contain all the information on the possible options. These are often called option tables, and can be built in Matlab using structures. An option table is typically a single variable, often called options, which contains a list of variables with their value. This value can be anything; numbers, strings, etc. The variables (called the fields of the structure) are set and accessed using a. : options.name = My name options.mark = 20 A = options.mark The last line sets A to 20. Clearly we can set any number of options inside a structure (options table) and then just pass the one variable to the routine. This bypasses the problem with passing multiple options; all that is needed is, within the function, to check whether the option has been set and if so to what value. The function isfield is used to see if an option has been set within the options table. 3 An alternative is to use the variable argument list; take a look at the varargin function. This is used, for example, by the MATLAB plot function, but I would not recommend it for your own codes. 6

2 TACTICS 2.5 Summary Example 5. Modify the function MyDivide so that it fails gracefully based on the (optional) parameters passed in the (optional) third argument. 1. If the option threshold is set then the function behaves as in example 4. 2. If the option threshold is not set but the option my infinity is, then if the divisor is too close to zero the value of my infinity should be returned. An example that implements this is given in MyOptionalDivide.m. Note that when you test this function you have to be careful to remove the value of threshold from the options table, as it takes precedence over my infinity. To do this use options = rmfield(options, threshold ); or similar. 2.5 Summary 1. Make all function names, variables, etc. meaningful. 2. Add comments to your function; in particular add a header comment. 3. Check the inputs to your function; if they do not meet your expectations give a helpful error message. 4. Make everything as intuitive as possible by minimizing extra arguments. 7

3 STRATEGY 3 Strategy The previous section concentrated on the low-level: how to write code that really did things, and how to do it in a way that can be easily understood and maintained. However, there is far more to writing good, easy to maintain programs that the low level tactics that we have looked at so far. In many ways the most important thing is to have a clear strategy of what you want to achieve, and to structure your code in a logical fashion based on that strategy. Often people are happy to fill in the gaps in how you are trying to do something, but only if it is obvious to them what you are trying to do. 3.1 Objects and variables In the previous section our task (dividing two numbers) was sufficiently simple that it was obvious how we would do it. With most programming tasks there is more than one way of approaching the problem, and some thought about what you are trying to do and how you are trying to do it can lead to large benefits, both in how you think about your code and how you explain it. An example comes by looking at a simple polynomial, y(x) = a 0 + a 1 x + a 2 x 2 +... a N x N N = a i x i. i=0 (1a) (1b) Starting from this viewpoint we have that the coefficients {a i } give a complete (and simple) description of the polynomial, and so a vector of N + 1 numbers containing these coefficients is sufficient to completely describe the polynomial. From this we can create a computer code that manipulates the coefficients directly to, e.g., evaluate this polynomial or its derivatives at a point, or to divide it by another polynomial. But with further thought we can easily see that there may be other, potentially better ways of describing the polynomial. For example, the monomial form y(x) = a N N i=0 (x b i ) (2) immediately gives you information about the roots of the equation, and the form y(x) = a N D i=0 (x b i ) m i (3) 8

3 STRATEGY 3.2 Planning and Pseudo-Code gives total information about the roots and their multiplicities. These sorts of forms are very useful when dealing with polynomials with multiple roots, or where distinct roots lie very close to each other. Another alternative is the nested form y(x) = α 0 + x (α 1 + x (α 2 + + x (α N 1 + xα N )... )) (4) which ensures that the polynomial can be evaluated with the minimum number of operations, which helps numerical stability. Which is the best representation? The answer is problem dependent. If you have to accurately evaluate a polynomial or its derivatives many times, e.g. using Horner s Algorithm, then you need the form of equation (4). If you are interested in the behaviour near roots or their properties, then you need the form of equation (3). It is rare that the simple forms of equation (1) will be what you want. For a quick, single purpose project that is all the information that you need. You decide on the representation of the polynomial based on the problem that you want to solve. You write your code for that representation, taking care to explain both the representation itself and the aspects of it that your code relies on. For a general project you think more deeply. The object that you are interested in is the polynomial, not its representation(s). There are many things that you might want to use a polynomial for, or operations that you want to apply to it. So there may be a variety of representations that you want to be able to use, in order to simplify the operations or algorithms that you apply. In a sense, this is like tensors and coordinate systems. The object is the tensor, which does not know (or care) about coordinates. Operations are in theory applied to the object. However, in practice it is often easier to concretely apply an operation in a given coordinate system. By being able to convert between representations, and by carefully separating out the objects from their representations, the big picture of your code will be easier to understand, debug, modify and extend. This idea of focussing on objects is at the core of Object Oriented Programming, which is now sufficiently old to have been absorbed into most programming languages and packages. Further techniques that make your life easier, such as inheritance, can be read about in many standard texts. (Inheritance is the idea that an object can live in many classes. For example, a shape object can be moved around or duplicated. A circle is a shape object with extra information, so can be moved around or duplicated using code written for a general shape object) 3.2 Planning and Pseudo-Code You will normally start with a task that you want to accomplish but only a limited amount of detail of the steps to take. The first stage, before writing code, is to 9

3 STRATEGY 3.2 Planning and Pseudo-Code plan what you are going to do. A useful technique that is often recommended, either once you have a plan or to help build one, is to go through a series of hand-written steps, starting from the big picture. At each stage you write down what you want the code to do. You then break each step into smaller steps that will accomplish this. An advantage of this technique is that you should by the end automatically have the appropriate comments to explain your code (although often the code produced should be sufficiently straightforward that the comments required are minimal). As an example we shall consider a simple task: to write a function that takes a matrix and returns the eigenvector corresponding to the eigenvalue with largest absolute value. There does in fact already exist a MATLAB command that does almost exactly this (eigs), but we shall ignore that in the following 4. We start by writing exactly what we want to do as a MATLAB comment. function V = maxeigenvector(a) This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. We then want to be a bit more concrete as to what we are actually doing, by defining the input and output. function V = maxeigenvector(a) Input : a square matrix A. Output: the eigenvector V corresponding to the eigenvalue with largest absolute value. In the case where there are multiple eigenvalues with largest absolute value, this will be the first one found. This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. As you can see we have already made two choices. The first is that we have restricted the input to square matrices. As we have restricted the input, we should put a check in the code on the input: function V = maxeigenvector(a) 4 The eigs command returns the largest eigenvalues and their associated eigenvectors usually the largest six. However, it can return confusing results if there is not a unique eigenvalue that has largest absolute value. 10

3 STRATEGY 3.2 Planning and Pseudo-Code Input : a square matrix A. Output: the eigenvector V corresponding to the eigenvalue with largest absolute value. In the case where there are multiple eigenvalues with largest absolute value, this will be the first one found. Check that A is a square matrix. This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. Secondly we have chosen what should happen in the complicated case where there are multiple eigenvalues with maximum absolute value. At this stage we need to reduce each stage in our pseudo-code to something more basic. We currently have two steps: checking the input and the body of the function itself. We start with the input checking step and break this down: function V = maxeigenvector(a)... Check that A is a square matrix. Check that A is numerical (and not a string or a structure) Check that A is a matrix (and not, for example, 3 dimensional) Check that A is square This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. Each of the three steps is sufficiently simple that you should expect there to be a MATLAB function for each, and indeed there is: isnumeric, ndims, and size do the job. At this point we can turn the pseudo-code in to real code as in the following: function V = maxeigenvector(a)... Check that A is a square matrix. if (not(isnumeric(a))) error( The input to maxeigenvector is not a numeric matrix! ) 11

3 STRATEGY 3.2 Planning and Pseudo-Code end if (ndims(a) = 2) error( The input to maxeigenvector is not a 2-dimensional... matrix! ) end if (size(a,1) = size(a,2)) error( The input to maxeigenvector is not a square matrix! ) end This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. As you can see we have removed the low level comments as the code is sufficiently self-explanatory. A similar procedure is applied to the actual body of the code. The steps that I would take are: function V = maxeigenvector(a)... This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. followed by function V = maxeigenvector(a)... This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. First compute the eigenvalues and associated eigenvectors of A Next find the maximum eigenvalue (by absolute value) Finally return the associated eigenvector. Again there are MATLAB functions or operations to do each of these steps; eig, abs, max, and array subscripting are required: function V = maxeigenvector(a)... 12

3 STRATEGY 3.3 Optimization This will be a function that will return the eigenvector corresponding to the eigenvalue with largest absolute value. First compute the eigenvalues and associated eigenvectors of A We should note that D1 is a MATRIX containing the eigenvalues on the diagonal. [V1,D1] = eig(a); Next find the maximum eigenvalue (by absolute value) The maximum absolute value is in C, but we do not care about it. The index array I gives the location of the maximum. [C,I] = max(abs(d1)); Finally return the associated eigenvector. V = V1(:, I(1)); 3.3 Optimization Optimization is the art of making your program run faster. According to the old saying there are two rules about optimization: 1. Don t do it. 2. (For experts only): Don t do it yet. The reason for this is encapsulated in another old saying: Premature optimization is the root of all evil. Both are exaggerated, but hold more than a grain of truth. The key reason for this is the truism is that optimization saves computer time, whereas you should be interested in saving person time. If it takes you 2 hours to improve a code that then runs in 10 minutes instead of 20 then you have lost time, not gained (unless, of course, you need to run the code many times). Most often you will find a good planning will minimize additional overhead. However, there are a few additional points that are MATLAB specific. 3.3.1 Vector operations MATLAB naturally applies operations to vectors and matrices wherever possible, such as > x = 0:0.01:4; > s = sin(x); 13

3 STRATEGY 3.3 Optimization The second line could have been written using a loop over the elements of the array s: > x = 0:0.01:4; > for i = 1:401 > s(i) = sin(x(i)); > end The use of a loop is, in some cases, more natural. However, it is usually less transparent (it certainly is above) and it imposes more structure than necessary: in the example above it is irrelevant in what order the operations take place, but by using a loop we have imposed an order, and this order may not be the fastest. By using vector operations we allow MATLAB to choose the fastest possible method. 3.3.2 Growing vectors The loop above has more problems in terms of efficiency. The problem is that the vector s has no definite size before the loop starts. Therefore every time the loop is executed MATLAB has to perform the following steps: 1. I am being asked to assign a value to s(i). 2. Does the vector s already have i elements? 3. No. 4. Therefore I must increase the size of s to contain i elements. 5. Then I assign a value to s(i). Although this is not the precise algorithm it uses, you see the problem: there is much more work than necessary having to go on inside the loop. Instead it is fastest to say before the loop what size s needs to be. The standard method for doing this is to initialize the vector (or array) to the zero vector (array) with the appropriate size; therefore a better loop structure is > x = 0:0.01:4; > s = zeros(size(x)); > for i = 1:401 > s(i) = sin(x(i)); > end Note that in other programming languages such as C or Fortran predefining the size of an array is often necessary, and indexing outside of the array will lead to failure at compilation time (if you are lucky), a code crash (in most cases), or garbage results (if you are really unlucky). 14

3 STRATEGY 3.3 Optimization 3.3.3 Use specialist tools If you have no choice but to optimize (if the program would take more time than you could possibly have available) then the first thing is to find out what part of your code takes the time. In MATLAB there is the Profiler, which will return detailed information on your script. To launch the profiler, go to the Editor and from the Tools menu select Open Profiler. There is considerable online help, but its use is straightforward. 15