Visualisation of Sport Rating Systems

Transcription

1 Visualisation of Sport Rating Systems Casper Thuis Bachelor thesis Credits: 18 EC Bachelor Opleiding Kunstmatige Intelligentie University of Amsterdam Faculty of Science Science Park XH Amsterdam Supervisor Christian Schaffner Institute for Language, Logic and Computation Faculty of Science University of Amsterdam Science Park XG Amsterdam June 28th, 2013

2 Abstract Rating and ranking teams is a vital part of sports and competitions. Ratings are used to determine which teams are allowed to play in the final and can be used for matching to support fair and fun play. Mathematical rating systems can be used to rate teams. However, people that play sports are not familiar with the mathematical concepts that are required to understand these rating systems. To support the use of mathematical rating systems, understanding of these rating systems within laymen is necessary. This thesis attempts to create a visualisation in order to enhance understanding of these systems. The tools for developing the visualisation are HTML, CSS, JavaScript and a JavaScript library called Data Driven Documents(D3). With the use of these tools a single rating system is visualised, namely Massey s method. The results indicate that a visualisation of the method can be created. Creating an interactive environment seems possible as well that can further enhance understanding. However, this thesis cannot validate whether the visualisation can enhance understanding within laymen, due to the lack of time and difficulty in determine understanding. Conclusively, further research needs be performed to determining whether the visualisation developed in this thesis can enhance understanding.

3 Contents Contents 1 Introduction Literature review Theoretical foundation Massey s method Visual variables Research methodology Data Programme structure or UML Implementation of Massey method Website framework Visualisation in D3.js Results 17 5 Evaluation Correctness Features Visualisation Variables Colour Position Size Value Understanding Discussion Positives Limitations Future research Conclusion Acknowledgement 27 References 28 A B Appendix Json File 29 Matlab code 32

4 C Matlab and JavaScript results 33

5 1 Introduction Sports are a considerable part of every day life for the Dutch population. (CH13) estimated that around 65% of the total population was involved in sports in There is a wide variety of popular sports, including soccer, hockey, frisbee, chess, tennis and numerous other sports. Important aspects of sport are sportsmanship and competition. Alongside competitiveness, a process of ranking teams or players is required to determine who is leading the competition. This thesis uses the following terminology quite frequently hence it defines rating and ranking systems. This terminology is retrieved from (Mas97; LM12). Ranking teams refers to the aspect of putting teams in order. Rating teams refers to giving a numerical value to teams that represent the teams strength relative to the rest of the teams in that league. If relatively few teams play against each other, a Full Round Robin would suffice to rank them accurately. (Sor99) defines a Full Round Robin as a tournament where every team is matched against each other. However, if there is an extensive number of teams, the Full Round Robin would be considered unreasonable or virtually impossible. It would be a very time consuming process. Hence, other systems of ranking can be used to determine the intermediate ranking of these teams after a few matches. To support matching and to represent the strength of a team or a players more accurately, a rating system can be used. Furthermore, these ranking systems are not just involved in ranking different teams but are also used for matching. The teams with approximately equal rating are matched against each other. This creates an overall fair and fun game, because opponents should have equal chance of winning. In conclusion, both aspects of ranking and rating teams are a vital part of sports. (LM12) describe how these rating and ranking systems are not only used in sports, but in numerous other aspects of everyday life, such as page ranking, hit lists and personalising commercials. Thus, rating and ranking systems are a considerable part of everyday life with or without people knowing it. The ranking and rating of teams does not only have different definitions but also have different properties. (Mas97) notes an couple of these differences. Firstly, teams that may be adjacent in a ranking system, are not necessarily adjacent in a rating system. Hence, ratings systems seem to contain more information about the current standings. In addition, rating systems give a representation of the strength of the team over the whole period, contradictory to some ranking systems who are solely based on win-lose outcome. The added benefit of ratings systems is that it can be used for ranking, whereas ranking cannot be used to rate teams. Ranking systems however seem to work intuitively for someone who has no knowledge of mathematics or statistics. Ranking is in line with how people look towards competition; person A is better than person B seems a natural comparison, whereas stating that person A is a 100 points better than person B can be seen as more confusing. In conclusion, rating and ranking can be used for similar purposes but are slightly different in their practical purposes 1

6 and in meaning. Since ranking a large number of teams can be a challenge (Sor99) different systems are developed. There are numerous simple systems developed, which will not all be discussed. Still, some simple systems of ranking will be explained to illustrate which sort of systems are widely used. (Sor99) highlights simple ranking systems in his paper. A Full Round Robin, as mentioned above, seems to be a fair and simple ranking method. However, the method has the disadvantage that it needs a high amount of games to determine the full ranked list. The Cup system is a ranking system where only the winners continue to the next round. This system determines the winning team quite fast, yet the winners might not be the best team due to performance issues in teams, for example having home field advantage. Having a home field advantage can give a 60% increase in likelihood to obtain a win in soccer matches (How15). Other performance issues can be in the form of injuries or a coach replacement. Therefore, (Sor99) considers the Cup system to be slightly unfair. Another system of ranking teams is the accumulative point systems, where teams gain 3 points by winning, 1 for a tie and 0 for losing. This system is widely used in Dutch sport competitions. A problem that (Sor99) underlines when using this system is that the strength of the teams are not taken into account. For example, it would be easy to gain a lot of points by winning from weak teams. Altogether, these non-mathematical based systems are simple and seem effective in finding a winning team. Nevertheless, as stated above, this thesis considers these systems to be less fair than more complex systems. As a result, a need for mathematical rating systems, that does not only rank the teams but also determine their strengths by giving them a rating, has arisen. While mathematical systems are considered to compute the rating of teams more fairly than simpler systems, there is still a problem with the overall trust and understanding of people who are subject to these systems. The Bowl Championship Series(BSC) uses these methods to calculate the ratings of American college football teams to determine which teams are allowed to play in the bowl finales (LM12). These bowl finales are a lucrative business because playing does not only mean obtaining a chance to win prize money but it also includes a change that value of the particular club could increase. The BSC still incorporates human ratings, which is contradictory to the reliability of mathematical systems. These human ratings are based on polls of coaches (LM12). Together they are combined with a number of ratings systems to create one combined rating. Based on this combined rating, there is determine which teams are allowed to participate in the bowl finales. This fact seems to indicate that people do not fully trust the use of mathematical systems. However, especially people can be considered to be biased towards teams. Possibly the distrust in these system is caused by the fact that people that do not fully understand these systems. These 2

7 system are mathematically or statically based and it would be likely that people do not understand them because they lack the knowledge to understand these systems. This incomprehension can especially be true when a certain scenario might arise. There are four teams in a league; team A is currently the leading team with a great difference in rating, team B and C have roughly the same rating, and team D is the weakest of the four and has a considerable lower rating than the other teams. Team B plays against team A and wins by 1 point. At the same time, team C wins from team D by one point. Most people would think that team B and team C would increase by the same amount in rating. However, because team B has won from a much stronger opponent, their rating will increase more than the rating of team C. This rating change might be somewhat counter intuitive and therefore incomprehension might arises. The implication of that scenario is that although these rating systems are considered fair for people that understand these systems, a significant portion of the people might deem this system as unfair. A rating system that does not appeal to people or is not understandable has no use in the sports world. Thus, if the sport environment desires to use these systems to establish an overall fair sport, people subject to these systems need to understand the working of these underlying algorithms. This thesis will research whether that is possible to achieve. Therefore, the following research question is sought to be answered: How can mathematical rating systems be made understandable for laymen? Is it possible to implement Massey s method as rating system and to visualise this system? Can Massey s method be made understandable through visualisation and interaction for laymen? 1.1 Literature review There are numerous mathematical and statistical rating systems and they all have disadvantages as well as advantages. (Mas97) has written a paper of a few methods to determine ratings. He describes these systems in a quite thorough manner and explains the mathematical details. In addition, (LM12) wrote a book concerning different systems to rate teams. The book gives a broad overview of goals that can be achieved with rating systems in general. Furthermore, it describes numerous methods in great detail and how these methods can be computed. Finally, it lists the advantages and disadvantages for both practical and theoretical purposes. (Sor99) also attempts to give a small overview of existing rating systems, but compared to the other writers, he examines less systems and his examination is more shallow. All research combined creates a quite extensive overview of ratings and ranking systems. An important conclusion which can be drawn from of the comparison of all three papers, is that there is 3

8 no correct method to rank or rate teams, because there seems to be no golden standard. Different rating system rely on different philosophies to achieve their goal. Furthermore, the advantages and disadvantages seem to be determined by the philosophy behind the method. An illustration shows this difference in philosophy when comparing Massey s method and Colley s method. Massey s methode takes the score of the match into consideration, while Colley s method does not. If the scores are included like in Massey s method, the advantage is that it can give information about how the game was played and incorporated this information into the rating of the teams. When excluding the score as in Colley s method, the advantage is that when teams play different game tactics, teams with an aggressive style would not benefit more than a team with a defensive style. Altogether, Massey s method could be considered somewhat biased because it can favour teams that ramp up the score by playing aggressive. In contrast, Colley s method would exclude relevant information that tells how a matched was played and how strong the teams are towards each other. This difference in philosophy also applies to other ratings systems. Thus, it seems that there is not one rating system that can be considered the best rating system. However, based on the situation and on which goals one wants to achieve, one rating system can be favoured over another system. By setting goals, one can choose systematically decide which system will be best for achieving these goals. The intention of this thesis is to create understanding among laymen of how ratings are computed and how scores affect can rating. Due to a lack of time, developing understanding of multiple ratings systems seems unreasonable. Therefore, the systematic approach as described in previous paragraph will be used to make a choice for which rating system is best, given the circumstances. There have been made attempts to examine differences between rating systems by (LM12; BDE + 13; CKLP11). One of the algorithms that is insensitive for small rank changes is Massey s method (CKLP11; LM12). By the lack of a golden standard, the Rating Percentage Index (RPI) is used as such, because it is used by the National Collegiate Athletic Association(NCAA) which is one of the largest sport association in United States (LM12). Similar conclusions are drawn by (BDE + 13). Additionally, Massey s method is one of the methods used in the BSC (LM12) in order to determine which teams which are allowed to play in the bowl finales. A more practical reason for choosing Massey s method is that it is a fairly easy algorithm to understand and implement. One of the practical downsides is that Massey s method has a high time complexity when solving it with the with linear method, O(N 3 ) (1), where N is the number of teams. However, the data set that is used in this thesis contains a small number of teams, namely 40 teams at most. Therefore, the time complexity is not considered to be an issue. It is important to emphasise that 4

9 choosing Massey s method is more or less arbitrarily, since many other rating systems can be considered to be made understandable as well. Creating understanding among people that have no or little mathematical knowledge can be achieved in different manners. One of the most straightforward solutions seems to be attempting to explain the mathematical details of Massey s method. (LM12) perform an excellent job at explaining Massey s method. However, this explanation does require at least a minimum understanding of mathematical concepts, such as linear algebra. Additionally, people in sport teams have probably little interest in learning such concepts, especially because it might be a time consuming task. Therefore, another manner of explaining, one that requires no mathematical terms, must be used. Visualisation has been helpful in creating understanding among people from the beginning of mankind, varying from cave drawings to art and schematic drawings. For this reason this thesis wants to enhance understanding through visualisation, because the visualisation seems to be generally be understood by everyone. (RR06; KKII05) both conclude in their research that visualisation can aid the learning process of mathematical concepts, such as learning the meaning of integrals. In addition, learning can be aided in the form of interaction. (PO00) showed that interaction can aid the learning process of students. The results also show that interaction aids the learning process better than reading a book. Both these phenomenon could also hold true for the visualisation of Massey s method. The field associated with visualisation of data is known as data visualisation which is discussed in (Car03; CM84). (CM84) conducted experiments to examine whether visualisation can be used to describe quantitative information, such as the difference in length between objects. The results of this experiment showed that people could get a rough estimation of how much two objects differentiated in length. The objects should substantially differentiate in length and position, to make sure participants could distinguish both objects. This experiment has obvious results, however it is also concluded that there is no difference in persons with or without technical background. This result seems important for this thesis because, the target group of this thesis would mostly likely lack the technical background, or in this case the mathematical knowledge. Despite the fact that the experimental research of (CM84) is quite outdated, this thesis consider that the results still hold true. Based on those results, a hypothesis is formed. Expected is that visualisation of Massey s method is possible. In addition, it is expected that Massey s method can be made understandable through visualisation and interaction. Data visualisation is used in this project to enhance understanding, however as stated in (CM84), the requirements to enhance understanding is that the vi- 5

10 Figure 1: Visual variable interpretation task, (Car03) sualisation is represented in such a manner, that it emphasises the difference or equality between objects correctly. (Car03) calls this representation of object properties, characteristics of visual variables. By these characteristics it is meant that every physical property of an object can be used for a particular corresponding interpretation. For example, in Figure 1 the visual characteristics are listed with the interpretations people can have with these characteristics. In this figure the characteristics value is shown. The value, by which is meant light intensity, can be used for the interpretations selection, association, order and length. However, it can not be used for a quantitative interpretation. (Car03) established quite an extensive overview of which properties of a object can be applied for which interpretations. These properties are called visual variables and are about the position, size, shape, size, value, colour, orientation and texture of an object. The interpretation are called, characteristics of the visual variables, and are discussed in his paper. They consist out of selection, association, quantity, order and length. These characteristics of visual variables are used to evaluated the visualisation of Massey s method. This thesis chooses to validate the visualisation on these principles to give an argumentation for design choices made. The visualisation of Massey s method needs to be publicly accessible, because ideally rating systems will be used widespread around the world. The Internet is accessible all over the world. In 2015, around 40% of the world population uses Internet (liv15). This means that the Internet is currently one of the most powerful media to reach an extraordinary number of people not bounded by 6

11 geolocation. Hence, the Internet is considered to be the best media to use for the visualisation. 2 Theoretical foundation 2.1 Massey s method As stated before, Massey s method will be used in this thesis for the calculation of ratings. The general goal of rating teams is to represent their strength by numerical values. (Mas97) wants to find these numerical value with the help of an equation that calculates the margin of victory of a particular match. This margin of victory represented by y, and could be calculated by subtracting the rating from team B of the rating from team A, r A r B = y K (2) where r a is the rating of team A, r b is the rating of team b and y K is the margin of victory of a particular match. However, the ratings are not yet known so the score of a match will be used to determine the ratings. This lead us to an important assumption of Massey s method which stated that the score difference in a particular match is expected to be approximately equal to the margin of victory. In other words, the score difference should be around the same value as the rating difference. Because of this assumption, the ratings could be determined by looking at the score differences. However, scenario s might arise that will lead to a violation of the assumption. Such a scenario might be as follows: there are three teams that all play against each other. Team A is victorious over team B, team B is victorious over team C and team C is victorious over team A. If this happens it is impossible to look at the score difference of the matches and determine the ratings based of those matches (Mas97). To solve this violation, an error is added to the equation, y i = (r a r b ) + e i. (3) This error compensates for the difference in actual match outcomes and expected outcomes. However, with this error added a method is needed to determine the ratings based on score outcome. Least square seems to be a suitable method. This method calculates a model that minimises the square difference between the data points and the calculated model. In other words, it sums the squared errors, nx S = e 2 i (4) i=1 where e i is the error. To use least squares method a system of linear equation needs to be drafted that represents the match outcomes. Every match can be represented as an equation. The equation, y i = x i1 r 1 + x i2 r x in r n + e i (5) 7

12 exists of n independent variables x ij which represent the teams. The possible values that x ij can have are 0,1 or -1 and y i represent the dependent margin of victory variable of a particular match. The e i represents the error. When a match is played, the variables in Equation 5 are initialised with a 1 when a particular team has won, with -1 when a particular team has lost and zero for the other teams that did not play in that match. If a two teams played a tie, they either get a 1 or a -1 but since the margin of victory variable is zero the equation will not be of influence. After initialising the outcome of a particular match, Equation 5 can be reduced to the following equation, y i = r a r b + e i (6) which is considerably smaller and similiar to equation 3. To isolate the error e i the equation can be rewritten as follows, e i = y i (r a r b ). (7) The least squares method minimises the error as presented in equation 4. There are multiple approaches to calculate the best fit for a linear system. This thesis has chosen for the linear approach. There also exist a non-linear approach for solving a linear system. However this non-linear approach can have some practical downsides, such as the chance of not finding a solution, or the possibility of finding multiple solutions. Because of the downsides, this thesis uses for the linear method. The equation for solving this system is as follows, W r = y (8) where W is a M by N matrix, r is a N size vector of unknown ratings, and y a size N vector of score differences. M stands for the number of matches played in the tournament and N for the number of teams. The unknown vector r contains the ratings of the teams when solved. In order to obtain the solution of r, W r = y (9) W t W r = W t y (10) one can be multiplied both side of the equation by W t. In effect, a new matrix X is created from W t W. (Mas97) notes that the diagonal of the X matrix, X ii, represents the number of matches that have been played by a particular team. The off-diagonal, X ij, (for i 6= j ), represents the negated number of matches of a particular team that has played against another team. The W t y part of the equation, which is called s, each represents the sum of the point differential from every game played by the ith team. Together a new equation can describe the system, X r = s (11) where, X is a N by N matrix, r and s are a N size vector. However, the current equation does not have an unique solution yet, because the linear systems is 8

13 not full rank. (Mas97). The linear equations are not linearly independent and causes the system to have an infinite number of solutions. Another view on this problem is that the linear solution has one degree of freedom left. To create an unique solution for the linear system, a constraint needs to be added. This constraint can be achieved by replacing the last column of the W matrix with 1 s and the last element of y with a 0. The constraint, 1 n NX r i = 0 (12) i=1, causes the rating average to be zero. The average can also be shifted by replacing the zero to any other real number. With the linear system being full rank and thereby solvable, the unknown rating vector, and thereby the solution to the system, can be calculated. To obtain the solution one has to isolate the unknown vector r by multiplying both sides with the inverse of the X matrix. Finally, the solution of the systems is a vector r that contains the ratings of all the teams. The ithvariable in the ratings vector r will be for the team associated with the ith variable in the equation 5. However, this method is not solvable for all tournaments. There are tournaments where certain teams are isolated from other teams. One can see this isolation as the matches being isolated from one another, or in graph theory terminology, the graphs are disconnected from each other, as shown in Figure 2. Another way of viewing this problem is that the teams are islands connected with each other by matches. If an island group does not have any connection or matches between them, comparing them is impossible due to lack of information. Or in graph terminology, when the graphs are disconnected, the system cannot compare these isolated teams because there is no information present about the relative strength between the teams. In effect, the linear approach to solve the system, cannot find a solution. Only the disconnected graphs on their own can be solved by the linear approach of least squares. Thus, to solve the separate systems a preliminary step is needed. This preliminary step is about determining which isolated teams are connected. The algorithm is as follows, one loops over every team, determines all the opponents of this team and keeps track of them in a list. When all the opponents have been found, it starts at the first opponent in the list and repeats this process. If new opponent are found that are not in the list, they are added to the list. This process repeats over all the opponents until the list is empty. If the list is empty all the opponents that have been in the list, including the team that one started with, are connected. One should continue this process until all teams are mapped. The time complexity of this algorithm is O(N 2 ), because every team will at most have all the teams as a opponent. If every team is mapped inside a island then the ratings can be determined by using least squares on each island separately. This is a systematic 9

14 way to counter the possibility that the linear approach of least squares method does not find a solution. In this project, this algorithm is not implemented because of how the data is structured. Nonetheless, this algorithm should be used when it is uncertain how the matches are connected. Figure 2: Connected versus Disconnected Graphs. The numbers stand for the teams and are called islands in the explanation. This thesis will define which features of Massey s method and other features not related to the method will be visualised. The first of the three features, has been proposed by (LM12) as a "what if" scenario for rating systems. This scenario states that the user can interact with the score of a particular match, to see how it impacts the rating of the selected team and the rating of other teams. Briefly said, the "What if" scenario is about changing scores to see the consequences of these changes on the teams ratings. To create this feature, match scores needs to be listed to the user. In addition, the ability to change those scores needs to be added. This interaction could probably contribute to the understanding of the method. The second feature that represents an important aspects of Massey s method, is the ability of comparing the rating difference and the score difference. As stated earlier, the main assumption of Massey s method is that this rating difference between team ithand team jthshould be the same value as the score difference. If this difference in value is not equal, it represents the error of the least squares solution. This error is a vital part of the methodology of Massey s method and should therefore be visualised. In line with the previous feature, another third feature can be added, which highlights the largest 10

15 error of a particular match to the user. This match with a high error margin can be more interesting for the user than other matches, since the high error indicates that the algorithm predicted a notably different outcome than the actual outcome of the match. In summary, all three features are listed below and aimed to be implemented within the visualisation. "What if" scenario, user can change the score and interact with the visualisation. A clear view of score difference and rating difference and the error between them. A highlighting of matches with a high error margin, to indicate that the algorithm predicated the outcome to be different. 2.2 Visual variables The visualisation needs to be evaluated on how well it upholds the visualisation principles that are defined by (Car03). In his paper he describes what interpretation characteristics are and which visual variable have those characteristics. In other words, there are certain visuals variables that can be used for different kinds of interpretations. For example, the visual variable colour can be used for an association interpretation, namely that all shapes with the same colour belong to each other. A more detailed explanation of all characteristics can be found in Table 1. In Table 2 is shown what visual variables have which characteristics. Table 1 and 2 are used to evaluate the visualisation. 3 Research methodology In this thesis a number of different stages have been run through to create the visualisation of Massey s method. The stages concern practical performed steps, information used, tools used and design choices. All stages are as follows: Data structure, Programme Structure or Unified Model Lanuage(UML), Implementation of algorithm in Matlab and Javascript, Website framework and user input, Visualisation of algorithm in D3.js java script library. The time allocated for this project was roughly two months and everything shown in the results was developed within this time span. The stages of the whole process will be explained in greater detail below. 11

16 Characteristics Selection Association Quantity Order Length Explanation The selection characteristic causes a particular object to be distinct form other objects. For example, the yellow colour of a circle will be distinguished from a red circle by a selective interpretation. The association characteristic causes particular objects to be considered a group. For example, a green coloured object will be considered a group with other green objects by an associative interpretation. The quantity characteristic causes a numerical relationship between objects. This quantity characteristic does not have to be precise but it can be seen as a ratio indication. For example, a line that is roughly 5 times bigger than another line causes a quantitative interpretation The order characteristic causes an ordering relationship between objects. For example, objects that have a different grey colour can be ordered by an order interpretation. The length characteristic is different from the other characteristics. This characteristic is about the maximum number of different forms that a visual variable can take and still be informative. For example, how many different sizes can be added while the circles will still be distinguishable. Table 1: A list of their important characteristics with the explanation.(car03). 3.1 Data The data used in this thesis is retrieved from leaguevine.com, which is a website for sporting communities of non-professional teams. It has numerous databases filled with tournament data for Ultimate Frisbee, rugby and other sports. An important feature for this project is that leaguevine has an is a API from which data can be extracted by an URL. This API can be used to create a simple communication port between the obtaining of data and the processing of data in an application. Hence, this process can be fully automated. The get-request loading time is not an issue because the data files are between 300kb and 600kb, which is rather small. The data used from leaguevine concerns multiple Frisbee tournaments from the Windmill event in the Netherlands from 2012 to An important reason for using these data sets is that they are preprocessed in Matlab files, in a similar honours project (SC13). The URL has the following form. 12

17 Variables Selective Associative Quantitative Order length Colour Well suited Well suited Not suited Not suited max 10 different colours Position Well suited Well suited Well suited Well suited Infinite, practically limited Size Well suited Ill suited Well suited Well suited max 20 different sizes Value Well suited Well suited Not suited Well suited max 10 different values Table 2: Characteristics of visual variables (Car03). Tournament name Teams Matches Windmill open Windmill mixed Windmill women Windmill open Windmill mixed Windmill women Table 3: Data set size of teams and matches fields=[id%tournament%game_site%start_time%%20swiss_round %20team_1_id%team_2_id%team_1%team_2%team_1_score team_2_score]&order_by=[start_time]&limit=200 Important aspects to notice are the tournament ID, the fields and the limit. The tournament ID is self explanatory and the limit defines the amount of matches obtained which is set on 200. The fields in the URL return information of which variables are requested. This thesis extracted the following information from: game ID, tournament information, game site, start time, swiss round number, team ID 1, team ID 2, team information 1, team information, 2, team score 1, team score 2. When this URL is requested by the user a JSON file is. This process which is shown in appendix A. The data is then processed by JavaScript to remove redundant information. The remaining information used and stored in an array in the following form. [match_id, team_1_id, team_2_id, team_1_score, team_2_score, round_number] The URL get-request is called by a jquery HTTP get-function. The jquery function is not an original JavaScript function and is obtained from the jquery.js library. The jquery.js, which is an extensive library that contains numerous functions. The jquery HTTP get-request is one of the methods used to load data from an external location and transfer it to be used locally. The data set consisting of six data sets from the Windmill event is shown in Figure 3. 13

18 3.2 Programme structure or UML The Unified Modeling Language(UML) is a model that illustrates what functions, objects and variables a system contains. The UML is the developing plan for this system. Since this thesis had not prepared such a plan beforehand, due to the fact that designing was performed step wise, the UML is not used as guideline to develop the system. However, it represents the outcome of the developing process. How the system is build is shown in the UML in Figure 3. Every block represents an object and has a name, a purpose, variables and functions that are shown in the grey, green, red and blue blocks respectively. On the right side there are also some functions, which are not bound to any object. An important aspect of the UML is that the tournament object contains all data related to the tournament. The objects also ensures that the data structures to calculate the ratings can be created. Furthermore, it contains functions that alter HTML content, returns opponents of the selected teams and score differences. In sum, this object holds the information about the tournament. A canvas object is another object which holds information about the visualisation. The variables of this object contain information of how the visualisation is represented. The functions of this objects creates and updates the visualisation based on the data given. Thus, it is an object that processes information of tournaments and translates that into a visualisation. Finally, the functions on the the right side of the UML are not bound to an object. These function processes the input of the user and calculate the ratings. 3.3 Implementation of Massey method Massey s method is a least square solution and it is not difficult to implement. However, the used JavaScript libraries have no information about the person that created the library and give no guarantee that the code is working correct. Hence, the JavaScript implementation should be validated. This thesis considers a Matlab implementation to be correct because the source is known and documented. It was obtained from a similar project (SC13) and has been sightly modified to the linear approach, as explained in the theoretical framework section. The Matlab code used is shown in Appendix B. The JavaScript implementation used for the visualisation of Massey s method uses a JavaScript library math.js, that extends the JavaScripts possibilities with functions that help with matrix operations. The earlier mentioned problem of the disconnected graphs is solved without the use of the proposed algorithm for finding isolated matches. The current data is organised in such a manner that when the teams have played the second round, every team connected with each other team, or as stated in graph terminology, the graph is fully connected. Therefore, only when determining the ratings in the first round the teams are 14

19 Figure 3: UML(Unified Modeling Language) overview 15

20 disconnect. All the teams will be isolated in pairs because they only played one round. Determining the ratings for the teams is performed by taking the margin of victory y i equation, newrating = (y i )/2 (13) and dividing it by two. The ratings of both teams are as follows: the rating of the team that won, is equal to the newrating variable. The rating of the team that lost is equal to the negated newrating variable. This represents the assumption of Massey s method perfectly, since the score difference between the two teams is exactly the same as the rating difference and the average of is 0. Thus, it would be the same as using Massey s method, yet it is easier to calculate. This is repeated for all teams and in result the ratings for the first round can be calculated. Altogether, there are two functions that calculate the least squares, one that is for fully connected matches and one that calculates the ratings of the first round in when the graph is disconnected. 3.4 Website framework Before the visualisation can be created, a website needs to be built to house the visualisation and to interact with the user. The website is built with standard HTML, CSS and a JavaScript code. However, a framework is used to aid this process, namely twitter bootstrap. Bootstrap allows a website to be developed in a significantly shorter time. For that reason, this project used bootstrap to create a basic website. The template used for the website is called the dashboard template and is visible in Figure 4. The dashboard area within the template is used as housing for the visualisation, the sidebar and the section area are used for users input. 3.5 Visualisation in D3.js The programming language used for the visualisation is the Data Driven Document or D3 in short. D3 promotes itself as being expressive, easily accessible and having high performance for data manipulation. D3 is designed by (BOH11). It makes data visualisation possible by loading data and creating objects for every data point one has loaded. D3 requires little capacity and it is usable in several browsers such as Firefox, Chrome, Safari, Opera, IE9+, Android and ios. Altogether, the visualisation has been made in D3.js for the above mentioned reasons. To create the visualisation Scalable Vector Graphics(SVG) objects are used. SVG defines objects in XML format and creates graphical objects, such as lines, circles and rectangles. D3 uses this these graphical objects to bind it to the data and thereby making it easy to create objects that are data related. 16

21 Figure 4: example of bootstrap template 4 Results In this thesis, the visualisation on the website can be regarded as the result. The following images 5, 6, 8 and 8 are retrieved from the website. For a complete view of the website and the visualisation, it is best to see it online. The code can be viewed on This is not shown in the Appendix because of the size of the code. The figure 5 shows the final product of the visualisation. There are several methods to represent the teams and the ratings, such as a graph or box plots. In this thesis a representation of line has been chosen to clearly illustrate the maximum and minimum rating. The left end of the line represents the minimum rating and the right end represents the maximum rating of the teams. All the teams are aligned on this line. This representation to visualise the teams, is based on (LM12, p.11). The teams are represented by circles and all have a radius of 10 pixels, with an exception of the team that the users selects, which has a radius of 20 pixels. The selection of a particular team determines which circles are filled with colour. Namely, the coloured circles are the selected team and the direct opponents of the selected team. The other teams, that are not direct opponents of the team, have a grey colour and a lower opacity. Every opponent has a vertical dotted line, that is connected to three horizontal lines. The first, a green horizontal line represent the difference in rating between the selected team and the opponent. The orientation of the line indicates a negative rating difference, when the line is on the left side, or a positive rating difference, when the line is on the right side. Next to the horizontal green lines a horizontal red line is drawn, indicating the actual difference in score between the selected team and the particular opponent. The length is equal to the difference in score. The orientation works the as same for the other line. The difference between the two horizontal lines is 17

22 Figure 5: Visualisation Figure 6: Table of match information the error that least squares method tries to minimise. This error is represented in the blue horizontal line. Thus, the error is the mismatch between the actual lengths of the green and red line, as can be seen in Figure 5. The user has several input options to interact with the visualisation. A team can be selected from a HTML text field with suggestion. Besides the selection of a team of interest, the user can interact with other aspects of the visualisation through multiple input buttons and menu s. For the possible user input and information, a table is created that lists all the matches of the selected team. The match information shown to the user consists of team names, scores, and round numbers. The table and the input field are displayed in Figure 6. Within this table, four buttons are added to increase or decrease the score of the teams. Thus, enabling the "what if" scenario that changes the visualisation. The user can also alter the tournament by selecting another tournament from the sidebar. The tournaments that can be selected are the six tournaments listed in Figure 3. A final aspect that the user can interact with is the round number. The user can select which round it wants to view and the visualisation changes accordingly. Figure 7 and 8 illustrate the view when round one and round five have been selected. 18

23 Figure 7: Round 1 is selected. Figure 8: Matches up to round 5 are selected. 19

24 5 Evaluation There are multiple aspects that need to be evaluated, namely: If the implementation of Massey s method is correct. If predefined features are implemented. If the visualisation makes uses of visualisation principles. If the visualisation enhances the understanding for people with laymen. 5.1 Correctness The results of both implementations of Matlab and JavaScript are listed in Appendix C. The output of the implementation show similar results. Despite the fact that a few rounding errors occurred in the JavaScript implementation, the overall implementation can be considered correct. 5.2 Features In the theoretical foundation section a list of features is discussed that are considered that would contribute to the understanding of Massey s method. The first feature is the "What if" scenario. The "What if" scenario allows the user to changes the scores and observer the impact of these changes. The effect of this feature can be clearly seen in the visualisation and works as expected. However, there still seems to be a bug in the second round. In the second round the user can change scores, however these score changes do not affect the ratings. This bug is currently unresolved. The second feature, the visualisation of the difference in scores, ratings and the difference between them is represented by the red, green and blue lines. This feature work correct and shows the differences in length. It also shows the quantitative values of these lines when the mouse is placed on these lines. For the evaluation we can conclude that this feature is implemented. The last feature, the highlighting of a large error margin is not included in the visualisation. Due to lack of time, this feature is not implemented into the website which is not satisfactory. This result is unsatisfactory because the user can potential gain information of which matches are more interesting than other matches. Altogether, all the implemented features have the desired effect on the visualisation because they mostly work correct, and fulfil the purpose for which they are created. 5.3 Visualisation Variables The following visual variables are used in the visualisation: colour, position, size, value. Each variable will be discussed separately. 20

25 5.3.1 Colour Colour has been used to indicate that certain teams have a connection, namely that the coloured teams are their opponents. This colour property is therefore used to create associative interpretation. Furthermore colours are used to differentiate the lines that represents the rating difference, score difference and error. It therefore also uses the selective characteristics to highlight that the lines have different meanings. These two characteristics are well suited for the colour variable. However, when there are too many opponents the selective strength of the colour might decrease, because the more colours there are the less different the colours might be. Fortunately, the number of different colours is little enough to still be informative. Overall, the colour have a positive effect on how informative the visualisation is Position The position of the circles, that represent the teams, is used to indicate the rating difference between the teams. Position is used to create a quantitative interpretation. Moreover, it can be viewed as an order interpretation, because the ratings can be translated into rankings as well. The length characteristics position is theoretical infinite but practically limited (Car03). By this is meant that although positions can be used to indicate every decimal place, it would not be informative if the positions are too close to each other. This effect is noticed when the team are close in rating and thus the position variable is partially informative in the visualisation Size The size of the circles is used to indicate that a certain team is selected and therefor the selective characteristic of size is used. This results can be clearly visible and thus have a positive effect on the visualisation Value The value might not be direct noticeable, but the grey circles have a lower opacity than the coloured circles. This is done to draw the attention to the direct opponents. Thus, value variable makes use of the associative characteristics of value. This effect is not clearly noticeable and thus is difficult to determine whether it has a positive effect on the visualisation. Altogether, the visual variables used in the visualisation make correct use of the characteristics that (Car03) mentioned in his paper. The position variable have problems when the team are close in rating and therefore can be less informative. In addition, the value variable might not be noticeable. Overall, the 21

26 visualisation makes correct use of most characteristics and it seems it contribute to an informative visualisation. 5.4 Understanding The intention of this thesis was to establish understanding of complex rating system with laymen. However determining whether the visualisation can aid in an understanding of Massey s method has not been preformed. The reasons for this will be discussed in the discussion. 6 Discussion This thesis tried to find an answer to the question: How can Massey s method be made understandable for laymen? To make the process more manageable, the question was divided in two sub questions. This thesis mainly focuses on the first sub question, namely whether Massey s method can be visualised. The question whether it is possible to make the visualisation understandable for laymen should be examined by an experimental research. Unfortunately, this is not within the scope of this research. Therefore, the research question can only be answered partially. The results show that this thesis has succeeded in creating a website that visualises Massey s method by using JavaScript. The implementation and the library is used are correct. This comparison is validated by comparing it to a Matlab implementation. Furthermore, two thirds of the predefined features are implemented in the visualisation. This thesis considers the three features necessary for aiding understanding. Firstly, errors play an important role in Massey s method, it was therefore decided to illustrate the errors in the visualisation. By omitting the errors, a significant part of Massey s method would be lacking. Secondly, it illustrates informative aspects of Massey s method that can give insights that would otherwise be more difficult to obtain, such as the "What if scenario". Lastly, the visualisation attempts to uphold the characteristics of the visual variables as explained in the evaluation. Therefore, this thesis concludes that Massey s method can be visualised by the means of JavaScript and D3 JavaScript library. However, this thesis cannot conclude that the visualisation has contributed towards more understanding of Massey s method among people that lack extensive mathematical knowledge. It cannot conclude this aspect because it was not possible to obtain results concerning this sub question. There are several reasons that this thesis did not attempt to validate this second sub question. The main reason is that the focus was on the aspect of creating a visualisation. The creation of visualisation itself takes a considerable 22