Visualizing the Ethiopian Commodity Market

Transcription

1 LiU-ITN-TEK-A--09/031--SE Visualizing the Ethiopian Commodity Market Jakob Rogstadius Department of Science and Technology Linköping University SE Norrköping, Sweden Institutionen för teknik och naturvetenskap Linköpings Universitet Norrköping

2 LiU-ITN-TEK-A--09/031--SE Visualizing the Ethiopian Commodity Market Examensarbete utfört i vetenskaplig visualisering vid Tekniska Högskolan vid Linköpings universitet Jakob Rogstadius Handledare Ahadu Woubshet Examinator Mikael Jern Norrköping

3 Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: Jakob Rogstadius

4

5 Abstract The Ethiopia Commodity Exchange (ECX), like many other data intensive organizations, is having difficulties making full use of the vast amounts of data that it collects. This MSc thesis identifies areas within the organization where concepts from the academic fields of information visualization and visual analytics can be applied to address this issue. Software solutions are designed and implemented in two areas with the purpose of evaluating the approach and to demonstrate to potential users, developers and managers what can be achieved using this method. A number of presentation methods are proposed for the ECX website, which previously contained no graphing functionality for market data, to make it easier for users to find trends, patterns and outliers in prices and trade volumes of commodieties traded at the exchange. A software application is also developed to support the ECX market surveillance team by drastically improving its capabilities of investigating complex trader relationships. Finally, as ECX lacked previous experiences with visualization, one software developer was trained in computer graphics and involved in the work, to enable continued maintenance and future development of new visualization solutions within the organization. i

6

7 Acknowledgments My special thanks go to Ahadu Woubshet for making time in a hectic business schedule to supervise my work, to Anteneh Mitiku for his contribution of much needed expertise in the area of market surveillance and his great interest and patience during the development of one of the software solutions presented herein, and to Alegntaye Yilma for his effort, patience and enthusiasm during all our hours of computer graphics training and discussion. I also want to thank Kadri Alfa for helping me adjust to life in Ethiopia and for being a great friend and neighbor. In addition, I would like to express my sincere gratitude to Markus Johnsson for all the practical assistance back in Sweden during my time abroad. Last but not least, I would like to thank the people at ECX who gave me their advice, support and time and who made this thesis possible. They are, in no particular order, Mussie Nadew, Bharat Kulkarni, Tilahun Bekele, Sisay Zerihun, Nardos Lemma, Iyob Tilahun and Daniel Tsegaye. This thesis was in part funded by John Lennings Stipendiefond för Norrköpings Näringsliv, Norrköpings Polytekniska Förening, Norrköping Campus Rotaryklubb 1 and Ethiopia Commodity Exchange. 1 Norrköping Campus Rotaryklubb works on a voluntary basis with executing humanitarian projects globally and acts to create understanding and peace between countries and individuals. iii

8

9 Contents 1 Introduction Problem Description Thesis Objectives Scope Sources Report Outline Background ECX Areas of Operation Warehousing and Grading Trading Operations Clearing and Settlement Market Surveillance Market Data Dissemination Stakeholders Rural Farmers Cooperative Unions Traders Exporters Foreign Buyers Media Researchers Policy Makers ECX Internal Market Data Dissemination Channels Feeds to International Vendors ECX Website Market Bulletin TV, Newspapers and Radio SMS and IVR Price Display Boards Data Kiosks Stakeholder and Dissemination Channel Summary Information Visualization & Visual Analytics Information Visualization Applied at ECX v

10 vi Contents 3 Development Tasks 15 4 Market Data Presentation on the ECX Website What is ECX Market Data? Methodology Scope Usage Scenarios Technical Limitations Related Work Requirement Summary Presentation Solutions Price Trend, Individual Symbols Context of Latest Market Update Price and Volume Trend, Entire Market Trade Volume Trend by Market Segment Evaluation Market Surveillance Application Methodology Aim Related work Features Data Import Trader Network Alerts Search Trade Price Difference Trade History Trade Volume over Time Total Trade and Top Evaluation Functional Performance Technical Performance Remaining Known Issues Sustainable Development 41 7 Discussion & Future Work Sustainable Development Market Data Presentation on the ECX Website Proposed Future Changes to the ECX Market Data Presentation Structure Internal Uses of Information Visualization at ECX Market Surveillance Application What Value Has the Thesis Work Added to ECX? Bibliography 49

11 Contents vii List of Figures 2.1 ECX open outcry trading floor (left) and traders signing a transaction (right) ECX website with solution for requesting historical trade data Price display boards located at a rural market site (left) and at the ECX trading floor (right) Six examples of information visualization methods. From left to right, top row: network graph, parallell coodinates plot with overlayed histograms and filter sliders, scatter plot. Bottom row: treemap, horizon graphs, density map Google s stock performance summary by sector (left) and part of the Financial Times stock search solution (right) Interactive time series graph provided by Yahoo Finance, showing index value, trade volume, selected technical analysis indicators and plotted time period The five implemented chart types for displaying time series of price data; line chart with glyphs indicating individual samples (a), line chart with semi-transparent area indicating day high and low (b), area only (c), candlestick chart (d), bar chart (e) Graph on the ECX website displaying price and trade volume history for two different symbols Small multiple line graphs added to the market update table on the front page of the ECX website symbols rendered as a grid of line graphs, with origin on the horizontal axis and grade on the vertical axis symbols rendered as a grid of horizon graphs, with origin on the horizontal axis and grade on the vertical axis Stacked chart displaying total weekly trade volume, grouped by origin The market surveillance application interface The effect of network filtering; all trade (left), only trade connections with more than 6 trades (right) One selected buyer with surrounding sellers. The values displayed on the lines indicate the ratio of the buyer s transactions that have taken place with the respective trade partner Alerts displayed for extreme trade focus (left) and anomalous prices (right) The small chart displaying transaction prices relative to the daily symbol average Trade history for a single symbol. Each dot represents a single trade, with area proportional to trade volume. Red dots belong to the selected trader Weekly trade volume for a single trader, grouped by symbol (top) and trade partner (bottom)

12 viii Contents 5.8 Pop-up windows for displaying total trade within the imported time interval, as well as the top ten sellers, buyers and pairs of traders

13 Chapter 1 Introduction 1.1 Problem Description Ethiopia is a developing nation situated on the horn of Africa, with a population of around 85 million and an economy still largely based on small-scale farming. As much as 80 percent of the total employment is provided by the agricultural sector and 60 percent of Ethiopia s exports are based on agriculture; mainly coffee, qat and various oilseeds [2]. Although large in absolute numbers, output per capita is low by international standards and the typical farmer lives far from the cities and owns a small piece of land on which he or she grows crops such as maize, wheat, beans, coffee, sesame, teff or qat, using old traditional methods. The harvest that is not consumed directly by the household is sold, but multiple factors lead to that sales revenue for the farmer is minimal. Harvests are seasonal and either need to be sold quickly or stored in proper warehouses. The seasonality and limited access to good storage facilities mean that regional supply spikes during the harvest season, giving the farmer little bargaining power. The local buyer is also often the farmer s sole source of market information and there is no way to tell if an offered price is fair or not. In contrast, the buyer, or one of all the other middlemen which the harvest will pass through before it reaches the consumer, has access to both storage facilities and plenty of information which tells him when and where to make his sell. Most of the final sales price that the consumer pays ends up in the pockets of these middlemen. In addition, lack of quality assurance causes goods to be inspected and repackaged every time they change hands, as this is the only way for buyers to know what they are getting. Naturally, such a system is highly inefficient and introduces huge overheads on the final market price. With this background, the Ethiopia Commodity Exchange (ECX) was founded in April 2007, with a vision to revolutionize Ethiopia s tradition bound agriculture through creating a new marketplace that serves all market actors, from farmers to traders to processors to exporters to consumers. This marketplace should be characterized by transparency, integrity, security and efficiency, none of which could be said to describe the traditional trade. To achieve this bold goal, the exchange must transform many parts of the Ethiopian economy. ECX has already built up solutions for warehousing, electronic banking, clearing and settlement of transactions and transfer of live market 1

14 2 Introduction data information that are previously unseen in the country. Trade at ECX began in May 2008 with maize and wheat. However, only a few months later a legislative decision was made by the government to transfer all coffee trade, with a few exceptions, from the four decades old Coffee Auction system to the new exchange, effectively redirecting the entire organization s capacity to this commodity for the near future. The decision came as the country s vital coffee export was facing an international crisis of confidence, with sales shrinking due to great problems in the areas of quality control, transparency and risk management. Now, after a year of trade operations, ECX has received both praise and criticism. As the latest market prices are now disseminated to a wide audience using media ranging from the ECX website to radio and electronic price display boards in rural areas, the middlemen have lost much of their upper hand in the trade. In a similar manner, the centralized electronic trade logs all transactions and thus makes it much more difficult for traders to avoid paying taxes on their profits. A shift from trade based on samples of coffee to trade of warehouse receipts and bank deposits has resulted in a no-default system with goods quality and quantity guaranteed by ECX. On the other hand, the same quality control has introduced a delay between when goods are deposited at an ECX warehouse and when they can be sold to the market. With all this done and after tens of thousands of transactions, spread over hundreds of coffee types and even more registered traders, ECX is now beginning to face a problem experienced by many modern data-intensive organizations; how to make all that collected information accessible and easy to understand in a way that stakeholders, both internally and externally, can use it to make better informed decisions. An increasingly popular approach to solving this problem is to somehow represent the data graphically, using tools and concepts from the academic field of information visualization. This field of research has roots that go back hundreds of years, but it has grown rapidly in the past few decades, as a consequence of great technological advances in data collection and storage which have led to that data is now often produced at much higher rates than it can be processed using traditional means. So far, little of this research has however spread to the developing nations, where better informed decisions are arguably of more value than anywhere else in the world. 1.2 Thesis Objectives Given the problems outlined in the previous section, the overall goals of this thesis are to: Identify general areas and specific problems at the Ethiopia Commodity Exchange where information visualization solutions can be successfully applied, to better meet needs and goals of the exchange and its stakeholders. Implement a few such solutions, to evaluate the validity of the approach and to illustrate both to users, managers and developers what can be achieved. Lay a foundation within the organization for sustainable maintenance and development in the area of information visualization.

15 1.3 Scope Scope ECX, although itself a modern IT organization, operates in a country with poorly developed infrastructure (IT, power and road networks), widespread analphabetism and low average levels of education; factors which all impose serious limitations on the applicability of information visualization solutions. It is not within the scope of this thesis to remove these limitations. Rather, the presented results are meant to highlight areas where such solutions can be successfully applied and to act as inspiration for future work. 1.4 Sources Information regarding the operations of the exchange, its stakeholders and their requirements has been provided by ECX staff and various internal publications. 1.5 Report Outline The remaining part of this thesis is organized as follows. Chapter 2 gives a background to the work, by further introducing the Ethiopia Commodity Exchange, its stakeholder groups and the different dissemination channels used to distribute ECX market data. The chapter then introduces the closely connected academic fields of information visualization and visual analytics and a concludes with a discussion of how these fields can potentially be of use to ECX. Based on this background, chapter 3 presents two software development tasks that were selected to be part of the thesis work. Chapter 4 covers different solutions suggested for graphically presenting market data to users of the ECX website and chapter 5 presents and informally evaluates a software solution developed for assisted market surveillance at ECX. Steps taken to enable continued development after the completetion of this thesis are described in chapter 6. Finally, chapter 7 presents conclusions and possible directions for future work.

16

17 Chapter 2 Background This chapter introduces the different services provided by ECX and describes its stakeholder groups and what they require from the exchange, with an emphasis on market information and services that can be improved by better information management within ECX. The chapter also describes the different distribution channels used to disseminate ECX market information and estimates the number of people reached by each media. A brief introduction is then given to information visualization and visual analytics, followed by a concluding discussion of how information visualization can be applied at ECX both internally and to support external dissemination of market data. 2.1 ECX Areas of Operation Most exchanges around the world have focused on providing a single service; a well functioning trading platform through which sellers and buyers can meet, discover prices and trade. ECX however operates in an environment where few important related services, such as warehousing, electronic banking and market data dissemination, function satisfactory and therefore the organization has made a decision to provide market actors with a complete end-to-end solution. This section offers a short overview of the different areas in which ECX operates Warehousing and Grading To sell agricultural products through ECX, a seller is required to deposit goods at an ECX warehouse. There the goods will labeled by type and origin (in the case of coffee) and be given a quality grade according to a standardized set of measurements. For coffee, grade 1 is highest and grade 10 is lowest. These combinations of labels (called a symbol) together identify the type of product that is being sold, e.g. WSDB3 for washed coffee from the Sidama B region of grade 3 or ULK8 for unwashed coffee from the Lekempti region of grade 8. Once the grading process is completed, the seller is given a warehouse receipt, which can later be sold to a buyer through the exchange. 5

18 6 Background Figure 2.1: ECX open outcry trading floor (left) and traders signing a transaction (right) Trading Operations The central service of ECX is its trading floor, designed as a so called open outcry system (Figure 2.1). This, as opposed to fully electronic trading, means that buyers and sellers meet in person in a trading pit, where they shout out their offers. Once two traders have agreed upon a deal, they proceed to formally register the trade agreement. An electronic system verifies that that the buyer has enough money deposited to cover the payment and that the seller has the sold goods deposited in an ECX warehouse before the trade is accepted by the system. Once accepted, an electronic warehouse receipt is sent to the buyer and money is transferred from the buyer s account to the seller s. To be allowed to trade through ECX, a trader must purchase a membership. Major producers, suppliers and exporters all have memberships, but most trade takes place between brokers who act on behalf of clients Clearing and Settlement To facilitate its automated trade reconciliation system described above, ECX collaborates with the major banks in Ethiopia to be able to issue transfers between the accounts of traders. A warehouse receipt is also automatically and electronically transferred to the buyer. While such systems are commonplace in many other parts of the world, it is the first such system that has been established in Ethiopia Market Surveillance The intended benefits of ECX over the previous Coffee Auction include increasing confidence and transparency in the market, reducing market risk and to ensure fair and legal trade. Therefore, ECX market surveillance attempts to keep track of market actors to discover different types of extraordinary behavior, which may initiate further investigation and possibly lead to expelling the trader from further access to the market Market Data Dissemination High market transparency can be defined as that every market actor has access to the information they are interested in or require to make informed market decisions.

19 2.2 Stakeholders 7 The ECX market data department therefore handles the tasks of interpreting current market trends and disseminating market information to the different groups of people along the chain of trade, from producers to middlemen to local or foreign buyers and analysts. 2.2 Stakeholders This section describes the different groups of people who in some way depend on the exchange, and what their needs are in terms of information or other services. Emphasis is given to needs that are in some way related to proper understanding of data collected at or by ECX Rural Farmers Agriculture is the backbone of the Ethiopian economy and 80 percent of the population is classified as farmers. Almost all agricultural output in Ethiopia is produced by small scale rural farmers whose produce goes mainly to feeding themselves and their families. Many of them grow crops that are currently traded on the ECX trading floor; mainly coffee, but as more commodities are introduced and as more produce ends up being sold through the exchange, more farmers will also become stakeholders to ECX. It is the goal of ECX to provide the Ethiopian farmers with access to: the latest prices for traded commodities, which act as reference prices for local trade, market trends, to make better predictions about for instance seasonal fluctuations in price and volume, fair prices discovered by the market (as opposed to set mainly by major market actors), information about the standardized grading system used when selling crops through ECX, especially for coffee where a little extra processing on the farmer s side can greatly increase the value of the harvest, a well functioning market, where payment and delivery (in terms of money, quality and quantity) functions in and efficient, transparent and reliable manner, Cooperative Unions It is common among Ethiopia s small scale farmers to form cooperative unions of around ten to twenty farmers. Such unions have more market influence than individual farmers and can also afford to have an office with for instance a computer and an Internet connection, which provides the members of the union with greater access to market information. A few cooperative unions choose to legally bypass the exchange and sell their produce directly to the export market, but in general, the unions share the needs of the farmers

20 8 Background Traders Most of the goods that are traded through ECX take the route from producer to exporter via one or many traders. Such traders can be either supplier companies that buy up production from farmers in their region of the country, or individual brokers who act as representatives for cooperative unions, supplier companies or exporters on the ECX trading floor. All traders require real-time access to the latest prices, as well as a deeper understanding of current market trends. Supplier companies and cooperative unions that sell their goods through the exchange require a secure and efficient warehousing system, access to information about grades and standards and reliable payments once a trade has taken place Exporters Ethiopia s export trade, in particular coffee, is strongly consolidated into a handful of major export companies. The exporters connect the local market to the international market, handle shipping and make sure the goods meet the requirements of the foreign buyer. Exporters need a market with good international reputation, high international penetration and high liquidity, in addition to access to the latest prices and trends. Exporters also benefit from the quality and quantity guarantees that ECX provides for all trades that go through the exchange Foreign Buyers To attract the attention and capital of buyers on the global market (both large actors such as Starbucks and Nestlé and smaller companies), the Ethiopian market needs to be highly transparent and reliable, as well as offer competitive prices Media Ethiopian newspapers, TV and radio all publish regular updates of ECX market prices. In addition to this, they and international media require access to data showing market trends and experts interpretation and analyses of the same, preferably in a format that is adapted to the individual media Researchers Private and public analysts in areas such as finance, agriculture and societal development wish to have access to long time series of high resolution data, which can be analyzed for conclusions that affect decision-making on many levels. ECX therefore needs to provide access to accurate historical data in an easy to use format Policy Makers Political decision makers in Ethiopia require an understanding of historical, current and future market economics to best be able to promote action that benefits the long term development of the country. ECX can aid in this process by providing a transparent, efficient and stable market with high international reputation.

21 2.3 Market Data Dissemination Channels ECX Internal The performance of many different functions at ECX depends greatly on individuals understanding of collected data about warehouse inventories, memberships, cash deposits, trades and market trends. The tools currently available to staff and managers typically allow extraction of individual data items based on some search criteria, but there is rarely any way to get an overview of the data or to extract patterns that are difficult to formulate in standard query languages. Tools which increase the accessibility of the information contained in all this stored data would likely help in finding operational bottlenecks and areas of improvement, as well as in improving both the efficiency and quality of the day to day work. 2.3 Market Data Dissemination Channels A wide range of different dissemination channels are used to spread ECX market information to the different stakeholder groups. This section briefly describes the different media, what information is disseminated through them and the number of people that rely on each for regular access to market information Feeds to International Vendors A limited number of high end users around the world subscribe to live ECX trade data feeds, through Reuters and Bloomberg. These feeds can then be input to different software tools specialized on handling high resolution financial data. Stakeholder groups targeted: Number of people targeted: Exporters, foreign buyers and researchers Hundreds ECX Website The main two purposes of the ECX website are to make the exchange accessible to the global market and to provide a comprehensive source of information for people who need to know how to trade on the exchange, what rules and regulations apply to different commodities and other operative information. In terms of market data, the website provides full access to prices and volumes for what has been traded, but the presentation is limited to tabular numerical data for the prices and volumes of the current day s trade (listed on the front page) and a form for requesting prices and volumes for past trade (Figure 2.2). No graphing tools are available to make the data easier to understand or to create overview of different market trends. Stakeholder groups targeted: Number of people targeted: Everybody except individual rural farmers Thousands Market Bulletin The ECX market bulletin is a one page document that is published weekly and contains a summary of recent market prices and comparisons with international market prices.

22 10 Background Figure 2.2: ECX website with solution for requesting historical trade data. The bulletin is also currently the only ECX publication that contains an expert s interpretation and analysis of recent market trends. Distribution takes place electronically through an list to which users can subscribe and physically through the ECX main office, where people can pick up a printed copy for free. Stakeholder groups targeted: Number of people targeted: Everybody except individual rural farmers Thousands TV, Newspapers and Radio The bulk of the ECX market data dissemination takes place through television, newspapers and radio, which together have almost full coverage of Ethiopia. Published market information in mass media is mostly limited to listing of the most recent market reference prices. Stakeholder groups targeted: Number of people targeted: People inside Ethiopia Tens of millions SMS and IVR By the end of 2009, ECX plans to deploy solutions for accessing the latest price information through toll free SMS (short message service) and IVR (interactive voice response) services. Such services can complement mass media by utilizing standard telephone lines to provide 24 hour access to updated information. Stakeholder groups targeted: Number of people targeted: Farmers, cooperative unions, traders and exporters Thousands to tens of thousands

23 2.4 Stakeholder and Dissemination Channel Summary 11 Figure 2.3: Price display boards located at a rural market site (left) and at the ECX trading floor (right) Price Display Boards A number of major market places around Ethiopia have been equipped with electronic price display boards (Figure 2.3), which are continuously updated with live prices from the ECX trading floor. Stakeholder groups targeted: Number of people targeted: Farmers, cooperative unions and traders Hundreds of thousands Data Kiosks Sometime during 2010, ECX plans to begin deploying a novel solution to market data dissemination, tailored to Ethiopia s rural needs - data kiosks. Each of these kiosks will provide a single village and its surroundings with access to updated information similar to what is published on the ECX website. The initial plan is to deploy 200 such kiosks, but if the project goes well, it may be extended to 5000 sites, which would cover most of the villages in the country. Stakeholder groups targeted: Number of people targeted: Farmers and cooperative unions Hundreds of thousands to millions 2.4 Stakeholder and Dissemination Channel Summary Table 2.1 shows a breakdown of stakeholder groups and the intended target audiences of the different channels used to disseminate ECX market information. Dissemination channels which are only available to parts of a stakeholder group, due to geographical or financial factors, are marked as rings. From this summary we can see that ECX market data dissemination reach is high, from rural producers to the international market and that further improvements are planned for the producers. While all stakeholder groups have access to recent reference prices from the market, access to market analysis and trends is so far mostly limited to stakeholders late in the chain of trade.

24 12 Background Dissemination channel Rural farmers Cooperative unions Floor traders Stakeholders Feeds to international vendors Website Market Bulletin TV Newspapers, radio SMS, IVR a Price display boards b Data kiosks in villages c Exporters Policy makers Media Researchers Foreign buyers a Deployment planned for December b 14 sites are currently operational out of a planned 200 by the end of c Deployment planned to begin during Table 2.1: Breakdown of stakeholder groups and the intended target audiences of the different channels used to disseminate ECX market information. 2.5 Information Visualization & Visual Analytics The scientific field of information visualization deals with using visual representations to help people analyze, understand and extract meaningful information from large collections of data, which usually lacks an inherent spatial representation. To achieve this goal, the visual representations employ a number of central techniques, including: mapping values to attributes such as position, color and size, which helps the user to create a mental model of the data, interactively letting the user drill down in the dataset to access the information at increasing levels of detail, using coordinated views to show different representations of the same data, or to show selected subsets of the data while retaining overview. Visual analytics is the science of analytical reasoning facilitated by such interfaces. People use visual analytics tools and techniques to synthesize information and derive insight from massive, dynamic, ambiguous, and often conflicting data; detect the expected and discover the unexpected; provide timely, defensible, and understandable assessments; and communicate assessment effectively for action [19]. Successful applications of these techniques can be found in a great variety of fields, including finance, medicine, environmental sciences, defense, statistics and IT. Figure 2.4 shows six examples of graphical representations which can be attributed to the information visualization field. These representations, or graphs, are a small

25 2.5 Information Visualization & Visual Analytics 13 Figure 2.4: Six examples of information visualization methods. From left to right, top row: network graph, parallell coodinates plot with overlayed histograms and filter sliders, scatter plot. Bottom row: treemap, horizon graphs, density map. subset of a wide range of representations suitable for bringing out different types of information; such as variations in time, relational structures, multivariate correlation or geographical distributions. Different graphs are often combined to balance their individual strengths and weaknesses. However, deployment of visualizations for information dissemination requires both that data containing this information has been collected and that sufficient distribution channels are in place. If the requirements of a specific presentation method are not met, the presentation needs to be adapted to the limitations of the environment in which it is to be used. An example of this is that visualizations intended to be distributed in print media cannot rely on interaction or animation to convey information and that colors may be limited to grayscale, but instead they can usually be produced in higher resolution than what is available on a computer screen. Another example is that the sample rate (e.g. in time or space) used when collecting the data imposes limitations on what can later be displayed Information Visualization Applied at ECX As described in the previous sections of this chapter, ECX is a highly data intensive organization. Many external stakeholders would benefit from improved understanding of complex market trends and internally most functions require good understanding of collected data about warehouse stocks, traded commodities, market actors and more. At the same time, the organization lacks adequate tools for creating this overview, both internally and externally. Information visualization and the related area of visual analytics are fields of research that can likely offer solutions to these problems, but which are previously unexplored in Ethiopia. Lack of exposure on all levels, from developers to potential end users, also means that nothing currently drives new development. As described in chapter 1, a central goal of this thesis is therefore to increase the general awareness among ECX staff and stakeholders of what is possible, so that further development can become driven by demand.

26 14 Background Discussions with ECX staff have outlined that visualization solutions likely can offer benefits within the following areas: Increase the general understanding among market actors of past and current market trends, such as movements in price, changes in supply and demand, seasonal changes in production volumes of different commodity types, local vs. international market trends and production trends between different commodity grades. Create an overview of past and current deposited warehouse volumes, separated by owners and commodity types; so that the ECX warehousing system can be adapted to better meet the requirements in terms of capacity and location. Improve the efficiency and capabilities of the ECX market surveillance department, which has need for great understanding of market trends and identification of patterns and outliers. Look at geographical production patterns to see how the Ethiopian agricultural sector performs over time. This should likely be done in collaboration with various authorities so that comparisons can be made with other indicators such as weather, population and income. Create an overview of the relationships and interactions between different ECX departments, such as between warehousing and membership or between trading operations and clearing and settlement (making sure that deliveries of goods and payments take place after a trade). This overview could be used for instance to look at the sources of income for the company and to find capacity bottlenecks in the chain of different systems.

27 Chapter 3 Development Tasks Two development tasks are selected to fall within the scope of the thesis work and to thereby meet the second objective defined in chapter 1.2. The first is to improve the presentation of public trade data on the ECX website, to better meet international and local needs and expectations. Although little new information will be added, better presentation tools should make the existing information far easier to understand and thereby more accessible to more people, which will increase market transparency. The second task is to develop a software application which can assist in the extremely time consuming day to day market surveillance work, in particular investigations that look for traders whose recorded trades in the past indicate behavior which is banned from the market. Depending on its success, such an application can potentially greatly increase the capacity and performance of the ECX market surveillance department, which ensures a better functioning market with higher integrity, i.e. lower sensitivity to manipulation from individuals or groups of market actors. Both tasks are selected for their likeliness to produce useful results within the scope of the project and for their high relevance to current ECX needs. 15

28

29 Chapter 4 Market Data Presentation on the ECX Website As described in section 2.3.2, the ECX website currently provides users with access to numerical trade data (daily prices, intraday price change and traded volumes for selected symbols and date intervals), but it is difficult to get an overview of current or historical market trends due to the lack of good presentation methods. Part of this thesis therefore looks into how the current presentation can be improved, with the goal of creating overview and making the information more accessible to users who find it difficult to interpret the large amounts of numbers. 4.1 What is ECX Market Data? Trade at ECX takes place during multiple trade sessions every day, each allocated to a specific group of commodity types. After each such session, the following prices are defined for every symbol (e.g. WSDB4) that was traded during the session. Opening price - a price representative of the first trades during the session. Closing price - a price representative of the last trades during the session. This price acts as a reference price for trade outside of ECX until the next session takes place. Day high - the highest price at which a symbol was traded during the session. Day low - the lowest price at which a symbol was traded during the session. Trade prices are measured in Ethiopian birr per ferasula for coffee and birr per quintal for grains. One ferasula equals 17 kg and one quintal equals 100 kg. Price increments are currently fixed at whole birr and for every session there is a preset price interval within which trades can take place, based on the previous session s closing price. Total daily trade volume is normally reported to the market measured in metric tons. The trade volume of individual trades is measured in lots, where one lot of coffee roughly equals the amount which can be loaded on one truck. There are two types of commodity trade that take place at commodity exchanges around the world. The first type is spot trade, i.e. trade of existing goods that are 17

30 18 Market Data Presentation on the ECX Website stored in a warehouse somewhere and can be delivered to a buyer immediately. The second trade type is futures trade, where traders purchase (and possibly resell) the rights to future harvest delivered at specified times. While the two forms of trade complement each other, ECX currently handles only spot trade. The ECX Market Data department deals with analyzing and reporting trade data to the various groups of market actors, in a timely, accurate and easy-to-interpret manner. Market transparency is highly dependent on well functioning dissemination of market data. 4.2 Methodology Development of the presentation solutions described in this chapter follows the steps listed below. It is desirable to have more direct involvement from all different user groups in the development process, but as this could not be arranged during the project, it will be left as a recommendation for future continued work. 1. Identify the information interests of different stakeholder groups. 2. Look at technical limitations and other factors that need to be taken into account. 3. Look at related work to see what visualization solutions are used by other exchanges and if any new promising methods have been presented in recent research. 4. Clarify 1-3 into requirements and guidelines. 5. Select solutions and implement. 6. Evaluate solutions with users. 7. Based on evaluation, possibly return to any of the previous steps to address discovered issues. 4.3 Scope As ECX already has staff in place to handle website development, the thesis work will be limited to developing presentation prototypes followed by implementing selected graph types in a code library which can be used to generate graphs from standardized input data. This library will then be used by ECX staff to deploy graphs on the website itself and to integrate them with existing website code and data repositories. 4.4 Usage Scenarios Section 2.2 described the different stakeholder groups who in some way depend on ECX market data. The people who use the ECX website as a source of information are typically found in the later stages of the chain of trade; some producers, cooperative unions and traders, but mainly exporters, foreign buyers, policy makers, reporting media and researchers. Many discussions with ECX staff, some with personal backgrounds as commodity traders, have together outlined that these stakeholder groups can be further simplified into three main usage scenarios which need to be considered. Although solutions for market data presentation should aim to support information gathering within these three scenarios, the scenarios themselves did not become clear until most

31 4.5 Technical Limitations 19 of the thesis work was already completed. Therefore, the solutions presented later on will be discussed in relation to these scenarios, along with the reasoning that was used when developing them. The first scenario is a producer (commonly a representative of a cooperative union, a large scale commercial farmer or a supplier who buys products from small scale farmers) who seeks information about the commodity type that he or she wants to sell on the market. The goal is to be able to time the sell decision to when the market conditions are most favorable to the producer. If prices were easy to predict, this would be simple, but in reality many factors contribute to future price changes and the trader needs enough information to be able to make an informed guess and to calculate the risks involved in the decision. The second scenario is for an active market trader, such as a broker who acts on behalf many producers, or a representative of a company that trades in many commodity types. This trader wants information about the overall state of the current market, which can be compared with previous knowledge to make out variations from the normal state. The trader wants to know about recent variations and trends in price and trade volume for different market segments, as well as factors which affect supply and demand, so that he or she can focus on trading commodities within segments that offer good profit margins. The third scenario is the planner, i.e. a major exporter, foreign buyer policy maker, researcher or market analyst, who tries to understand high level market trends over long periods of time. This includes price fluctuations, movements in supply and demand between market segments, periodical events such as harvest seasons and deviations from these cycles. 4.5 Technical Limitations Due to the environment in which ECX operates, a number of technical limitations exist. First of all, users inside Ethiopia are accessing the website almost exclusively using a dial-up connection. This not only means that content takes longer time to load compared to in a broadband dominated environment, but also that the cost of requiring users to have installed a browser plug-in (e.g. Adobe Flash Player, Microsoft Silverlight, Windows Presentation Foundation or Java) is much higher. In fact, even the web server is limited to around 256 kbps peak upload speeds, making low bandwidth usage a high priority. It is also important to keep in mind that any developed solution sooner or later needs to be maintained and extended by ECX developers. Currently, all development takes place in Microsoft.NET environments and no staff has more than very limited experience using any language (or accompanying class library) suitable for highly interactive web content, such as Java or ActionScript. Although it is possible to train staff in using new development platforms,.net should be the first choice and other platforms can be selected only if they offer benefits that are worth the investment in training.

32 20 Market Data Presentation on the ECX Website Figure 4.1: Google s stock performance summary by sector (left) and part of the Financial Times stock search solution (right). 4.6 Related Work Many providers of different types of market data currently operate on the Internet; mainly exchanges (both for stocks, currencies and commodities) and third party news agencies. Some of the most significant providers - Bloomberg [1], Financial Times [18], Yahoo Finance [21], Google Finance [6], CME Group [3], NASDAQ [12] and Metastock [11] - have been selected to act as a sample of common presentation practices. When displaying information about more than a few symbols at a time, tables are the prevalent presentation method. Sometimes these tables are extended with a visual aid consisting of a colored arrow which indicates a short time positive or negative trend, usually the last day s change. The tables can usually be sorted by some criteria, such as price change, to get a list of the best or worst performing symbols in the market. Two websites offer some additional functionality for providing a context in which to interpret the values. Google offers an aggregated sector summary which shows the performance distribution of stocks grouped into different sectors (Figure 4.1), although the variation among stocks within each listed sector is much greater than that between different sectors. Most sectors also show a distribution around zero, which together with the variability indicates that this grouping corresponds poorly to actual performance clusters among stocks. The Financial Times instead extend their data tables with tooltip price trend line graphs shown when the user moves the cursor over a symbol in the table and they offer a novel search function (Figure 4.1) where the user can specify search criteria by visually selecting segments of histograms of different attributes and end up with a list of stocks that match the search. The website also contains macromaps for currencies and markets, where countries and market names are placed on a geographical map and colored based on their performance. In addition to the tabular data, all sites offer some type of presentation of time series data for single symbols, ranging from static server-side generated line graphs [21, 1, 3] to more or less interactive Flash charts [21, 6, 18, 12] and fully fledged standalone applications available for download to paying customers [1, 11]. Figure 4.2 illustrates an interactive time series graph provided by Yahoo. Some of the sites also let their users adjust the graphs by switching between different time intervals and different chart types (line, area, candlestick and bar). Noteworthy extensions to the common functionality are integration of technical analysis tools [21, 3, 11] and links to relevant news articles automatically placed along the timeline [6]. Typical graph interactivity, where available, is to display the exact values for the time sample at the mouse cursor s

33 4.6 Related Work 21 Figure 4.2: Interactive time series graph provided by Yahoo Finance, showing index value, trade volume, selected technical analysis indicators and plotted time period. position. The Metastock software stands out by offering the by far widest selection of graph types as well as technical analysis tools, but also comes at a significant cost. To summarize, all of the examples listed here focus on providing expert users, normally active traders, with access to the latest price changes in the market and lists of the current best and worst performing symbols. Sorted tables are used to highlight extreme performers (high and low) and time series data is presented for selected symbols or indexes, sometimes with options to compare with indexes, other symbols or different technical analysis overlays. The standard chart types are line, candlestick, bar and area charts and trade volume is often presented together with price trends. The similarity between different providers likely means that frequent users have a very easy time moving from site to site. None of the examples, perhaps with the exception of Google and the Financial Times, attempt to provide a decent overview of the entire market or of individual market segments, targeted at irregular or non-expert users. There is also an enormous focus on presenting long lists of numbers which can be daunting for any non-expert user approaching the service. While academia has come up with a couple of different approaches to addressing this lack of overview, commercial applications of the theory are still relatively limited. Implementations of horizon graphs, heat maps, stack graphs, scatter plots and more are available from commercial actors such as Panopticon [13] and Tibco Spotfire [17] and to some extent through open source frameworks such as Prefuse [14], Prefuse Flare [15] and GAV [5]. Some work also addresses the poor performance of classical market segment classifications (e.g. IT, healthcare), including [16] which presents a self-organizing map (SOM) of line graphs in which different stocks are assigned coordinates in 2D-space.

34 22 Market Data Presentation on the ECX Website 4.7 Requirement Summary Based on the above usage scenarios, technical limitations and related work, the following points will act as guidelines when designing different solutions for market data presentation. The solutions should give the user a good understanding of current and historical market trends for daily opening, closing, high and low prices, as well as trade volume and supply and demand for different commodity types. Graphs should be easy to interpret, even for people with very little experience interpreting common graphical representations of trade data. Where applicable, presentation methods should follow international standards. The solutions must be accessible using a dial-up connection. The presentation tools should preferably be developed in.net, as this is the only platform that the ECX developers has experience with. If a different platform is selected, the benefits must outweigh the increased difficulty of continuing development of the solution after the thesis project is completed. Initial investment cost should be kept at a minimum. 4.8 Presentation Solutions This section presents the different suggested extensions to the existing market data presentation functionality on the ECX website Price Trend, Individual Symbols When work began, it was assumed that the standard functionality provided by other actors (raw data updates, price history for selected symbols) could be treated as a "minimum specification" for the ECX website as well and that extensions would be made to better target the needs of the wider range of user groups. The first step therefore became creating a webpage for display of time series containing daily price and volume data for one or a few different symbols at a time. This was also expected to be a natural step to take for users of the existing solution, which only displayed numerical values in a table (Figure 2.2). The necessity to provide this service to website users of all categories motivated picking the graph technology with lowest requirements on the client side; static images rendered on the server. The simplicity of static images also allows for very rapid graph implementation, producing quick results that can later be re-implemented using other technologies if direct interaction with the graph proves to be necessary. Figure 4.3 shows five different graph types which have been implemented. Note especially the differences in depicting the strong downwards trend in the beginning of the interval, followed by a rather strong upwards trend two weeks later. A layered approach was used when implementing the graph, so that from a technical point of view any of the representations can be combined in the same graph. The first type is the widely recognized line graph, with optional symbols that indicate individual time samples. This graph type only displays a single indicator value

35 4.8 Presentation Solutions 23 (closing price), but it is the easiest to interpret and it only requires one horizontal pixel per time sample, which allows for display of long time series with high resolution. The second graph type is a line graph which has been extended with display of daily high and low prices. The line shows closing price, while the top of the brighter area represents daily high price and the bottom the daily low. Sample indicators are again optional. Addition of the area makes it possible to see the entire range of prices, while still putting emphasis on the closing price. If the closing price line from the previous method is removed, one is left with a graph that displays only the daily high and low, focusing on showing the price range within which trades have taken place. This is the third method. A fairly complex but well established presentation method is the Japanese candlestick graph [10], which shows daily opening, closing, high and low price using a box and a line. The graph emphasizes intraday movements, i.e. price changes within each trade session, and is the fourth implemented graph type. The last graph, a bar chart [10], is again both complex and well established. All four daily price values are displayed using a vertical line for the high-low range and two horizontal lines for opening and closing price. The glyph is colored red or green depending on the intraday change. This graph places equal emphasis on all the different types of information presented in the other graph types. This and the candlestick graph both use five to seven pixels per sample, resulting in that time series cannot be as high resolution as for the line graph. The vertical axis of the graph has been placed on the right hand side, to be able to place a label showing the last day s exact closing price on the axis. Figure 4.3: The five implemented chart types for displaying time series of price data; line chart with glyphs indicating individual samples (a), line chart with semi-transparent area indicating day high and low (b), area only (c), candlestick chart (d), bar chart (e).

36 24 Market Data Presentation on the ECX Website Figure 4.4: Graph on the ECX website displaying price and trade volume history for two different symbols. Trade volume can be used to estimate the level of market support for a price, or the likeliness that that a price level will be sustained in the future, and it is plotted using either an area graph or a bar graph. Both of these graph types are again fairly established standards for showing this information. Figure 4.4 shows how composite graphs are generated on the ECX website based on user selected parameters. The graph in the figure is generated using the following default settings; line graph with area showing price, glyphs for indicating days with trade and area display of trade volume. The figure shows six months of data, while the default is three. The two time series are for grade eight unwashed Lekempti coffee produced during two different years, as indicated by the color legend at the top of the graph. Below the graph is a data table with the numerical values represented by the graph (not fully shown in the figure). This webpage for displaying price and volume trends for individual symbols addresses mainly usage scenario number one Context of Latest Market Update The start page of the ECX website has for a long time contained a table with a summary of the current day s trade, much like what is presented by other exchanges. A problem with this solution is however that any updates need to be interpreted in the context of previous days values. Tufte discusses this issue in [20] and suggests: At the heart of quantitative reasoning is a single question: Compared to what? Small multiple designs, multivariate and data bountiful, answer directly by visually enforcing comparisons of changes, of the differences among objects, of the scope of alternatives. For a wide range of problems in data presentation, small multiples are the best design solution. The table was therefore extended with a small multiples solution using line graphs to show the past month s closing prices, as illustrated in Figure 4.5.

37 4.8 Presentation Solutions 25 Figure 4.5: Small multiple line graphs added to the market update table on the front page of the ECX website. The graphs not only show the price trend of the individual symbols, but also the level of correlation between product grades within a commodity group, such as washed Sidama coffee of different grades. To a lesser extent it is also possible to observe similarities (or lack thereof) between coffee of different origins. Each graph image is linked to the corresponding larger price and volume graph (previous section), to provide the user with drill down functionality. This solution mainly addresses usage scenario number two; the frequent trader who requires updates to their existing knowledge of the current state of the market Price and Volume Trend, Entire Market The small multiple line graphs on the front page are a space efficient way of providing context to the daily update, but they do not provide market overview as the front page only displays the subset of symbols that have been traded during the current day. Overview requires looking at all parts of the market simultaneously and to achieve this two alternative solutions have been proposed to ECX. Work on web deployment has begun, but neither has yet gone public on the website. Both methods aim to create overview, while retaining sufficient detail for the user to make out outliers. A small multiples solution is again used to present the price information, but this time in a grid with commodity type on the horizontal axis and grade on the vertical axis. Each small graph is again also linked to the corresponding larger price and volume graph, providing drill down functionality. The first solution uses a line graph to show price trend and graph background color opacity to show the percentage change between the first and last sample in the graph. Green and red represent positive and negative total change, respectively. The background color makes it possible to roughly estimate the magnitude of the variations in the line graph and also makes it easy to pick out the symbols with the greatest or smallest change within

38 26 Market Data Presentation on the ECX Website Figure 4.6: 200 symbols rendered as a grid of line graphs, with origin on the horizontal axis and grade on the vertical axis. the plotted time interval, much like in a heatmap. By comparing the different grades of a specific coffee type, it is also possible to see converging or diverging price trends among the different grades. As it can be assumed that higher grades will always go at higher prices than lower grades, coffee types that have a more negative trend for high grades than for low grades exhibit a converging trend (i.e. the price gap between grades decreases). In contrast, coffee types that have a more positive trend for high grades than for low grades show a diverging trend (the price gap increases). Figure 4.6 shows what this solution can look like for 200 symbols. ECX currently has around 500 different tradable symbols defined, thus a web page form could be used to let the user limit the display to only a subset of symbol groups that she wants to look at; e.g. only washed coffee or only coffee produced during a specific year. The main reason why this solution is not yet publicly available is that there are many different possible ways of sorting the symbols, not only by origin, grade and processing type, but also regular/under screen (in the case of coffee) and current/previous harvest year. User evaluations are required to find the sort order that best lets users make the comparisons which are of most value to them. An input form can then be designed to let users switch between different grid sort orders and different time intervals to plot. The second proposed solution, illustrated in Figure 4.7, complements the first by replacing the line graph with a horizon graph displaying price change in percent from the first sample. This is a more complex representation mainly intended for demanding expert users such as exporters and market analysts, in which it is possible to accurately estimate the magnitude of the price change shown in the graph to within a few units [7]. Both of the two methods above are recommended to be supported by a second set of graphs which show trade volume over time. The suggested graph type to use for this is a horizon graph with a user adjustable number of bands (by default only one, resulting in an area graph). However, regardless of the representation used, the vertical value axis range should be shared among all the volume graphs, to make it possible to distinguish highly traded symbols from those with little or close to no trade. Using the same grid layout for the price and volume graphs makes it possible to compare the location of "hotspots" within the two grids, in addition to observing the volume

39 4.9 Evaluation 27 Figure 4.7: 200 symbols rendered as a grid of horizon graphs, with origin on the horizontal axis and grade on the vertical axis. distribution among the different grades and types. A last recommendation for the design of the final web page is to include a form where the user can select the plotted time interval and the sort order of the columns (alphabetically, by total trade volume or by average overall price change). These solutions aim to provide information useful in scenario two and three; the active trader and the planner Trade Volume Trend by Market Segment High level movements in trade volume between different market segments over long periods of time are interesting for market analysts and decision makers; as such trends are closely connected to the economy of the producers and the regions in which they operate. Producers and traders can also use the information to look at periodic fluctuations in trade volume, which influence when it is favorable to sell or buy different commodity types. One solution for presenting this information is to use a stacked area graph, illustrated in Figure 4.8. The figure shows weekly trade of export coffee (washed and unwashed) from when coffee was introduced on the exchange at the beginning of December 2008 to the 25th of May 2009 when the figure was created. The implementation currently being finalized for deployment on the ECX website allows display of total weekly trade volume or trade value (volume multiplied by price) for coffee. The trade can be grouped by origin, grade or symbol and filtered to show only export, washed, semi-washed, unwashed or local coffee. This solution targets mainly the third usage scenario, the planner, but also to lesser extent scenarios one and two. 4.9 Evaluation Although user evaluations can be immensely helpful when designing visualization solutions, no such evaluations have been performed other than with ECX staff (and even then only informally). ECX employees are however not at all representative of most of the website user groups and therefore website usage statistics have been looked at for indications of whether the published solutions are adequate or not. As only the page

40 28 Market Data Presentation on the ECX Website Figure 4.8: Stacked chart displaying total weekly trade volume, grouped by origin. for display of price trends for individual symbols and the small multiple graphs on the front page are presented to the public thus far, the remaining solutions cannot yet be evaluated in this manner. In January 2009, a website survey was published on the ECX website which indicated that the around half of the external website visitors were exporters living in Addis Ababa, who accessed the website multiple times a week using a dial up connection. The remaining half was split fairly equally among all the other stakeholder groups. While the statistical significance of this survey is low (only 37 people responded), it still shows that most users likely fall into usage scenario number two and three. During a period of one month, from late April to late May, the page for presentation of historical prices was accessed 320 times (excluding requests from ECX and web crawlers). Visitors came from 117 different IP addresses, but for Ethiopian users accessing the site on dial up, IPs are dynamic and of little value when tracking individual users. 40% of the requests came from Ethiopian users. What is more interesting is that almost 90% of all the requests were for graphs with settings that indicate that the user navigated to the page using one of the small multiple line graphs on the website s start page (line graph with area showing the past month s price and volume for a single symbol). During the month, no individual symbol has been plotted more than 15 times and no individual graph more than seven times (different date intervals are counted as different graphs). Around 5.3% of the total number of page views for the entire ECX website was for the page for presentation of historical prices. In short, close to no users make use of the graph selection interface to plot information of special interest. Without further user evaluations, it is difficult to know what the reasons are behind this low usage frequency, but the explanation could be one or a combination of the following: Users feel that the interface for changing graph types, time interval and plotted symbols and indexes is difficult to understand or to use. The graphs do not provide information that is of value to the website users, regardless of the settings used to generate them. This could in turn be due to either that the graphs themselves are poorly designed, or that the solution works

41 4.9 Evaluation 29 well for the group of users for which it was designed, but that very few members of this group access the website. The information on the front page matches the users needs very well, and the use of the separate graph page is limited to displaying a selected subset of the information in higher resolution. Using graphs to represent data is a relatively new concept on the Ethiopian commodities market and the current low usage frequency is due to that the users are unfamiliar with the benefits offered by the graphs. In time, the new tools will make their way into the work procedures of more people. If it is true that most website users who seek access to market data fall within usage scenarios two and three, the overview solutions currently under development will likely attract more visitors than the published page for looking at price history of individual symbols. Expanding the price history page with more information than only price and volume could also make it much more useful when trying to understand what caused the observed prices or price trends. The great range of standardized grades and origins for coffee at ECX results in that each of these individual symbols gets a very small part of the total trade. These low individual trade volumes in turn mean that daily price ranges are very small, often being limited to a single value for each symbol. This fact was not considered when selecting ways to graphically represent the data and especially the graph types illustrated in Figure 4.3 (d) and (e) are unsuitable for displaying the current real trade data. The strengths of these representations could be better utilized if trade could be grouped by week when displaying long time series. Although being an international standard, the focus on closing prices means that there is no way to see how the daily trade volume was distributed between the day s highest and lowest price. A way to solve this could be to render the daily trade prices using some kind of histogram, where the height of the graph corresponds to trade volume. Showing a volume weighted average price could perhaps serve the same purpose.

42

43 Chapter 5 Market Surveillance Application The second software development task undertaken within the scope of this thesis, as outlined in chapter 3, is a software tool for assisting in the day-to-day work of the ECX market surveillance department. This chapter describes the development process and the final product and informally evaluates its performance with its main current user. 5.1 Methodology Development of the software tool described in this chapter took place iteratively in close collaboration with staff from the market surveillance department. The staff members contributed by phrasing questions that were difficult to answer using available methods and by providing feedback and testing of the application. As new questions were raised, new additions were made to the application so that they could be answered, followed by informal user evaluation which highlighted areas of further improvement. 5.2 Aim Market surveillance is a complex area that deals with finding many types of undesired market behavior and cases of system abuse. Some of the more data-intensive tasks are finding and investigating cases of potential cross trading, price fixing and other forms of market manipulation, as well as getting a general overview of the trade and to create mental models that can help in distinguishing between normal behavior and outliers in the data. So far, the only software tools at the market surveillance department s disposal to help in these tasks are Microsoft Excel and an in-house developed software application for retrieving database records of past trades. The tools are not sufficient and a software application will be developed that is tailor-made to simplify data-intensive investigations performed by the department. During the development process, the following questions act as guidelines for the selection and design of different features in the application. 1. Who trades with whom and what do they trade? 2. Are any people trading overly much with each other? 3. Are any people consistently trading above or below average market prices? 31

44 32 Market Surveillance Application 4. Who trades commodity type X? 5. Who are the dominant traders, for all commodities and for commodity type X? 6. Which are the most well established trade paths from sellers to buyers? 7. What is the total amount of trade for commodity type X during time period T? Combinations of these questions are central in the investigations conducted by the department and all of them are highly time-consuming to answer using the available investigation methods; often so time-consuming that they are in practice impossible to investigate. For most of the questions, the problem is also that the lack of overview means that investigation can only be initiated after suspicion has been raised concerning the actions of an individual trader. It is highly desirable to instead be able to approach the issue from the other end, i.e. to explore the data for indications of suspicious behavior, regardless of which trader it belongs to. 5.3 Related work Specialized software tools for market surveillance are in use at exchanges around the world. However, no publications were found about any such tools that are relevant to the ECX commodity spot market, which is likely both due to that surveillance is a sensitive topic and that the software solutions have been developed by commercial actors rather than academia. 5.4 Features This section describes the different features of the developed software application Data Import ECX stores information such as trade time, seller and buyer member and client, price, volume and type of commodity for every trade that takes place on the trading floor of the exchange. The normal mode of data import to the software application is to connect to this database and retrieve all trade records within a specified time interval. Alternatively, the user can import data stored in a Microsoft Excel document, which allows for exploration of custom datasets as well as a way to use the application when direct database access is for some reason unavailable Trader Network The central view of the software application is a self organizing network of nodes, and connections between them. Nodes represent traders (buyer and seller members or clients) and connections represent all trades between two traders. The view is implemented in C# (.NET 3.5) and Direct3D as an interactive system of masses connected by springs in a two-dimensional coordinate system. Figure 5.1 illustrates what the view looks like when populated with a test dataset representative of two months of coffee trade. Due to the sensitive nature of the real trade data, this test dataset will be used for the figures in this report. Buyer names are colored dark grey and seller names

45 5.4 Features 33 Figure 5.1: The market surveillance application interface. are colored green, while the lines that connect traders are colored with a gradient from green (seller) to brown (buyer). The green and brown colors are selected to match the shirts which are worn by the two trader groups while in the trading pit. The size of trader names and the width of connections are scaled according to either the trade volume or the number of trades that is represented by each item, depending on the user s choice. The user can also zoom and pan in the view to look at specific sections of the network, as well as grab and pull nodes to either help the algorithm sort out the connection overlaps quicker, or to manually re-sort the nodes according to some custom criteria. The view can be filtered to exclude all but the most well established trade connections, which is done by setting visibility thresholds for the minimum number of trades or the minimum number of lots that a link represents. Figure 5.2 shows the effect of this filtering, where the number of displayed connections from the test dataset has been reduced from over 3000 to around 200 and the number of traders from over 1300 to around 150. This filtering makes it clear that the market represented by the test dataset is dominated by a few major buyers, who buy from a large number of small sellers. The view can also be filtered to show only trades of a particular commodity type (e.g. coffee, sesame), processing type (e.g. washed, unwashed) or coffee type (e.g. washed Sidama, unwashed Forest), or a combination of the three. These filter options are dynamically populated using the retrieved data, which means that although the filter labels are based on coffee, new commodity types introduced in the future will be displayed as filter options by the application. From the network graph, the user can drill down in the dataset by selecting individual traders (nodes) or pairs of traders (connections) to get further information about

46 34 Market Surveillance Application Figure 5.2: The effect of network filtering; all trade (left), only trade connections with more than 6 trades (right). Figure 5.3: One selected buyer with surrounding sellers. The values displayed on the lines indicate the ratio of the buyer s transactions that have taken place with the respective trade partner. them. When the user selects a trader by clicking on them, the application displays trade percentages (Figure 5.3) for connections that make up more than ten percent of the trader s trade, measured in trade volume or number of traders. A table view containing each trade that belongs to the selected trader also shows up at the bottom of the screen. Traders who have acted as both sellers and buyers show up twice, but with an appended suffix of (B) for buyers and (S) for sellers Alerts Questions two and three posed by the market surveillance team (are any people trading overly much with each other, or at prices consistently above or below the rest of the market) are dealt with using a system of automated visual alerts. Highly focused

47 5.4 Features 35 Figure 5.4: Alerts displayed for extreme trade focus (left) and anomalous prices (right). trade is defined as that an individual has traded a high percentage of their number of trades or a high percentage of their traded volume with a single other trade partner. A numerical threshold exists so that traders with very little trade are ignored, as they would otherwise always be highlighted by this system (a trader with a single trade always traded exclusively with a single trade partner). Price anomalies are calculated by first computing the volume-weighted average price for each symbol and each day, followed by subtracting this average from each trade s price. This gives an absolute price difference for each trade from the daily average price, which is also converted into a percentage difference from the daily average. The alert system looks at the absolute maximum (positive or negative) for both these values, among all trades between two traders (one connection between nodes). Connections that contain a trade with a price that is too far from the daily average according to a user specified threshold are highlighted. Highlights in blue indicate an unusually high price and highlights in red an unusually low price. Figure 5.4 illustrates both types of alerts. It is important to note that neither of these alerts necessarily imply that a trader has taken part in any kind of unauthorized behavior. However, the alerts attract the attention of the surveillance analyst, who can then investigate the matter further and either dismiss the alert as a false positive or take appropriate action. All numerical thresholds for the alerts can be adjusted by the user Search A search field for trader names is available in the application to let the analyst quickly navigate to a specific trader of interest Trade Price Difference When the user selects a trader or a connection in the network view, the trades belonging to the selected item are plotted in a small graph in the top right corner of the main

48 36 Market Surveillance Application window (Figure 5.5). The plot shows individual trades as dots, with area proportional to trade volume, with time on the x axis and price difference in Birr from the daily volume weighted symbol average on the y axis. A dot with a value of zero on the vertical axis means that a trade took place at the average price for that symbol that day. Dots with the same color represent trades of the same symbol. The purpose of this plot is to give the analyst a quick indication of whether the trader has a tendency to go for prices that are different from the market average or not, and whether the trader tends to sell or buy in small or large quantities. For deeper analysis, the analyst can bring up the trade history view discussed next. Figure 5.5: The small chart displaying transaction prices relative to the daily symbol average Trade History By double clicking on a trader or connection between traders, the user can bring up a drill-down view of the selection, presented in Figure 5.6. This view is a time plot showing all trades for a single selected symbol and the user can switch between all symbols that have been traded by the selected trader. Each dot represents a single trade, with area proportional to trade volume. Trades that belong to the selected trader are highlighted in red. Displayed are also volume-weighted five working-day moving average prices for the symbol (blue line) as well as for all the symbols with the same processing type (green line). A gauss kernel is used when computing the moving averages, to bias the values towards the trades during the day in the middle of the interval. The graph implementation consists of extending the time graph from the graph library used on the website (see chapter 4) with a layer that shows samples as colored dots. The axes, grid lines and line graphs are all already part of the library. The user can zoom in and out in time by scrolling with the mouse wheel, while right clicking on the graph toggles between showing the entire trade history and only showing trades from the clicked date. Functionality exists to print the graph Trade Volume over Time Within the trader drill-down view (a pop-up window), it is also possible to look at the total trade volume over time for the selected trader or connection. This is rendered in the same way as market volume trend is rendered on the ECX website (described in section 4.8.4) and trade can be grouped by commodity type, processing type, coffee type or trade partner. Trade volumes are summed up to weekly totals to reduce the graph noise introduced if plotting daily trade, as few traders trade each of their traded commodity types on a daily basis. Figure 5.7 shows the trade of one trader, grouped by symbol (top) and by trade partner (bottom). Three trends are visible; first, the trader s total trade volume has decreased over time; second, the trader has moved from

49 5.4 Features 37 Figure 5.6: Trade history for a single symbol. Each dot represents a single trade, with area proportional to trade volume. Red dots belong to the selected trader. Figure 5.7: Weekly trade volume for a single trader, grouped by symbol (top) and trade partner (bottom). trading both unwashed Jimma ( UJM ) and washed Limmu ( WLM ) towards only trading low-grade unwashed Jimma; and third, recently the trader has mostly traded with Issak Ibrahim Aji Total Trade and Top 10 The user can bring up another window that for each commodity type displays the market s total trade volume, number of trades and number of traders. In case the user has filtered the network graph to only show a specific commodity type, that filter setting will apply also to the total trade list and the list switches to displaying values for individual symbols. It is also possible to get lists of the top ten sellers, buyers and connections, by trade volume and by number of trades. These lists are also filtered using the same commodity type filter as for the network view. The two windows are illustrated in Figure 5.8 and the information presented in both of them can be printed from within the application.