Managing Tempora Financia Data in an Extensibe Database * Rakesh Chandra and Arie Segev Water A. Haas Schoo of Business University of Caifornia at Berkeey and Information and Computing Sciences Division Lawrence Berkeey Laboratory Berkeey, CA 94720 emai: crakeshqcsr.b.gov, segevocsr.b.gov Abstract 1 Introduction Compex financia products and trading appications are difficut to mode and impement in conventiona commercia databases due to tempora, object, rue and other data support requirements. Extensibe database systems provide a better soution because of the abiity to define compex data types, manipuate data of type procedure, define rues, operators and access methods to optimize these operators. The paper discusses the design issues in modeing financia trading systems in extensibe DBMS. The compexity of financia products is anayzed and strategies for modeing these products are pro posed. Operators reevant to financia trading are discussed aongwith access methods to optimize these operators. The paper describes an impementation of a financia trading system in POSTGRES. Keywords: Tempora Databases, Compex Objects, Rue Processing, Extensibe Databases. Tbia work wa rupported by an NSF Gmnt IRI-9116770 aud by the Appied MJhematicd Scienceo Rcseaxzh Progrun of the Office of Energy Research, U.S. Department of Energy under Contract DEACO3-76SFWOQS. Pennirrion to copy witiout fee a or part of tir materia ir granted provided tat tac copier are not made or distributed for direct commercia advantage, tac VLDB copyrigt notice and tae tite of the pubitaiion and iir date appear; and notice in given taat copying ir by pemaierion of tae Ver) Large Data Base Endowmeni. To copy oiaetwire, or to repubir, reqmiree e fee and/or rpecia permiraion from tac Endowment. Proceedings of the 19th VLDB Conference Dubin, Ireand, 1983. Since 1971, after the breakdown of the Bretton WoodswANK system of fixed exchange rates, financia markets have seen a sharp increase in the fuctuation of interest rates and exchange rates. The past 20 years have aso seen rapid advances in information technoogy that make it possibe to coect and process arge amounts of data. These events, when couped with the sophistication of financia theory, have created a marketpace of a vast array of financia products that cater to different investment needs. Financia Trading Appications, which are meant to faciitate trading in these products, have aso become very compex because : There are a variety of compex financia products avaiabe and the dynamic nature of the market, couped with a decrease in reguation, has created a situation in which new products are introduced often and od ones discontinued. Trading strategies are baaed on numericay intensive procedures and compex mathematica reationships between financia products. The decrease in the cost of teecommunications and the increased reiabiity of networks make profitabe trading opportunities avaiabe ony for short periods of time. The proiferation of financia products, increase in information and rapid advances in technoogy keep trading houses and investment firms under constant pressure to deveop new ad-hoc appications for financia trading support. These appications tend to be expensive, invove the dupication of effort to a arge extent and are for the most part, product specific. The investment is often wasted if the product is discontinued. In addition, the non-uniformitv of annications makes it very difficut. 302
for the firm to get a cear picture of its overa risk at any point in time. Commercia databasea are unabe to hande the compexity of financia products and trading appications because they are speciaized for the creation, manipuation and processing of fixed-format snapshot records rather than tempora data. Extensibe database systems [CARE871 provide an appropriate environment for the deveopment of high-performance financia trading appications. The primary goa of this paper is to describe the compexity of financia data and focus attention on strategies for designing trading appications using extensibe databases. Whie current extensibe database systern prototypes support many features required for the impementation of such appications, this paper identifies requirements that are essentia but acking in current systems. 1.1 Reevant Research Rapid advances in technoogy have changed the way investment firms do business. This has been we documented in [BAUE92] and [SPEC88]. [BAUE92] discuss the impact that computers have had on the abiity of investment firms and trading houses to quicky anayze numerous trading strategies. [SPEC88] discuss the roe computers and trading systems had in the Oct. 19, 1987 crash. The presence of databases in trading systems was discussed in [ABBOSB], [PEIN88] and [SAMM87]. [ABBOSS] point to the rezponse time requirements of a database that faciitates stock-trading. [PEIN88] and [SAMM87] present rea-ife experiences gained from a study of a arge, high voume stock trading system that used a standard reationa DBMS. The main focus of this paper is to demonstrate how extensibe databases can be used to buid reiabe, high- performance financia trading appications. There are many prototype extensibe systems incuding POST- GRES [STON9Oa], Exodus[CARE88], Starburst[LIND87] aud Ode[AGRA90]. A more compete discussion of the capabiities required by trading systems and the support ing features avaiabe in these prototypes is presented in Section 3 and Section 4. [SEGE87] provided a convenient way to ook at tem- pora data through the concepts of Time Sequence and Time Sequence Coection. Tempora data modes are designed to capture the compexities of many timedependent phenomena. Tempora data modeing and representation have been extensivey studied in the iterature in [CLIFS I], [GADI88], [SEGE88a], [SNOD87], [WUU92]. A gossary of tempora concepts can be found in [JENS92]. This paper modes tempora objects found in the financia domain and discuszea the impementation of the mode in an extensibe database. The rest of the paper is organized as foows. Section 2 presents two exampes of financia data and reated trading appications. This section highights the compexity of the data and the difficuty of designing trading ap pications using traditiona databazes. It aso describes the functionaity necensary to support a trading appication. Based on the exampes presented in Section 2, Section 3 gives an outine of the design of a financia trading appication using an extensibe DBMS. This section aso discusses the functionaity avaiabe in existing object-oriented and extensibe databasea with reference to the functionaity required for this appication. Section 4 shows how a trading appication can be designed and impemented in POSTGRES Section 5 summarizes the contributions of the paper and discusses issues for further research. 2 Trading Appications and Financia Data This section introduces finance concepts and terminoogy necessary to understand the functionaity of a financia trading appication. It highights the compexity of financia data and the reated appication. The functionaity required of a database that can support financia trading is aso discussed. It shoud be noted that for this paper, the terms financia product and financia instrument are synonymous. Term Structure A debt instrument is a promissary note that evidences a debtor/creditor reationship. In such a reationship, one party borrows funds from another party and the borrowing party promises to pay the funds, together with interest. An exampe of a debt instrument is a US Govt %easury Bond (T-Bond). The ength of time ti the debt instrument matures is caed its term-temaurity or term. Since there is a chance that the borrower wi fai to make timey payments of interest and/or principa on the debt, the associated risk is quantified and caed the defaut risk. Each bond of a given term is priced by the market and this price can be converted by vauation arithmetic into a yied. The reationship between yied and term is caed the term-structure of interest rates. When graphed, it is known as a yied curve. The yied curve/term-structure varies over time due to the fuctuation of interest rates. Traders, financia product designers and economists are interested in the shape of the yied curve at a particu- ar point in time (c&-sectiona data) and aso in the changes in the yied curve over a period of time (timese&a data). 303
If the term-structure were to be modeed in a database, the foowing database features are required : Capabiity of creating data types of arbitrary compexity using base data types and other compex data types. An exampe of a compex data type is the data type time-series which is a 2 dimensiona array of foat and time. The term-structure at time t, is a data structure of type time-series. The termstructure over a period of time is a compex data type and is a 2 dimensiona array of time-series and time. This feature aows the creation of hierarchica structures that specify reationships between different data objects. This incudes the capabiity to specify different kinds of term-structures as instances of the object term-structure, e.g., USGovt- Treasury term-structure, Corporate term-structure. Capabiity of defining procedures that can take objects of type time-series as arguments, e.g., drawyied-curve(term-structure). Capabiity of defining rues that specify reationships between objects. In addition to the database features mentioned above, other features needed to mode compex financia objects are : Capabiity of defining rues that act as constraints on the vaues of attributes of an object. A rue syntax that aows the expression of events based on the state of objects in the database, externa events, time-based events, triggering of other rues and the execution of procedures. Capabiity of expressing tempora conditions in rues. Capabiity of supporting data of type procedure. Active rues that check for conditions rather than ony being event driven. Capabiity of specifying the sequence of rue execution. An exampe of a financia contract is now discussed to iustrate the requirements mentioned above. Option Contract A ca option on an underying asset, say a stock, grants its purchaser the option to purchase a specified number of shares of the stock from the seer of the option. If the option is cassified as a European Option, this right is good ti the expiration date. For simpicity, this is the ony case discussed. The ife of an option is the time Attribute Asset Sire Expiration Months Exercise Price Exercise Period Lust Trading Day Expiration Day Settement Day STOCK OPTION CONTRACT RueIJ Stock Symbo, e.e., BMW SO sh& of the &derying stock Next Three Months u we u corert 2 quartery expiration months. The quartery expiration months are March, June, September, December. For each expiration month, there must be at east 3 Ca Options at east one exercise price in-the-money, at east one exercise price at-the-money and at eaut one out-of-the money. Exercise pricer separated by DM 5 intervas. Last Ikading Day of the option The third Friday of the expiration month if that day in an exchange trading day, otherwise the exchange day preceding this Friday The exchange trading day foowing the ast trading day Two exchange days after the exercise Figure 1: Part of an Option contract between the date of purchase and the expiration date. The price at which the purchaser can buy the stock is caed the ezercise price and wi be denoted as K. To buy this right, the purchaser must pay an option premium. The stock price at any point in time wi be denoted as S, and if S > K, then the option is said to be in-the-money. If S = K, the option is at-the-money and if S < K, the option is out-of-the-money. Figure 1 shows the attributes of an option contract that is traded on the Frankfurt Stock Exchange and the rues governing the instantiation of these attributes. These attributes incude the underying et& and the size which are easiy modeed in a database as text and integer respectivey. The ast trading day is an attribute that rep resents the ast day on which the option can be bought or sod on the exchange. This is derived from the expiration month based on the foowing rue the third Friday of the expiration month if that is an exchange trading day, otherwise the exchange trading day immediatey preceding this Friday. In this rue, exchange trading days refer to the days on which trading occurs on the Frankfurt Stock Exchange. The rue ceary demonstrates the need for a concise anguage to express tempora conditions and the capabiity of the database to understand these rues and manipuate tempora objects. The database must aso give the user the capabiity of defining coections of time points, e.g., syntax to define the set of time points that 304
constitute the exchange trading days, An impicit part of the contract is its cash fow pattern. At the boundary points, the cash fow cacuation is simpe and is given by the foowing rue : On date the contract is bought canh fos - - (preniuu + transaction costs) On wpiration data ii (Stock price of underying asset (S) > axerci8e price (K)) then cash tou - S-K ere cash fos - 0 Cash fow patterns can be very compex and are usuay expressed in a high eve programming anguage. A risk-profie is a two-dimensiona array that records the change in cash fow with changes in the price of the stock. It is generated from the cash fow pattern and the database shoud be abe to either maintain the risk-profie which entais supporting objects of the type 2 dimensiona foat array or maintain the ogic to derive the risk-profie. Maintaining the ogic to compute the risk-profie and cash fow pattern can be accompished by supporting data of type procedure and rues that trigger these procedures bwed on certain conditions. The correct procedure to vaue an option contract is determined by the cash fow pattern during the ife of the option. This procedure is cated the vauation scheme. Vauation schemes are essentiay modues of code written in high-eve programming anguages. Since the vauation scheme is dependent on the cash fow pattern, the database must be abe to store the vauation scheme associated with the option and execute this procedure when necessary. An additiona requirement is that the database rue schema must ensure that a parameters necessary for the vauation scheme, e.g., stock price voatiity and risk-free interest rate, are estimated before the vauation scheme is triggered. In addition to the rues described above and in Figure 1, the database must aso store rues that define reationships between financia contracts. If there is a mathemat- ica reationship between the vaue of two or more financia contracts, it can be defined as a rue in the database. A vioation of this rue opens up a potentiay profitabe arbitrage opportunity. (Arbitrage is defined as the simutaneous buying and seing of equivaent portfoios to obtain a riskess profit.) Since such opportunities aren t avaiabe for very ong, the rue shoud have a constraint associated with it. This constraint wi either recheck the arbitrage condition before committing the transaction of buying and seing the two portfoios or have a maximum bound on the time that the transaction can take. In this section, we discussed the compexity of financia objects and the reated trading appication. The next section discusses the design of a financia trading appication using an extensibe database. 3 XYading Appications using Extensibe Databases The discussion in the previous section makes it cear that the design of a trading appication requires a database that provides data management, object management and knowedge management capabiity. Object management entais efficienty storing and manipuating compex data types. Knowedge management refers to the capabiity of storing and enforcing rues to refect the semantics of the appication [STONSOb]. Extensibe DBMSs have this capabiity because : They provide the capabiity of adding compex data types to the base types of foat, int and char. This aows compex financia products to be modeed in the database, thus creating a uniform and centra store for financia products and financia data. They provide the faciity to decare new operators on base and compex data types. This aows the definition of procedures that can take financia products as arguments. An exampe is an operator to compute the term-structure from the database of bonds. Section 4 discusses two such important operators. They provide faciities for impementing new access methods designed to optimize user-defined operators. This feature aows the creation of indexes that can faciitate operators on financia products. One such index is discussed in Section 4. Rues can be defined and their execution sequence controed. This aows knowedge of the financia appication to be buit into the system. If rues are processed efficienty appications can avoid the expense and performance probems due to inefficient appication programs. Reationships between financia products can be expressed and active rues created to faciitate arbitrage trading. Rues aso make it possibe to construct aerters that can be used to ca the attention of traders to unusua activity or an important news item. Extensibe DBMSs aso provide a natura environment for buiding trading appications because a buiding bocks approach to appication deveopment can be adopted. This approach preserves investment in software by encouraging the moduar design of appications, promoting generaity in the design of modues, aowing reuse of existing modues and enforcing a standard interface for modues. The foowing discussion focuses on important eements of trading appications and database modeing issues. 305
Eements of a Trading Appication The important eements of a trading appication that are to be modeed in a database are : (a) tempora objects, e.g., time-series such as the price of a stock over time, (b) cross-sectiona objects, e.g., objects whose timevarying characteristics are not recorded by the database, (c) rues, e.g., expression of the fact that the vaue of an option must be recomputed when the stock price changes by some pm-specified amount, (d) methods or procedures, which are essentiay modues of code deveoped in higheve programming anguages such as C++. Exampes of methods are vauation schemes for options and procedures for computing the term-structure. Both rues and methods add domain knowedge to the database and reduce the amount of appication code required. The other eements are (e) caendars that describe sets of time points ike a particuar date/time or time intervas ike years and (f) externa objects which are convenient abstractions for objects that are not defined in the appication. An externa object coud be an externay updated information source such as a stock ticker or data feed coming from other ocations not running the same appication. It shoud be noted that the above cassification is not a disjoint partition of the eements of a trading appication. For exampe, a rue coud change over time. This makes the rue a tempora object as we. A cass is a coection of these objects. For exampe, the option contract described in Section 2 coud be modeed as a cass consisting of tempora objects, rues, methods and crosssectiona objects. The foowing discussion provides a detaied specification of features that must be buit into an extensibe DBMS. The most important of these features are caendars, tempora objects and rues. Caendars The discussion of option contracts in Section 2 highighted the need for a powerfu anguage and agebra to express natura anguage time-based expressions. Financia trading appications must aso be abe to trade on a goba basis. Trading around the word requires knowedge of the different trading days and trading hours on different exchanges. A system of caendars and reationa operators is used to achieve this functionaity. [SOO92] first introduced extensibe caendric systems. Caendric systems are coections of caendars and operators. They aso discuss a tookit that aows the definition of new caendars and caendric systems. Our modeing of caendars uses the simpe set based agebra defined in [LEBA86]. Operators for these caendars incude interva based reationa operators ike overaps, meets, precedes and operators that faciitate deriving caendars from other caendars. Caendars and the reated operators are usefu for (a) defining the time points at which tempora objects have vaues, (b) defining the tempora ogic for rues that have triggers based on time, e.g., rues that aert traders that certain options are to expire at time T or in the interva [T,, Te, (c) describing sets of time points or time intervas, e.g., AMERICAN-BUSINESS-DAYS z DAYS-IN-YEAH - WEEKENDS - HOLIDAYS, (d) defining constraints, e.g., suppose no new option contracts can. be introduced if any current options are to expire in 10 exchange trading days. To express this rue, caendars are used with appropriate operators to express the set of days, ezchangt trading days. Then reationa operators wi operate on the derived caendar to express the ogic of the rue. Exampes are presented in Section 4, (e) representing natura anguage time-based expressions, e.g., the 3 d Friday of the month, (f) aowing different semantics for date arithmetic, e.g. the yieds on some bonds are computed based on the actua number of days between two dates but with the assumption that the year aways has 360 days and (g) maintaining vaid time in databases. This system of caendars aow a concise representation of time points and intervas and make it unnecessary to physicay store time points associated with tempora objects. Thus, it is imperative for data manipuation operators, that have tempora objects as arguments, to work in cose association with the caendar system. In Section 4 we discuss the impementation detais of caendars, reationa operators and a data manipuation operator, The discussion aso presents the time agebra used to denote time-based natura anguage expressions. Tempora Objects Tempora objects track the environment over time. It shoud be emphasized that a tempora object is a generic term for both a simpe time-series and compex tempora objects. A time-series can be considered to be a sequence of vaues in the time domain for a singe entity instance [SEGE87], e.g., stock prices. A coection of n-ary vectors grouped together to represent a semantic unit is aso a tempora object but wi be referred to as a compex tempora object. An exampe of a compex tempora object is a Company Baance Sheet. The individua items ike Gross Saes, Cost of Goods Sod and Operating Expenses are time-series that are recorded at the same time points and are reported together. Rues and procedures that have been changed over time are aso considered to be tempora objects. These genera semantics of tempora objects aow us to consider versions as a specia case of tempora objects. Since it is possibe to construct compex tempora objects from sets of simpe time-series, the discussion wi 306
focus on time-series. Time-Series modeing and representation arc an integra part of modeing compex financia objach rrnc wi b(* ciscuhnc:d in dcai beow. Part of the foowing functionaity cau be supported by (ROSE911 and [WUU92]. Each time-series is essentiay an n-ary vector and is associated with a set of user-defined information. This information (M) is cassified into (a) information that must be present with every time-series (M,) (if not suppied by the user, appropriate defauts are used) and (b) information optionay suppied by the user (M,). it4, consists of : 1. 3. 4. Name : The identifier of the time series to be used in data retrieva and data manipuation routines. 2. Caendar/Granuarity : a set of pre-defined time points. This item specifies the caendar with which the time-series is associated. For exampe, the time-series IBM-DAILY-CLOSING woud be associated with the caendar AMERICAN-BUSINESS- DAYS. This means that on every day in the caendar AMERICAN-BUSINESS-DAYS, the time-series shoud have a vaue. Granuarity is a specification of the points in time in the defined caendar that can potentiay have data vaues [SEGE87]. The defined caendar wi thus determine the granuarity of the time-series. The advantage of associating a timeseries with a caendar is that there is no need to physicay store the individua time points with the vaues of the time-series. When the timeseries is retrieved due to a query, the individua time points can be generated using the specification of the caendar. This is especiay advantageous for time-series with arge ifespans. Since the individua time points are not saved on disk, there are arge savings in disk space utiization. Thus, a time points of the timeseries are physicay stored ony when the caendar cannot be pm-defined. This is possibe in the case of randomy updated time-series ike tick-by-tick stock prices. Exception-Set : is a set of time points (within the caendar) on which vaues of the time-series are not recorded. For exampe, even though IBM-DAILY- CLOSING shoud be recorded on every day in the caendar AMERICAN-BUSINESS-DAYS, there may be an important announcement on a particuar day that stops trading in the stock. Thus the vaue of the time-series is not recorded on that day. The exception-set wi incude such time points. Thus, the actua caendar for a time-series is the set difference of Caendar and Exception-Set. be 00. The ifespan is used in conjunction with the caendar and exception-set to generate the set oftime points for which the tirnc-series has vaues. 5. Update Mode : This indicates whether the timeseries is derived from another time-series(s) or is base data. If the series is derived, the rue for update is specified here. Time-series are aowed to have a hybrid update mode. For exampe, a time-series recording the vaue of an option wi change whenever the price of the underying stock changes (price is derived) and aso when the option is traded on the market. In the atter case, the price is not derived and is determined by the vaue at which the option was exchanged on the trading foor. A more detaied treatment of the Update Mode is provided in [ETZI92]. 6. Frequency : This specifies the frequency with which the time-series is updated. The time of update refers to the vaid time. Vaid time is defined in [JENS92] as the time when the fact is true in modeed reaity. Frequency is aways specified with respect to the caendar with which the time-series is associated and may be a non-trivia function on the set of time points in this caendar. For exampe, suppose EMP, a time-series which records the eve of empoyment in the country, has the Caendar/Granuarity : the ast day of the month uness the day is a hoiday in which case it is the preceding business day. The frequency of EMP woud be monthy. If a time-series is derived from other time-series, the frequency woud be the frequency of the base data or some function of it. For exampe, consider the time-series DJIA and DJIAHILO. DJIA, the Dow Jones Industria Average, is a weighted average of the price of a given set of stocks. It is computed every time the price of a component stock changes. Thus, it is a derived time-series with the same frequency of update as the base data. On the other hand, DJIAHILO, which is a time-series that contains the daiy high and ow vaues of the DJIA, has a daiy frequency which is different from the frequency of its base data. It is important to stress the difference between frequency and granuarity. A time-series is said to be reguar [SEGE87], if it contains a vaue for each time point in the time-series ifespan. In a reguar time-series, the granuarity is the same as the frequency. In this case, the exception-set is a nu set. 7. Type [SEGE87] : The type of a time-series determines how to derive vaues of the time-series at time points where the vaue isn t expicity specified. Lifespan : This indicates the start time and end time An exampe of a one-dimensiona time-series vector of the time-series. The end time can be specified to with the associated user-defined information is the ob- 307
servations of a country s Gross Nationa Product (GNP). The caendar associated with GNP is a function of the AMERICAN-BUSINESS-DAYS caendar. There is no exception-set defined for this time-series. GNP is not derived from any other time-series and thus its update mode is Base Data. The frequency of update is quartery and refects the doar vaue of the sum tota of economic activity in the quarter. The type of the time-series GNP is user-defined. This means that user-defined functions wi be used to determine the vaue of GNP at time points where it has not been expicity recorded. For exampe, the GNP on Apri 30rh (vaid time) is not recorded in the time-series. This coud be derived by a function which uses the previous vaues of GNP or through a function which uses other economic indicators. Rues (discussed beow) can be used to define the type of a time-series and buid in the desired eve of compexity. A detaied treatment of time-series modeing in databases can be found in [SEGE92]. Rues Rues are usefu for testing integrity constraints, maintaining consistency, versioning, materiaized views, updating derived data [STONSOb] and monitoring the database for specific events [DAYASS]. In the framework of the trading appication, the functionaity demands that a rue be a 6-tupe < Rue - List, Caendar, Event - Condition, Action, Transaction Couping, Constraint >. Each component of the 6-tupe is expained beow : Rue-List : This is a coection of rue ids and is used to group rues that must be executed in sequence. The position of a rue within the rue ist determines the order of execution. Caendar : The caendar associated with a rue defines the time points/intervas when the rue is active. As noted in the discussion on caendars, this gives the user unimited fexibiity in specifying time-based rues. For exampe a rue can be fired at a specific point in time, e.g., on Wednesday at 10 a.m., at certain intervas of time, e.g., every 5 hours, at specific points in time, e.g., Mon, Wed, Friday, and aways, in which case they become active rues. The defaut vaue for the caendar is nu which means that uness specified, rues wi be event-driven. An Event s scope is defined as the set of objects that determine the occurrence of the event. Thus, the scope can be a set of rues (for rue-triggered events), methods (for events based on the execution of procedures) and caendars (for time-based events). The condition can be based on the current state of database objects or historica states. The cvcntcondition that triggers a rue is specified by using eements of the events scope, a condition and ogica connectives ike and, or, excusive-or. For exampe, a trigger can be based on the state of an object and time. Actions are either rues or methods. Thus, actions can trigger other rues, execute methods, update database objects and perform any database functions that can be done through a method. Actions are aso aowed to update rues. The utiity of this functionaity in trading appications is iustrated by the foowing exampe. In times of great uncertainty, it woud make sense to compute the term-structure often. Thus the rue for computing the term structure woud have the syntax Every T minutes do compute-term-structure, where T woud have a sma vaue. But in times of ower voatiity, an active database woud update the vaue of 7, so that the term-structure is computed ess often. This frees up system resources for use in other tasks. Transaction Couping : This defines the couping bctween the event and action in the rue. A transaction is an ordered set of methods and rues bounded by a begin transaction and commit/abort. Based on this definition of a transaction, four types of couping can be defined [GEIiA92] (a) immediate : action is executed immediatey after the event is recognized in the same transaction. This is the defaut vaue. (b) deferred : The action is executed just prior to the commit of the transaction that recognizes the event, (c) dependent : The action is executed as a separate transaction but ony after the transaction that recognizes the event has committed, and (d) independent : The action is executed as a separate transaction with no dependency on the transaction that recognizes the event. The atter transaction coud abort or commit without affecting the action. Constraint : This is a simpe Rue (see beow) or caendar and is used to enforce timing constraints on the execution of rues. If the constraint is a caendar and if the transaction-couping is either immediate, deferred or dependent, it indicates the maximum time that can eapse between the beginning of the event transaction and the commit of the action transaction. If the time constraint is not met, the action is aborted, A simpe rue wi reexecute the event transaction and check the resut of the event with the vaue that was previousy obtained. The defaut vaue of the constraint is nu, which means that no constraint is appicabe to the rue uness specified. 308
In this section, we have provided a detaied specification of the important eements of a trading appication, The foowing section discusses the choice of POSTGRES as the extensibe database to impement our ideas and describes important aspects of the impementation. 4 Impementat ion Athough recent work in tempora databases [ROSEO], [ROSE93], [SU91],[WUU92], describe very usefu functionaity not avaiabe in existing prototypes, they are not fuy impemented and integrated with other features such as abstract data types and extensibe access methods. Consequenty, we imited the impementation aternatives to those discussed beow. We empoyed five criteria in choosing an extensibe system. These were (a) support for rues, (b) preference for a mode that was an extension of the reationa paradigm, (c) presence of a fast path capabiity that aowed the creation of indexes to optimize any operators that we defined, (d) persistent programming anguage access and (e) avaiabiity. Our four aternatives were Ode, Starburst, Exodus and POSTGRES. We soon reaized that the genera rue capabiity required for the trading appication w&s not avaiabe in any data mode and that ony POSTGRES woud aowed us to modify the source code to impement this capabiity. Ode provides persistent programming anguage access though Ott but the non-avaiabiity of the source code and the fact that it is based on the C+t object paradigm made it an unattractive aternative. Starburst[HAASSO] is an extensibe reationa DBMS that provides the capabiity to create compex objects, new storage methods, optimization of new operators, specification of the storage method for tabes and a genera rue capabiity [WID092]. We found the fast path capabiity more difficut to use compared to POSTGRES. Exodus[CARE88] incudes two basic components - The storage object manager which provides concurrent and recoverabe access to object of arbitrary size and the type manager that has a set of base types which can be extended by users. Exodus aso provides ibraries of database system components for access methods and version management. It provided the E impementation anguage and a generator that produces a query optimizer and compier from the description of the avaiabe operations and methods. Between POSTGRES and Exodus, we chose the former because of the avaiabiity and the fact that we had worked with this mode before and understood the design and impementation we. Aso Exodus provided no basic rue capabiity. Overview POSTGRES is a next generation extensibe reationa DBMS with genera mechanisms that can be used for 8e- mantic data modeing. These mechanisms incude (a) abstract data types which are used to support compex objects (b) data which can be of type procedure and (c) rues. The primary goa was to impement caendars, tem- pora objects and rues. The impementation of tempora objects is done by using the POSTGRES feature of decaring compex data types. Caendars are imp]& mented by using stored procedures and user-defined operators. The POSTGRES Rue System is not adequate for the demands of the trading appication. It must be extended to incude (a) event specification that incudes time-based events, triggering of rues and/or execution of procedures, (b) decouping the action part of the rue from the event if specified, (c) ordering the sequence of execution of a set of rues and (d) imposing time constraints and rechecking the event condition before committing the action transaction. POSTGRES aows user-defined op erators and access methods. Operators reevant to this appication were defined and appropriate access methods designed to optimize these operators. For brevity ony caendars and access methods are discussed in this section. 4.1 Caendars The impementation of caendars invoved creating the data type interva and set of intervas, which are caed Caendars. The agebra on which the impementation is based was formay introduced in [LEBA86]. A Caendar is formay defined as a structured set of intervas and the Order of a caendar is defined as a measure of the depth of the structured set. Thus, the set S = {(/I, u), (12, uz),..., (I,,, u,)} is a caendar with or- der 1 whie R = {&,. - -, S,,,} where Si = {(Ij, uj)) is a caendar of order 2. A caendar of order 0 is simpy a set of numbers. A set of basic caendars, e.g., YEARS, HOURS, were created as system defined caendars and the reationships between them were expressed in a tabe which had the format, {Caendar, Caendars, ist}. Here Caendari is a text variabe and the ist is an order 0 caendar. For exampe, to express the reationship between YEARS and MONTHS, the entry in this tabe woud be {YEARS, MONTHS, 12}, which means that 12 MONTHS 3 YEARS. The reationship between YEARS and DAYS is more compicated because of a eap year every 4 years. Consequenty, the entry in the tabe is : { YEARS,DAYS,(365,365,366,365)}, which means that in the first year there are 365 days, the second year has 365 days and so on as the ist specifies for four years, after which the same pattetn is repeated. The reationships between the system defined caendars are based on a start date which was taken as Jan 1, 1970 (the start date on 309
.Ianuuy ow.buw 1083 s M Tu w m F & atidt&bn s hi TU w m pq &=-- Rdax.d Dwbn 1 2 Figure 2: Resut of strict and reaxed division Genericay, a reationa operator (Op) takes two itrtervas to generate a third interva. POSTGRES is easiy extended to support reationa operators ike intersection cover, overaps, during, meets, contains, < and < operators since the semantics of these operators are we defined. Two new operators, the division operator and seection operator, were introduced to faciitate manipuation of caendars. The division operator takes a caendar of order 1 as its eft argument, an interva aa its right argument and generates a caendar of order 1 as the resut. If the right argument is a caendar, then it operates on every interva in the caendar. For each reationa operator, there are two interpretations of the division operator. Formay the strict division (:) operator is defined as : the UNIX system). To create a new caendar two different operators were defined. The first operator is caed generate and takes the arguments start time, end time and a ist of numbers. C : Op :< t,, ts >Z {cn < t,, tc > I(c E C)A(C op < t,,t, >)I The operator generate creates the foowing caendar : The reaxed division (.) operator is defined as : gencrate(t,,t,; i&;.-.;int,)= {(T.,T,+int),(T,+int,T,+int+intz),..., (T, + Ci<ninti,T, + C;,,inti + intcrua~),..-} where T, is the start time and T, is the end time. In generate the ist of numbers is considered a circuar ist and the operation is carried out ti the end time is exceeded. The operator, generate, is iustrated with the foowing exampe. YRS-SINCE-1987 = generate(jon--87; Jan-3-92; 365,366,365,365) = { (1,365), (366,731), (732,1096), (1097,1461), (1462,1826), (1827,1829)} where the second eement in the caendar, {(366,731)}, denotes that the second year, 1988, began 366 days from Jan 1, 1987 and ended 731 days from Jan 1, 1987. It shoud be noted that January, 1, 1987 is taken as 1. The second operator to create a new caendar is based on the same ogic as the system tabe that stores reationships between system defined caendars. This operator is caed caoperate and it takes the arguments caendar and a ist of intervas. The operator, caoperate(c, T,; x; x2;... ; x,), where C is the caendar from which the new caendar is to be derived, woud create a new caendar whose first interva is a union of the first x1 intervas of caendar C, the second interva is the union of the second 22 intervas of C and so on. The ist is appied as a circuar ist. caoperate is iustrated by the foowing exampe. If YEARS is the system defined caendar f (L365), then caoperate(yeaw *; 7), woud give the caendar of weeks in the year since : cop. < t,, 1, >i {ci(c E C) A (c op < t,, t, >} where the interva (--00, 00) is excuded from the resuting sets. If weeks in the year 1993 are : {WEEKSn {(-4,2),(3,9),...) and {Jan - 1993 = {(1,31)}}, the eements in the caendars WEEKS : during : Jan-1993 and WEEKS.during.Jan-1993 are iustrated in Figure 2. The operator seection, denoted by [z]/c, seects the zth interva from the caendar C. The operator recursiueseection, denoted by [z],/c, seects the xth interva from the caendar C recursivey, ti the resut is an order 0 ist. Specificay if C is an order 2 caendar, the tth interva is chosen from each eement. If C is an order 1 caendar, the zth eement of each interva is chosen. 4.2 Operators and Access Methods POSTGRES provides the capabiity of defining operators, written in high-eve programming anguages, to the database and then using them in the query anguage. In this section, we describe three operators reevant to financia trading. The first operator, ComputeTS operates on the database of bonds to create the term-structure. The Transformation operator converts a time-series from its current frequency to another frequency. POSTGRES aso aows a fast path capabiity to access its internas for creating new indexes and access paths to optimize the userdefined operators. The access method used to optimize the ComputeTS is described in detai. An agorithm for (YEARS,*;7)~{(1,7),(8,14),(15,21),...,} the Transformation operator is aso discussed. In ComputeTS(B,Range,Defaut,Category), B indi- Here * indicates an arbitrary end time. cates the set of bonds that are to be used to compute 310
Figure 3: Adding records to Main-Memory Index the term-structure. Range is the range of time between which the term-structure is to be computed. Defaut indicates the defaut risk for which the term-structure is computed and is determined by the investment grade rating assigned to it by credit rating agencies. Category indicates the type of bonds to be used in the computation, e.g., Corporate, Government, Mortgage. The computation invoves the steps of seecting the set of bonds that have a given defaut risk and category type from among the space of bonds, B. If B is unspecified, the entire database of bonds is used and for each bond in the previous set, computing the yied, i, by using the price of the bond, the vector of cash fows, CF, and the expiration date of the bond. To compute the yied of a bond, the remaining term, the cash fows and the current bond price are required. This operator returns a twodimensiona array of yied and term. From the formuas described above, it is cear that the computation of the term-structure is a numericay intensive procedure. It is further compicated by the fact that the fuctuation of bond prices on a minute-to-minute basis requires recomputation of the term-structure very often. Thus, the data structure that is used to optimize the computation of a term-structure must aow (i) parae computation of parts of the term-structure, (ii) the choice of the degree of accuracy required in term-structure construction such that as the accuracy desired decreases, the response time improves and (iii) computation of ony a sma portion of the term-structure with an improvement in response time. The main memory data structure shown in Figure 3 is proposed to optimize the computation of the term-structure. The hash-tabe contains a possibe combinations of bond-rating and bond category. A hash-function, /I(), is appropriatey chosen to avoid any coisions between these combinations. Each eement in the hash-tabe points to a ist of buckets. Each bucket is characterized by a remaining term range (RTM). A bucket with RTM -5 woud contain bonds of a specific category and bond rating with remaining term between 1 and 5 years. Each bucket record contains information on a particuar bond incuding its remaining term, date of expiration and cash fow if the cash fow is simpe. It aso contains a pointer to the disk bock where the bond detais are physicay stored. Information on the number of eements in the bucket is aso maintained dynamicay so that the buckets can be reorganized when necessary. The advantage of this data structure is that it aows term-structures of different ranges to be computed by different processors. For exampe, if the term-structure for bonds of AAA rating and the category corporate, is to be determined for the range -30 years and two processors are avaiabe, the job coud be divided among the two by sending a snapshot of the data structure for 1-15 years to one processor and the snapshot for 16-30 years to the other. Partia term-structures wi be computed by each processor and the resut is concatenated. In addition, this data structure aows the construction of a term to a user-specified degree of accuracy. The data structure must be updated whenever (a) a new bond is issued and (b) a bond expires/defauts or is caed (forciby expired by the issuer) and (c) periodicay to correcty refect the RTM. Buckets are aowed to grow a maximum of N, after which they are spit and the records distributed equay over the new buckets. Figure 3 shows the case when a new bond record is added and the number of records in the appropriate bucket is 12. Since N ~13, it causes the creation of a new entry in the bucket ist and an equa redistribution of records between the two buckets. Buckets with the same RTM are recombined ony when the number of records in each bucket is ess than [N/m. When the number of records in contiguous buckets with different RTM have ess than [N/m each, and neither bucket has an adjacent bucket with the same RTM, the buckets are coaesced and the RTM of the resuting bucket is expanded. The vaues of N and m are dependent on the frequency of insertions and deetions in the index. The operator, Transformation(TS,F), is used to convert time-series from the existing frequency to another. TS is a time-series and F is the frequency to which TS is to be converted. Conversions from a ower frequency to a higher frequency are aowed ony if the semantics of the transformation are cear. Since the time points of a time-series are not expicity recorded, the transformation operator uses the caendar, exception-set, ife* pan and frequency of %a time-series to convert it into a time-series with another frequency. To express the transformation operator agebraicay the foowing definitions are introduced : 311
The set of system defined caendars is CO. domain(c0) re(c, Cz) indicates the reation between caendars 6, Cz, e.g., re(months, DAYS) = (DAYS,*;31,28,...) rank(c) is used to assign an ordering to the system defined caendars, e.g., tank(seconds) = 1 and rank(hocjrs) = 3. hcc(c, cz), the highest common caendar, is formay fined as ~CC(CI,C~) = {ci rank(ci) 2 rank(cj), VCi, Cj, (re(c, Ci) A (re(cs, Ci) A (re(c, Cj)h (re(c2,cj)) A (i # i- For exampe, hcc(decade, YEARS) = YEARS. ca(frequcncy): maps every frequency to a system defined caendar. Formay, {ca(frequency) I+ cc E CO}. Given these definitions, we are now in a position to formay define the trans f ormaion operator. Transformation(TS, F) = {TS(i), Vi E ([n],/coll)} where, {COLL = ((TS.C).oueraps. re(ca(f), hcc(ca(ts.f), ca(f1))))) where TS.C and TS.F are the caendar and frequency of the Time-series that is being transformed. TS(i) is short form for the vaue of the time-series at the time point i and [n& is short form for the seection operator that seects the ast item from the caendar. Note that this seection operator is appied recursivey ti the caendar is an order 0 caendar. This is iustrated with the foowing exampe : A time-series with weeky frequency and the caendar AMERICAN-BUSINESS-WEEKS is to be transformed into a time-series with a monthy frequency. Since there is no simpe reation between the caendars WEEKS and MONTHS, the agorithm for the Transformation operator is used. Using the definition of transformation and the system-defined functions, we have ca(monthy) = MONTHS, ca(weekg) = WEEKS, and hcc(months, WEEKS) = DAYS. re(months, DAYS) = (DAYS, *; 31,28,31,30,...) and (TS.C) =AMERICAN-BUSINESS-WEEKS, which is a caendar of weeks in the year. &5), (8,12), (35,18), (22,28), (2% 33), (37,401, (43,47), (50,54), (57,81), - + *a 1 Then {(TS.C).overaps.(DAYS, *; 31,28,31,30, e. e)) resuts in a caendar of order 2 : t{(, 5), (8,12), (15,18), (2292% (2% 31)) {(32,33), (37,40), (43,47), (50,54), (57, w),...,i de- From this caendar, the ast interva is seected and resuts in an order 1 caendar : {(29,31), (57,69),..., }. Since the seection operator wi be appied recursivey ti the caendar is of order 0, the ast eement is scected from each interva, resuting in an order 0 caendar : {31,59, *. *, }. The transformed time-series woud then be the vaue of TS at each time point in this caendar. 5 Concusions and Further Research Fuctuations in interest and exchange rates, rapid advances in information technoogy and financia theory, have created a marketpace of a vast array of compex financia products. Financia trading appications, which are meant to faciitate trading in these products have aso become very compex. Because of the constant pressure to keep up with the market, investment firms and trading houses are forced to create product specific trading systems that are discarded as soon as the product is discontinued. To preserve the investment, a buiding bocks approach to appication deveopment shoud be adopted. Extensibe database systems provide an environment for deveoping fast high-performance appications. The main objective of this research is to focus attention on stratc giea for designing trading appications using extensibe databases. The contributions of this paper incude (a) anaysis of the compexity of financia products and design of strategies for modeing them in extensibe databases. This incudes a compete specification of tempora objects, rues and caendars, (b) introduction of operators reevant to financia trading, (c) design of access methods to optimize these operators and (d) impementation of the design in POSTGRES. We are ooking at the foowing areas for further research: Introduction and optimization of financia trading operators. The paper described two important operators reevant for financia trading and access methods to optimize these operators. A detaied study of financia trading shoud suggest the basic operations that can be used as buiding bocks for more compicated operations. We are compiing this ist of operators, so that appropriate access methods for the optimization of these operators can be deveoped. Storage methods for tempora objects encountered in trading appications is an open probem. There have been severa proposas in the iterature for efficient storage and retrieva of tempora and mutidimensiona data but it is not cear which proposa is 312
the best or whether a competey new approach is re- 16. quired. We are currenty doing a performance anaysis of storage structures based on typica queries that are encoutered in financia anaysis. 17. References 2. 3. 4. 5. 7. 10. 11. 12. 13. 14. 15. [AGRASO] Agrawa, R. and Gehani, N. H., ODE (Ob ject Database and Environment): The Language and Data Mode, Proc. ACM SIGMOD 1989, Portand Oregon, 1989, ~~-36-45. [BAUE92] Bauer, R. J. and Liepins, G.E., Genetic Agorithms and Computerized Trading Strategies, in Expert Systems in Finance, O Leary, D.E., and Watkins, P.R. ed., EIsevier Pubishers, 1992, pp.89-100. [CARE871 Carey, M (ed.), SpeciaI Issue on Extensibe Database Systems, Database Engineering, June 1987. [CARE881 Carey, M., et.a. The Architecture of the EXODUS Extensibe DBMS, in Readings in Database Systems, Stonebraker M. 1990, Morgan Kaufman. 6. [CLIF87] Cifford, J. aud Croker, A. The historica reationa data mode HRDM and an agebra based on Iifespans*, in Proc. Third Internationa Conference on Data Engineering, pp. 528-537, Los Angees, February 1987. [DAYA88] Daya, U., et. a., The HiPAC Project: Combining Active Databases and Timing Constraints, Proc. ACM SIGMOD Record Vo. 17, No. 1, March 1988. 8. [ETZI92] Etzion, O., Ga, A., Segev, A., Tempora Sup port in Active Databases, Proc. of the Zud Int. Conj. on Information Technoogy and Syrtemr, 9. [GAD1881 Gadia, S.K., The Roe of Tempora Eements in Tempora Databases, Data Engineering Buetin 7, pp. 197-203, 1988. [GEHA92] Gehani, N., Jagadish, H. V., ShmueIi, O., Event Specification in an Active Object-Oriented Database, Proc. of ACM SIGMOD 1992, pp. 81-90. [HAASSO] Haq L., et. a., Starburst Mid-Fight: As the Dust Cears, IEEE Transactions on Knowedge and Data Engineering, Vo. 2, No. 1, Mar. 1990. [JENS92] Jenson, C.S., Cifford, J., Gadia, SK., Segev, A., Snodgrass, R.T., A Gossary of Tempora Database Concepts, ACM SIGMOD Record, Vo 21, No. 3. [LEBA86] Leban, B., McDonad, D., and Forster, D., A Representation for Coections of Tempora Inter&s, in Proc. of the AAAI-1986, Sth Int. Conf. on Artificia Intehgence, pp. 367-371, 1986. [LND87] Lindsay, B., A Data Management Extension Architecture, Proc. ACM SIGMOD 1987, San Ran- &co, CA, 1987. [MANK92] Mankiw, G., Macroeconomics, Worth Pubishers, New York 1992. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. [PEN88] Pein, P., and Sammer, H., High Contention in a Stock Trading Database: A Case Study,, in Pm. ACM SIGMOD 1988, May 1988, pp. 260-268. [ROSESI] Rose, E., and Segev, A., TOODM - A Tempora, Object-Oriented Data Mode with Tempora Constraints, Proc. of the 10 th Int. Conj. on the Entiry- Reationship Approach, San Mateo, CA, 1991. 1. [ABB088] Abbott, R. and Garcia-Moina, Hector, 18. [ROSE931 Rose, E., and Segev, A. UA Tempora Object- Scheduing Rea-time Transactions, Sigmod Record, Oriented Agebra and Data Mode, Forthcoming in Vo. 17, No. I, March 1988, pp.71-81. ECOOP93. [SAMM87] Sammer, H., Onine Stock Trading Systems: Study of an appication, in Proceedings of Spring COM- PCON 87, San Francisco, pp. 161-163. [SEGE87] Segev, A., and Shoshani, A., A Logica Modeing of Tempora Databases, in Proc. ACM SIGMOD Conference 1987. [SEGESSa] Segev, A., and Shoshani, A., The Reprosentation of a Tempora Data Mode in the Reationa Environment, Lecture Note; in Computer Science, Vo 339, M. RafaneRi, J.C. Kensin, and P. Svensson (eds.), Springer-Verag, pp. 39-61, 1988. [SEGE92] Segev, A. and Chandra, R., A Data Mode for Time-Series Anaysis, in Proc. of Workshop on Current Issues in Databases and Appications, Rutgers University, Sept. 1992. [SNOD87] Snodgrass, R., The Tempora Query Language TQue, ACM TODS, Vo 12, No. 2. [SOOS] Soo, M., Snodgrass, R., Dyreson, C., Jensen, C.S., and Kine, N., ArchitecturaI Extensions to Sup port Mutipe Caendars, TempIS Technica Report 32, Computer Science Department, University of Arizona. Revised May 1992. [SPEC88] Voecker, J., et. a., How computers heped stampede the stock market, in SPECTRUM, Ott, 1988. [STON99a] Stonebraker, M., The Impementation of POSTGRES, IEEE Transactions on Knowedge and Data Engineering, March 1990. [STONSOb] Stonebraker, M., Jhingran, A., Goh, J., Potamianos, S., On Rues, Procedures, Caching and Views in Data Base Systems, Proc 1990 ACM SIGMOD Conj. on Management of Data, June 1990. [SU91] Su, Y.H.S., Chen, H. M., A TemporaI Knowedge Representation Mode OSAM /T and its Query Language OQL/T, Proc. of 17 h Int. Conj. on Verg Large Databaases, pp. 431-442, September, 1991. [WID092] Widom, J., The Starburst Rue System: Language Design, Impementation, and Appications, in Data Engineering, Vo1.15, No.-4, Dec. 1992. [WUU92] Wuu, G.T.J, and Daya, U. A Uniform Mode for Tempora Object-Oriented Databases, 8 h Int. Conj. on Data Engineering, pp. 584-593. 313