Trading rule extraction in stock market using the rough set approach

Transcription

1 Tradig rule extractio i stock market usig the rough set approach Kyoug-jae Kim *, Ji-youg Huh * ad Igoo Ha Abstract I this paper, we propose the rough set approach to extract tradig rules able to discrimiate betwee bullish ad bearish markets i stock market. The rough set approach is very valuable to extract tradig rules. First, it does ot make ay assumptio about the distributio of the data. Secod, it ot oly hadles oise well, but also elimiates irrelevat factors. I additio, the rough set approach appropriate for detectig stock market timig because this approach does ot geerate the sigal for trade whe the patter of market is ucertai. The experimetal results are ecouragig ad prove the usefuless of the rough set approach for stock market aalysis with respect to profitability. Key words: the rough set approach, tradig rule, stock market timig 1. Itroductio quatitative data. Some researchers ivestigated the issue of predictig stock idex futures market. Trippi ad For a log time, stock market predictio is logcherished desire of ivestors, speculators, ad idustries. Although several studies ivestigated to predict price movemets i stock market, fiacial time series too complex ad oisy to forecast. May researchers predicted the price movemets i stock market usig artificial itelligece (AI) techiques durig past decades. Kimoto et al. (1990) used several learig algorithms ad predictio methods for the predictio systems of Tokyo stock exchage prices idex (TOPIX). Their system used modular eural etwork to lear the relatioships amog various factors. Kamijo ad Taikawa (1990) used recurret eural etwork ad Ahmadi (1990) employed backpropagatio eural etwork with geeralized delta rule to predict the stock market. Yoo ad Swales (1991) also performed predictio usig qualitative ad DeSieo (1992) ad Choi et al. (1995) predicted daily directio of chage i S&P 500 idex futures usig ANNs. Duke ad Log (1993) executed daily predictio of Germa govermet bod futures usig feedforward backpropagatio eural etwork. Recet research teds to iclude ovel factors ad to hybridize several AI techiques. Hiemstra (1995) proposed fuzzy expert systems to predict stock market returs. He suggested that ANNs ad fuzzy logic could capture the complexities of the fuctioal mappig because they do ot require the fuctioal specificatio of the fuctio to approximate. A more recet study of Kohara et al. (1997) icorporated prior kowledge to improve the performace of stock market predictio. Tsaih et al. (1998) itegrated the rule-based techique ad the ANNs to predict the directio of the S&P 500 stock idex Graduate School of Maagemet Korea Advaced Istitute of Sciece ad Techology

2 futures o a daily basis. Previous research usig AI techiques almost predicted the price of every tradig day, week, ad moth. It is more importat, however, to determie stock market timig, whe to buy ad sell stocks, tha to predict the price movemet for everyday because ivestors i stock market geerally do ot trade everyday. If ivestors trade their stocks everyday, they are charged to tremedous amout of fee for trade. Market timig is a ivestmet strategy which is used for the purpose of obtaiig the excess retur. Traditioally excess retur is achieved by switchig betwee asset classes i aticipatio of major turig poits i stock market (Waksma et al., 1997). I this paper, we propose a ew rule-based techique, so called rough set approach, to determie market timig for stock market. Rough set approach is very valuable to extract tradig rules. First, it does ot make ay assumptio about the distributio of the data. Secod, it ca geerate profitable market timig because it ot oly hadles oise well, but also elimiates irrelevat factors (Ruggiero, 1997). I additio, the rough set approach appropriate for detectig stock market timig because this approach does ot geerate the sigal for trade whe the patter of market is ucertai. The rest of the paper is orgaized ito five sectios. The ext sectio reviews the basic cocept of the rough set approach ad their applicatios i busiess. I the third sectio, we explai the tradig rule extractio ad review the cocept of market timig i stock market. I the fourth sectio, we describe the research data ad execute experimets. I the fifth sectio, empirical results are summarized ad discussed. I the followig sectio, coclusios ad future research issues are preseted. 2. The rough set approach ad their applicatios i busiess The rough set cocept proposed by Pawlak (1982) assume that there is some iformatio which ca be associated with every object of the uiverse. The same iformatio ca characterize the objects, ad these are idiscerible i the view of available iformatio about them. The so-called idiscerible relatio from this view is the mathematical basis of the rough set theory. Ay set of idiscerible objects is called elemetary set ad forms a basic atom of kowledge about the uiverse (Dimitras et al., 1999). Ay subset of the uiverse ca either be expressed precisely i the view of the atoms or roughly oly. I the latter case, a certai subset ca be characterized by two ordiary sets which are called lower ad upper approximatios. For each cocept X the greatest defiable set cotaied i X ad the least defiable set cotaiig X ca be computed. The former set is called a lower approximatio of X ad the letter is called a upper approximatio of X (Pawlak et al., 1995). The lower approximatio cosists of all of he objects that certaily belog to the cocept while the upper approximatio cosists of ay objects that possibly belogs to the cocept (Ruggiero, 1997). The cocept X is called rough if the upper approximatio is ot equal to the lower approximatio of X (Jagielska et al., 1999). Geerally, the kowledge about objects ca be represeted i the shape of a iformatio table. The rows of the table are objects, the colums are attributes ad the itersectios of rows ad colums are filled with attribute values. The importat poit i the rough set approach is the discovery of the depedecies betwee attributes. It ca be assumed that the set of attributes depeds o the set of attributes. Aother importat cocept is the reductio of attributes. This process is performed for fidig the miimal subset assurig the same quality of classificatio. The attributes i iformatio table are composed of two kids of attributes, the coditio attributes ad the

3 decisio attributes. As for coditio attributes ad decisio attribute, there are aother measure of depedecy, which ca represet full depedecy ad partial depedecy. The reducts i the rough set approach is sets that cotai the same elemets as other sets but possess the fewest attributes (Ruggiero, 1997). Elimiatio of superfluous attributes ca help to extract strog ad oredudat classificatio rules (Jagielska et al., 1999). Itersectio of all the reducts is called Core. The core is a collectio of the most relevat ad strog attributes (Dimitras et al., 1999). As the rough set approach to data aalysis ca hadle imperfect data full of ucertaity ad vagueess, it atoes for other approaches dealig with data ucertaity such as probability theory, evidece theory, ad fuzzy set theory, etc. Usig this approach oe aalyses oly facts hidde i data, it eeds o additioal iformatio about data ad does t correct icosistecies maifested i data, istead rules extracted are categorized ito certai ad possible forms. The applicatios of this approach o busiess area are maily o the evaluatio ad predictio of bakruptcy ad market research. These works were made maily by the Greek iformatio about bakruptcy. Oe of the earliest applicatios was doe i the work of Slowiski ad Zopouidis (1995). They used the rough set approach to evaluate bakruptcy. Slowiski et al. (1997) compared rough set approach with discrimiat aalysis for predictio of compay acquisitio i Greece ad Dimitras et al. (1999) compared it with logit aalysis to predict busiess failure. The rough set approach, however, may ot be proper i the evaluatio of bakruptcy because of the excessive productio of rules for the predicted cases. I the work of Slowiski et al. (1997), 24 rules were give to the 30 cases. I additio, the case will occurs that the rules solely by the rough set approach. This problem is crucial i the field of bakruptcy predictio because all the judgmet is to be give to all ew cases to come. 3. Tradig rule extractio ad market timig Extractig tradig rules from the stock market is alterative way of detectig market timig. Market timig is a ivestmet strategy which is used for the purpose of obtaiig the excess retur. Traditioally excess retur is achieved by switchig betwee asset classes i aticipatio of major turig poits i the stock market (Waksma et al., 1997). Detectig market timig meas determiig to buy ad sell to get a excess retur from tradig. The market timig systems usually employs profitable rule base to capture the major turig poits. It is very importat to determie stock market timig tha to predict the price movemet for everyday because ivestors i stock market geerally do ot trade everyday. If ivestors trade their stocks i everyday, they are charged to tremedous amout of fee for trade. I additio, although there are a ifiite umber of possible tradig rules by which we could trade, it seems that oly a few of them would have made a profit. There has bee much debate regardig the developmet of tradig systems usig historical data. We agree that the future is ever exactly like the past, however, a commo ivestmet approach is to employ systems that would probably have worked well i the past ad that seem to have a reasoable chace of doig well i the future. So, we defie a goal of system as fidig tradig rules for market timig which would have yielded the highest retur over a certai period. extracted caot be applicable to the holdout samples

4 4. Research data Note) C: Closig price, L: Low price, H: High price, The research data used i this study is KOSPI 200 from May 1996 through October KOSPI 200 is the uderlyig idex of KOSPI 200 future which is the first derivative istrumet i Korea stock market. Futures are the stadard forms that decide the quatity ad price i the certified market at a certai future poit of time. The geeral fuctios of futures market are supplyig iformatio about future price of commodities, speculatio ad hedgig (Kolb ad Hamada, 1988). We collected samples of 660 tradig days. May previous stock market aalyses have used techical or fudametal idicator. I geeral, fudametal idicators are mostly used for log-term tred aalysis while techical idicators are used for short-term patter aalysis. I this research, we use the techical idicators as iput variables. Nie techical idicators are selected by the review of domai expert ad past research. Table 1 gives selected iput variables ad their formulas. MA: Movig average of price, M t : ( H t L t + C t ) SM t : i = 1 M t i + 1, D t : i = 1 M +, 3 t i + SM 1 t, Up: Upward price chage, Dw: Dowward price chage The rough set approach eeds the data which are discretized i advace because the rule extractio process does ot has the discretizatio algorithm. The thresholds of discretizatio are preseted i Table 2. These thresholds are selected by the review of the textbook about techical aalysis i stock market. Table 3 presets summary statistics for each variable. Category Features A1 (Stochastic %K) [0, 25] (25, 75] (75, ) A2 (Stochastic%D) [0, 25] (25, 75] (75, ) A3 (RSI) [0, 30] (30, 70] (70,100] A4 (Mometum) (-, 0] (0, ) A5 (ROC) (-,100] (100, ) A6 (A/D Oscillator) (-,0.5] (0.5, ) A7 (CCI) (-,0] (0, ) Features Names of feature Formulas A1 Stochastic %K C t L 100 H L A2 Stochastic %D 1 % K t i A3 <Table 1> Techical idicators (Achelis, 1995; Gifford, 1995; Chag et al., 1996; Edwards ad Magee, 1997; Choi, 1995) RSI (relative stregth idex) i = A4 Mometum C t C t 4 A5 ROC (rate of chage) 1 i= 0 1 i= 0 Ct 100 C t A6 A/D Oscillator H t Ct H t Lt A7 A8 CCI (commodity chael idex) OSCP (price oscillator) Up Dw ( M t SMt ) ( Dt ) MA 5 MA 10 MA A9 Disparity 5 days C t 100 MA 5 5 t i t i A8 (OSCP) (-, 0] (0, ) A9 (Disparity 5 days) (-, 100] (100, ) <Table 2> Discretizig threshold (Murphy, 1986; Achelis, 1995; Gifford, 1995; Chag et al., 1996; Edwards ad Magee, 1997; Choi, 1995) Features Max Mi Mea Stadard Deviatio A1 (Stochastic%K) A2 (Stochastic %D) A3 (RSI) A4 (Mometum) A5 (ROC) A6 (A/D Oscillator) A7 (CCI ) A8 (OSCP) A9 (Disparity 5 days) <Table 3> Summary statistics

5 5. Experimetal Results decreased more tha 58% durig the modelig period, we ca fid rules that would yield the high level of profit As metioed earlier, i this study, the rough set were it used to trade stocks o a give set of historical data. approach is used to fid the profitable rules. It produces followig results. First, the core of attributes is empty. This implies that o sigle attribute perfectly explai the characteristics of decisio classes. This may be owig to complex relatioship amog attributes i stock market. Secod, 280 reducts are obtaied from experimet. Derived reducts ad their stregth are preseted i Appedix. The stregth meas the umber of objects satisfyig the rule. The rough set approach produces several umber of reducts. To extract rules from the several sets of reducts, Dimitras et al. (1999) used followig two criteria. First, the reduct should iclude as small a umber of attributes possible. Secod, the reduct should ot miss the attributes judged by the domai expert as the most sigificat for evaluatio of the firms. These criteria, however, are arbitrary ad subjective oes. I this study, we use followig criteria to extract rules. First, we select reduct which has the largest stregth. The largest stregth meas may objects of historical data support the rule. The reducts selected are twelve reducts which are the reduct #1 ~ #12 i Appedix. These reducts are supported by 60 stregth respectively. The we extract rules from each selected reduct. I the process of extractig rules, we permit 51% of the miimum level of discrimiatio ad 21 of the miimum stregth for each rule. The we apply the results of extracted rules to our tradig strategy i Table 4. I the process of applyig the rules, the reducts which do ot produce Buy sigal are excluded because these reducts do ot produce profit or loss ay more. Fially, the reduct #1, #2, #3, #4, #6, #9, #10, #11, ad #12 are selected ad they are applied to outof-sample data to validate the geeralizatio power. If today s sigal is UP, The If the previous day s decisio was BUY, The SELL stocks; Else BUY stocks; If today s sigal is NONE (If ay rules do ot matched to today s market coditio) or DOWN, The If the previous day s decisio was BUY, The SELL stocks; Else do ot trade. <Table 4> Tradig strategy Tradig profit eared from simulated tradig results are preseted i Table 5. While the uderlyig idex decreased more tha 58% durig the modelig period, we ca fid rules that would yield the high level of profit were it used to trade stocks o a give set of historical data. The rules derived by modelig data are applied to validatio period to verify the effectiveess of the proposed approach. I the derived tradig rules sets, the reduct #3 produces the best performace for validatio period. While the uderlyig idex decreased about 17% durig the validatio period, the tradig strategies followed by the derived rules get 31.5% for the reduct #3 (the derived rules set #3 i Table 5) of tradig profit. This meas the ivestor who follows the rules from the reduct #3 ca ear 48.3% of profit durig the validatio period tha the ivestor who follows buy ad hold strategy. The descriptio of rules of the reduct #3 are illustrated i Table 6. The other rules i Table 5 are alteratively good tradig rules although there is mior differece i simulated performace. These experimetal results demostrate that the rough set approach is promisig method for extractig profitable tradig rules. Tradig profit eared from simulated tradig results are preseted i Table 5. While the uderlyig idex

6 Followig Buy & Hold Accumulated profit (assume iitial ivestmet of 10,000 wo) (the accumulated rate of retur / the excess retur tha followig buy ad hold strategy) Modelig period (May 1996 November 1997) 4,167 wo (-58.3% / 0%) Validatio period (Jauary 1998 October 1998) 8,317 wo (-16.8% / 0%) advatage of the proposed approach is that tradig rules geerated by the rough set approach produce the tradig sigals oly whe the rules are fired. Each of the rules extracted produces the sigals less tha 20% of time. The rough set approach do ot produce the tradig sigals Followig the rules set 1 (Reduct #1: {A1, A2, A6, A8}) Followig the rules set 2 (Reduct #2: {A1, A2, A8}) 10,872 wo (8.7% / 67.0%) 9,539 wo (-4.6% / 53.7%) 12,480 wo (24.8% / 41.6%) 12,660 wo (26.6% / 43.4%) whe the patter of market is ucertai because the selectio of reduct ad the extractio of rules are cotrolled by the stregth of each reduct ad rule. This is Followig the rules set 3 (Reduct #3: {A1, A3, A6, A8}) Followig the rules set 4 (Reduct #4: {A1, A6, A8}) 10,830 wo (8.3% / 66.6%) 10,177 wo (1.8% / 60.1%) 13,153 wo (31.5% / 48.3%) 11,816 wo (18.2% / 35.0%) very importat to detect market timig because market timig is detected by capturig the major turig poits i data. I additio, ivestors i stock market geerally do Followig the rules set 5 (Reduct #6: {A3, A6, A8}) Followig the rules set 6 (Reduct #9: {A4, A7, A8}) 9,881 wo (-1.2%/ 57.1%) 9,689 wo (-3.1% / 55.2%) 10,744 wo (7.4% / 24.2%) 12,140 wo (21.4% / 38.2%) ot trade everyday owig to tremedous amout of fee for trade. The derived rules form the experimets are Followig the rules set 7 (Reduct #10: {A4, A8, A9}) Followig the rules set 8 (Reduct #11: {A5, A7, A8}) 9,430 wo (-5.7% / 52.6%) 9,689 wo (-3.1% / 55.2%) 12,688 wo (26.9% / 43.7%) 12,140 wo (21.4% / 38.2%) alteratively good tradig rules although there is mior differece i simulated performace. The experimetal results are very ecouragig ad prove the usefuless of Followig the rules set 9 (Reduct #12: {A5, A8, A9}) 9,430 wo (-5.7% / 52.6%) 12,688 wo (26.9% / 43.7%) the rough set approach for stock market aalysis with respect to profitability. <Table 5> The profit of buyig ad sellig simulatios We eed a more extesive validatio process because the future is ever exactly like the past. I Elemetary coditios Decisio class No. A1 A3 A6 A8 A10 Stregth # Up 29 # Dow 21 # Up 47 # Dow 21 # Dow 21 # Dow 33 # Dow 104 <Table 6> Descriptio of derived rules from the reduct #3 7. Cocludig remarks additio, the rough set approach, however, does ot has the geeral guidace for the discretizatio of cotiuous data, we eed a comparative study about the effects o results of differet methods for discretizatio icludig domai kowledge, equal frequecy approach, ad etropy miimizatio method. Refereces Achelis, S. B., Techical Aalysis from A to Z, Probus This study iteds to mie profitable tradig rules usig the rough set approach for Korea Stock Price Idex 200 (KOSPI 200) futures. The rough set approach is very valuable to extract tradig rules because it ca be used to discover depedecies i data while elimiatig the superfluous factors i oisy stock market data. Additioal Publishig, Ahmadi, H., Testability of the arbitrage pricig theory by eural etworks, Proceedigs of the IEEE Iteratioal Coferece o Neural Networks, 1990, pp Chag, J., Jug, Y., Yeo, K., Ju, J., Shi, D. ad Kim, H.,

7 Techical Idicators ad Aalysis Methods, Seoul, Korea, Jiritamgu Publishig, Choi, J., Techical Idicator, Seoul, Korea, Jiritamgu Publishig, Choi, J. H., Lee, M. K. ad Rhee, M. W., Tradig S&P500 stock idex futures usig a eural etwork, Proceedigs of the 3 rd Aual Iteratioal Coferece o Artificial Itelligece Applicatios o Wall Street, 1995, pp Dimitras, A. I., Slowiski, R., Susmaga, R. ad Zopouidis, C., Busiess failure predictio usig rough sets, Europea Joural of Operatioal Research, Vol. 114, 1999, pp. 263~280. Duke, L. S., ad Log, J. A., Neural etwork futures tradig - A feasibility study, Adaptive Itelliget Systems, Elsevier Sciece Publishers, 1993, pp Edwards, R. D. ad Magee, J., Techical aalysis of stock treds, Chicago, Illiois, Joh Magee Ic., Gifford, E., Ivestor s Guide to Techical Aalysis: Predictig Price Actio i the Markets, Lodo, Pitma Publishig, Hiemstra, Y., Modelig structured oliear kowledge to predict stock market returs, I Trippi, R. R. (Eds.), Chaos & Noliear Dyamics i the fiacial Markets: Theory, Evidece ad Applicatios, Irwi, 1995, pp Jagielska, I., Matthews, C. ad Whitfort, T., A ivestigatio ito the applicatio of eural etworks, fuzzy logic, geetic algorithms, ad rough sets to automated kowledge acquisitio for classificatio problems, Neurocomputig, Vol. 24, 1999, pp Kamijo, K. ad Taigawa, T., Stock price patter recogitio: A recurret eural etwork approach Proceedigs of the IEEE Iteratioal Joit Coferece o Neural Networks, 1990, pp Kimoto, T., Asakawa, K., Yoda, M. ad Takeoka, M., Stock market predictio system with modular eural etworks, Proceedigs of the IEEE Iteratioal Joit Coferece o Neural Networks, 1990, pp Kohara, K., Ishikawa, T., Fukuhara, Y. ad Nakamura, Y., Stock price predictio usig prior kowledge ad eural etworks, Iteratioal Joural of Itelliget Systems i Accoutig, Fiace ad Maagemet, vol. 6, 1997, pp Kolb, R. W. ad Hamada, R. S., Uderstadig Futures Markets, Scott, Forema ad Compay, Murphy, J. J., Techical Aalysis of the Futures Markets: A Comprehesive Guide to Tradig Methods ad Applicatios, New York, New York Istitute of Fiace, A Pretice-Hall Compay, Pawlak, Z., Rough sets, Iteratioal Joural of Iformatio ad Computer Scieces, Vol. 11, 1982, pp Pawlak, Z., Grzymala-Busse, J. Slowiski, R. ad Ziarko, W., Rough sets, Commuicatios of the ACM, Vol. 38, No. 11, 1995, pp Ruggiero, M. A., Cyberetic Tradig Strategies: Developig Profitable Tradig Systems with Stateof-the-art Techologies, Joh Wiley & Sos, Ic, Slowiski, R. ad Zopouidis, C., Applicatio of the rough set approach to evaluatio of bakruptcy risk, Iteratioal Joural of Itelliget Systems i Accoutig, Fiace ad Maagemet, Vol. 4, 1995, pp. 27~41. Slowiski, R., Zopouidis, C., ad Dimitras, A. I., Predictio of compay acquisitio i Greece by meas of the rough set approach, Europea Joural of Operatioal Research, Vol. 100, 1997, pp. 1~15.

8 Trippi, R. R. ad DeSieo, D., Tradig equity idex futures with a eural etwork, The Joural of Portfolio Maagemet, 1992, pp Tsaih, R., Hsu, Y. ad Lai, C. C., Forecastig S&P 500 Stock idex futures with a hybrid AI system, Decisio support Systems, 1998, pp Waksma, G., Sadler, M., Ward, M. ad Firer, C., Market timig o the Johaesburg Stock Exchage usig derivative istrumets, Omega, Iteratioal Joural of Maagemet Sciece, Vol. 25, No. 1, 1997, pp Yoo, Y. ad Swales, G., Predictig stock price Appedix performace: A eural etwork approach, Proceedigs of the IEEE 24 th Aual Coferece o Systems Scieces, 1991, pp No. Reduct Stregth 1 {A1, A2, A6, A8} 60 2 {A1, A2, A8} 60 3 {A1, A3, A6, A8} 60 4 {A1, A6, A8} 60 5 {A2, A9} 60 6 {A3, A6, A8} 60 7 {A3, A7} 60 8 {A3, A9} 60 9 {A4, A7, A8} {A4, A8, A9} {A5, A7, A8} {A5, A8, A9} {A2, A3, A8} {A1, A2, A3, A6} {A2, A3, A6, A8} {A2, A6, A8} {A1, A2, A6} {A3, A7, A9} {A7, A8, A9} {A1, A2, A3} {A2, A3, A6, A9} {A1, A3, A4, A6, A8} {A1, A3, A5, A6, A8} {A2, A3, A6} {A3, A4, A6, A7, A8} {A3, A5, A6, A7, A8} 54 No. Reduct Stregth 27 {A1, A2, A3, A8} {A3, A4} {A3, A5} {A1, A4, A6, A8} {A1, A5, A6, A8} {A3, A4, A7, A9} {A3, A5, A7, A9} {A4, A6, A7, A8} {A5, A6, A7, A8} {A1, A4, A8} {A1, A5, A8} {A3, A6, A7} {A3, A4, A6, A8, A9} {A3, A5, A6, A8, A9} {A1, A4, A6, A7, A8} {A1, A5, A6, A7, A8} {A1, A3, A4, A8} {A1, A3, A5, A8} {A1, A3, A6, A7, A8} {A4, A6, A8, A9} {A5, A6, A8, A9} {A1, A2, A3, A9} {A2, A3, A6, A7} {A3, A4, A7, A8, A9} {A3, A5, A7, A8, A9} {A1, A3, A6, A7} {A4, A7, A9} {A5, A7, A9} {A6, A7, A9} {A1, A3, A8} {A2, A4, A6, A8, A9} {A2, A5, A6, A8, A9} {A1, A2, A9} {A1, A2, A3, A7} {A1, A2, A4, A6} {A1, A2, A5, A6} {A1, A3, A4, A6, A9} {A1, A3, A5, A6, A9} {A2, A6, A7, A8} {A1, A3, A4, A7, A8} {A1, A3, A5, A7, A8} {A2, A3, A6, A8, A9} {A2, A8, A9} {A3, A4, A6, A9} {A3, A5, A6, A9} {A2, A3, A7} {A1, A3, A4, A6, A7} {A1, A3, A5, A6, A7} {A1, A4, A6, A8, A9} {A1, A5, A6, A8, A9} {A2, A3, A9} 35

9 No. Reduct Stregth 78 {A1, A3, A4} {A1, A3, A5} {A2, A3, A6, A7, A8} {A2, A4, A6, A7, A8} {A2, A5, A6, A7, A8} {A1, A2, A4, A7, A8} {A1, A2, A5, A7, A8} {A1, A2, A6, A7, A8} {A1, A4, A6, A7} {A1, A5, A6, A7} {A2, A7, A8} {A1, A3, A6, A8, A9} {A1, A3, A6, A9} {A1, A4} {A1, A5} {A2, A3, A8, A9} {A1, A3, A4, A6} {A1, A3, A5, A6} {A1, A3, A6} {A2, A3, A4, A6, A9} {A2, A3, A5, A6, A9} {A2, A6, A7} {A3, A4, A6, A7, A9} {A3, A5, A6, A7, A9} {A1, A2, A4, A8} {A1, A2, A5, A8} {A1, A2, A6, A9} {A2, A7, A9} {A4, A7, A8, A9} {A5, A7, A8, A9} {A1, A3, A4, A6, A7, A8} {A1, A3, A5, A6, A7, A8} {A2, A3, A4, A6, A7} {A2, A3, A5, A6, A7} {A3, A6, A7, A8} {A1, A2, A4, A6, A8, A9} {A1, A2, A5, A6, A8, A9} {A3, A6, A8, A9} {A3, A6, A9} {A1, A2, A3, A6, A7} {A2, A3, A4, A8, A9} {A2, A3, A5, A8, A9} {A2, A3, A4, A7, A8} {A2, A3, A5, A7, A8} {A2, A6, A8, A9} {A3, A7, A8, A9} {A1, A2, A6, A7} {A1, A3, A4, A8, A9} {A1, A3, A5, A8, A9} {A2, A3, A4, A6, A8} {A2, A3, A5, A6, A8} 21 No. Reduct Stregth 129 {A2, A8} {A2, A7} {A3, A4, A6, A7} {A3, A5, A6, A7} {A4, A6, A9} {A5, A6, A9} {A1, A2, A3, A6, A9} {A1, A2, A4, A6, A7, A8} {A1, A2, A5, A6, A7, A8} {A1, A2, A6, A8, A9} {A1, A3, A4, A6, A8, A9} {A1, A3, A5, A6, A8, A9} {A1, A3, A6, A7, A8, A9} {A1, A3, A7, A8, A9} {A1, A2, A4, A6, A7} {A1, A2, A4, A6, A8} {A1, A2, A5, A6, A7} {A1, A2, A5, A6, A8} {A1, A3, A4, A7, A8, A9} {A1, A3, A5, A7, A8, A9} {A2, A3, A6, A7, A8, A9} {A2, A4, A6, A7} {A2, A5, A6, A7} {A1, A3, A4, A7, A9} {A1, A3, A5, A7, A9} {A2, A3, A4, A6, A7, A8} {A2, A3, A4, A7, A9} {A2, A3, A5, A6, A7, A8} {A2, A3, A5, A7, A9} {A1, A2, A3, A7, A9} {A3, A4, A6, A8} {A3, A5, A6, A8} {A1, A2, A3, A4, A6, A8} {A1, A2, A3, A5, A6, A8} {A1, A4, A7, A8} {A1, A5, A7, A8} {A2, A3, A4, A7, A8, A9} {A2, A3, A5, A7, A8, A9} {A3, A4, A7, A8} {A3, A5, A7, A8} {A1, A2, A3, A4} {A1, A2, A3, A5} {A2, A3, A7, A8} {A2, A4, A6, A8} {A2, A4, A7, A9} {A2, A5, A6, A8} {A2, A5, A7, A9} {A2, A7, A8, A9} {A3, A4, A8, A9} {A3, A4, A9} {A3, A5, A8, A9} 11

10 No. Reduct Stregth 180 {A3, A5, A9} {A1, A2, A3, A4, A8, A9} {A1, A2, A3, A5, A8, A9} {A1, A2, A7, A8} {A2, A3, A4, A7} {A2, A3, A5, A7} {A1, A2, A3, A4, A6} {A1, A2, A3, A5, A6} {A1, A2, A8, A9} {A1, A6, A8, A9} {A2, A4, A7, A8, A9} {A2, A5, A7, A8, A9} {A1, A2, A4, A8, A9} {A1, A2, A5, A8, A9} {A1, A4, A8, A9} {A1, A5, A8, A9} {A1, A6, A7, A8} {A2, A3} {A3, A6, A7, A8, A9} {A1, A2, A3, A4, A7, A8} {A1, A2, A3, A5, A7, A8} {A1, A2, A3, A6, A8, A9} {A1, A2, A3, A6, A8} {A1, A2, A4, A6, A9} {A1, A2, A5, A6, A9} {A2, A3, A7, A8, A9} {A2, A3, A4, A6, A8, A9} {A2, A3, A5, A6, A8, A9} {A2, A6, A9} {A1, A2, A3, A4, A6, A7} {A1, A2, A3, A4, A8} {A1, A2, A3, A5, A6, A7} {A1, A2, A3, A5, A8} {A1, A2, A4, A6, A7, A9} {A1, A2, A5, A6, A7, A9} {A1, A3, A8, A9} {A1, A4, A7, A8, A9} {A1, A5, A7, A8, A9} {A1, A8} {A2, A4, A6, A7, A9} {A2, A5, A6, A7, A9} {A2, A6, A7, A8, A9} {A1, A2, A4} {A1, A2, A5} {A1, A3, A6, A7, A9} {A1, A3, A7, A8} {A1, A4, A6, A7, A9} {A1, A5, A6, A7, A9} {A4, A6, A7} {A5, A6, A7} {A2, A3, A4, A9} 3 No. Reduct Stregth 231 {A2, A3, A5, A9} {A2, A4, A7, A8} {A2, A5, A7, A8} {A1, A2, A3, A4, A6, A9} {A1, A2, A3, A4, A7} {A1, A2, A3, A5, A6, A9} {A1, A2, A3, A5, A7} {A1, A2, A3, A8, A9} {A1, A2, A4, A7} {A1, A2, A5, A7} {A1, A2, A6, A7, A9} {A1, A2} {A1, A3, A4, A6, A7, A9} {A1, A3, A4, A7} {A1, A3, A5, A6, A7, A9} {A1, A3, A5, A7} {A1, A4, A6, A9} {A1, A5, A6, A9} {A1, A6, A7, A8, A9} {A2, A3, A4, A8} {A2, A3, A5, A8} {A3, A4, A7} {A3, A5, A7} {A1, A2, A3, A4, A7, A8, A9} {A1, A2, A3, A5, A7, A8, A9} {A1, A2, A3, A7, A8, A9} {A1, A2, A4, A7, A8, A9} {A1, A2, A4, A7, A9} {A1, A2, A5, A7, A8, A9} {A1, A2, A5, A7, A9} {A1, A2, A6, A7, A8, A9} {A1, A2, A7} {A1, A3, A4, A9} {A1, A3, A5, A9} {A1, A3, A7, A9} {A1, A3, A9} {A1, A3} {A1, A4, A6} {A1, A5, A6} {A1, A6} {A2, A3, A6, A7, A9} {A2, A3, A7, A9} {A2, A4, A6, A9} {A2, A5, A6, A9} {A3, A6, A7, A9} {A3, A8} {A4, A6, A7, A9} {A5, A6, A7, A9} {A6, A8, A9} {A7, A9} 1