Data Center Power System Reliability Beyond the 9 s: A Practical Approach

Transcription

1 Data Center Power System Reliability Beyon the 9 s: A Practical Approach Bill Brown, P.E., Square D Critical Power Competency Center. Abstract Reliability has always been the focus of mission-critical power system esign. While a rigorous mathematical approach to reliability is necessary to achieve the best results, the techniques use can be misunerstoo or misapplie. Further, those practical elements so crucial to successful implementation are too-frequently overlooke. This paper gives an overview of the theoretical unerpinnings of power system reliability analysis an the limiting real-worl factors that must be use to temper any rigorous mathematical approach. 2. Introuction Moern ata center power systems represent the ultimate in optimization for reliability. This is a necessity, since the computer an IT equipment which these power systems serve are very sensitive to even momentary loss of power. At the highest level are those systems which qualify as Tier IV systems per the Uptime Institute classification system [2], which boast a representative site availability of 99.99%, or 4 nines. But just what oes availability really mean? Can a single number like this really give all of the necessary information about the reliability of a power system? What about iniviual component statistics such as MTBF how o they come into play? Are there other features, such as maintainability, that have an effect on just how often there will be problems with the power to critical equipment? The answers to these questions can be surprising. Funamentally, reliability analysis for electric power systems is similar to such analysis for any other engineere system. Such analysis is preicate upon the fact that no engineere system can operate inefinitely without failure. The failure of a component within the system is treate as a ranom process, for which there are mathematical tools for characterization an analysis. Using these tools an making certain assumptions yiels further characterizations that can be applie to the system as a whole. However, these characterizations ( availability is one of them) can be misuse, or, more often, over-use. An, concentrating upon a single characterization, such as availability, can cause the wrong conclusions to be mae concerning certain practical aspects of the system, such as maintenance. In orer to get a broa perspective of just what keeps such a system up an running with reliability that is best-in-class, one must look beyon the 9 s at the practical aspects of reliable power system operation an how it is achieve, while at the same time knowing how the rigorous mathematical approach that quantifies the 9 s works an what its limitations are. With such a perspective, ecisions can be mae that further the cause of keeping the power up, rather than iminishing it. 3. The 9 s The Mathematics of Reliability The language of reliability is the mathematics of probability an statistics. Unfortunately, it isn t the simplest language to unerstan, as one must have a working knowlege of calculus to grasp the intricacies involve. The approach herein will be to cover the basics without extensive mathematical equations. This will give a basic acquaintance with the concepts to most reaers, while the more ambitious reaer can explore further the information in the Appenix an the reference materials given in section 6 for etaile answers. In particular, reference [3] contains a soun introuction to the subject. Revision 2 2/07 Page of 3

2 3.. Component-Level Reliability Central to reliability is the concept of probability. Central to probability is the concept of an event. Simply state, an event is something which happens. Because the context herein is reliability, an example event coul be component A fails (we won t worry about the efinition of failure for now; more on that later). If component failure is a ranom process, then some probability can be assigne to it, for example the probability that the component will fail in the next instant of time. Probabilities may be given in terms of a number between 0 an or in percentage form % = 0.0. If the probability that component A fails in the next year is %, for example, at any time uring the next year there is a % chance that component A will fail. Another example might be the number of years between failures of component A. Such a figure is a number that behaves accoring to the laws of probability, an is referre to as a ranom variable. In our case, we will concentrate on the ranom variable that represents the number of years between failures of component A. Such a ranom variable has certain properties. One of these is probability ensity, which for a given number takes on the value of the probability that the ranom variable is equal to that number. For the ranom variable we have efine here, the value of the probability ensity for a given number t is the probability that t is exactly the number of years until component failure. Many types of probability ensity exist, among them the Gaussian or normal ensity which is the frequently-encountere bell curve use extensively in statistics. Using the probability ensity one may also efine, via calculus, a probability istribution which for a given number takes on the value of the probability that a ranom variable is less than or equal to that number. Because the physical phenomenon to be ealt with in reliability analysis is that of component failure, it woul seem logical to assume that the longer a given component is in service, the more likely failure woul become. State another way, the preicte amount of time between failures for a component is more likely to be less than the time in service as the time in service increases. The exponential probability ensity an its associate exponential probability istribution are one way to characterize such behavior. A etaile mathematical escription of the exponential probability ensity an istribution is given in the appenix. For our purposes here, is it sufficient to know that the exponential probability ensity an istribution are commonly use when escribing the behavior of component failure. Assuming that the ranom variable efine above (the number of years between failures of component A) behaves accoring to the exponential probability ensity an istribution leas to the assumption that there is a constant mean time between failures an associate failure rate for component A. The formal efinitions for MTBF an Failure Rate are [3]: MTBF: The mean exposure time between consecutive failures of a component. Failure Rate: The mean (arithmetic average) number of failures of a component or system per unit exposure time. The most common unit in reliability analysis is hours (h) or years (y) Therefore, the failure rate is expresse in failures per hour (f/h) or in failures per year (f/h). This creates a funamental issue, since the use of constant MTBF an Failure Rate in reliability analysis is base upon the assumption that the probability ensity an istribution functions for the time between failures are exponential. Further, in applying the efinitions for MTBF an Failure Rate unless a sufficiently long exposure time is use the results may be overly optimistic. Both of these limitations come into play in practical situations in the ata center, as will be explaine herein. Another frequently-use characteristic of a component is the mean time to repair or MTTR, often given the esignation r an efine as [3]: Revision 2 2/07 Page 2 of 3

3 Mean Time to Repair (MTTR): The mean time to replace or repair a faile component. Logistics time associate with the repair, such as parts acquisitions an crew mobilization, are not inclue. It can be estimate by iviing the summation of repair times by the number of repairs, an, therefore, it is practically the average repair time. The most common unit in reliability analyses is hours. In aition to this basic concept, other maintenance an repair-relate terms are efine at the component level, incluing [3] : Mean Time to Maintain (MTTM): The average time it takes to maintain a component, incluing logistics time. MTTM is primarily a measure of the preventative maintenance frequency an urations urations System-Level Reliability Engineere systems can be consiere to consist of multiple components, each of which has its own set of failure characteristics as escribe above. System-level reliability analysis consiers how these components are connecte to form the system, an uses this information to calculate the various aspects that escribe the reliability of the system. There are a number of methos available to o this. Several methos are escribe in etail in [3]. The traitional series/parallel combinations that support the metho of minimal cut-sets are escribe in the appenix. For our purposes here, it is sufficient to recognize that, whatever metho we use, we have the capability to calculate certain reliability inices for the system. The most common of these is availability, efine as [3]: Availability: The ability of an item uner combine aspects of its reliability, maintainability, an maintenance support to perform its require function at a state instant of time or over a state perio of time. From this rather vague efinition come two ifferent types of availability [3]: Inherent Availability (Ai): The instantaneous probability that a component or system will be up or own. Ai consiers only owntime for repair ue to failures. No logistics time, preventative maintenance, etc., is inclue. Operational Availability (Ao): The instantaneous probability that a component or system will be up or own, but iffers from Ai in that it inclues all owntime. Inclue is owntime for unscheule (repair ue to failures) an scheule maintenance, incluing any logistics time. It shoul be note that Ai an Ao may be efine for a system as a whole or any component of a system. A secon set of inices eal with owntime, efine as [3]: Mean Downtime (MDT): The average owntime cause by scheule an unscheule maintenance, incluing any logistics time. Repair Downtime (Rt): The total owntime for unscheule maintenance (excluing logistics time) for a given time perio. 4. Beyon the 9 s Practical Consierations The results of a rigorous mathematical approach to reliability are values for the reliability inices, the most common of which are escribe in section 3. In a ata center these are commonly calculate at the point of supply to the critical computer an IT equipment thus the term loapoint reliability. Such calculations, when viewe in the context of the assumptions mae in calculating them, are very powerful tools. However, misuse or over-use of such numbers can Revision 2 2/07 Page 3 of 3

4 lea to a false sense of security. Herein are given some practical aspects that must be use to temper the results of any rigorous reliability analysis. 4.. Absolute Reliance on Availability Availability, as efine in section 3, is the most often-quote specification regaring the reliability of a ata center. A much sought-after goal is 5 nines, or % availability. But, what oes this really mean? With % availability, in a given year only 0.00% of the time, or 5 minutes, 5.36 secons, will the system (or loa point within the system) be out of service. Is this one outage of 5:5.36? Or, is it 5 outages of minute per outage, plus another 5.36s outage? One outage of 0:30.72 every two years? In reality, availability is too vague a figure to make such specific preictions when use alone. The repair owntime or, if available, the mean owntime (both are escribe in section 3) shoul be use along with the availability to escribe the reliability of the system. What is even more worrisome is that there are two types of availability, Inherent availability an Operational Availability, as escribe in section 3. Operational Availability provies a real-worl measure of the availability of the system. Inherent Availability provies a way of comparing system esigns without factoring in maintenance an logistics concerns, which can vary from facility to facility. If a reliability stuy only calculates the Inherent Availability, the true Operational Availability of the system will likely be lower than the calculation shows. Also, traitional methos for calculating availability are only accurate if component MTBF s for the system hol true an remain constant throughout the component lifetime. In other wors, in some cases the calculate reliability inices from a system reliability stuy are only a snapshot of the system availability given the conitions assume in the stuy. In reality, the system reliability ecreases over time as the sum aggregate effect of component aging, as illustrate in Fig.. The true Operational Availability is as given in the figure, i.e., mean uptime ivie by the sum of mean uptime an mean owntime. This can be lower than the availability preicte by a rigorous reliability analysis if maintenance logistics an non-constant component MTBF s are not taken into account in the analysis. Fig. Illustration of system reliability changes with time [2] 4.2. Component MTBF s A major assumption mae in most rigorous reliability analyses is the MTBF for each system component. This has a ramatic effect on the outcome of the analysis, an therefore vali ata is essential. In the absence of specific ata from the component manufacturer survey ata, such as presente in [3] an [4],can be use; inee, the use of such ata is recommene in lieu of manufacturer-specific ata unless that manufacturer s component will be use on the project in Revision 2 2/07 Page 4 of 3

5 question or unless the manufacturer ata contains worst-case ata for all of the potential component manufacturers whose components coul be use on the project. The first ifficulty with component failure rates is the fact that, in reality, component MTBF s are not constant. The MTBF of a component changes over the lifetime of the component. The nature of this change is shown graphically in the bathtub curve of Fig. 2. The perio of heightene failure rate at the beginning of component life is known as infant mortality. Hopefully, failures associate with infant mortality are caught uring the commissioning process (more on this below). The perio of heightene failure rate at the en of component life is known as wearout. The possibility of wearout makes it very necessary to follow the manufacturer s recommene maintenance scheule in orer to avoi a lowering of the reliability of the system (more on this below also). Between the infant mortality an wearout points the failure rate is approximately constant. It is this perio of time, an this perio of time only, for which the results of a reliability analysis which utilizes constant component MTBF s is vali. Constant MTBF assumption vali only uring this perio Fig. 2 Illustration of time-varying nature of component failure rate The secon issue with the use of constant MTBF s is that, for most types of components, it is simply either very ifficult or impossible to give such a number without survey ata. Most component types have many failure moes, an these failure moes are not necessarily inepenent of one another. Attempts to supply such ata base upon any metho other than fiel history ata (which takes years to buil up in orer to have a vali time basis for the calculation) or a rigorous reliability stuy of the component in question can result in overlyoptimistic results. The thir ifficulty with the use of constant failure rates is their misuse if something goes wrong. Suppose component A in a ata center power system fails an causes an outage to the critical loa. Is the component to blame for the outage? On the surface, possibly; however, shouln t the system have been esigne to take into account a single point of failure? Also, if the ata center has been up an running for year, an this was the only failure of component A that year, is the failure rate of component A failure per year? From the en-user point of view, possibly; from a statistical point of view, no. Instea it woul affect the over-all failure rate tabulate for that component by a manufacturer. In many cases, any failure is unacceptable is the attitue towar such component failures, however it must be pointe out that if no component ever faile (a istinct impossibility!) ata center availability woul always be 00%. The concern shoul be, instea, the historical failure rate of this component, up to an incluing recent ata; if the failure rate is significantly higher than reference survey ata such as given in [3] an [4], this coul be an issue. If this is not the case, the concern shoul be whether or not the esign takes the component failure rate in question into account. Another aspect of component failure rates to take into account is the efinition of failure. In establishing failure rates for components, the manufacturer an the en-user must agree on this Revision 2 2/07 Page 5 of 3

6 point, otherwise component failure rates have no common basis of meaning. Per [3], the efinition is: Failure: The termination of the ability of a component or system to perform a require function. Preictably, isagreements can arise regaring the precise meaning of this efinition. For example, suppose component A acts in a manner that causes the power system to go from a higher reliability state to a lower reliability state, but oes not cause an outage. Is this a failure? Arguments exist for both yes an no answers to this question. One potential reason is the stigma associate with the wor failure. Component failure oes not equal a efective component! As has been iscusse herein, failure is a phenomenon associate with any engineere component, an is a function of usage history an maintenance as well as quality of manufacture Component Maintenance A hallmark of well-esigne ata center power systems is maintainability. The necessity of maintainability is unavoiable: In orer to keep component failure rates to their normal, nearlyconstant values, maintenance must be performe. Otherwise, the wearout line of the bathtub curve of Fig. 2 may be crosse, with a resulting increase in failure rate. One important point regaring system maintainability comes from the fact that the reliability of the system is not constant. This fact is illustrate in Fig.. Reliability ecreases over time. Ieally, when reliability reaches the lowest acceptable level, maintenance is performe to bring the system back to an acceptable level, an the process repeats. In reality, maintenance is rarely manage to this egree of etail, but accoring to the component manufacturer s maintenance scheule. The time uring repair is shown as mean owntime in Fig., an this is the same mean owntime escribe in section 3.. For a maintainable system, the reliability of the system uring the repair time oes not go to zero as shown in the figure, but rather to some lower but non-zero level. The better the maintainability of the system, the higher the reliability level uring maintenance. Such maintainability is achieve by parallel power paths, proper switching evices to allow component isolation, sufficient working space, etc. Another aspect of component maintenance is the nee for benchmarking. During maintenance, tests are performe, such as the high-potential tests, thermal scanning, etc. The results of such tests are the most meaningful when they are tracke over time, rather than simply counte as pass/fail for a given test. An abrupt change in a test result usually signals a problem, an the best way to notice such a change is by comparing with test results from previous maintenance perios. Computerize storage of such recors can facilitate this process. Maintenance can also help pinpoint abnormal sources of component eterioration, such as overloae circuits, improperly set protective evices, changing voltage conitions, etc. To avoi builing maintainability into a ata center power system will allow reuction of construction costs. However, without maintainability component outages are unavoiable either scheule, uring maintenance, or unscheule, ue to component failure Commissioning The importance of commissioning cannot be over-emphasize. True commissioning is ifferent from simple startup of components, which only tests the component in question, usually per preefine stanar proceures. At the highest level, commissioning tests whole systems an across systems to make sure all components work together properly. The manufacturer s startup proceures for a given type of component are simply esigne to bring that component up to the Revision 2 2/07 Page 6 of 3

7 point that it can be energize. True commissioning takes a system-level approach, with the goal being to ensure that the facility is functioning accoring to its intene purpose. Commissioning is an engineering-base activity because it requires custom proceures to be evelope base upon engineering knowlege of the system. It shoul test real-worl conitions. Often, interoperability problems can only be foun through commissioning. No system can perform to its true potential unless this important process is performe, an performe correctly. Failure to perform commissioning can lea to system reliability inices that are significantly lower than those preicte by a reliability analysis Emergency Proceures an Traine Maintenance Staff Emergency contingency proceures are a must to allow speey resolution of power system issues while minimizing the impact on critical loas. Such proceures shoul list, step by step, actions to be taken for a given type of emergency. Unfortunately, such proceures are not the norm for ata centers, an further, even if they are their use is epenent upon traine maintenance staff. Given the complexity of most ata center power systems, it woul be logical that the typical ata center woul have a maintenance staff that is extremely familiar with the system an large enough to cope with major emergencies. Surprisingly, this is not the case. In many instances, maintenance is contracte, with minimal on-site staff to cope with emergencies. In some cases, even the on-site staff are not familiar with system operation, instea relying upon component manufacturers to supply this familiarity. The result is that when an emergency oes occur, even if emergency proceures are in place there is no one familiar enough with the system to properly implement them. Having aequate system ocumentation is also a must. Data centers are ynamic environments, an as critical loa components an the aitional infrastructure to support it are ae this ocumentation must be maintaine. Unfortunately, in some cases up-to-ate ocumentation, such as a single-line iagram, oes not exist. If require, the services of the original engineer of recor for the facility shoul be retaine to keep system ocumentation up to ate. Having such ocumentation available in key easily-accessible locations (such as posting single-line iagrams for easy reference) is also a must. 5. Summary Data centers are ynamic systems, an therefore any type of analysis that provies a snapshot picture of the performance of such a system may give overly-optimistic results. Rigorous reliability analyses, when use properly to compare alternate system esigns or rank the relative performance of ifferent facilities, are powerful tools. However, other uses of the results of such analysis, such as attempting to guarantee the availability of a given facility over time, can lea to overly-optimistic results unless real worl factors are taken into account in the analysis. In orer to maximize the reliability of any ata center power system, recognition of such factors such as the over-reliance on availability figures, complexities associate with component failure rates, the importance of commissioning an maintenance, an the nee for aequate emergency proceures an traine maintenance staff is a must. Being cognizant of these real-worl factors, going beyon the 9 s an implementing accoringly, gives the best chance for success in keeping ata center power systems up an running. Revision 2 2/07 Page 7 of 3

8 6. References [] Tajali, R., Maintaining the Long-Term Reliability of Critical Power Systems, available at [2] Turner, P.W., Seaer, J.H., Brill, K.G., Tier Classifications Define Site Infrastructure Performance, 2006, available at [3] IEEE St , IEEE Recommene Practice for the Design of Reliable Inustrial an Commercial Power Systems [4] Hale, P.S., Jr., Arno, R.G., Survey of Reliability an Availability Information for Power Distribution, Power Generation, an HVAC Components for Commercial, Inustrial, an Utility Installations, Inustrial an Commercial Power Systems Technical Conference, 2000, Conference Recor, 7- May 2000, pp Revision 2 2/07 Page 8 of 3

9 7. Appenix Introuction to Reliability Mathematics As state in section 3, the language of reliability is the mathematics of probability an statistics. This section will give a eeper look into some of the concepts presente in section 3 for the reaer who is intereste in learning more about this subject. The approach herein will be to cover the basics without using calculus; this means that certain results will be presente without showing how they were erive. For further etail, it is recommene that the reaer consult reference [3]. 7.. Component-Level Reliability We will efine the ranom variable X as the time between failure of component A, in years. The value of the probability ensity of a ranom variable, given a certain number t, is the probability that the ranom variable is equal to that number.. For the ranom variable X this is written mathematically as P(X = t) an is the value of the probability ensity, for a given number of years t, that t is exactly the number of years until component failure. Using the probability ensity one may also efine, via calculus, a probability istribution which for a given number takes on the value of the probability that a ranom variable is less than or equal to a given number, written mathematically as P(Xb t). The exponential probability ensity is efine as: P(X t = t) = λe λ () The associate exponential probability istribution is foun from the exponential probability ensity using calculus an is: P(X λt t) = e (2) The term λ in () an (2) is simply a number. It is interesting to note the behavior of the exponential probability ensity an istribution as y increases the probability ensity starts at a value of λ an ecreases to zero, while the probability istribution starts at a value of 0 an increases to. This is illustrate graphically in Fig. 3. If the exponential probability ensity/istribution were applie to the ranom variable X (which we have efine above as the number of years until failure of component A), the value of the probability ensity of X per (),for a given number t, woul be the probability that the number of years until the failure of component A is equal to t. Similarly, the value of the probability istribution of X per (2), for a given number t, woul be the probability that the number of years until failure of component A is less than or equal to t. If the preicte amount of time between failures is more likely to be less than the time in service as the time in service increases, then the form of the exponential probability istribution per Fig. 3 b.) makes intuitive sense, since the larger the value of y ( the number of years in service), the higher the probability that the time between failures is less than t. The physical meaning of the exponential probability ensity is harer to visualize intuitively in this case; it is sufficient to know for our purposes here that the two are relate through calculus. Revision 2 2/07 Page 9 of 3

10 l Increasing t ---> Increasing t ---> Fig. 3 a.) Graphical representation of a.) Exponential Probability Density b.) Exponential Probability Distribution One more thing may be etermine from the exponential probability ensity/istribution as applie to X. Via calculus, the mean or expecte value of X may be etermine. This represents the value of X that woul be etermine over many observations of the failure of component A. The result: The mean value of X = /λ, which a constant. An since X was efine as the time between failures of component A, the number /λ represents the Mean Time Between Failures, or MTBF, for component A. The number λ itself is known as the failure rate of component A. Both of these terms were introuce in section 3. The mean time to repair (MTTR) was also introuce in section 3. Herein, the MTBF is referre to with the variable an the MTTR is referre to with the variable r. b.) 7.2. System-Level Reliability Metho of Minimal Cut-Sets As state in section 3, there are several methos to calculate the reliability inices for a power system. Herein we introuce the metho of minimal cut-sets as it is the simplest to unerstan an allows a goo introuction to how such analysis is performe. In the metho of minimal cut-sets, a cut-set is a set whose failure alone will cause system failure an a minimal cut-set is a cut-set with no sub-set of components whose failure alone will cause system failure. In a power istribution system, a minimal cut-set can be consiere to be a portion of the power path from the source to the loa which, if remove, woul cause power to the loa to be lost. Insie a single minimal cut-set there may be a number of parallel paths, all of which must fail for the entire cut-set to fail. This is illustrate in Fig. 4: Revision 2 2/07 Page 0 of 3

11 Component Component 2 Component m Fig. 4 Visualization of a minimal cut-set The minimal cut-set in Fig. 4 consists of m components, all of which must fail for the cut-set to fail. The probability of failure of the cut-set is the probability that all components fail. Consier, then, the jth component, where j =,2,3,..,m. Assuming that the component can be characterize with a constant MTBF (i.e., assuming an exponential probability ensity for the time between failures) an a constant MTTR, the component can be consiere to be working for an amount of time equal to the MTBF, an not working (i.e., faile) for an amount of time equal to the MTTR. The probability of failure of the jth component can be calculate as: Average Downtime of Component j MTTR j rj Pj = = = (3) Total Time Perio Consiere MTTR + MTBF r + j j j j The system may be consiere to consist of a number of cutsets. If the number of cutsets is n, consier the ith cutset, where i =,2,3,..,n. Now, assuming component inepenence the probability that the entire ith cut-set fails, C i, may be calculate as prouct of all of the probabilities of failure of the components, each of which was calculate per (3): C = P P P L P (4) i 2 3 n For a given cut-set, suppose that the probability of failure C i has been calculate. This can be thought of in terms of cut-set failure MTBF i an a cut-set MTTR r i : C i Average Downtime of Cutset i ri = = (5) Total Time Perio Consiere r + i i How are i an r i to be etermine? One coul think of the reciprocal of i as a repair rate (repairs per year, for example). Logically, the repair rate for the cutset woul then be the sum of the repair rates for each component of the cutset. State mathematically, this is: Revision 2 2/07 Page of 3

12 r r = L i i r r2 r3 rm = r + r 2 + r 3 + L r m (6) Now we have a way of calculating the cutset MTTR r i. It is only a matter of solving (5) for the cutset MTBF i at this point: Ci i = ri (7) Ci A given minimal cut-set is escribe as first orer if it has one component, secon orer if it consists of two components, etc. The entire power istribution system may be visualize as the total number n of minimal cut-sets connecte in series. The ientification of the minimal cut-sets is via a Failure Moe an Effects Analysis (FMEA). As an approximation, however, higher-orer cut-sets may be neglecte since their probability of occurrence is low. A visualization of a power system as series-connecte minimal cut-sets is given in Fig. 5: Minimal Cutset Minimal Cutset 2 Minimal Cutset n Fig. 5 Visualization of a power system as minimal cut-sets in series In Fig. 5 the system consists of n minimal cut-sets in series, each with an associate probability of failure (C i ), MTBF ( i ), an MTTR (r i ). The probability of failure of the system, then, is the probability that any one of the minimal cut-sets fails. This is, in general, the sum of all of the iniviual cut-set probabilities, minus the sum of the probabilities of any combination of simultaneous cut-set failures. If the probabilities of simultaneous cut-set failures are low, then the probability of failure of the system, S, is: S = C (8) + C2 + C3 + KCn The probability of system failure, S, may be thought of as a function of the system MTBF,, an system MTTR, r: Revision 2 2/07 Page 2 of 3

13 r S = (9) + r The reciprocal of the MTBF for the system is the system failure rate, efine in section 3. Logically, the system failure rate for the series combination of cutsets is the sum of the iniviual failure rates of the cutsets. State mathematically, this is: = L + n = L + n (0) With a way of calculating the system MTBF, one has only to solve (9) for the system MTTR, r: S r = () S The system reliability inices may now be calculate as: Inherent Availability Ai = S (2) Re pair Downtime Rt = r (3) Such calculations can be automate, an this type of analysis is typically one on the computer via reliability software rather than by han ue to the number of cut-sets involve. In most cases several points in the system, usually critical loa points, constitute ifferent system outputs. This further necessitates computer analysis ue to the number of calculations involve. In performing such analysis, careful consieration must be given to just what constitutes a failure, as this will affect the values of the component MTBF s. Revision 2 2/07 Page 3 of 3