The Comparison and Seection of Programming Languages for High Energy Physics Appications TN-91-6 June 1991 (TN) Bebo White Stanford Linear Acceerator Center P.O. Box 4349, Bin 97 Stanford, Caifornia 94309 USA This paper discusses the issues surrounding the comparison and seection of a programming anguage to be used in high energy physics software appications. The evauation method used was specificay devised to address the issues of particuar importance to high energy physics (HEP) appications, not just the technica features of the anguages considered. The method assumes a knowedge of the requirements of current HEP appications, the data-processing environments expected to support these appications and reevant non-technica issues. The anguages evauated were Ada, C, FORTRAN 77, FORTRAN 90 (formery 8X), Pasca and PL/. Particuar emphasis is paced upon the past, present and anticipated future roe of FORTRAN in HEP software appications. Upon examination of the technica and practica issues, concusions are reached and some recommendations are made regarding the roe of FORTRAN and other programming anguages in the current and future deveopment of HEP software. I. Introduction The programming anguage to be used for any software appication is a critica determinant of the speed of software deveopment, the ease of software maintenance and the portabiity of software to other systems. Many anguage comparisons have appeared in computer science and programming iterature. A arge portion of these comparisons have been conducted on the anguages in situ. Litte, it seems, has been written about how anguages shoud be evauated and assessed with respect to specific software projects. Physicists have ong been recognized as among the most knowedgeabe of natura scientists with respect to the technica aspects of computing. Advancements in high energy physics have been driven by advancements in computer hardware and software technoogy and vice versa. There is significant cross-over of physicists into computer-reated tasks. Physicists and deveopers of physics software generay have the expertise to make a choice of deveopment programming anguages. This paper describes an exercise in which a number of computer programming anguages were evauated specificay for high energy physics (HEP) appications. The method devised for this evauation is based upon a knowedge of the requirements of the appications invoved, the dataprocessing environments expected to support those appications and additiona surrounding technica and non-technica issues. This method, coincidentay, cosey paraes the feasibiity and requirements anaysis common to many software engineering methodoogies. Therefore, the goas of this study were, Contributed to the InternationaJ Workshop on Software Engineering, Artificia Inteigence and Expert System for High Energy and Nucear Physics, Lyon ViJJeurbanne, fiance, March 19-24, 1990 Presented at the Conference on Computing in High-Energy Physics, Oxford, Apri 1989
O to systematicay evauate candidate programming anguages specificay for high energy physics appications; and O to evauate the roe of programming anguages in future HEP computing environments. II. Methodoogy In order to perform a meaningfu anguage comparison, it was necessary to define specific anguage evauation eements. These evauation eements define the anguage comparison environment and insure that ony the meaningfu features of the candidate anguages are considered. A concise definition of these eements hopefuy reduces the possibiity of programming anguage bigotry and the comparison of irreevant anguage features. For this study, the foowing eements were identified: O Technica specification of a generic HEP programming appication; O Identification of candidate programming anguage features necessary to satisfy these specifications; O candidate programmin g anguage fit in HEP software environments; O Anticipated growth and future deveopment of the candidate programming anguage. III. Identification and Comparison of Reevant Language Features High energy physics data reduction and anaysis programs provide good exampes of highy numerica-intensive, batch-oriented scientific programming appications. An anaysis of authorsoicited, typica off-ine programs used at SLAC and CERN was used to compie a ist of common processes required by such appications. This ist incuded O input/output of binary fies O 64-bit foating-point arithmetic O operations using compex data types O vector and matrix arithmetics O data structure manipuation O access of common bocks of data O direct addressing of dimensioned variabes (as distinct from matrix operations and incuding non- zero ower bounds for arrays) O separate or independent compiation of subprograms and cross modue checking 2
O subprogram parameter passing by vaue and reference and the abiity to pass routines as p-ters O access to ibraries of mathematica functions O access to histogramming services O access to graphics services O extended fie or database services (e.g., partice tabes). This ist of processes was used to identify the candidate programming anguages to be evauated in the study. It was expected that the candidate anguages woud either have features-which woud map directy to the processes on the ist or woud have access to toosets which woud suppement their functionaity. It was aso decided that the candidate anguages woud be we-estabished (i.e., standardized to some degree), scientific and agorithmic. The anguages which met these criteria were Ada, C, FORTRAN 77, FORTRAN 90 (formery 8X), Pasca and PL/. The next step was to derive a wishist of anguage features from the ist of typica HEP software processes. Specific candidate anguage features were then compared against this ist. The resut is iustrated in Tabe 1, The Detaied Language Features Comparison. References in this tabe provide carification and indicate aternatives to deprecated features. In this way each candidate anguage can be reaisticay evauated in terms of its standard and non-standard features, impementation-dependent features and extensions and toosets commony avaiabe in HEP computing environments. * Ada has the broadest base of features of the anguages being discussed. It has an extensive range of array definitions and permits subprograms to use arbitrariy sized arrays. Ada does not permit the user to define the interna representation needed for muti-dimensiona arrays. Intrinsic array operations are not defined within the anguage and are viewed as an extension in an Ada package if required. Ada offers extended capabiities in the area of bit manipuation on the order of enumerated types. Rea data types are defined within the anguage and support for extended precision and compex arithmetic can be provided through packages in a reasonabe way, since overoading of operators aows the mathematica syntax to be preserved. However, Ada does not recognize the need for at east two (singe and extended precision) structuray different foating point arithmetics. Ada was obviousy designed with conscious emphasis on sound software engineering. It has the best abiity of the candidate anguages to hide data and routines. It is especiay strong in demanding a carity of exposition of the data and routines which may be imported to, and exported from, any program unit by providing cross-modue checking. Ada provides a
wide range of I/O faciities though a set of pmdefmed packages, such as DIRECT-IO, TEXT-IO, or LOW LEVEL-IO. The with and use constructs can be used for computation. * C is a anguage with functionaity simiar to Pasca, but it is much ess strict in the consistency checking of program units. LINT must be used to achieve the kind of cross modue checking that Ada and Pasca offer. Data hiding is avaiabe at the same eve as Ada. C defines I/O through a standard ibrary (ibrary stdio). C defines rea and doube precision data types in the anguage, and support for compex arithmetic can be provided using the typedef and struct faciities, but without retaining the mathematica syntax. C manages fairy we in the area of bit manipuation. Bit fieds of variabe widths can be incorporated into user-defined structures, and a pointer type can be used to ocate the structure at a given address in memory. Sets are not defined. Enumerated types are not defined in Kernighan and Ritchie but are in the ANSI draft standard. The anguage provides a very usefu preprocessor for performing such operations as macro definition and ibrary incusion. * FORTRAN 77 provides reasonabe support for array handing and manipuation. It is the ony one of the candidate anguages which defines rea, doube precision and compex data types as part of the anguage. FORTRAN 77 contains a very cear definition for formatted and unformatted J/O to termina, printer and mass storage. However, it is cear that FORTRAN 77 acks a great dea of the functionaity that the other anguages offer, and itte type checking and no cross modue checking. There is no data hiding beyond oca variabes. In reaistic HEP appications, FORTRAN 77 must be suppemented by additiona software toos. At CERN, the whoe concept of ZEBRA is a manifestation of one of FORTRAN 77 s needs (i.e., data structure manipuation). FORTRAN 90 has directy addressed many of these deficiencies of FORTRAN 77. * Pasca, as impemented by both DEC and IBM, is very simiar to Ada from the stand-point of computing itsef. It offers the abiity to hide data and code at the same eve and cross modue checking. It does not offer muti-programming as part of the anguage. Pasca provides rather simpistic I/O faciities in the Jensen and Wirth defmition which are not as comprehensive as those of FORTRAN 77 and do not provide for unformatted (binary) data. Extended I/O faciities are common in impementation-dependent compiers. Extensions in the area of array manipuation aeviate many of the probems which exist in the Jensen and Wirth definition. Bit manipuation is possibe in an arcane, poory documented manner. Rea data types are defined in the anguage, but doube precision and compex must be emuated in user-defined data structures. * PL/ is a very rich anguage in terms of its constructs, neary on a par with Ada. It offers data hiding at the same eve as Pasca. Like Ada, PU offers a anguage feature for exception 4
handing. It offers rather itte, however, in terms of type checking and no cross modue checking. According to the anguage definition, concurrent computation is part of the anguage. However, this feature was not avaiabe in the IBM and DEC compiers used. Compex and doube precision data types are defined as part of the anguage. Bit functions are aso supported. Sets and enumerated types are not defined in the anguage. PLJ does provide a very powerfu preprocessor which provides such capabiities as incuding text from an externa ibrary, conditiona compiation of sections of the source program, macro deveopment and variabe name repacement. The technica issues aone woud appear to suggest the superiority of any of the candidate anguages over FORTRAN 77. However, it is cear that this woud obviousy not be a reaistic concusion. From a historica perspective no programming anguage evauation coud be conducted with the prospect of competey repacing FORTRAN in HEP appications. This presumption is substantiated in the evauation of practica issues. IV. Programming Language Fit (Practica and Non-Technica Issues) The foowing categories of practica issues reevant to coaborative HEP software appications have been identified: O The Historica Roe of FORTRAN For a its ineegance, and ack of safety features, it seems certain that FORTRAN wi remain the main anguage for HEP code we into the 199Os... Computing at CERN in the 1990s The widespread adoption of FORTRAN by the physics community probaby came about because it was the best approximation to the genera purpose anguage, capabe of abstracting most of the computer-oriented ideas that one physicist wished to express to another (i.e., the ingua franca). Miions of ines of program code written in FORTRAN are a vauabe foundation of the roe of computing in high energy physics. Much of this code has proven itsef over the course of many years to be accurate, reiabe, efficient and fexibe enough to be used as the requirements of experiments and even as physics itsef have changed. Concurrenty, the shortcomings of the anguage have ed to the production of software ibraries that aeviate many of its defects, and represent a huge investment in accumuated expertise. From a historica perspective it is more reaistic to evauate programming anguages in the HEP environment in such a way as to compement FORTRAN appications. To fit Ada, C, Pasca or PL/ into a HEP appication of any consequence 5
wi depend on the abiity of that anguage to communicate effectivey with modues, ibraries and other program units which were deveoped in FORTRAN. O Code Portabiity and Maintainabiity Code portabiity is one of the major practica issues in a HEP experimenta environment. Experiment coaborators are iteray seconds away from one another via network. Programs, subprograms, ibraries, etc. are easy and necessary to share. Many of the advantages of a coaborative environment woud be ost if software portabiity presented a major probem. The idea programming anguage woud operate independenty of the hardware and the operating system within which it functions. Programs written in that anguage woud execute with minimum modification on a coaborative systems and yied identica resuts. The portabiity of software written in a high eve anguage depends upon the avaiabiity of the appropriate compiers on the target machines or on the existence of a compier that is itsef portabe and which can be moved easiy to new machines at a cost far ess than that required to produce it initiay. Moreover, for portabiity to be successfu, a common subset (or diaect) of the high eve anguage must exist among the compiers. Given this scenario, the programmer has a maximum degree of independence and the functionaity aowed by the portabiity of software. 6
O Inter-Language Communication It is not uncommon in the evoution of a software system to want to program a new appication in one progr amming anguage whie sti maintaining the use of existing ibraries that have been programmed in a different anguage. In most cases the recoding of existing software soey for compatibiity cannot be cost justified. This concept of inter-anguage communication has payed a very important historica roe. Subprograms and ibraries programmed in assember anguage offer greater efficiency to appications programs written in higher eve anguages. The increasing demand for program portabiity across mutipe computer systems has pushed the demand for inter-anguage communication to the higher eve anguage eve from the assemby anguage eve. Separate compiation of program modues aows for compatibiity at the source code eve if the modue interfaces are precisey defined. Effective inter-anguage communication deetes the need to Ye-invent the whee and aows the capabiity to integrate the od and the new in a satisfactory manner. In a HEP environment, this is a measure of the fit to existing FORTRAN ibraries, graphics and database faciities. O Language Standardization Language standardization activity must be considered for projects with engthy ife cyces. The technica direction assumed by the standardization committee for a programmin g anguage is of critica importance. Standards activities shoud attempt to preserve investments in software written in the anguage and to create new standards with as high a degree of compatibiity as possibe with previous standards. At issue is the question of object code compatibiity and source code compatibiity. V. Preiminary Concusions A survey of the technica and practica features of the candidate anguages ed to the foowing preiminary concusions: O A of the candidate anguages are mature; O Ada has the broadest base of standardized features; O C has broad base of features couped with high portabiity and avaiabiity factors; O FORTRAN 77 has a the HEP data types native; functionaity has a strong dependence on extensions and additiona software toos especiay with data structures; O FORTRAN 90 hods great promise, but when wi reiabe compiers be avaiabe? O Many important Pasca features are impementation-dependent; O PI.,4 is a very rich in features; avaiabiity coud be a major concern. 7
It is obvious from these concusions that not one of the candidate anguages is ceary superior for HEP appications when considering technica and practica issues. Yet, it does become cear that the size and compexity of HEP software systems is forcing the HEP community to confront the notorious Software Crisis. It may have been more reaistic to address the question Can an evauation of programmin g anguages for HEP appications hep improve programmer/physicist productivity and increase the reiabiity and maintainabiity of HEP software systems? VI.What About Muti-Language Systems? Instead of expecting (or hoping) a singe programming anguage woud be an obvious choice for HEP appications, perhaps a more reaistic future ies in muti-anguage systems? As was indicated earier, FORTRAN pioneered muti-anguage programs with Assember routines. However, portabiity requirements have pushed the issue from assember to higher order anguages. A of the candidate anguages have some capabiity for inter-anguage communication. Opening the door for muti-anguage systems aows compex and coaborative HEP software projects to: O choose the best too (anguage, tooset, ibrary, software product, etc.) for the job at hand; O choose the best programmer/software designer for the too; O take better advantage of the technoogy and expertise in current computing. VII. Concusions The foowing genera concusions were reached as a resut of this programming anguage comparison and evauation: we O The common assumption that FORTRAN wi remain the main anguage for HEP code into the 90 s is a vaid, but conservative, one; O HEP software deveopment in aternative anguages and muti-anguage systems shoud be encouraged if it can be proven to be a sound software design decision.
Goa of This Study To systematicay evauate candidate programming anguages spec$cay for high energy physics appications; and To evauate the roe of programming anguages in future HEP computing environments 9
For a its ineegance, and ack of safety features, it seems certain that FORTRAN wi remain the main anguage for HEP code we into the 199Os... Computing at CERN in the 1990s FORTRAN is probaby the ony perknnia standard which wi never be questioned. Trends in Computing for HEP I don t know what the anguage of the year 2000 wi ook ike but I know it wi be caed FOR- TRAN. Tony Hoare If HEP wishes to keep to its eve of achievement, credibiity and exceence, then it needs an injection of bright young computer-wise scientists and engineers. Pa010 Zanea 10
The Tower of Babe, Not a New Acceerator Design 11
Language Evauation Eements Technica specification of the programming appication Anaysis of candidate programming anguage features Programming anguage fit in the appication environment Growth and future deveopment
Typica/Required Processes in HEP Appications input/output of binary fies 64-bit foating-point arithmetic operations using compex data types vector and matrix arithmetics data structure manipuation access of common bocks of data direct addressing of dimensioned variabes (as distinct from matrix operations and incuding nonzero ower bounds for arrays) separate or independent compiation of subprograms 13
subprogram parameter passing by vaue and reference and the abiity to pass routines as parameters access to ibraries of mathematica functions access to histogramming services access to graphics services extended fie or database services (e.g., partice tabes). 14
Wishist and Detaied Language Feature Comparison Features Ada m Fgx Prrca PL/I address arithmetic arbitrary array bounds arbitrary function return v&e argument by name or position array bound checking TC4y Of StNCtWCS array operations biaary I/O bit ogic ca by reference ca by vaue compex variabes co-t computation uoss rnodue argumcat checking data hiding descriptive variabe aames doube precision dynamicaanory cnumcratcd data types aception handing formatted I/O ~acm3i;=r.~ - &*a...*...ci.*- --.- - inrcr-hrguagc communicajon oca procedures mutipe subprw catria pass arbitrary wtrix pass arbitrary -dimensiona array pass structure to subprogram pointers to functions pointers to variabes prepr- ItXWSiOt routine hiding sets static variabes StiJg.5 strong type cbccking subprogram argument checking user generic functions variabe equivaence variabe initiaization variabe range checking no yes DO ;k m no Yes yes Ye a0 Ya a0 a0 y!3) ya (21) & gi) (29) a0 Ya yes & EL (42) t0 a0 a0 yes &, II0 110 Y= Yes a0 15
References (1) s; (4) (9 (6) 0 69 (9) w (11) 02) (13) 04) (15) (16) (17) (18) w9 m (21) 2 (34) i2 if2 (29) m (31) (32; WI WI (35) (36) (37) (38) (39) w. (41) (43) (43) w (45) WI (47) WI (4% w-9 61) (52) (53) w tokoowu bar-0 in ref. [11 ony sitnpk variabks or poinms; ANSI draft standard aso incudes strwutrcs (81 USUgCOCiCfUOCiOOS may be vritm suus bit string. sttitq at* pointer, some uprcssions DEC - yes: IBM - with IAD faciity DEC-yea;IBM-uo uac operator ovdoadiry @wrod io a package) cao assign an array to anotbcr. but 00 opentioos on arrays as units impcmcntation dqwndatt be INTENT of a dummy argument may be specified can use typedef and struct, but without mathcmatic.ai syntax 00, must be emuated with raords defined in the anguage defitioo: not impemented by DEC or IBM 00. occusay to UK LINT yes, if INTERFACE bocks arc used 00, ony in oca procedures DEC - yes; IBM - oo (ony 8 characters aowed) no, must be emuated no, ItsczEBRA -- no, io ref. m; yes io the ANSI draft standard (81 impementation apenden ya for I/O impkmtntation depxknt DEC-yu;IBM-no,uscZEBRA defii via INTERFACE PRAGMA; not aways impancntod DEC - yes; IBM - yes; standard defiition - unknown. EXTERNAL decaration defined via EXTERNAL rod FORTRAN decarations; impaneotatioo dependent yes, with parameters of the DECLARAion and PROCEDURE statements and ony with specific anguages unknown uoknown pass array of pointers to arrays; both must start at 0 pass array of poiota-s to arrays; turo off range checking ower bound must be 0; no bound checking possibe DEC(IS0) - yes; IBM(ANS1) - no; turn off range checking DEC - yes; IBM -no not in the sense of C, pass routine at FORTRAN eve oq use ZEBRA conditiooa compiation, decaration import, and simpe exica substitution of constants with arithtnedc arc part of the W3-e yes, MORTIWN is one exampe substitutioo of constants with atithtnetic is supported by DEC and IBM 00. ony as oca procedures impementation dependent canbccodcdasamodule. hnpuncntation dependent yes for simpe variabes. no for routines if IMPLICIT NONE is used 00, must use LINT uocbeckd-woversion or variant record variaot record or union variant record or union impementation dependent 16
Practica Issues The FORTRAN egacy Port abiity Maintainabiity Standardization (features vs. extensions) Avaiabiity of knowedgeabe users Fit with current technoogy, avaiabe toos and projected hardware environments 17
Preiminary Concusions I A of the candidate anguages are mature Ada has the broadest base of standardized features C has broad base of features couped with high portabiity and avaiabiity factors FORTRAN 77 has a the HEP data types native; functionaity has a strong dependence on extensions and additiona software toos esp. with data structures FORTRAN 90 hods great promise, but when?? Many important Pasca features are impementationdependent PL/ is a very rich in features; avaiabiity coud be a major concern
HEP Programming an.d the Software Languages Crisis Can an evauation of programming anguages for HEP appications Improve programmer productivity? Increase the reiabiity and maintainabiity of HEP software systems? 19
Can Software Engineering Hep? The software project ife-cyce: 1) Feasibiity and Requirements Anaysis 2) Logica Design 3) Detaied Design 4) Coding 5) Impement at ion 6) Maintenance 20
Structure Charts A too often used in the Detaied Design phase Defines the skeeton of the fina system in terms of subprogram structures, data structures, etc. Heps to separate interface specification from impement ation detais Demonstrates the maintenance view of the system MAIN Leve 1 Routines Leve 2 Routines 21
What About Muti-Language Systems?? FORTRAN pioneered muti-anguage programs with Assember routines Portabiity requirements have pushed the issue from Assember to Higher Order Languages. Aows: The best too for the job The best designer for the too 22
Inter-Language Communication Ada c defined via INTERFACE PRAGMA; not aways impemented DEC - yes; IBM - yes; standard definition - unknown _ FORTRAN 77 EXTERNAL decaration FORTRAN 90 EXTERNAL decaration Pasca defined ia EXTERNAL and FOR- TRAN dvecarations; impementationdependent PL/ yes, with parameters of the DEC- LARATion and PROCEDURE statements and ony with specific anguages 23
ILC Compatibiity Issues data type conventions: it is imperative that data type matching occurs with parameters passed to modues; array conventions: caing conventions: it is necessary to ktiow how mutidimensiona arrays are mapped into memory by the caed and caing anguages; it is necessary to know how parameters are passed to modues, how vaues are returned and how re ister and memory management is a anded. 24
FORTRAN e C Data Typing c TYPe int * ong int * foat * doube * foat [2] Struct Compex * doube[2] Struct Doube Compex * char * struct CHARACTER{ char * text; int ength; 1 FORTRAN Type INTEGER*2 INTEGER*4 REAL*4. REAL*8 COMPLEX COMPLEX COMPLEX*16 COMPLEX*16 LOGICAL*1 CHARACTER*(*) 25
Concusions II The assumption that FORTRAN wi remain the main anguage for HEP code we into the 90% is a vaid, but conservative, one; HEP software deveopment in aternative anguages shoud be encouraged if it can be proven to be a sound software design decision.