Jennifer Bützler, Christina Bröhl, Christopher M. Schlick

Transcription

1 Age-robust cognitive ergonomic design of network diagrams in project management software: An empirical study of the effect of continuous edges on execution time and error rate Jennifer Bützler, Christina Bröhl, Christopher M. Schlick Chair and Institute of Industrial Engineering and Ergonomics, RWTH Aachen University, Aachen, GERMANY Network diagrams in project management software (PMS) are a useful method when planning projects. However, working with these diagrams can quickly become confusing when dealing with complex project plans. This effect is often more pronounced for older persons due to age-related changes in perception, cognition and motor control. Nevertheless, project planning is frequently accomplished by older employees due to their knowledge and experience. Continuous edges can support perception and decision making when working with complex network diagrams. This paper describes an age-differentiated study in which different types of edge design were investigated. In order to improve external validity, the study combined different tasks and different levels of abstraction and aggregation. Reaction times and error rates were analysed as dependent variables. The results of this study reveal that the work with network diagrams can benefit from an ergonomic edge design. A continuous edge design led to significant improvements in performance for all age groups. Practitioner Summary: A review of commercial project management software solutions shows that the design of edges in network diagrams significantly differs between the products and there is no standard design. Therefore, the effect of continuous edges on comprehension of network diagrams in PMS was investigated in this study with respect to age. Results showed that the use of continuous edge design results in better performance regarding execution times and leads to lower error rates for all age groups. Keywords: network diagram, project management, ergonomic design, age-robust design 1. Introduction Not only the working environment in general, but in particular the field of project management, is currently dominated by two trends in Europe - the increasing use of software systems and the demographical change. Due to their knowledge and experience project planning and controlling is often accomplished by older project managers. Because of the high complexity of the planning tasks and the connection to risks, the use of project management software can provide an essential support of the planning process. One frequently used planning method are network diagrams, which are able to clearly show relationships between activities on the base of graph theory on the one hand, but on the other hand can become confusing especially for complex project plans. Therefore, working with network diagrams places particularly high demands on the visual perception and cognitive processing of the presented information. Furthermore, we carried out an age differentiated usability study of the de facto standard of PMS and found that there were significant ergonomic deficits regarding the visual design of network diagrams (Bützler et al. 2013). Regarding age-robust software design it has to be taken into account that the aging process goes along with changes in the perceptual, cognitive, and motor system. The process of aging is often accompanied by a reduction of the visual field (Werner et al. 2010) and a loss of visual acuity (Schieber 2006). The changes in the cognitive system concern e.g. the decline in capacity of the visual-spatial and verbal working memory (Park 2002) as well as a decline in selective attention (Kramer and Madden 2008). For visual search a shift in the speed accuracy tradeoff can be observed: older people typically need longer but also make fewer mistakes (Rogers and Gilbert 1997; Strayer and Kramer 1994). Regarding the motor system an increase in movement time for pointing movements can be observed with increasing age (Vercruyssen 1997). In order to adapt software systems to these age-related changes, a number of authors proposed guidelines of how to design web pages for older users (e.g. Chadwick-Dias et al. 2007; Czaja and Lee 2007; National Institute on 1

2 Aging 2009). Empirical studies for example focus on the development of age-robust prototypical client applications (Arnott et al. 2004; Hawthorn 2003), web-pages (Chadwick-Dias et al. 2003; Chevalier et al. 2007) and project management software (Jochems 2010). The findings underline how important it is to use both guidelines as well as empirical studies when designing software for the elderly. Within the field of graph drawing specific aesthetic criteria are known that enhance graph comprehension (DiBattista et al. 1999, Kaufmann et al. 2001). One aesthetic criterion deals with the reduction of bends in edges. Thereby continuous edges are created. Empirical studies show that depictions possessing a good continuity are perceived easier and as a whole unit (Beck et al. 2005, Field et al. 1993). This is also underlined by the Gestalt law of continuity (Koffka 1935). Nevertheless, this criterion has not been proven empirically for project management software. In addition, a review of commercial project management software solutions shows that the design of edges in network diagrams ranges widely between the different products and there is no standard design. Edges consisting of horizontal and vertical lines are used often but they possess bends in edges and therefore are discontinuous. However, according to the Gestalt law of continuity it is assumed that continuous edges are perceived better. Therefore, the effect of continuous edges on comprehension of network diagrams in PMS was investigated in this study with respect to age. Based on the described literature we expect that a continuous edge design leads to shorter execution times and lower error rates than a discontinuous design in successor identification and related tasks. Regarding the age group we expect an increase of execution time with age but at the same time due to a speed-accuracy tradeoff we assume error rates to decrease with age. 2. Method An age differentiated empirical study was conducted to investigate different types of edge design in network diagrams. The investigation was carried out in four parts as the aim was to combine basic analysis with practical relevance covering three different types of tasks (Figure 1). In the first part, a highly abstract and aggregated diagram was presented to investigate the effects of edge design within a setting where the diagram was shown on only one screen area and no information was distracting. The information of the activities was given abstractly by using numbers. In the second part aggregation was decreased by displaying the diagram on ten screen areas to increase the size of the activities so that a change between these different screen areas was necessary to solve the task. By using an application-oriented network diagram based on the popular Metra Potential Method (MPM) notation (Kerbosh and Schell 1975) in the third part, the information density was enlarged resulting in a lower abstraction. In part one, two and three the activities were labeled by numbers, whereas in part four the activities were labeled following the work breakdown structure of a realistic project plan. A predecessor and successor identification task had to be solved in part one, two and three whereas in part four, an interpretation task was conducted. 2.1 Design Performance was evaluated in terms of execution time and error rate while accomplishing predecessor identification, successor identification and interpretation tasks with different types of edge design in a network diagram. Within the predecessor identification task participants were instructed to search the predecessor(s) of the prior defined activity and select this/these with a mouse click. After that they had to confirm their answer with a finish -button. For the successor identification task, the assignment was to count the successor(s) of the prior defined activity, select the numbers of successors in an input box with a mouse click and confirm the selection with the finish -button. Within the fourth part a more practical task was chosen. In this interpretation task, the participants had to identify the activities that define the critical path and select these with a mouse click. Like in the prior tasks, confirmation of the selection was done by pressing the finish -button. A network diagram with 91 activities was chosen. With respect to the potential loss in visual acuity with increasing age, the font size was set for all parts to 22 (Vetter et al. 2010). The ratio between activity height to width was 1:2 for all parts of the experiment (part 1: h=1 cm, w=2 cm; part 2/3/4: h=2.25 cm, w=4.5 cm). The horizontal and vertical distance between the activities was set to 0.5* activity width. For predecessor and successor identification task the prior defined activity was generated randomly to prevent learning effects. 2

3 Each position was presented once within one age group and for each condition. For the predecessor task the first position and for the successor task the last position was excluded. Figure 1. Depiction of the different parts of the experiment, the level of abstraction and aggregation, the task types and the investigated network diagrams. The experimental analysis was based on a full factorial design with two within-subject factors for the predecessor and successor identification task and one within-subject factor for the interpretation task. Moreover, the age group served as between-subject factor. The within-subject factor edge design was investigated in three levels for all tasks (Figure 2). Two continuous edge designs were tested against the defacto design that included conventional bends. The first continuous design gained continuity by using smooth edges and the second design used diagonal links between activities. Figure 2. Investigated types of edge design. For the predecessor and successor identification task, the level of abstraction and aggregation (3 levels as described above) was investigated as the second within-subject factor. For the interpretation task only the fourth part of the task was used. Furthermore, the sample was divided into three age groups (Group I:

4 years, Group II: years, Group III: years) which served as between-subject factor for all tasks. This resulted in a 3x3x3 full factorial design for the predecessor and successor task and a 3x3 full factorial design for the interpretation task with repeated measures on the within-subject factors. Dependent variables were the execution time and the error rate, which is defined as the number of errors (wrong answers) occurred in relation to the number of executed tasks. 2.2 Participants Altogether, 90 subjects participated in the experiment. They were paid volunteers aged from 25 to 68 years. 36 younger subjects aged between 25 and 39 years (mean=29.23, SD=3.15), 36 subjects between 40 and 54 years that build the medium age group (mean=46.87, SD=4.13) and 36 persons between 55 and 68 years (mean=62.53, SD=3.16) were investigated. We did not investigate subjects older than 68 or younger than 25 in order to get a representative age range for persons working with project management software. Regarding experience with projects and project management, 76.7% of the younger age group, 56.7% of the medium age group and 53.3% of the older age group have ever been involved in project management in their job but only 6.7% of the younger, 13.3% of the medium and 13.3% of the older age group rated their experience with project management as high. Within the younger age group, 33.3% of the participants have ever used project management software and within the medium and older age group 16.7%. Regarding the experience with project management software, participants of all age groups had low or little experience with PMS. 2.3 Apparatus The experiment was conducted using a 17 TFT-monitor. The viewing distance was set to 50 cm and the illumination was kept constant at 300 lx. A self-developed software was used, in that the network diagram was depicted in the main part of the screen and the instruction was displayed permanently in the lower screen area. Changing between different screen areas was realized via a mouse click on the arrow on the left respectively right screen area. Pressing this arrow activates a slide to the next screen area similar to the gestural sliding navigation on touch screens. 2.4 Procedure In the beginning, the participants had to fill in a questionnaire regarding their age and experience in project management. Then the participants were seated in front of the monitor. Before each part of the experiment, the participants watched a short introduction video in that the task and the visualization of the corresponding part were explained. After that the participants were given a short training during which they could ask questions and were corrected by the instructor in case of occurring errors. Participants were instructed to fulfill the task as fast and as correctly as possible. All participants solved the different parts and tasks in the same sequence as depicted in Figure 1 starting with part one and the search task. For part one, two and three, subjects had to fulfil three predecessor identification and three successor identification tasks with the first experimental condition, then for the second and so on. In order to reduce switching costs the tasks were presented block wise and the participants were additionally informed about the change of task in between the two tasks. The sequence of the layout presentation was randomized between subjects. In part four participants had to solve two interpretation tasks for each condition. The data was analysed by factorial analyses of variance with repeated measures (ANOVA). The age group of the participants served as a between-group factor and the level of significance was set to α=0.05. Post-hoc tests were performed using Bonferroni correction. 3. Results 3.1 Predecessor identification task Regarding the execution time we found a significant age effect (F (2;87) =20.968; p<0.001) with a strong effect size of ω²=0.38. Participants of the medium (p<0.001) and the older (p<0.001) age group needed longer to 4

5 execute the task than participants of the young age group (Figure 3, left). Furthermore the middle-aged participants were faster than the older participants (p=0.044). For the level of abstraction and aggregation we also found a significant effect (F (1.338; ) = ; p<0.001; ω²=0.38). The execution time was lowest for the highest level of abstraction and aggregation and highest for the medium level (p<0.001 for all pairwise comparisons, Figure 3, right). It is assumed that the lower execution time for the lowest level of abstraction and aggregation than for the medium level is due to a training effect as all subject conducted the tasks first with the highest level, then the medium and lastly with the lowest level. For the factor edge design, the highest execution time occurred for the discontinuous edge design V1 with an average of ms (SD= ms), followed by the continuous edge type V3 (M= ms; SD= ms). The lowest execution times showed for the continuous edge type V2 (M= ms; SD= ms). However, statistically this effect was not significant. Figure 3. Execution time for the predecessor identification task as a function of age (left) and level of abstraction and aggregation (right). Regarding the error rate a significant effect of the age group was found (F (2;87) =5.100; p=0.008; ω 2 =0.12). The errors that were made by the older participants (M=12.0%; SD=21.7%) were significantly higher than the amount of errors made by the younger age group (M=4.7%; SD=12.3%; p=0.006). The difference between the medium (M=9.0%; SD=18.3%) and the older age group as well as the medium and younger age group was statistically not significant, but a trend can be observed descriptively that shows a higher error rate with increasing age group. These results do not indicate the occurrence of a speedaccuracy trade-off. Concerning the level of abstraction and aggregation, the ANOVA did not show a significant effect on the error rate (L1: M=8.6%; SD=17.0%; L2: M=8.1%; SD=17.2%; L3: M=8.9%; SD=20.0%). For the edge type a similar tendency was found for the error rate than for the execution time (V1: M=9.6%; SD=19.2%; V2: M=7.8; SD=18.0%; V3: 8.3%; SD=17.1%). However, also this effect was statistically not significant. 3.2 Successor identification task For the successor identification task we found a significant effect of the age group on the execution time (F (2;87) =27.026; p<0.001; ω 2 =0.46). Post-hoc pairwise comparisons showed that younger participants were significantly faster than middle-aged (p<0.001) and older participants (p<0.001). In addition, participants of the oldest age group were significantly slower than participants of the medium age group (p=0.007) (Figure 4, left). Furthermore, a significant effect for the level of abstraction and aggregation was found (F (1.862; )= ; p<0.001; ω 2 =0.43). A significantly lower execution time was found for the highest level of abstraction and aggregation (M=5813 ms; SD=3040 ms) than for the medium (M=12041 ms; SD=7076 ms; p<0.001) and for the lowest level of abstraction and aggregation (M=10260 ms; SD=4722 ms; p<0.001). The execution times for the medium and the lowest level of abstraction and aggregation did also differ significantly (p<0.001). Moreover, we found a significant interaction between age group and level of 5

6 abstraction and aggregation (F (3.723; ) =12.605; p<0.001). Analysis showed an ordinal interaction and therefore the main effects can be interpreted unambiguously. Regarding the factor edge design we also found a significant effect on the execution time (F (1.715; ) =4.321; p=0.020; ω 2 =0.01). The execution time was higher with the discontinuous edge design V1 than with the continuous edge design V2 (p=0.026). Descriptively V1 showed also a higher execution time than the continuous edge design V3. Comparing the two continuous edge types, the shortest execution time was found for V2 (Figure 4 right). Figure 4. Execution time for the successor identification task as a function of age (left) and edge design (right). Concerning the error rate, the ANOVA showed a significant main effect of the age group (F (2;87) =3.873; p=0.033) with an effect size ω 2 =0.08. The participants of the younger age group made significantly less errors than the participants of the older age group (p=0.030) (Figure 5 left). The level of abstraction and aggregation had no significant effect but the data showed a tendency towards higher error rates with increasing level of abstraction and aggregation (L1: M=3.2%; SD=10.3%; L2: M=4.3%; SD=12.6%; L3: M=4.6%; SD= 14.1%). Furthermore, a significant effect was found for the edge design (F (1.606; ) =3.601; p=0.039; ω 2 =0.02). The error rate was significantly higher for V1 than for V2 (p=0.008) (Figure 5 right). Differences between V1 and V3 as well as between V2 and V3 were statistically not significant. Figure 5. Error rate for the successor identification task as a function of age (left) and edge design (right). 3.3 Interpretation task For the interpretation task the ANOVA showed a significant effect of the age group on the execution time 6

7 (F (2;87) =24.687; p<0.001; ω 2 =0.44). Participants of the older age group had significantly higher execution times than participants of the medium (p=0.005) and younger age group (p<0.001). Moreover, the younger age group showed lower execution times than the medium age group (p=0.001) (Figure 6 left). As in the successor identification task, we also found a significant main effect of the factor edge design in the interpretation task (F (1.787; ) =3.619; p=0.034; ω 2 =0.01). A significantly higher execution time was found for the discontinuous edge design V1 than for the continuous design V3 (p=0.027). Descriptive analysis also showed a higher execution time for V1 than for V2. The comparison between V2 and V3 showed the lowest execution times for V3 (Figure 6 right). Figure 6. Execution time for the interpretation task as a function of age (left) and edge design (right). Regarding the error rate, a significant main effect of the age group was found (F (2;87) =24.687; p<0.001; ω 2 =0.44). Younger participants made significantly less errors (M=2.9%; SD=13.7%) than participants of the medium (M=10.0%; SD=25.1%; p=0.001) and older age group (M=20.0%; SD=36.6%; p<0.001). Moreover, the medium age group made less errors than the older age group (p=0.005). These results indicate no speed-accuracy tradeoff for the interpretation task. Concerning the edge design no significant effect on the error rate was found. However, descriptively most errors occurred when working with the discontinuous edge design V1 (M=13.3%; SD=29.8%) whereas the continuous edge designs resulted in lower error rates (V2: M=10.0%; SD=26.2%; V3: 9.4%; SD=26.9%). 4. Discussion For all tasks the results of the study reveal significant effects of the age group on execution time and error rate. These effects occur consistently for all levels of abstraction and aggregation, meaning that the results hold for high and low information density as well as for one or more screen areas. With increasing age participants needed longer to conduct the tasks and made more errors. Interestingly these results do not confirm the age-dependent shift in the speed accuracy tradeoff often reported in literature (Rogers and Gilbert 1997; Strayer and Kramer 1994). Concerning the type of edge design we found for the successor and interpretation task that a continuous edge design was preferable regarding execution time and error rate. Furthermore, we found a taskdependency regarding the question which kind of continuous design leads to the highest performance. For the successor identification task, the continuous design with smooth edge resulted in the shortest execution times and least errors whereas for the interpretation task the continuous diagonal design was preferable. Furthermore, the results of the predecessor identification task yielded a similar tendency regarding the edge design than the successor task. These effects occurred consistently for all levels of abstraction and aggregation. In conclusion, the results of this study reveal that the work with network diagrams can benefit from ergonomically designed edges. Based on the described results we recommend a continuous edge design as this leads to shorter execution times and less errors for all investigated tasks. This is true for diagrams in 7

8 which information is reduced and that can be depicted in one screen area but also for diagrams that show detailed information and are spread via different screen areas. Regarding the type of continuous design, it has to be investigated which type of task has to be conducted primarily. These results apply to all age groups and confirm a design-for-all approach. Acknowledgements The research was funded by the German Research Foundation according to the transfer project (SCHL 1805/6-1). References Arnott, J. L., Khairulla, Z., Dickinson, A., Syme, A., Alm, N., Eisma, R., and Gregor, P interfaces for older people. In Systems, Man and Cybernetics, 2004, IEEE International Conference on Vol. 1, IEEE Beck DM, Pinsk MA, Kastner S Symmetry perception in humans and macaques. Trends in cognitive sciences 9(9): Bützler, J.; Bröhl, C.; Jochems, N.; Schlick, C Age Differentiated Usability Evaluation of Project Management Software In Human System Interaction (HSI), 2013 The 6th International Conference on, IEEE, Chadwick-Dias, A., Bergel, M., and Tullis, T. S Senior surfers 2.0: a re-examination of the older web user and the dynamic web. In Universal Acess in Human Computer Interaction. Coping with Diversity Springer Berlin Heidelberg. Chadwick-Dias, A., McNulty, M., and Tullis, T Web usability and age: How design change can improve performance. In ACM SIGCAPH Computers and the Physically Handicapped No , ACM. Chevalier, A., Dommes, A., Martins, D., and Valérian, C Searching for information on the web: Role of aging and ergonomic quality of website. In Human-Computer Interaction. Interaction Design and Usability Springer Berlin Heidelberg. Czaja, S.J., and Lee, C.C Information technology and older adults. In A. Searsand J. Jacko (Ed) The humancomputer-interaction handbook CRC Press.DiBattista, G., Eades, P., Tamassia, R., und Tollis, I. G Graph Drawing - Algorithms for the Visualization of Graphs. Prentice Hall. Field DJ, Hayes A, Hess RF Contour integration by the human visual system: Evidence for a local association field. Vision research 33(2): Hawthorn, D How universal is good design for older users? In ACM SIGCAPH Computers and the Physically Handicapped No , ACM. Jochems, N Altersdifferenzierte Gestaltung der Mensch - Rechner- Interaktion am Beispiel von Projektmanagementaufgaben. PhD thesis, RWTH Aachen. Kaufmann, M. und Wagner, D Drawing Graphs - Methods and Models. Springer Kerbosh, J. und Schell, H Network planning by the Extended METRA Potential Method. Technical Report KS-l.l, University of Technology Eindhoven, Department of Industrial Engineering. Koffka K Principles of Gestalt psychology. Harcourt, Brace, Oxford, England. Kramer, A. F., and Madden, D. J Attention. In F. I. M. Craik and T. A. Salthouse (Eds.) The Handbook of Aging and Cognition. Psychology Press. National Institute on Aging Making your web site senior friendly. National Institute on Aging and the National Library of Medicine. Park, D. C., Lautenschlager, G., Hedden, T., Davidson, N. S., Smith, A. D., and Smith, P. K Models of visuospatial and verbal memory across the adult life span. Psychology and Aging, 17(2), Rogers, W. A., and Gilbert, D. K Do performance strategies mediate age-related differences in associative learning? Psycholgy and Aging, 12(4): Schieber, F Vision and aging. In J. Birren and K. Schaie (Eds.), Handbook of the Psychology of Aging, ,. Elsevier Academic Press. Strayer, D. L. and Kramer, A. F Aging and skill acquisition: Learning-performance distinctions. Psychology and Aging, 9(4): Vetter, S., Jochems, N., Kausch, B., Mütze-Niewöhner, S., and Schlick, C Age-induced change in visual acuity and its impact on performance in a target detection task with electronic information displays. Occupational Ergonomics 9(2): Vercruyssen, M Movement control and speed of behavior. In A. Fisk und W. Rogers (Eds.), Handbook of Human Factors and the Older Adults San Diego: Academic Press. Werner, J. S., Schefrin, B. E., and Bradley, A Optics and vision of the aging eye. In M. Bass, J. M. Enoch, and V. Lakshminarayanan (Eds.), Handbook of Optics. Volume III. Vision and Vision Optics. McGraw-Hill. 8