Online Visual Landscape Assessment Using Internet Survey Techniques

Transcription

1 Online Visual Landscape Assessment Using Internet Survey Techniques Michael ROTH 1 Introduction The conservation, preservation and development of the variety, particularity and beauty of nature and landscapes are an important historical root of nature conservation and landscape planning (cf. GASSNER, 1989; FISCHER-HÜFTLE, 1997; WÖBSE, 2002, p. 167). To achieve these aims, the scenic beauty and visual quality of landscapes have to be evaluated. For visual landscape assessment, landscape planners can use either non-empirical expert methods or empirical methods (surveys). Both types of methods have serious disadvantages: The so called expert methods normally represent the assessment of one person, which can lead to arbitrary results depending on the scheme chosen from the huge variety of possible scenic quality estimation methods (GRUEHN 2001). The empirical methods for gathering data on visual landscape quality cause an immense practical effort, resulting in either small samples or high costs. The situation mentioned (non-validated expert methods on the one hand and highly laborious and expensive empirical methods on the other) has led to the insufficient consideration of visual aspects in environmental planning (HEHL-LANGE & LANGE, 1992; KÖHLER & PREISS 2000; BREUER 2001; NOHL 2001; JESSEL & FISCHER-HÜFTLE 2003). Since the mid-nineties of the last century, we are in the midst of an Internet revolution in experimental research (REIPS, 2002). In landscape evaluation, especially visual quality assessment, the aforementioned revolution hasn t taken place yet. Landscape planning is just about to discover the vast potentials of digital media as means of public participation in the planning process (cf. von HAAREN et al., 2002). So it is about time to expand the knowledge on the use of up-to-date Internet survey techniques in empirical visual landscape assessment, which was the main goal of the study presented in this paper. 2 Method To (a) test the objectivity, reliability and validity of visual landscape assessment gained through Internet surveys and (b) find out whether there are (demographic, methodological or technical) factors affecting the rating of the participants, two main steps had to be taken: First, suitable criteria had to be found to judge the validity of the Internet survey and the data acquired by it. The results of a recent on-site survey and photograph-based survey (GRUEHN et al., 2003, unpublished) were taken for this purpose, because the validity of in-

2 2 Michael Roth field and photograph-based scenic quality evaluations is acknowledged by experts and demonstrated by scientists (cf. HUNZIKER & KIENAST, 1999). Second, a web questionnaire was developed incorporating up-to-date standards for Internet-based experimenting (e.g. REIPS, 2002; ADM, 2000, 2001a and 2001b) into landscape perception research. Figure 1 shows an example of the online questionnaire. Fig. 1: Screenshot of the online questionnaire. Several open source/freeware software systems were used for the creation of the online survey: Hyper-Text Markup Language (HTML) web pages, small JavaScript programs (used for client-side response time measurement, user input check-up and hardware

3 Online Visual Landscape Assessment Using Internet Survey Techniques 3 identification) and programs written in Practical Export and Reporting Language (PERL) running on the Apache web server to generate dynamic web-pages (online questionnaire) and to save the transferred data to a MySQL database running on another server. The technical realization is illustrated in figure 2. Fig. 2: System architecture of the online questionnaire. As no commercial software was used, no licence fees had to be paid. No plug-ins or special software was needed on the participants (clients ) computer. Several other means were taken to ensure the acceptance of the online survey and to increase the number of participants by reducing the amount of dropout (for the first three techniques cf. REIPS, 2002): high-hurdle technique (e.g. collection of personal data at the beginning, decrease of the amount of text page per page), warm-up technique (practice of rating before the real experiment start), one-item-one-screen design (each rating on a separate web page), incentive (raffle of two gift certificates for books, 10 each), response time measurement.

4 4 Michael Roth The obligation to maintain secrecy was ensured by the possibility to leave all demographic data fields blank and the separate storage of demographic data and evaluation results. For identification only the address, and for localisation of the participant only the postal code or the country, was used. To investigate potential methodical biases resulting from different means of input, every second person was presented a text field to directly enter their scoring instead of the graphical rating scale with numeric marks shown in figure 1. The participants had to evaluate up to 17 digital landscape photographs regarding 25 criteria that were all taken from either the German Federal Nature Conservation Act or various scenic beauty evaluation methods used in landscape planning. The experiment was online from November 4, 2003 for two months. Respondents were acquired through advertisements on several web sites, in online forums and newsletters. A documentation version of the web questionnaire can be visited at in the Internet. 3 Results 3.1 Participants 774 people visited the online survey s web site. 558 of them viewed further pages of the online questionnaire. 424 started with the evaluation of the landscape photos and 321 completed the evaluation of at least one photo regarding the 25 criteria presented. Overall, 1477 complete evaluations of photos were collected. The age of the participants ranged from 11 to 78 years, the average age was nearly 30 years. (Coincidentally) 50 % of the participants were male; the other 50 % were female. Most of the participants came from Germany (337), Austria (29), Switzerland (9), but even though the questionnaire was only presented in German language some participants came form Italy (2), the USA (1), Finland (1), Japan (1) and Turkey (1). The participants differ from the general public regarding their average age (just under 30 years), their above-average school graduation and above-average professional graduation. Only about one quarter of the participants deal with landscape assessment occupationally. 3.2 Dropout analysis Dropout analysis is important to judge the generalizability of the online survey s results. As the participants could stop the experiment whenever they wanted, and demographic and technical data was collected right at the start and each rating and response times were saved separately (and immediately after the rating) to the database, dropout could be analysed regarding the dependence on all these factors.

5 Online Visual Landscape Assessment Using Internet Survey Techniques 5 None of the technical factors (screen resolution, colour depth, operating system, browser) of methodological factors (type of rating input) was found to have a significant influence on dropout (n = 558, α = 0,05). From the demographical factors, only age, nationality and the role of nature and environment for the participant s life were found to have significant (p 0,05), but very small (0,008 ε² 0,04), influences on dropout. Using COHEN S (1988) thresholds for effect sizes as cited in BORTZ (1999, p. 137), these effects have only low practical relevance. 3.3 Response time analysis The filling in of the web questionnaire for the assessment of 2 or 3 different landscape photographs took about 10 to 15 minutes. The rating for one criterion took between 4 and 6 seconds (medians of response times) except when a new photograph was presented (the median of response time then was 11 seconds). Several factors have a strongly significant (p 0,001) influence on response times: Participants using a text field for data input needed 15 seconds less for the assessment of one photograph (25 criteria). The average time for the assessment of one photograph decreases with the course of the survey from about 3 minutes (first photograph) to about 2 minutes (17th photograph). Women need about 25 seconds less for the complete rating of one photograph. Users of bigger screens (higher screen resolution) and newer browsers need less time for the evaluation of the landscape photographs. 3.4 Objectivity of the online survey Regarding the objectivity of online visual landscape assessment, we have to distinguish between the view of the individual and the view of a group or population. PLACHTER et al. (2002, p. 364) define methods of landscape assessment as objective when they produce results that are independent of the evaluator. On an individual level, preferences, previous knowledge, experiences, expectations etc. play an important role for scenic quality estimation. On a group level, visual landscape assessment is nearly independent of the factors used for group formation, as the results of the online survey show: sex, nationality, request for information on the results of the study, participation in the raffle, school graduation, professional occupation with visual landscape assessment, screen resolution, colour depth, operating system, browser, type of input instrument, sequence of photographs, response time and web site linking/referring to the internet survey 1 have either insignificant (p > 0,05) or irrelevant (ε² < 0,04) effects on the evaluation results. The independence of the evaluation results of the type of input instrument (graphical rating scale, text field) corresponds with the results of HUNZIKER & KIENAST (1999) who found out that landscape preference values are independent of the evaluation method (pair comparison/direct scoring). 1 The referring web site can be used in the so-called multiple site entry technique (REIPS, 2002) to calculate an estimate of biasing potential through self-selection of the participants.

6 6 Michael Roth With the other factors (age, geographical origin within Germany, importance of nature and environment for the personal life, frequency of outdoor trips) the statement mentioned above (no significant or no relevant effect) is true for 21 to 24 of the investigated 25 criteria. The strongest significant influence explains about 6,5 % of the variance of one criterion (ε² = 0,065). The effect of the geographical origin within Germany hasn t been considered problematic, because the sample of respondents is representatively distributed throughout the 16 German federal states (Kendalls τ b = 0,669, p < 0,005). 3.5 Reliability of the online survey The reliability of the online survey was tested with 15 participants who filled in the questionnaire for a second time after a certain span of time (few days up to nearly two months). These people produced 84 duplicate photograph evaluations (25 criteria each). Reliability was computed both on an individual and on a group level. Spearman s rho (rank correlation) and Pearson s r (absolute correlation) were used as reliability measures and produced similar results. Figure 3 shows the reliability values for each criterion on individual and group level. For all criteria except confused and typical, satisfying results for reliability have been achieved (p < 0,01). Fig. 3: Results of the reliability test (rank correlation for each criterion on first and second assessment). This figure also shows the 25 criteria used for visual landscape assessment.

7 Online Visual Landscape Assessment Using Internet Survey Techniques Validity of the online survey The validity of the online survey and its results was tested by comparing (correlating) the results of the online study with the ones of the on-site survey and the colour print-based survey. Again, Spearman s rho and Pearson s r were used and produced very similar results. Validity (Pearson s r > 0,4, according to the thresholds given by BORTZ & DÖRING (2003, p. 201)) was achieved for 14 of the 25 criteria, high validity (Pearson s r > 0,6) for 11 amongst those (cf. table 1). Through cluster analysis, these criteria could be grouped into experience dimensions (cf. Bauer et al., 1979) according to the categories mentioned in the German Federal Nature Conservation Act in connection with scenic qualities. Those are: variety ( diversified, variform ) and beauty ( aesthetic, beautiful, total aesthetic value ). The potential criteria to record the peculiarity of landscape ( typical, characteristic, peculiar ) have not proved to be valid or must be doubted because of the ambiguous meaning (positive and negative) of the criteria peculiar(ity) in German and English language and the lack of information needed to assess the characteristic (geographical and historical background knowledge on the site presented on the digital photograph). The dimension of visual naturalness, often used in scenic quality estimation methods instead of beauty (e.g. ADAM et al., 1986), could be validly recorded through the use of the criteria natural and original. Tab. 1: Results of the validity test showing the correlation of group means for each photo/landscape (*** p < 0,001, ** p < 0,01, * p < 0,05) criterion online survey on-site survey correlation (Pearson s r) online survey photo-based survey diversified 0,793*** 0,888*** aesthetic 0,601* 0,681** snug 0,723** 0,691** beautiful 0,665** 0,755*** generic 0,499* 0,566* graceful 0,625** 0,670** variform 0,797*** 0,882*** peculiar 0,593* 0,558* wild 0,644** 0,720** depressing 0,838*** 0,877*** natural 0,683** 0,695** monotonous 0,696** 0,767*** original 0,502* 0,652** total aesthetic value 0,617** 0,707**

8 8 Michael Roth 4 Conclusions and Outlook The Internet survey has proved to be a cost-efficient, objective (on group level), reliable and valid instrument to gather data on landscape perception and visual landscape assessment. Results of such a survey have a high generalizability because of the independence of most demographic, methodological and technical factors and the ease of drawing a representative sample regarding the geographic distribution of participants. For those reasons, online scenic quality evaluation can be used to acquire broadly empirically based assessments of scenic quality needed for planning purposes and environmental impact assessment. This democratic legitimation (and validation) of visual landscape assessment can help to give visual aspects a higher weight in environmental planning. Landscape assessment experts are not made dispensable by the method presented, because it needs to be professionally prepared, conducted and interpreted. The evaluation of the peculiarity of nature and landscapes can according to the results of this study not be assessed by the general public through Internet surveys, because a certain level of expert and background knowledge is needed for this purpose. Furthermore, online visual quality estimation delivers data useable for fundamental research as well as the development, improvement and validation of non-empirical scenic beauty estimation methods, especially GIS-based procedures (e.g. Bishop & Hulse, 1994; Hunziker & Kienast, 1999; Augenstein, 2002; Roth, 2002). Up-to-date standards and methods of Internet-based experimenting were applied. The research taken enhances past landscape preference studies using the Internet (e.g. Bishop, 1997; Wherrett, 1999), because not only one aspect (colour difference or overall landscape preference), but 25 criteria representing different experience and perception dimensions were investigated and the results of the online survey were validated by using the ones of a recent on-site study. Further research should be done to examine the effect of other landscape representations in the Internet (e.g. 360 panoramic images, video, visual landscape simulations, 3D models or interactive presentations) and cross-cultural comparisons of online visual landscape assessment. 5 References Adam, K., Nohl, W. & Valentin, W. (1986): Bewertungsgrundlagen für Kompensationsmaßnahmen bei Eingriffen in die Landschaft. Düsseldorf. ADM Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.v. (Ed.) (2000): Richtlinie für Online-Befragungen. Frankfurt am Main. ADM Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.v. (Ed.) (2001a): Standards zur Qualitätssicherung für Online-Befragungen. Frankfurt am Main. ADM Arbeitskreis Deutscher Mark- und Sozialforschungsinstitute e.v. (Ed.) (2001b): Checklist for Clients Commissioning Online Surveys. Frankfurt am Main.

9 Online Visual Landscape Assessment Using Internet Survey Techniques 9 Augenstein, I. (2002): Zur Berücksichtigung des landschaftsästhetischen Potentials in der planerischen Umweltvorsorge: Entwicklung eines GIS-gestützten Verfahrens am Beispiel des Regierungsbezirkes Dessau. Doctoral thesis, Rostock University, Faculty of Agricultural and Environmental Sciences. Bauer, F., Franke, J. & Gätschenberger, K. (1979): Flurbereinigung und Erholungslandschaft. Empirische Studie zur Wirkung der Flurbereinigung auf den Erholungswert einer Landschaft. Bundesminister für Ernährung, Landwirtschaft und Forsten (Ed.): Schriftenreihe B: Flurbereinigung, Heft 68. Landwirtschaftsverlag, Münster-Hiltrup. Bishop, I.D. (1997): Testing perceived landscape colour difference using the Internet. Landscape and Urban Planning 37 (3-4): Bishop, I.D. & Hulse, D.W. (1994): Prediction of scenic beauty using mapped data and geographic information systems. Landscape and Urban Planning 30 (1-2): Bortz, J. (1999): Statistik für Sozialwissenschaftler. 5th edition. Springer, Berlin. Bortz, J. & Döring, N. (2003): Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler. 3rd edition. Springer, Berlin. Breuer, W. (2001): Ausgleichs- und Ersatzmaßnahmen für Beeinträchtigungen des Landschaftsbildes. Vorschläge für Maßnahmen bei Errichtung von Windkraftanlagen. Naturschutz und Landschaftsplanung 33 (8): Cohen, J. (1988): Statistical Power Analysis for the Behavioral Sciences. 2nd edition. Erlbaum, Hillsdale, NY. Fischer-Hüftle, P. (1997): Vielfalt, Eigenart und Schönheit der Landschaft aus der Sicht eines Juristen. Natur und Landschaft 72 (5): Gassner, E. (1989): Zum Recht des Landschaftsbildes. Natur und Recht 11 (2): Gruehn, D. (2001): Einschätzung der Bedeutung von Landschaftselementen für das Landschaftserleben. In: Paar, P. & Stachow, U. (Eds.): Visuelle Ressourcen - Übersehene ästhetische Komponenten in der Landschaftsforschung und entwicklung. ZALF-Bericht 44. Zentrum für Agrarlandschafts- und Landnutzungsforschung e.v., Müncheberg. Gruehn, D., Roth, M. & Kenneweg, H. (2003, unpublished): F&E-Studie "Entwicklung eines Ansatzes zur Einschätzung der Bedeutung von Landschaftselementen für das Landschaftserleben als Grundlage für die Beurteilung des Landschaftsbildes". Abschlussbericht i.a. des Sächsischen Landesamtes für Umwelt und Geologie. TU Berlin. Hehl-Lange, S. & Lange, E. (1992): The insufficient consideration of visual aspects in environmental planning. Association of European Schools of Planning, VI. AESOP- Congress, Stockholm. Hunziker, M. & Kienast, F. (1999): Potential impacts of changing agricultural activities on scenic beauty - a prototypical technique for automated rapid assessment. Landscape Ecology 14 (2): Jessel, B. & Fischer-Hüftle, P. (2003): Bewältigung von Eingriffen durch Verkehrsvorhaben in das Landschaftsbild. Naturschutz und Landschaftsplanung 35 (12): Köhler, B. & Preiss, A. (2000): Erfassung und Bewertung des Landschaftsbildes. Grundlagen und Methoden zur Bearbeitung des Schutzguts "Vielfalt, Eigenart und Schönheit von Natur und Landschaft" in der Planung. Informationsdienst Naturschutz Niedersachsen 20 (1): 1-60.

10 10 Michael Roth Nohl, W. (2001): Landschaftsplanung. Ästhetische und rekreative Aspekte. Konzepte, Begründungen und Verfahrensweisen auf der Ebene des Landschaftsplans. Berlin, Patzer-Verlag. Plachter, H., Bernotat, D., Müssner, R. & Riecken, U. (2002): Entwicklung und Festlegung von Methodenstandards im Naturschutz. Ergebnisse einer Pilotstudie. Bundesamt für Naturschutz (Ed.): Schriftenreihe für Landschaftspflege und Naturschutz 70. Landwirtschaftsverlag, Münster-Hiltrup. Reips, U.-D. (2002): Standards for Internet-Based Experimenting. Experimental Psychology 49 (4): Roth, M. (2002): Möglichkeiten des Einsatzes geographischer Informationssysteme zur Analyse, Bewertung und Darstellung des Landschaftsbildes. Natur und Landschaft 77 (4): Von Haaren, Chr., Knickrehm, B., Kunze, K. & Redslob, M. (2002).: Interaktiver Landschaftsplan. Verbesserungsmöglichkeiten für die Akzeptanz und Umsetzung von Landschaftsplänen. Bundesamt für Naturschutz (Ed.): Angewandte Landschaftsökologie 43. Landwirtschaftsverlag, Münster-Hiltrup. Wherrett, J.R. (1999): Issues in using the Internet as a medium for landscape preference research. Landscape and Urban Planning 45 (4): Wöbse, H.H. (2002): Landschaftsästhetik. Ulmer Verlag, Stuttgart.