Recent Advances in Electrical and Computer Engineering

Transcription

1 An Evaluation of the Quality of the B2C Sites in Saudi Arabia Lilac A. E. Al-Safadi Department of Information Technology College of Computer & Information Sciences King Saud University Abstract: - Over the past decade, the Kingdom of Saudi Arabia has taken number of steps towards developing its Information Technology sector. Both Business-to-Consumer (B2C) and the Business-to-Business (B2B) e- commerce initiatives were introduced throughout the country. However, no much attention was given to the quality of these developed systems. Our previous work proposed a Quality Evaluation Model (QEM) that can help developers design and produce e-commerce systems of high quality targeted at Saudi end users. In this paper, we present the results of a comprehensive assessment and evaluation of the characteristics and attributes of the top ten B2C Saudi sites using the developed QEM. The main goal of this work is to show how far Saudi B2C e-commerce sites meet the quality characteristic like functionality, usability, reliability, efficiency from the user point of view. Results show that most Saudi companies use e-commerce to present their products or branding through the web. A significant work still needs to be undertaken in order to meet the international best practices of B2C sites. The evaluation identifies the key quality areas that can be adopted by Saudi B2C site designers to lead to the situation improving significantly. Key-Words: - E-commerce; B2C; Quality Evaluation; Evaluation Model; Quality Engineering; Saudi Arabia. 1 Introduction Electronic Commerce (e-commerce) in the Kingdom of Saudi Arabia is still in its early stages. While 30% of businesses and 13% of government organizations buy online, only 8% of businesses sell online [1]. Businesses (B2B selling) are the most common target group for companies selling online. Consumer selling (B2C) and government selling (B2G) also rate highly, with at least two-thirds of the companies selling to these entities. In terms of target group revenue share, B2B is estimated at 54%, B2C at 28%, and B2G at 18% [1]. Saudi Arabia is the biggest market in the region for companies selling online. Developing systematic and disciplined methods and tools for evaluating these sites is essential. These methods should identify which attributes and characteristics should be used to obtain meaningful indicators for assuring specific evaluation goals from a specific point of view. The evaluation of local and international ecommerce sites helps Saudi businesses that are trying to maintain quality control of sites. Up until now, there have been few attempts to assess Saudi e-commerce sites. Al-Otaibi and Al- Zahrani [2] have analysed the practices and characteristics of Saudi e-commerce websites in order to measure the tendencies of Saudi organizations towards e-commerce. Eid [3] identified the factors that influence the extent to which Saudi consumers trust, are satisfied with, and are loyal towards B2C e-commerce. These studies do not present a complete quality model and do not identify the quality criteria and their levels of importance from online users perspectives. Alsafadi and Garcia [4] proposed a quality evaluation model (QEM) for evaluating B2C e- commerce applications. In this proposed model, the views of users of the Saudi market were considered in indentifying which characteristics and attributes influence the quality of B2C applications. QEM is a quantitative method for evaluating the quality of B2C sites. The method is based on ISO quality standards. The objective of QEM is to answer an important question: which of the site functions is most important for the success of a B2C site? Figure 1 shows a screenshot of a Saudi store page where some attributes used in QEM are highlighted. Table 1 below shows the criteria used for evaluating the quality of B2C e-commerce applications and the weight of each criterion. s were assigned to the quality factors by end users and experts in the field in Saudi Arabia, using an evaluation questionnaire. The table will be explained in more details in next section. The QEM tool supports the administration of site evaluation. The tool allows entering a score for the characteristics of the B2C application and automatically aggregates the elements in order to calculate a global quality indicator for each site. In this paper, we will assess and evaluate the characteristics and attributes of the top ten B2C Saudi sites using the developed QEM. The main goal of this work is to show how far Saudi B2C e- ISBN:

2 commerce sites meet with quality measures like functionality, usability, reliability and efficiency from the user s point of view, and wish to help in discovering absent features or poorly implemented features. Accordingly, recommendations for improvement are provided The rest of this paper proceeds as follows. In Section 2, we describe the research methodology and the evaluation process to which QEM adheres. In Section 3, we analyze the evaluation results. Finally, some concluding remarks and recommendations are drawn in section 4. With-Sale promotion Appetizer promotion Fig. 1. A screenshot of a Saudi store where some attributes are highlighted Table 1. Quality requirements tree for evaluation of B2C sites and associated weight for each requirement [4] Functionality Quality Factors 1. Searching and retrieving Website search mechanisms quick search advanced search Average level of retrieve customization Navigation and Browsing Navigability orientation average of links per page Navigational control objects Presentation permanence and stability of controls presentation permanence stability of sub-sites controls Level of scrolling vertical scrolling horizontal scrolling Navigational prediction link title with explanatory help link comment or description Quick browse Product Product description Textual description contents and structure product image product evaluation and comparison product rating related products recommendation catalog download facility product personalization Product ranking top products best seller products Purchase purchase mode Online shopping basket shopping basket continues buying feedback edit/recalculate feature Payment provider full integration of payment provider forms of payment Sadad Credit card Bank transfer Paypal Google Checkout Offline printable check-out form fax/telephone/ purchase quick purchase check-out features check-out security canceling feedback purchase policies Purchase cancellation policy Return policy Shipping and handling policies Payment policy Recent purchase (gift service) multiple delivery options Customer E-subscriptions account facility Account availability Account security Account setting wish list Promotion with-sale appetizer promotion (e.g. contests, miles, etc.) Order management order history order tracking Services Discussion forums Surveys/Polls Newsletter 0.34 Usability Factors 1. Global Site Understandability table of contents sitemap alphabetical/subject index text labels picture clickable images guided tour for first time visitors Online Feedback and Help Features Global Help Help Search Help Purchase Help Check-Out Help Contact details Directory Telephone/Fax Directory Post Mail List FAQ Feature Real-time customer service Comments/Suggestions Interface and Aesthetic Features main control Presentation permanence and stability of main controls Direct controls Main control Search control Browse control Account control Shopping basket control Indirect controls permanence control's stability Style issues Link color style Aesthetic designs Miscellaneous Features Foreign Language Support What's New Feature 0.49 Reliability Factors 1. Link Errors Broken links Invalid links Unimplemented links 0.33 ISBN:

3 2. Miscellaneous Errors and Drawbacks different browsers browser independent dead-end web nodes destination nodes under construction Accuracy and Relevance of Information testimonials references or links physical address of the office 0.36 Efficiency Factors 1. Performance quick pages quick checkout and payment Accessibility Information accessibility text-only version readability by deactivating browser image feature image title global readability browsers' version even those which do not support frames mobile device accessibility Research Methodology In this study, a dataset of Saudi B2C e-commerce site is identified. This dataset was selected from the top Saudi e-commerce sites listed in Al-Otaibi and Al-Zahrani [2]. For each site in the dataset, referred to as a case in this paper, we have evaluated the main page as well as the sub pages. The research we have conducted was performed in two steps. At the first step, the dataset were screened and filtered. Each URL was processed by automatically retrieving a number of pages, starting from the home URL and then following hyperlinks within each domain. The results for each site were stored for later analysis. At this stage a number of sites were filtered out when sufficient data was not obtained for reliable investigation. This situation can arise for a variety of reasons: it might be no longer available, it might not be B2C, it might not be Saudi, it might require registration/login; or it might be intrinsically a very small site. At the next step, the first ten ranked sites in the dataset, listed in table 2 below, were evaluated using the QEM. Table 2. Cases used in our experiment Site No. name URL S1 Jarir Bookstore S2 Arabian Advanced System S3 Saudia Airline S4 Saudi American Bank S5 National Computer Systems Co. www. Nashirnet.net S6 InRiyadh www. InRiyadh.com S7 Sahara Net S8 Mstml Online store www. Mstml.com S9 Al-jazirah newspaper S10 Asharaq Al-Awsat The QEM focuses on the following quality characteristic requirements: functionality, usability, reliability and efficiency with the aim of assessing the level of accomplishment of these characteristics. Each quality characteristic is decomposed in multiple levels of sub-characteristics, and in turn, a sub-characteristic was refined in a set of measurable attributes, as shown in table 1. QEM aggregates the measured attribute values. The evaluation process in the methodology used here is adopted from Olsina [5] and follows the following steps: 1) Evaluation of elementary criterion; 2) Aggregation of elementary criterion into global criterion; and 3) Results analysis and recommendations. In the evaluation of elementary criteria, each attribute was assessed by assigning a numerical value which can be interpreted as the degree of requirement satisfied, where 0% denotes a totally unsatisfactory situation, whereas 100% represents a fully satisfactory situation. Some criteria were assigned binary values where 0 denotes existing whereas 1 represents not existing. Each elementary criterion was scored by multiplying the attribute weight listed in table 1 by the degree of satisfaction. The aggregation of elementary preferences yields to the global quality preference. The global quality preference represents the global degree of satisfaction of all involved requirements. Table 3 below shows the assessment results obtained for the top 10 Saudi B2C sites, denoted as S1 S10, categorized by the main quality characteristic requirements: functionality, usability, reliability and efficiency. Table 3. The global quality preference for the top 10 Saudi B2C sites W S1 S2 S 3 S4 S5 S6 S7 S8 S9 S10 A Functionality Usability Reliability Efficiency Total (global quality preference) The first numerical column represents the importance of characteristics from the customer s perspective, adopted from QEM. In addition, the table compares the score of each case with the score given to one of the most popular international B2C sites which is Amazon; denoted as column A in the table. The scores were assigned to Amazon as part of an experiment conducted to test QEM [4]. To ease the interpretation of preferences, results were aggregated into three categories or ISBN:

4 acceptability levels, namely: unsatisfactory (from 0 to 40%), marginal (from 40 to 60%), and satisfactory (from 60 to 100%). The highest possible score for any one B2C site is 1 (100%). The highest ranked B2C Saudi site in the banking industry received a score of 0.7 (70%). The site scored highly on evaluation because of the availability of online transactional functions, history management, and the presence of relevant information such as reference to other sites and physical office information. In addition, the site is clear, easy to navigate, and provides mobile accessibility. 3 Interpretation of results After the first part of the research, we have concluded that many Saudi B2C sites have not yet developed a mature mechanism for transactions; except for the industries of banking, travel and some ICT online services. Saudi B2C sites focus more on informative mechanisms, although the information on the sites was generally up to date. Most Saudi companies use e-commerce to present their products or branding through the web. An analysis and comparison of the results yields to the following: 3.1 Functionality Functionality describes the set of attributes that satisfy the stated or implied needs of customers. The QEM comes with a comprehensive set of functions. Table 1 shows the importance of each function to users. The Functionality high-level characteristic is composed in sub-factors such as Searching and Retrieving, Navigation and Browsing, Product, Purchase, Customer, Promotion, Order Management and Services. Table 4 shows the results of the functionality quality preference of the cases. Table 4. Results of Functionality quality preferences for top 10 Saudi B2C sites Functionality Factor Searching & retrieving Navigation & Browsing Product Purchase Customer Promotion Order management Services As per QEM, the overall weight of the functionality criterion which reflects the importance of that criterion to Saudi users is 26%. The results of the experiment show that among the top 10 ranked Saudi B2C sites, the highest functionality score was 15%. This was received by a site in the banking industry. Below is an analysis of factors that yield to the overall evaluation. Searching and Retrieving Functions - Searching is available in 90% of cases, while advanced searching is only available in 60% of cases; and - Only 30% of sites enable customization of retrieved results Navigation and Browsing Functions - 70% of cases received a satisfactory level of acceptance in terms of navigation and browsing issues; - Almost all cases pay attention to the permanence and stability of navigation and browsing controls; - 90% of cases provide quick browse functionality; and - No case provides navigational prediction functions. These functions are represented by supporting link titles with explanatory help, comments or descriptions. The QEM shows a 50% score of the importance for both functions. Linking titles with explanatory help ranked first in the list of most desirable navigation and browsing functions in [4]. Products The product criteria focuses on product description, product evaluation and comparison, product rating, product ranking, products recommendation, product personalization and catalog download facility. In general, cases scored very poorly in this respect as the only criteria they address is the product description. The details follow: - In terms of the content of online material, almost all the cases have made extensive effort on placing information online for public access in terms of textual description and product image; - No case provides product evaluation and comparison, product rating, product ranking, products recommendation, or product personalization; and - 10% of cases provide catalogue download facility Purchase Functions Purchase features focus on two main subcharacteristics: Purchase Mode and Purchase Policies. The majority of the Saudi B2C sites are ISBN:

5 primarily informative and do not have the functionality for online transactions. Online transactions are limited to three industries; banking, travel and ICT online services. Regarding the Purchase Mode, customers showed more preference for online over offline modalities with an increase in security confidence. The details are given below: - 50% of cases provide online purchase modes; - 30% of cases use online forms of payment systems and 40% offer offline forms of payments; - Credit cards and SADAD payment systems ( are the most common payment methods, with 30% of the sites accepting these. Bank transfers are next, with acceptance by 20% of the sites. Other online payment systems such as PayPal and Google checkout are not accepted by any of the sites; - As for purchasing policies, 30% provide purchase cancellation policies, 20% provide a return policy, none provide shipping and handling policies, 50% provide a payment policy, and 10% provide a recent purchase policy. None of them provide delivery options. Customer Functions The customer functions focus on the availability of electronic subscription, account management and wish lists. 70% of the sites offer online subscription to the site, 50% offer account management facilities, and none offer wish list management. Promotion Functions The QEM focuses on two types of promotion, withsale and appetizer promotions (contests, miles, etc.). Among the cases, 40% of sites offer appetizer promotion, and 20% offer with-sale promotions, while customers scored the importance of the former type of promotion as 0.53 and the later as Order Management Functions On these sites, Order Management Functions are limited to order history. No site provides order tracking functions as none provides delivery functions. Services Functions Site services include discussion forums, surveys and polls and newsletters. 30% of cases offer online discussion forums, and 20% offer surveys and newsletters. In general, customers gave a 10% score of importance to service functions and these services were ranked as being just above average. 3.2 Usability Usability refers to the ease of use for the e- commerce site. Usability is an extremely important factor in e-commerce since it affects the number of visitors and stickiness (duration) of their stay at the site. Usability affects the visitor acquisition which is measured by the number of visitors who show interest in the site by registering or subscribing. These measurements are an important direct source of revenue for online systems. The Usability highlevel characteristic is composed of sub-factors such as Global Site Understandability, Feedback and Help Features, Interface and Aesthetic Features, and Miscellaneous Features, as shows in table 5 below. Table 5. Results of Usability quality preferences for the top 10 Saudi B2C sites Usability Factors Global Site Understandability Online Feedback and Help Features Interface & Aesthetic Features Miscellaneous Features As per QEM, the overall weight of the usability criterion which reflects the importance of that criterion to Saudi users is 26%. The results of the experiment show that among the top ten ranked Saudi B2C sites, the highest usability score was 12%. This score was received by a site in the banking industry and another online newspaper. It represents 53% of customers' expectations. Below is an analysis of factors that yield to overall evaluation. Global Site Understanding This factor contains the attributes that may help visitors to obtain a quick understanding of the structure and the content of the site. This includes a table of contents, a site map, an alphabetical/subject index, text labels, pictures and images, and a guided tour. B2C users gave a score of 25% for the importance of this factor. The highest score among our test set was 17%, which was received by an online market. The details follow: - No site has a table of contents; - 40% of sites have a site map; - No site has an alphabetical or subject index; - 70% of sites have text labels; - 95% of sites have pictures, 40% of these sites provide clickable images; and ISBN:

6 - No site provides a guided tour for first time visitors. Online Feedback and Help Features Feedback and Help Features are other important characteristics of usability which obtained 24% score of importance by B2C Saudi users. These techniques aim to build and strengthening the relationship with customers. The results of our experiment shows that Saudi B2C sites scored poor in this criteria and that the highest score obtained was 9% by the online Saudi market. The details follow: - No site has Global Help on the home page; - No site has search help, purchase help, or Checkout help; - No site offers real-time customer service; - 30% of the sites have FAQ, and a means of receiving visitors comments and suggestions; and - As for the contact details, 90% have an address but none provide a full directory. 80% have their telephone and fax contact, and 70% have mail online. Interface and Aesthetic Features Interface and Aesthetic features focus on the presentation permanence and stability of direct and indirect controls as well as style issues. These features obtained a 25% score of importance by Saudi B2C users. The results of our experiment show that Saudi B2C sites scored poorly in this criteria and that the highest score obtained was 9% by the online Saudi market. The details follow: - 90% of sites have presentation permanence and stability of main control, 45% have search control, 30% have Browse control, 50% have account control and 20% have shopping control. Only 20% of sites provide permanence of their indirect controls; - As for style issues, 80% of sites provide link colour style and almost all sites provide some designs with various aesthetic qualities. The style is uniform in 90% of the sites. Few online catalogues had no aesthetic features but these catalogues were filtered out due to their size and simplicity; - 70% provide English/Arabic content. None have support for more than 2 languages; and - 30% provide a What s New feature 3.3 Reliability The characteristic of reliability defines the capability of the system to maintain its service provision. This has a direct effect on customer satisfaction and thus on online purchase decisions. Table 6. Results of Reliability quality preferences for top 10 B2C Saudi sites Reliability Factors Link Errors Miscellaneous Errors & Drawbacks Accuracy & Relevance of Information The evaluation quality methodology shows that customers give almost equal importance to information that boosts the credibility of a site; such as testimonials, references or links and physical address of office. On the other hand, our experiment found that 30% of cases have links and reference to another site, and that physical contact information is quite prevalent and available in 60% of cases. None of the sites provides testimonials. - Most of the sites (84%) have received high scores in the evaluation of the reliability; and - In 7% cases we found link errors and in 15% deficiencies such as destination nodes under construction. 3.4 Efficiency Efficiency is concerned with the system resources used to provide the required functionality, such as the amount of disk space, memory, and network, which affects the overall level of performance. Efficiency obtained a 23% score of importance by customers. A system can be called efficient if the user can access the relevant website promptly and from anywhere. Therefore, the Efficiency high-level characteristic is composed of two main sub-factors; Performance and Accessibility as shown in the table below: Table 7. Results of Efficiency quality preferences for top 10 B2C Saudi sites Efficiency Performance Accessibility Considering the Efficiency characteristic of Saudi B2C sites, the highest score in our experiment cases scored 21% and was obtained by the bank, while the lowest score was 11% and was obtained by a bookstore and online content provider. However, all the scores are average and above. The efficiency criteria measure the performance and ISBN:

7 accessibility of sites. Efficiency aspects in cases are analysed below: - In general all cases support quick pages, and the transitionary sites support quick check out methods; - The support for text-only version is not present on any of the sites; - Almost 90% of the sites omit alternative text for images. Where provided, it was often found to be inappropriate or ineffective; and - Although all the sites are accessible via smart phones, only 30% of cases support a special version for mobile phones. While over 50% of Saudi population aged 16 years and below and currently have access to the Internet from any location, only 30% of the sites support mobile accessibility. It is important for organizations to understand their target audiences information and service needs. 3.5 The Global Evaluation The final necessary process is aggregating the elementary scores to obtain a global quality indicator for each site in our experimental cases. The score is computed as follows: Global Indicator = SUM (Elementary Indicator ) Elementary Indicator = Component * Degree of Satisfaction. Regarding the aggregation process, it follows the hierarchical structure of the requirement tree, from bottom to top. The global preference represents the global degree of satisfaction of all the requirements involved. The global preference of our cases is listed in table Conclusion and Discussion Al Safadi and Garcia [4] proposed QEM, a userdriven evaluation methodology and a tool to assess and evaluate of e-commerce B2C systems. The tool works as a guideline for designers to follow in order to increase the numbers of visitors to sites and therefore generate revenue. The existence of the tool has shed light on important characteristics and attributes an e-commerce designer should focus on. Although the methodology is objective rather than subjective, i.e. quantitative rather than qualitative, the subjective components cannot be avoided completely In this paper we have explained the extent to which the Saudi B2C sites meet the users requirement. By using QEM we have discovered absent attributes, absent sub-characteristics, or requirements poorly implemented, and haven accordingly provided recommendations for improvements. In this study we have observed that the existence of many sub-characteristics and attributes are related to the different application domains. The Banking and Travel industry focuses on the informational and transactional functions of their sites. The retail industry is far from meeting the international quality level when compared to sites like Amazon, or the local quality level when compared to the importance of weights assigned by Saudi users. Urgent changes needed would include focusing on product related functions offering product evaluation and comparisons, product rating, related product recommendations, product ranking and product personalization. In general, Saudi sites lack intelligence functionalities (prediction, adaption, and optimization). Another area would be offering transactional functionalities and order management. Immediate actions that can be taken by site designers involve adding online help, site maps, tables of content, and a site tour for first visits. References: [1] IT Report 2010 On the Internet Ecosystem in Saudi Arabia, The Communications and Information Technology Commission (CITC). [2] Al-Otaibi M. B.; Al-Zahrani, R.M.: E- commerce adoption in Saudi Arabia: an evaluation of commercial organizations websites. Int. Conf. Information Technology in Asia (CITA) (2003) [3] Eid, M. I.: Determinants of e-commerce customer satisfaction, trust, and loyalty in Saudi Arabia. J. Electron. Commer. Res. (1 February 2011) [4] Al Safadi, L. and Garcia, R. ISO9126 Based Quality Model for Evaluating B2C E- Commerce Applications A Saudi Market Perspective. Under Review [5] Olsina, L. Measuring Web application quality with WebQEM. IEEE Multimedia, Volume: 9, Issue: 4, ISBN: