This article appeared in a journal published by Elsevier. The attached

This article appeared in a journal published by Elsevier. The attached

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and educat...

449KB Sizes 0 Downloads 10 Views

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright

Author's personal copy

Expert Systems with Applications 37 (2010) 8685–8693

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

A fuzzy-AHP-based technique for the decision of design feature selection in Massively Multiplayer Online Role-Playing Game development Ying-Fu Lo a,*, Ming-Hui Wen b a b

Department of Marketing and Logistics, China University of Technology, Hsin-Chu 303, Taiwan User Research Group, heroxhero.com Inc., Hsin-Chu City 300, Taiwan

a r t i c l e

i n f o

Keywords: Decision making Fuzzy membership Analytic Hierarchy Process Massively Multiplayer Online Role-Playing Games Human factors

a b s t r a c t The Massively Multiplayer Online Role-Playing Game (MMORPG) industry is growing rapidly. It is very important to understand how to design a successful MMORPG that can satisfy its customers. A MMORPG is composed of many design components; therefore, the game developers cannot perfect all of the design factors due to the constraint of development resources and the time-to-market pressure of the product. The purpose of this study is to help game developers to make the trade-off decision of where to invest their limited resources to implement critical design factors. We proposed a fuzzy-AHP-based technique and constructed a hierarchical system structure for MMORPG design, which includes two systems, nine design components and 36 design features for the design factor trade-off problem. A questionnaire with a pair-wise comparison data input format was employed to gather users’ assessments of the relative importance of the design criteria. Two hundred and twenty-one Taiwanese MMORPG players participated in this online survey. Our results identified five critical design components that contributed to 66% of users’ sense of important MMORPG design features. The study also recommended the top 10 important design features, which can fulfill about 40% of users’ expectations in implementing the critical components. Specific design guidelines were also suggested for MMORPG implementation. Ó 2010 Elsevier Ltd. All rights reserved.

1. Introduction Massively Multiplayer Online Role-Playing Games (MMORPGs) are modern games that provide a persistent 3D virtual world to support thousands of people playing together on the Internet (Ducheneaut, Yee, Nickell, & Moore, 2006). According to a marketing research report by DFC-Intelligence, the online game industry has created a big market share in today’s entertainment industry (DFC-Intelligence, 2006). This genre of games, such as World of Warcraft, Lineage and EverQuest, has spawned a multi-billion dollar market and has attracted over 16 million subscribers worldwide (Woodcock, 2008). Therefore, the MMORPG industry is no longer a youth-oriented business but creates a digital entertainment lifestyle popular among youth and adults alike. However, although the MMORPG industry is growing rapidly, many game companies struggle to become profitable because there are so many new games introduced to the market each year. Researchers have stated that only a few games can survive in the competitive game market (Hsu, Lee, & Wu, 2005). Therefore, game companies cannot delay in finding a good product design strategy for increasing their competitive advantage and enabling them to * Corresponding author. E-mail address: [email protected] (Y.-F. Lo). 0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.06.059

survive in the game market (Fitzsimmons, Kouvelis, & Mallick, 1991). MMORPGs are developed by groups of specialists in the fields of art, programming and design. These teams design various different systems such as animation, sound and music, story, databases and servers and programming (Kelly, 2004). In addition, each of these systems has several design features, including several implementation levels as well as different costs (Hsu, Lee, & Wu, 2006). In general, game developers do not perfectly implement all of the features of the games, due to the constraints of development resources (e.g., human resources and money) and the time-to-market pressure of the product. Based on this research background, the game developers do not consider the importance of these design features on an equal footing. They must identify the relative importance of the various game design features in order to allocate the resources of product development. However, the game developers cannot rank the importance of these design features only according to their previous experience and personal expectations, as earlier research on the development of MMORPGs has indicated that the perceived importance of game features differed significantly between the developers and the users (Choi, Kim, & Kim, 1999). Some design features that the game developers consider to be important may not be considered important by the users. Therefore, according to the

Author's personal copy

8686

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

philosophy of product development, designers should follow a user-oriented approach. This is, designers must understand the users’ basic needs as well as their initial expectations of the product (Norman, 2002). Studies have investigated the topic of game design from a useroriented approach (Choi et al., 1999; Malone, 1981; Yee, 2006). In MMORPGs, Choi et al. (1999) classified the game design factors into users’ perceived fun and cognitive fun. They proposed a hierarchical structure to model users’ overall experiences of fun and then suggested several design features to implement the concept of fun into the games. Another study was conducted by Yee (2006), who classified the motivational factors of online game players into three main components (achievement, social and immersion) in order to understand the reasons influencing players to stay in the game world longer. However, the studies conducted by Choi and Yee discussed the game design from the perspective of the users’ psychological needs, focusing on the users’ levels of sensation and cognition, such as challenge, fantasy and achievement. Their results shed light on the nature of users’ expectations of game play, but they did not provide a link between these psychological factors and the game system’s physical features. Thus, it may not be easy for game developers to use their findings to realize and implement these psychological factors in the games. In order to bridge the gap between users’ psychological needs and game system features, some heuristics design guidelines have already been suggested in educational games (Malone, 1981), action games (Fabricatore, Nussbaum, & Rosas, 2002), strategic games (Hsu, Wen, & Wu, 2007) and online games (Choi & Kim, 2004). These studies suggested useful design guidelines to link the users’ psychological needs to the system requirements and also proposed possible design features that can be used to fulfill users’ psychological needs in the game. However, these studies were focused only on defining the design features to fulfill the users’ psychological needs in the game. They did not consider the relative importance among these game features. Therefore, their results help the designers to identify relative design features for game development, but they fail to support game developers in making tradeoff decisions among these features or in allocating their available resources for game design. A study that can support game developers’ trade-off decisions among design features has been introduced in action game design by Hsu et al. (2005). They identified 39 design features of action games and applied neural network techniques to determine the relative importance of these design features in appealing to game buyers. The top 10 design features identified in their study can fulfill about 50% of users’ expectations. Their results shed light on the relative importance of design features in action game development. With an understanding of this information, game developers can plan an efficient design strategy to pay more attention and invest more resources in implementing those critical design features. However, their work was concentrated on the design of action games, which is very different from that of online games because actions games are more focused on demonstrating users’ perceptual and motor skills and only allow one or two users to play together. Thus, their findings cannot be directly applied to MMORPG design. The purpose of this study was to identify the relative importance of MMORPG design features with a user-oriented assessment approach. In order to provide an overall view of the game system, the design features that we discuss in this study include both game content system (e.g., sound and light effects) as well as value-added system (e.g., customer services and virtual item shops). We applied a hybrid methodology that combined fuzzy logic techniques and the Analytic Hierarchy Process (AHP) to provide a systematic tool to analyze users’ sense of importance of the design features. The results of this study determined the relative importance of online

game design features, which can be applied as a design strategy by game developers making trade-offs between the design features during the product development process. The AHP technique helps decision makers to decompose a multi-criteria problem into a hierarchical model. This structuralized model represents the top-level goal as well as the various sub-elements of the decision problem. In this study, based on the hierarchical model constructed, users can evaluate the relative importance of each game feature by simply using a pair-wise comparison method. Thus, the users’ mental workloads employed in multi-criteria comparisons are reduced, and the results are more precise (Miller, 1956). The study also compared the rated importance scores of each design feature to reveal the relative importance of each design feature (Saaty, 1980). On the other hand, the fuzzy technique, introduced by Zadeh (1965) to emulate a human’s ability to make judgment by using imprecise information, was also applied in this study. The users’ sense of the importance of game design features is determined by referring to their previous gaming experiences at the cognitive and perceptual level, which are difficult to precisely describe or clearly define. A fuzzy set can help us to transfer the rough descriptions of user experience to precise numbers for future data analysis. 2. Systematic structure of online games In this study, we constructed a hierarchical model to represent the systematic structure of MMORPG features by reviewing the relative literature and by analyzing existing games. There are 36 design features identified in this study. Twelve MMORPG players and two human factors specialists who have professional online game skills and experience assisted us in conceptualizing these 36 design features as high level components though the KJ method (Kawakita, 1975; Scupin, 1997). The MMORPG components discussed in this study can be divided into two main systems: the game content system and the value-added system. The game content system refers to the fundamental elements used in constructing an online game, which includes such design features as the character system, environmental setting, sound/light effects, user-task design and game world rules in this study. The value-added system refers to the features/services that can enrich the game world and create extra benefits to the company. The value-added systems here include systems such as a virtual item cash shop, social community system, user interface design and quality of services. The hierarchical structure of MMORPG design in this study is shown in Fig. 1. 2.1. Game content system The game content system discussed in this study included five design components: character system, environmental setting, sound/light effects, user-task design and game world rules. Each design component played a different role in the online games and included several specific design features. The character system plays a mediating role between the user, the game content and the other players. In online games, users can have one or more virtual body through role-playing the character(s). During the game play, users control their characters to accomplish their in-game goals at the personal and social levels (Yee, 2006). For example, users use their character to explore the game world, to fight with monsters and to accomplish in-game quests (Kelly, 2004). In addition, users also use their character to deal with other players in the virtual world, by using a simulated face-to-face metaphor (Moore & Ducheneaut, 2007). We therefore determined the users’ sense of the importance of the character sys-

Author's personal copy

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

Design Objective

MMORPG Game Design

Design Systems

Value-Added Systems

Quality of Services

User Interface Design

Social Community System

Virtual Item Cash Shop

Game World Rules

User-Task Design

Sound/Light Effects

Environmental Setting

Design Features

Character System

Design Components

Game Content System

36 Design Features

Fig. 1. Hierarchical system structure of MMOPRG design.

tem by measuring its five representative design features through the following statements: (1) the design of the character system in this game is differentiated, (2) this game provides various skills for the characters, (3) this game allows me to personalize my character, (4) the design of the character is complete, and (5) the design of the character’s appearance is attractive. Every online game is built on a theme that provides an environment in which users, non-player characters (NPC), monsters and other elements live (Gilbert, 2007). The design feature environmental setting includes, for example, the geographic setting, the space/ time background setting and the storyline setting. A high quality environment design creates an immersive experience for the users, which can increase a user’s sense of engagement in the game world. Research has indicated that players are motivated to explore the gaming environment in order to meet their need for curiosity and the pursuit of fun (Malone & Lepper, 1987). We therefore determined the users’ sense of the importance of the online gaming environmental setting by measuring its four representative design features through the following statements: (1) I feel immersed in the game world, (2) the scale of the game world is unlimited, (3) the scene design is vivid, and (4) the background story is attractive. The design of sound and light effects in online games is focused on enriching the users’ sensory experiences (Malone & Lepper, 1987). Today, the design quality of sound and light effects in online games is as excellent as in movies. Most online games are now built on a high fidelity 3D virtual world and employ world-class symphonic groups to record their music. The audio-visual quality can be one of the most important features in designing a successful game (Norman, 2004). We therefore determined the users’ sense of the importance of the sound/light effects by measuring its three representative design features through the following statements: (1) the design of the animation is vivid, (2) the game music is always suitable for the context, and (3) the game provides high resolution pictures. The design of user tasks in online games, also called ‘quests’ or ‘missions,’ includes both short-term and long-term goals that users need to accomplish by following the storyline. The in-game task design is tightly related to the users’ gaming experience, such as their sense of fun (Malone & Lepper, 1987), achievement (Yee, 2006) and challenge (Hsu et al., 2007). Due to the unbounded nature of online games, the game system should provide a suitable number of task sets and give appropriate difficulty levels for each task, in order to optimize the users’ gaming experience. We therefore determined the users’ sense of the importance of the user tasks by measuring its five representative design features through the following state-

8687

ments: (1) the quests are clearly guided, (2) the system allows multiple users to solve problems together, (3) the rewards of the quests are attractive, (4) the difficulty of task design is suited to my skill level, and (5) new game quests are released regularly. Although the characters’ actions in the online game world are simulated human–computer interactions, users interact with all of the game elements by using their characters. In order to organize the game world, the game world should be operated by a set of predefined rules. Game world rules provide a payoff table to manage the input and output relationships between the users and the other in-game components (Olsen, 2003). For example, the game world rules define the difficulty of living in the game world (e.g., experience-point requirements for progressing the character) and set up the balance of player–player and player–system interactions (e.g., PvP fighting or hunting of a monster). We therefore determined the users’ sense of the importance of the game world rules by measuring its four representative design features through the following statements: (1) the progression of character levels is predictable, (2) the power of character talents is balanced, (3) the game pace is appropriate, and (4) the design of the reward system is fair. 2.2. Value-added system The value-added system discussed in this study included four components: a virtual item cash shop, social community system, user interface design and quality of services. Each component also included a few design features and played different roles in meeting user requirements in online games. As compared to traditional business models that ask users to pay a monthly subscription fee for playing the game (e.g., World of Warcraft and EverQuest), to sell virtual items through the virtual item cash shop is a new business model for creating revenue to the online game companies (van Lent, 2008). In this business model, the game company offers a free game to the users. Users can download a free game package via the Internet without paying any monthly subscription fees. Game publishers sell functional or decorative virtual items that users can buy for cash and use in the game world to provide extra benefits to their avatar. We therefore determined the users’ sense of the importance of the virtual item cash shop by measuring its three representative design features through the following statements: (1) the virtual items provided by the cash shop varied, (2) the virtual items are useful, and (3) the appearance of the virtual items is attractive. A social community is a core factor linking users who have the same goals and interests in the virtual world. Design features that support users’ cybersocial activities include online forums, friendship management systems and player-matching systems. Earlier research has indicated that people who are deeply engaged in a social community will have an increased sense of belonging to the game and will be motivated to play the game longer (Block, 2008; Pisan, 2007). We therefore determined the users’ sense of the importance of the social community by measuring its three representative design features through the following statements: (1) community-related features can well support in-game social activities, (2) social features are easy to use to manage users’ online friendships, and (3) this game provides a real-time message system to support dynamic social interactions. User interface (UI) design is related to the level of usability when the users interact with the game though the user interface. The design features related to the user interface include such features as shortcut bars, macro commands and direct manipulation techniques. Online games use avatar-mediated interaction metaphors to enable the users to make commands to their character and to perform in-game actions such as attacking, running and chatting (Moore & Ducheneaut, 2007). Therefore, a user-friendly interface

Author's personal copy

8688

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

design in the online game will decrease the user’s frustration when interacting with the interface. We therefore determined the users’ sense of the importance of the user interface design by measuring its three representative design features through the following statements: (1) the user interface flow is easy to understand, (2) the function of the user interface is easy to use, (3) the game allows users to manipulate functions by using a graphical user interface (GUI), and (4) the metaphor and online guidance are clear. Online gaming is a business of fun. Game publishers not only earn profits by selling the game software package, but also charge regular subscription fees from the users (van Lent, 2008). That is, online game businesses provide not only a product, but a service as well. Therefore, the quality of services (QOS) in the online gaming industry is very important. The QOS in online games can be divided into two levels: the technical level (e.g., network bandwidth and server stability) and the customer level (e.g., user account maintenance, official website updating and customer service). A high quality of services is expected to increase customer satisfaction (Luo, Liu, Shao, Lu, & Ye, 2006). We therefore determined the users’ sense of the importance of the quality of services by measuring its five representative design features through the following statements: (1) the game server is fast and stable, (2) the subscription fee is acceptable, (3) game systems are upgraded regularly, (4) the official website of this game provides useful information, and (5) the customer service center is friendly and helpful.

3. Methodology 3.1. The fuzzy-AHP approach In order to determine the best design strategy for online game feature selection, the fuzzy-AHP technique was employed to determine the relative importance of the various online game design features in this study. The fuzzy-AHP approach is a hybrid methodology that combines fuzzy set theory and AHP. Fuzzy set theory (Zadeh, 1965) is a mathematical theory designed to model the concepts of vagueness, imprecision and uncertainty in human cognitive processes (Ma, Chen, & Wu, 2007). Fuzzy set theory and fuzzy logic have been widely used in a variety of applications that are controlled by humans and that are difficult to precisely define (Klir, Clair, & Yuan, 1997). In addition, the AHP approach provides a systematic process for determining the relative importance of a set of elements in a multi-criteria decision problem. In AHP, a decision problem is decomposed into a hierarchical structure model that includes several decision components. After that, the relative importance of each decision component can be determined from the users and is then assigned a score calculated by using a pair-wise comparison method (Kurttila, Pesonen, Kangas, & Kajanus, 2000). The conventional AHP method has been widely used for modeling and solving multi-criteria decision-making problems (Dag˘deviren & Yüksel, 2007). However, sometimes the AHP criteria are difficult to clearly define or describe, so it is not easy for the decision makers to make correct judgments between the criteria. To overcome this drawback of AHP and to resolve the vagueness of the AHP criteria, Laarhoven and Pedrycz (1983) proposed a hybrid method, called fuzzy-AHP (FAHP), that combined the benefits of both the fuzzy set theory and AHP methods. The FAHP inputs the

Extremely important

Very strongly important

Strongly important

Moderately important

triangular fuzzy numbers into a pair-wise comparison matrix to support decision makers’ assessments on alternatives with respect to each attribute (Ayag˘, 2005). Specifically, FAHP reinterprets decision makers’ assessments from ‘‘extract values” to ‘‘interval values,” which can better represent the decision alternatives, as compared to the conventional AHP method. Finally, by aggregating the fuzzy utilities of each alternative and comparing their ranks, the relative importance of decision alternatives can be identified. FAHP has been widely used to solve the problem of Multi-Attribute Decision-Making (MADM) in the product design process. It can help product developers to determine the relative importance of design alternatives with limited information (Chen & Hwang, 1992; Kahraman, Cebeci, & Ulakan, 2003). For example, in the product conceptual design process, Kwong and Bai (2002) proposed a fuzzy-AHP approach to determine the importance weights of customer requirements in quality function deployment. In the product development stage, Lee, Lau, Liu, and Tam (2002) applied fuzzy-AHP methodology to modular product design. Their findings provided critical information on product module selection to support decision-making during the product development stage. 3.2. Proposed approach First, we asked online gamers to evaluate the relative importance of the nine design components proposed in the hierarchical system structure of MMORPGs. After that, a fuzzy-AHP methodology was applied to determine the relative importance weights of the design components. The detailed process of data collection and the proposed fuzzy-AHP model will be described in the following section. 3.2.1. Questionnaire design and data collection A questionnaire with a pair-wise comparison data input format was employed to gather users’ assessments of the relative importance of the design systems (game content system and valueadded system), design components (nine components) and design features (36 features). For each item, we applied a nine-point ratio scale suggested by Saaty (1990), which shows the user’s assessment among such options as equally important, moderately important, strongly important, very strongly important and extremely important. Fig. 2 shows an example of the pair-wise comparison between the game content system and the value-added system. Two hundred and twenty-one Taiwanese MMORPG players participated in this online survey. The survey was advertised on several online game forums in Taiwan to recruit volunteers for this study. After a primary data analysis, the incomplete and outlier data were deleted. We finally obtained 200 valid samples to be used in this study. 3.2.2. Proposed fuzzy-AHP model Our proposed fuzzy-AHP approach included seven steps. First, we used triangular fuzzy numbers to construct the fuzzy comparison matrix. Second, we integrated the collected data of users’ assessments of each decision element by using the fuzzy average method proposed by Buckley (1985). Third, we computed the fuzzy weight of each decision element by using the Approximation Method introduced by Buckley (1985). Fourth, the Center of

Equally importan

Moderately important

Strongly important

Very strongly important

Game content system

Extremely important

Value-added system Fig. 2. Example of the nine-point pair-wise comparison ratio scale.

Author's personal copy

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

Gravity Method, a defuzzifying method proposed by Tzeng and Teng (1993), was performed to defuzzify the weight of each decision element. Fifth, we normalized the weights of all of the decision elements. Sixth, we aggregated each level of the proposed hierarchical model of online games and calculated the relative value of the fuzzy weight for each element at the feature, component and system levels. Finally, we computed the consistency index (CI) and consistency ratio (CR) for each fuzzy comparison matrix. The seven steps of the proposed fuzzy-AHP approach in this study will now be described in detail. Step 1: Establish the fuzzy comparison matrix. ~ to 9 ~ to repe ij from 1 This study used triangular fuzzy numbers M resent the results of users’ assessments of the pair-wise comparisons between each of the decision elements (including systems, components and features) (see Table 1) by constructing a fuzzy positive reciprocal matrix M. The proposed fuzzy comparison matrix was defined as follows:

e ij  M ¼ ½M M is the fuzzy positive reciprocal matrix

e ij ¼ ðLij ; M ij ; Rij Þ M

ð1Þ

Lij is the left value of the fuzzy membership function of the collected subject assessments of design feature j of decision element i, Mij the middle value of the fuzzy membership function of the collected subject assessments of design feature j of decision element i and Rij the right value of the fuzzy membership function of the collected subject assessments of design feature j of decision element i

e ij ¼ 1= M e ji ; M

8i; j ¼ 1; 2; . . . ; n

ð2Þ

Step 2: Integrate the collected data of users’ assessments of each decision element. There are many possible approaches for integrating the collected subjects’ assessments to calculate the triangular fuzzy number. In contrast to some studies that applied statistical parameters such as the minimum, maximum, mean and mode to represent the fuzzy numbers, this study applied the geometric mean method proposed by Buckley (1985) to integrate the collected data of users’ assessments of each decision element. The computing process is defined as follows:

  ~ 1ij  m ~ 2ij      m ~ nij ~ ij ¼ ð1=nÞ  m m

ð3Þ

~ ij is the integrated triangular fuzzy numbers, m ~ Nij the value of the m pair comparison of the collected subject assessments of design feature j of decision element i and n the number of subjects. Step 3: Compute the fuzzy weight.

Table 1 Membership function and definitions of fuzzy numbers.

8689

After integrating the collected data and calculating the corresponding triangular fuzzy numbers, this study applied the Approximation Method proposed by Buckley (1985) to compute the fuzzy weight. The formula of the Approximation Method (Tzeng and Teng (1993)) for computing the fuzzy weights is defined as follows:

e i ¼ ða ~i1  a ~i2      a ~in Þ1=n ; 8i ¼ 1; 2; . . . ; n Z  1 fi ¼ Z ei  Z e1  Z e2      Z en W

ð4Þ ð5Þ

e i is the geometric mean value of the triangular fuzzy number, a ~ij Z the triangular fuzzy number of row i and column j in the fuzzy pof i the fuzzy weight of each row of the sitive reciprocal matrix and W fuzzy positive reciprocal matrix. Step 4: Defuzzify the decision elements. The weights of the decision elements, including systems, design components and design features, were represented by fuzzy values. Therefore, the defuzzification process can assign a distinct number to each of the decision elements. We applied the Center of Gravity Method of defuzzification to calculate the center of gravity of the triangular fuzzy number. Given a triangular fuzzy e ¼ ðLij ; M ij ; Rij Þ, the defuzznumber and its three sides, denoted by A ified weight DFij can be calculated by using the following formula:

DF ij ¼ ½ðRij  Lij Þ þ ðM ij  Lij Þ=3 þ Lij

ð6Þ

Step 5: Normalize the defuzzified weights. In order to compare the importance among different decision elements at different levels, we first normalized the defuzzified weights. The definition of the normalized weights (NWi) of each decision element at each level can be defined by the following formula:

NW i ¼ DF ij

.X

DF ij

ð7Þ

Step 6: Calculate the synthesized weight for each of the components at each level. After Step 5, we calculated the normalized weights of each decision elements at each level. However, to determine the priority of each element, it is still necessary to synthesize weights for each decision element in each level. The larger the value of the synthesized weight, the higher the priority of the element will be. The definition of synthesized weights of each decision element at each level can be defined by the following formula:

NW K ¼ NW i  NW ij  NW ijk

ð8Þ

Step 7: Check for consistency. The purpose of the Consistency Index (CI) is to measure the overall inconsistency for the proposed hierarchy and for each decision element. In addition, the Consistency Ratio (CR) was also calculated to describe the consistency of the pair-wise comparisons. The equations for calculating the CI and CR for each decision were defined as follows:

Fuzzy number

Membership function

Definition

~ 1 ~ 2

(1, 1, 2)

Equally important

(1, 2, 3)

~ 3 ~ 4

(2, 3, 4)

Between equally and moderately important Moderately important

where kMax is the maximum eigenvalue; n is the number of decision component

Consistency Ratio ðCRÞ ¼

~ 5 ~ 6

(4, 5, 6)

Between moderately and strongly important Strongly important

~ 7 ~ 8

(6, 7, 8)

~ 9

(8, 9, 10)

(3, 4, 5)

(5, 6, 7)

(7, 8, 9)

Consistency Index ðCIÞ ¼

Between strongly and very strongly important Very strongly important Between very strongly and extremely important Extremely important

kMax  n n1

CI RI

ð9Þ

ð10Þ

Here, RI is the average index for randomly generated weights obtained from a table of random consistency indices. To judge the consistency of the pair-wise outputs, if CR 6 0.1, then the output of the pair-wise comparison is sufficiently consistent. On the other hand, if CR > 0.1, then the results of the pair-wise comparison are inconsistent.

Author's personal copy

8690

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

Objective level

System level

Design component level

CI

CR

Important factors of online game design CI = 0.072 CR = 0.000

Game content CI = 0.078 CR = 0.070

Environmental setting Character system Sound/light effects User-task design Game world rules Virtual item cash shop Social systems User interface design Quality of services

0.092

0.082

0.096

0.086

0.055

0.094

0.074

0.066

0.090

0.096

0.056

0.096

game content (60.37%) is higher than that of value-added systems (39.63%). The relative importance weights among the game content, ordered from highest to lowest, are character system (30.02%), environmental setting (25.36%), sound/light effects (16.30%), user-task design (15.14%) and game world rules (13.17%). On the other hand, the relative importance weights among the value-added systems, from highest to lowest, are user interface design (30.23%), virtual item cash shop (27.70%), social systems (21.93%) and quality of services (20.13%). We calculated the synthesized weights of the nine components in the system level in order to provide an overall view of all design components. The relative importance rankings from highest to lowest were found as follows: character system (18.13%), environmental setting (15.31%), user interface (11.98%), virtual item cash shop (10.98%), sound/light effects (9.84%), user-task design (9.14%), social interaction systems (8.69%), quality of services (7.98%) and game world rules (7.95%). The results of the fuzzy-AHP analysis also revealed the relative importance of the 36 design features. The highest rated design feature, various character skills, contributed about 4.39%, and the lowest rated feature, friendly and helpful customer service, was weighted at about 0.95% of the overall synthesized feature weights. Table 3 shows the weights and synthesized weights of the two systems, nine design components and 36 design features.

0.054 0.065

0.094 0.072

4.2. Consistency check

0.091

0.081

4. Research findings and decision-making 4.1. Relative importance of online game design features The results of the fuzzy-AHP analysis revealed the relative importance of online game design features. Table 2 shows the rankings of the design systems, components and features of MMORPGs. At the system level, the relative importance weight of

Table 2 Consistency Index (CI) and Consistency Ratio (CR) at each design level.

Value-added systems CI = 0.083 CR = 0.092

Finally, we calculated the values of the Consistency Index (CI) and Consistency Ratio (CR) to ensure reliability within the various levels of systems, design components and design features. Overall,

Table 3 The weights of systems, components and design features. Systems

Components and weights

Synthesized weights

Design features

Feature weight

Synthesized feature weight

Game content 0.6037

Environmental setting (0.2536)

0.1531

Character system (0.3002)

0.1813

Sound/light effects (0.1630)

0.0984

User-task design (0.1514)

0.0914

Game world rules (0.1317)

0.0795

Immersive game world Unlimited game map Vivid scenes Attractive background story Differentiated character systems Various character skills Character personalization Completed professional system Attractive appearance Vivid animation Music matched to context High resolution pictures Clear task guidance Cooperative mission and tasks Attractive rewards Appropriate task difficulty Regularly released new quests Well-defined character progression Balanced power of character talents Appropriate game pace Fair reward systems

0.2557 0.2455 0.2629 0.2358 0.2268 0.2421 0.1683 0.1614 0.2014 0.2722 0.3171 0.4107 0.2392 0.1863 0.1928 0.2032 0.1785 0.3482 0.2138 0.2406 0.1974

0.0392 0.0376 0.0402 0.0361 0.0411 0.0439 0.0305 0.0293 0.0365 0.0268 0.0312 0.0404 0.0219 0.0170 0.0176 0.0186 0.0163 0.0277 0.0170 0.0191 0.0157

Virtual item cash shop (0.2770)

0.1098

Social systems (0.2193)

0.0869

User interface design (0.3023)

0.1198

Quality of services (0.2013)

0.0798

Variety of items in cash shop Useful items Attractive items Complete social support Easy-to-manage friendships Real-time social interactions Understandable user interface Easy-to-use user interface functions Graphic user interface support Clear metaphor and guidance Fast and stable game server Acceptable subscription fee Regular upgraded game system Useful information on official web Friendly and helpful customer services

0.3392 0.3902 0.2707 0.3278 0.3245 0.3477 0.3243 0.2845 0.2167 0.1744 0.1971 0.1928 0.2394 0.2514 0.1193

0.0372 0.0428 0.0297 0.0285 0.0282 0.0302 0.0389 0.0341 0.0260 0.0209 0.0157 0.0154 0.0191 0.0201 0.0095

Value-added systems 0.3963

Author's personal copy

8691

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693 Table 4 The top 10 critical design features. Ranking

Design components

Design features

1 2 3 4 5 6 7 8 9 10

Character system Virtual item cash shop Character system Sound/light effects Environmental setting Environmental setting User interface Environmental setting Virtual item cash shop Character system

This game provides various skills for characters The virtual items are useful The design of the character system in this game is differentiated The game provides high resolution pictures The design of the scene is vivid I feel immersed in the game world The user interface flow is easy to understand The scale of the game world is unlimited The virtual items vary The design of the character appearance is attractive

Total

the results of the consistency analyses revealed that all of the CI and CR values in this study were lower than 0.1. This result represents a high level of consistency in our subjects’ responses when they compared the relative importance of each pair of decision elements at each design level. To take a look at each design level, a consistency check was carried out between the game content system and the value-added system at the objective level; the CI was 0.072 and the CR value was 0.000. The consistency index values of the game content system, calculated by its five design components, were 0.078 (CI) and 0.070 (CR). The consistency index values of the value-added system, calculated by its four design components, were 0.083 (CI) and 0.092 (CR). Table 2 shows the specific values of the consistency analysis at each design level. 4.3. Critical design factors for online game development At the conceptual level, according to the results of our analysis, this study attempted to identify the design factors that contribute the most to users’ assessments. The top five important design components contributed 66.24% of the importance weights of the nine design components; they are the character system (18.13%), environmental setting (15.31%), user interface (11.98%), virtual item cash shop (10.98%) and sound/light effects (9.84%). With regard to the implementation of online games, the top 10 design features identified in this study contribute about 40% of the importance weights of all 36 design features. The top 10 critical design features are listed in Table 4. 5. Discussion and conclusion 5.1. Critical design factors for online game development Based on the relative importance of design features as assessed by the users, this study identified five critical design components for MMORPG development, the character system, environmental setting, user interface design, virtual item cash shop and sound/light effects, as the critical factors for online game development. The five important design components were also included in the top 10 important design features identified in this study. We further attempted to infer why these factors played an important role in players’ online gaming experiences. The design component character system is the most important factor, as determined in this study. These interactions, including those at the human–computer level (e.g., interacting with game content) and human–human level (e.g., interacting with community members), in the virtual world occur through an avatar-mediated metaphor (Moore & Ducheneaut, 2007). Therefore, the users are tightly connected with their virtual body and apply it to perform all in-game tasks. Thus, the character system is focused on sat-

Synthesized feature weight (%) 4.39 4.28 4.11 4.04 4.02 3.92 3.89 3.76 3.72 3.65 39.78

isfying the users’ desire for ‘role-playing’ (Yee, 2006). Therefore, the users’ ‘role-playing’ motivation may account for why the character system received a high importance rating in this study. In implementing the character system, game developers can refer to the three critical design features, namely, various character skills, differentiated character systems and attractive character appearances, that were identified among the top 10 features in this study. The environmental setting is another important factor that received a high score in this study. The environmental setting in the virtual world is related to users’ cognitive curiosity. Research has shown that incompleteness and paradoxes of the environmental setting in the virtual world can affect users’ cognitive curiosity and satisfy users’ intrinsic motivation for exploring the game world (Malone & Lepper, 1987). In addition, earlier research has also pointed out that a well-implemented environmental setting in a virtual world can provide an immersive experience to the users so that the distance between the real world and the virtual world can be reduced in the user’s mind (Yee, Bailenson, Urbanek, Chang, & Merget, 2007). Again, a high-fidelity environmental setting in the virtual world can also enhance the users’ social presence so that the social distance among users and their friendships in the virtual world can be decreased (Ducheneaut et al., 2006). These reasons might account for why the design component environmental setting plays an important role in MMORPG design. In implementing the environmental setting, game developers can focus on the three critical design features identified in this study: immersive game world, unlimited game map and vivid scenes. The design component user interface design was also found to contribute to the users’ sense of priorities in online game design. A usable interface will not frustrate users, but will help provide the users with a positive use experience. The user interface design may be a significant factor because it is related to the users’ sense of control during game-playing. Earlier research has found that the users’ motivation for game-playing is influenced by their sense of control (Malone, 1981). Novak, Hoffman, and Yung (2000) have defined the users’ sense of control as referring to the influence, dominance and guidance capabilities that the users have in the games. Users who gain a high sense of control may feel that they have a high degree of liberty, being allowed to do anything in the games, which in turn motivates them to play the game. Based on our findings, we believe that a successful game must provide a usable user interface to support users’ in-game tasks and to allow them to meet their in-game goals. To design a usable user interface, the designers should consider the compatibility between the users’ requirements and system features (Norman, 2002). Designers can also refer to the identified critical feature, defined by the statement ‘the user interface flow is easy to understand,’ as the basic design principle in this study. The design component virtual item cash shop also contributed significantly to the users’ sense of priorities in online game design. Most online games provide a cash shop that sells fantastic and

Author's personal copy

8692

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693

powerful virtual items to support users’ in-game activities. By applying the virtual items to the characters in MMORPGs, users can personalize their characters and represent their personal identities (Ducheneaut, Wen, Yee, & Wadley, 2009). For example, they can buy weapons, shields and armor from the cash shop to enhance their characters’ ability to become powerful and efficient. In addition, they can purchase new packages of hairstyle, hair color and skin tone to customize their character’s appearance, in order to become more ideal and attractive (Ducheneaut et al., 2009). Research has found that the users’ virtual identity can influence their motivation for game play (Yee, 2006). We therefore infer that the design component virtual item cash shop received high importance ratings from the users because it can assist the users in building their personal identity in the virtual world and because it can motivate them to continue to participate in the game. To develop a beneficial virtual item cash shop in MMORPGs, we recommend that the designers pay more attention to ensuring that the types of virtual items in their cash shop are ‘varied’ and to making sure that those items are ‘useful’ to the users. Finally, the design component sound and light effects was also identified as an important factor that contributes to the players’ sense of priorities in MMORPG design. It has been shown that the design of music in computer games can affect users’ immersive experiences, such as danger, excitement, and relaxation (Lipscomb & Zehnder, 2004). This experience was focused on satisfying users’ needs for sensory stimulation and curiosity on both the visual and audio level. Research has found that users’ sensory curiosity can increase their motivation to play computer games (Malone, 1981). Therefore, online games that have a high quality of sound and light design can affect users’ emotional states and can provide them with fantastic episodes that they have not yet experienced. This may explain why the design component sound and light design obtained a higher assessment importance score from the users. The critical design feature identified in this study to support the implementation of sound and light design in MMORPGs is the presentation of ‘high resolution pictures.’ 5.2. General conclusions This study proposed a fuzzy-AHP-based technique to solve a trade-off decision problem among online game design factors. We attempted to explore the relative importance of various game design factors and suggested an efficient portfolio of design features for game development. We constructed a hierarchical system structure for MMORPGs that included two systems, nine design components and 36 design features. Our results indicated that the relative importance weight of game content is higher than that of value-added systems. Underlying the two main systems, five components were identified as the critical design factors and contributed about 66% of users’ sense of priorities in MMORPG design. The study also recommended 10 important design features, which can fulfill about 40% of users’ expectations in implementing the five critical components through specific design guidelines. In general, our findings suggest a heuristic design strategy for MMORPG development so that the game developers can invest their resources (including time, money and human resources) on those factors that our findings suggest as the most effective. Again, we discussed the critical design components contributing to the users’ motivation for game-playing by referring to the existing literature. Therefore, this study provides game developers with not only suggestions of game design features, but also with a deeper understanding of the effects of the design factors on users’ personal experiences. This study acknowledges the following research limitations. First, concerning cultural issues, this study was conducted in Taiwan, and the question of whether the findings obtained from

Taiwanese participants can be applied to represent the requirements of users from other countries should be examined in further research. Second, although our results provide some important design guidelines for enhancing the quality of online game design in order to meet users’ expectations, these design guidelines still need to be discussed in detail because each guideline may contain several options and implementation levels. Therefore, we suggest that future research should attempt to define specific design rules from a practical standpoint.

References Ayag˘, Z. (2005). A fuzzy AHP-based simulation approach to concept evaluation in a NPD environment. IIE Transactions, 37(9), 827–842. Block, P. (2008). Community: The structure of belonging. San Francisco, CA: BerrettKoehler Publishers. Buckley, J. J. (1985). Fuzzy hierarchical analysis. Fuzzy Sets and Systems, 17(3), 233–247. Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute-methods and applications. New York: Springer-Verlag Inc. Choi, D., & Kim, J. (2004). Why people continue to play online games: In search of critical design factors to increase customer loyalty to online contents. CyberPsychology and Behavior, 7(1), 11–24. Choi, D., Kim, H., & Kim, J. (1999). Toward the construction of fun computer games: Differences in the views of developers and players. Personal and Ubiquitous Computing, 3(3), 92–104. _ (2007). Developing a fuzzy analytic hierarchy process Dag˘deviren, M., & Yüksel, I. (AHP) model for behavior-based safety management. Information Sciences, 178(6), 1717–1733. DFC-Intelligence. (2006). The online game market. . Ducheneaut, N., Wen, M. H., Yee, N., & Wadley, G. (2009). Body and mind: A study of avatar personalization in three virtual worlds. In 27th annual CHI conference on human factors in computing systems (CHI 2009) Boston, MA. Ducheneaut, N., Yee, N., Nickell, E., & Moore, R. J. (2006). ‘‘Alone together?”: Exploring the social dynamics of massively multiplayer online games. In 24th annual CHI conference on human factors in computing systems (CHI 2006), Montréal, Québec, Canada. Fabricatore, C., Nussbaum, M., & Rosas, R. (2002). Playability in action videogames: A qualitative design model. Human–Computer Interaction, 17(4), 311–368. Fitzsimmons, J. A., Kouvelis, P., & Mallick, D. N. (1991). Design strategy and its interface with manufacturing and marketing: A conceptual framework. Journal of Operations Management, 10(3), 398–415. Gilbert, D. (2007). Hacking world of warcraft (ExtremeTech). New York: John Wiley & Sons, Inc.. Hsu, S. H., Lee, F. L., & Wu, M. C. (2005). Designing action games for appealing to buyers. CyberPsychology and Behavior, 8(6), 585–591. Hsu, S. H., Lee, F. L., & Wu, M. C. (2006). An integrated approach to achieving optimal design of computer games. Expert Systems with Applications, 31(1), 145–149. Hsu, S. H., Wen, M. H., & Wu, M. C. (2007). Exploring design features for enhancing players’ challenge in strategy games. CyberPsychology and Behavior, 10(3), 393–397. Kahraman, C., Cebeci, U., & Ulakan, Z. (2003). Multi-criteria supplier selection using FAHP. Logistics Information Management, 16(6), 382–394. Kawakita, J. (1975). The KJ method – A scientific approach to problem solving. Technical report. Tokyo: Kawakita Research Institute. Kelly, R. V. (2004). Massively multiplayer online role-playing games: The people, the addiction and the playing experience. Jefferson, NC: McFarland & Company. Klir, G. J., Clair, U. S., & Yuan, B. (1997). Fuzzy set theory: Foundations and applications. Upper Saddle River, NJ, USA: Prentice-Hall, Inc. Kurttila, M., Pesonen, M., Kangas, J., & Kajanus, M. (2000). Utilizing the analytic hierarchy process (AHP) in SWOT analysis – A hybrid method and its application to a forest-certification case. Forest Policy and Economics, 1(1), 41–52. Kwong, C. K., & Bai, H. (2002). A fuzzy AHP approach to the determination of importance weights of customer requirements in quality function deployment. Journal of Intelligent Manufacturing, 13(5), 367–377. Laarhoven, P. J. M., & Pedrycz, W. (1983). A fuzzy extension of Saaty’s priority theory. Fuzzy Sets and Systems, 11(3), 229–241. Lee, W. B., Lau, H., Liu, Z. Z., & Tam, S. (2002). A fuzzy analytic hierarchy process approach in modular product design. Expert Systems, 18(1), 32–42. Lipscomb, S. D., & Zehnder, S. M. (2004). Immersion in the virtual environment: The effect of a musical score on the video gaming experience. Journal of Physiological Anthropology and Applied Human Science, 23(6), 337–343. Luo, L., Liu, J., Shao, L., Lu, W., & Ye, M. (2006). A context-aware smart-call-center solution: Improving customer service for online games. IBM System Journal, 45(1), 145–160. Ma, M.-Y., Chen, C.-Y., & Wu, F.-G. (2007). A design decision-making support model for customized product color combination. Computers in Industry, 58(6), 504–518. Malone, T. W. (1981). What makes computer games fun? BYTE, 6, 258–277.

Author's personal copy

Y.-F. Lo, M.-H. Wen / Expert Systems with Applications 37 (2010) 8685–8693 Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning and instruction, III: Conative and affective process analysis (pp. 223–253). Hillsale, NJ: Lawrence Erlbaum Associate, Inc. Miller, G. A. (1956). The magic number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97. Moore, R. J., & Ducheneaut, N. (2007). Doing virtually nothing: Awareness and accountability in massively multiplayer online worlds. Computer Supported Cooperative Work, 16(3), 265–305. Norman, D. A. (2002). The design of everyday things. New York: Basic Books. Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books. Novak, T. P., Hoffman, D. L., & Yung, Y. F. (2000). Measuring the customer experience in online environments: A structural modeling approach. Marketing Science, 19(1), 22–42. Olsen, J. M. (Ed.). (2003). Game balance and AI using payoff matrices. Hingham, Massachusetts: Charles RiverMedia. Pisan, Y. (2007). My guild, my people: Role of guilds in massively multiplayer online games. In Proceedings of the 4th Australasian conference on interactive entertainment, Melbourne, Australia.

8693

Saaty, T. L. (1980). The analytic hierarchy process: Planning, priority setting, resource allocation. New York: McGraw-Hill. Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48, 9–26. Scupin, R. (1997). The KJ method: A technique for analyzing data derived from Japanese ethnology. Human Organization, 56(2), 233–237. Tzeng, G. H., & Teng, J. Y. (1993). Transportation investment project selection with fuzzy multiobjectives. Transportation Planning and Technology, 17(2), 91–112. Van Lent, M. (2008). The business of fun. Computer, 41(2), 101–103. Woodcock, B. (2008). An analysis of MMOG subscription growth, version 23.0. . Yee, N. (2006). Motivations of play in online games. CyberPsychology and Behavior, 9(6), 772–775. Yee, N., Bailenson, J. N., Urbanek, M., Chang, F., & Merget, D. (2007). The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. CyberPsychology and Behavior, 10(1), 115–121. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.