MULTIMEDIALE LEERMIDDELEN --- HOOFDSTUK

Mental models

Acquisition of a mental model by means of computer simulation

Het opbouwen van mentale modellen bij het werken en leren met computersimulaties


Door: R. Koffijberg, mei 1996, Zeist/Enschede. Begeleiding: ir. J. Vader (begeleider bij Origin/IMCC), dr. ir. F.B.M. Min (afstudeerbegeleider) en prof. J.J.C.M. Moonen (supervisie).

Deze tekst is van oorsprong een literatuurscriptie in het kader van de studie Toegepaste Onderwijskunde bij de Universiteit Twente.

Aan deze page wordt nog gewerkt. Als u de volledige tabellen en de plaatjes wilt hebben, bel of mail ons dan even, dan sturen we u die toe.

1. INTRODUCTION

This paper tries to connect two topics: the concept of a mental model and computer simulation. Both are very broad terms, which will be limited to a workable definition. A mental model will be viewed upon as a kind of visual mental structure that facilitates the understanding of a system. Computer simulation is seen as a special kind of open learning environment. The central question in this paper is: How should model driven computer simulation facilitate the development of a mental model ?
Chapter one deals with the question: What is a mental model ? To answer this question the literature about mental models is regarded. A general description of mental models is provided. Because one of the characteristics of mental models is their visual structure, the imagery debate is shortly addressed. The benefits of a mental model are summed up and finally a definition is formulated.
The second question is: How do people acquire a mental model ? Different strategies that people use in this everyday process are described. These strategies do not include any instruction. It is also possible to design instruction for generating a mental model. Some relevant experiments are described. An important conclusion is that people benefit from diagrammatic representations.
The third question is: What is computer simulation?. In this paper computer simulation is regarded as a learning tool. Not every computer simulation contains a mathematical, underlying model. The so-called model-driven computer simulation does have such a model. This is interesting because a comparison with a mental model can be made. Now that all these subquestions are answered, the question: How should model driven computer simulation facilitate the development of a mental model ? can be inferred. The answer to this question lies in the use of simple models, visualization of information and a mental model as a learning goal.


2. MENTAL MODELS

There are many ways to model the presence of knowledge in the human brain. One of them is a mental model. This is a kind of mental picture, from which people are able to draw conclusions. Another way to represent knowledge is as a collection of mental sentences or propositions. This is done in production systems. This approach has been very successful, mainly because computers proved to be successful media for implementing these systems. Proponents of this theory are called propositionalists. Their ideas are summarized in paragraph 2.2. If propositionalist are right about their view that a picture-like form of thinking is not possible, the concept of a depictive mental model becomes highly speculative. To prove the depictive way of thinking, which is fundamentally different from the propositional way, Kosslyn uses neuroscientific research results (paragraph 2.3). A mental model is worth teaching, if a learner can benefit from itís use. The advantages of having an appropriate mental model are described in paragraph 2.4. This chapter starts with one of the first thoughts of mental models.

2.1 In general
The first article on mental model theory was published by Kenneth Craik (1943) who stated:

... If the organism carries a Ďsmall-scale modelí of external reality and of its possible actions within its head, it is able to try out various alternatives, conclude which is the best of them, react to future situations before they arise, utilize the knowledge of past events in dealing with the present and the future, and in every way to react in a much fuller, safer, and more competent manner to the emergencies which face it.

In general, mental models are considered as internal representations of reality. Three characteristics are frequently mentioned in various descriptions. The first characteristic is that mental models are closer to images than they are to verbal descriptions, in other words they are more signs than symbols (Payne, 1988).The second characteristic is that mental models can be run. This means that a mental simulation in the mindís eye can produce qualitative inferences immediately. The last characteristic of mental models is that they rely heavily on experiential knowledge. In a often quoted article, Norman (1983) makes some general observations on mental models that people have:

1. Mental models are incomplete
2. Peopleís abilities to run mental models are limited
3. Mental models are unstable; people forget the details
4. Mental models do not have firm boundaries: similar devices and operations get confused with one another
5. Mental models are unscientific representations, people maintain superstitious behavior patterns, this means that a mental model doesnít necessarily have to correspond to reality.
6. Mental models are parsimonious, often people do extra physical operations rather than the mental planning that would allow them to avoid those actions

An example in a study of Schaubel et. al. (1991) makes these observations clear. In this study novels are asked to discover the function of a few black boxes in a computer simulated electric circuit called Voltaville. Schaubel et. al. were able to identify four different mental models differing in properties of entities, relations between entities, and causes and effects that played a role in effecting change in a circuit. Poor performers on the posttest (5 out of 22 subjects) possessed a simple local model: they divided the black boxes in two categories. Things that work (make the lightbulb light) and things that didnít work. They didnít try to find out whether the black boxes could be anything else than a battery. These models are incomplete (observation 1) and unscientific (observation 5). Subjects made illogical assumptions. They included irrelevant properties as size and weight of the black boxes in their mental models. At the highest level of performance students (3 out of 22) held a causal model. They looked beyond (a) the surface structure of the boxes, (b) the literal classification system suggested by the labels on the boxes and (c) the teleological associations suggested by the bulb (e.g. the purpose of a circuit is to light the bulb). The students abstracted the understanding that, regardless of their names, all of the components served either as voltage sources or resistors. They were able to integrate all variables in circuits with more than one black box. In summary, lowerlevel models were local and piecemeal, whereas models at higher levels were progressively more cohesive and integrated. The least sophisticated models focused on the surface structure of the task materials, where the more sophisticated models focused on progressively more principled notions concerning the functions and electrical properties of components. The way conceptual objects were organized in the models varied as well, with increasing differentiation, elaboration and higher order classification serving as features of the more sophisticated models. ( Schauble et. al., 1991, p213.)
Concluding, a system in reality may lead to a mental model. Norman calls the system in reality a target system.






Figure 1.Instruction or observation from a target system leads to a mental model.

To acquire a mental model of a system which is not easily observed (e.g. molecules, national economy, energy), a conceptual model has to be presented to a learner. This is a mental model of an expert, condensed in some medium, e.g. a text, a computer program or a drawing.
Norman (1983) distinguishes for this purpose a target system, the conceptual model of that target system, the userís mental model of the target system and the scientistís conceptualization of that mental model. A conceptual model is invented to provide an appropriate representation of the target system, appropriate in the sense of being accurate, consistent and complete. These characteristics are not necessarily held by a mental model. A scientistís conceptualization of a mental model is a mental model of a mental model. Norman introduces a notation of these concepts:

t target system
C(t) a conceptual model of the target system
M(t) a mental model of the target system
C(M(t)) a conceptualisation of a mental model










Figure 2. The different models according to Norman (1986)

In a later discussion of these issues, Norman (1986) also introduces a design model, a userís model and a system image. Other authors also contributed to what Streitz (1988) calls the Mental Model Zoo. Streitz tries to systemize the different views on mental and conceptual models.
He starts with a functionality f. This functionality can be realized by a system S, leading to a systemís realization of the functionality, S(f). A functionality cannot exist without a realization of this functionality. Streitz argues that a functionality can be realized in different systems. His assumption is that people can abstract the basic features of a function from their experience with a variety of system realizations.
The mental model of this functionality calls Streitz U(f). U stands for user. This model is based on experiences with a variety of the realizations, Si (f). This is a different concept from the mental model Norman mentioned. His mental model in the terminology of Streitz would be: U(S(f)). This model is a second order model. For conceptual models, Streitz wants to distinguish between models of designers, D(f) and those of psychologists P(f). This distinction accounts for the difference in conceptualization in models implemented in S(f) and those studies by psychologists. Thus, Streitz considers four models; U(f), S(f), P(f) and D(f).







Figure 3. The first order model (f) and 3 second-order models D(f), S(f) and U(f) according to Streitz (1988).

If U(f), S(f), P(f) and D(f) are seen as operands and S, U, D and P are seen as operators, a range of second-order models appears when the operators are applied to the operands. This situation is represented in the last four rows of Figure 4. The first row contains the first-
order models.


S
U
P
D
f

+ + very relevant
+ relevant
0 not relevant
- doesnít exist
* self reflection

Figure 4. Classification of first and second-order models of a functionality f. (Streitz, 1988).

If scientists talk about mental models of users, a third-order model is needed: P(U(S(f))); the psychologistís mental model of the users mental model of the systems realization of a functionality. This model is influenced by the P(S(f)) and the P(f).
Communication about a third-order model will result in a fourth-order model. One can question the usability of these and higher order models.
The central model in this study is the userís mental model of the systemís realization of the function f; U (S(f)). This is the model the user refers to when actually interacting with a system.
The concept of a mental model is used in different fields of science and lends itself for many definitions. A diversity of terminology is used to refer to different models for example: surrogates, metaphors and glass-box machines. A surrogate is a mental model that has the same input/output functions as a system in reality, but itís internal causal structure is different. Itís obvious that a surrogate carries the danger of malfunctioning in some (maybe important) situations. A metaphor is a comparison between a unknown system in reality and a known system in the mind of a user or learner. Knowledge of one domain is transferred to another domain. A glassbox-model lies in the middle of the continuum from surrogate to metaphor. The description of this model is not very clear: The glass box model share the mimicry quality of surrogates whilst offering a semantic basis for understanding the system, as do metaphors, by providing information about the internal processing and mechanics of the system. (Bibby, 1992).
One explanation for this diversity is that different disciplines of cognitive psychology and cognitive ergonomics/human factors independently developed a notion of a mental model (Goei, 1994). Another explanation is that the notion of a mental model can be molded in any form a researcher would like to have. Bainbridge (1992) concludes that the domain in which the mental model is used, largely affects the elements that are considered as necessary.
A problem with the concept of mental models is the form in which they are represented in the head. It is not likely that they are similar to photographs; it would require a homunculus to perceive these pictures (Johnson-Laird, 1980). In general, two perspectives on knowledge representation and problem solving can be found in literature; a propositional one and a depictive one. The propositional one is described in the next paragraph, the depictive view in paragraph 2.3.

2.2 The propositional perspective
Before the theories of mental models were introduced, the classical view on skill acquisition was dominated by the propositional viewpoint. In this view cognition is modeled by productionsystems that are easily simulated by computer programs.

Propositionalists see cognition as a collection of facts and rules represented in a sentence-
like structure in the head. They differ from natural language sentences, a reason why it is sometimes difficult to put an idea into words. Another argument for a difference with natural language is that young children are able to understand concepts they are not able to explain. Some propositionalist allow the formation of a picture on the basis of propositions. This mental picture can not contain more information than the propositions itself. (Stillings et. al., 1987).
Newell and Simon (1972) popularized a way of representing knowledge, which has become a universal standard in psychology (Payne, 1988). Their system is known as the production system representation. Production systems are convenient languages to model learning. Learning is seen as a transition from novice to expert. An expert does know the same as a novice, but he is able to use his knowledge in a more efficient way.
To understand this, two kinds of knowledge are distinguished: declarative knowledge and procedural knowledge. Declarative knowledge consists of facts, concepts and principles. Procedural knowledge is knowledge about how to process or manipulate information to accomplish tasks. According to Anderson (1987), procedural knowledge accounts for all mental and physical activities.
The knowledge base of an expert and a novice can be the same (Shih & Alessi, 1993). It consists of declarative knowledge. The expert has more procedural knowledge, in production system terms: more specific production rules. The production rules consist of domain specific procedural knowledge. This knowledge is acquired through a synthesis of domain general procedural knowledge and domain specific declarative knowledge.
Production systems consist of a set of rules (productions), that communicate by way of a working memory. Productions in a production system are condition-action pairs that specify what action occurs under which condition. A novice in this view uses general-purpose, weak-search methods to solve a problem or acquire a cognitive skill. These search strategies can be made more efficient and faster. This is done by various mechanisms that mutate production rules. These mutating mechanisms model the learning mechanisms that occur with practice. (Payne,1988).
The modular structure of production systems allows the adding of new code to the existing system, without complete rewriting of this system. Provided that there is sufficient redundancy in the ruleset, it is also possible to delete rules without distortion of the reasoning ability of the system. These qualities make the system suitable to model the knowledge-structures of a learner. Since the early designs, many refinements have been introduced, particularly in the pattern matching and conflict resolution strategies. (Payne, 1988). Learning theories have been automated by specifying mechanisms that create new production rules from old, using a variety of transformation techniques. The fundamental approach of the systems is the same. The initial production system solves problems in a domain by some combination of weak search methods (which require only general knowledge). This starting production is designed to model novice problem solving performance. As the novice production system solves problems in a domain, learning mechanisms derive new productions that encode specific knowledge about the domain in their antecedent conditions. The new productions take precedence in the solution of prior problems, allowing knowledge-base heuristics to control search. Expert performance is thus modeled by productions which encode methods for accomplishing tasks (Payne, 1988).
The described approach has been criticized. This critique provides arguments in favor of a more realistic theory of knowledge representation. Payne noticed three major shortcomings of the proposistional view; it (1) neglects context effects, (2) neglects to model mistaken performance, and (3) neglects the role played by rich conceptual representations (Payne, 1988).

(1)Theories have concentrated on the mechanisms of change rather than the conditions of change. The conditions of change lay in a cognitive context.
(2)Mistaken behavior is modeled by adding mal-rules to the rule set. A mal-rule is a rule that leads to an incorrect process of reasoning. A major problem with mal-rules is that they oversimplify the psychological status of regular mistakes. Not all regular mistakes need be caused by buggy procedures. A second problem is that not all problematic misconceptions will be manifest in regular misbehaviors.
(3)The last shortcoming is not soluble within the framework of production system representations. Experts know more than can be easily represented as a collection of method-generating rules; they can reason and argue about their expertise, they can cope with novel problems and their skill transfers to seemingly separate domains.

The conclusion of Payne is:
To interrelate conceptual knowledge with procedural skills appears to demand more powerful constructions than mere networks of facts.


2.3 Building blocks of mental models
The subsystems in the theory of mental imagery are the building blocks for the theory of mental modeling. The debate between proponents of propositional representations and proponents of mental imagery is shortly addressed. With the publication of Image and Brain Kosslyn (1994) tries to resolve the debate.

The central theme in mental model theory is visualizing. People are able to see in the minds eye. They can perform mental simulations to predict target system behavior. The human ability to imagine objects that are not present, corresponds to introspective observations.
Mental imagery is a necessary condition for the proof of mental modeling theories. If mental imagery does not exist, the theory of mental models becomes highly speculative. In other words, mental imagery is the foundation of mental modeling. Imagery is a fundamentally different approach than the one taken in the paragraph on the propositionalist viewpoint. A similarity between both approaches is that they assume that humans do not think in sentences nor in pictures. There is some sort of language of thought. The debate is whether humans are able to think in picture-like structures. The question whether people are able to think in sentence-like structures is not a topic of discussion.
The processes of visualizing are studied in the field of mental imagery.
...imagery is a special purpose component of the cognitive architecture containing representations and processes that are dedicated to processing certain kinds of visual information and that are distinct from the aspects of the architecture that support propositional representations. (Stillings et. al, 1987, p.36)
A sequence of mental transformations makes it possible to predict the behavior of a target system. (A target system is a system that is imagined). This predictive power is one of the characteristics of a mental model. Another resemblance between imagery and mental models is the picture-like form of representation, which appeals to introspective observations. Kosslyn (1973) tried to prove the existence of mental imagery by scanning experiments. He focused on the so-called privileged properties; properties of depictive representations which are not shared by propositional representations.
The subjects in his experiment were asked to close their eyes and visualize previously memorized drawings. For example, subjects might close their eyes and visualize a boat with a motor on the left end, a porthole in the middle and an anchor on the right end. Then, they would focus attention on one end of the object, for example the motor. Next they would hear the name of a part (such as porthole, but also mast and anchor which were not present) and were to look for the named part on the imagined object. The subjects were to press one button if they could see the named part and another if they could not. The further along the object they had to scan, the more time was required, even though the object was no longer present and the subjectsí eyes were closed.
The results of these experiments were seen and understood differently by propositionalists. The time the subject needed was not used to scan the image, but to work down several lists of mental propositions. According to this reasoning, the subjects memorized objects by formulating a series of linked propositions.
As more scanning and mental rotation experiments were conducted, the propositional researchers had to add more rules, lists and symbols to their theories. This means that the propositionalistsí theory lost predictive power. The modifications were according to Kosslyn, ad hoc and post hoc.
In his later work, Kosslyn (1994) uses a cognitive neuroscience approach to explore the concept of mental imagery. His goal is to characterize the subsystems that are responsible for imagery in terms of input, operation and output of a set of neurons.
The mechanisms of imagery are interwoven with those of visual perception. It is therefore that Kosslyn starts with a study of visual perception. He distinguishes between relatively well understood low-level vision, driven on stimuli input and high-level vision, based on previously stored information about properties of objects and events. This last form of perception has deep and pervasive effects during perception.
...I will argue that imagery plays an important role during ordinary perception. (p.53)
He identifies five classes of visual perception abilities his theory has to account for. The goal ... is to infer the subsystems that confer our ability to identify objects (1) in different locations and at different distances; (2) with different, sometimes novel, shapes (3) when the input is impoverished; (4) as specific exemplars, including ones that are not typical for the category; and (5) when we see objects embedded in scenes. Each of these abilities corresponds to a set of more specialized abilities. (p.76)
In the end of his book Kosslyn gives an overview of 16 subsystems of which the mental imagery ability is comprised (p.380). He is able to describe input, operations and output of these systems and gives a suggestion of the localization in the brain. The empirical basis behind his theory is too extensive to describe within the limitations of this paper. It consists of observations of brain damaged patients, experiments with mental rotation, scanning and zooming (also with blind subjects), PET-scans during mental tasks, recognition tasks, research on categorization, prototyping and familiarity, and other research of the past fifteen years.
Phylyshyn is a hard-core propositionalist, and one of the main opponents of Kosslyn in the so-called imagery debates. His critique on Image and Brain is fundamental: ... But the basic problem still stands: as long as the research questions continue to be ill- posed, the problem about mental images will remain unsolved, regardless of how munch brain (or other) data is collected. (Phylyshyn, 1994).
Another critique on Kosslyns book is more positive.
... Evidence for shared mechanisms in imagery and perception is obviously relevant to the debate about the nature of images: if images share brain areas with perception, and these areas are hard-wired (that is, topographically organized) to represent shape depictively in perception, it follows that these areas represent shape depictively in imagery. This hypotheses is further supported by evidence that imagery not only activates homologous topographically organized areas in the human brain but is impaired when these areas are damaged. In summary, the central issue of the imagery debate can now be stated in concrete terms: do areas of the brain that depict visual information represent visual mental images? Yes is an educated answer. How are patterns of activation in these areas formed, manipulated, and used during imagery? With its schematic approach Image and Brain addresses these questions in a precise and challenging, yet enjoyable way. (Georgopoulos, 1995)
A part of Kosslyns work deals with image maintenance, the ability to maintain an image over a period of time which makes it possible to use it in mental operations.
Image maintenance lies at the heart of the use of imagery in reasoning: such tasks usually require at least a few seconds to perform - and hence if one cannot maintain the image, it is useless. For example, one might be told that Sally is smarter than Sam, Sam is dumber than Sheila, and Sheila is dumber than Sally. If one images each person as a dot on a line, with the smarter people being further to the right (and can remember which name goes with which dot), it is easy to decide who is smartest, who is dumbest, and so forth. I think of his sort of imagery as one type of mental model, of the sort discussed by Johnson-Laird (1983). (Kosslyn, 1994, p. 324)
Kosslyn (1994) identifies three reasons why not all theorists are convinced that mental models represent the actual processes in the brain better than production rules do. The first is that their intuitions have developed over years of working with computers trying to program their ideas in LISP. Second, the successes of Artificial Intelligence impressed researchers so much that they saw no reason for change. The last reason is the attractivity of the apparent simplicity of a single-representation system.

2.3.1 Conclusion
Kosslyn tries to prove mental imagery with the results of neuroscientific research. This seems a successful approach although hard-core propositionalists are not convinced. The subsystems of the brain that are responsible for depictive thinking are outlined in a systematic way, but more research is needed to prove the correctness of this particular model. The ability of people to think depictive is sufficiently proven. This means that the mental model theory can be build on an empirical basis. This way of thinking is not so easily modeled in a system, or simulated on a computer as propositional thinking is.

2.4 Benefits of a mental model
Is there a good reason for investing mental effort in acquiring a mental model? This is the central question in this paragraph. There have been reported influences on problem solving, on learning speed and on inferring procedures.

The way in which knowledge is represented has an influence on the outcome of problem solving. During an experiment with 36 high school and college students, Gentner and Gentner (1983) asked their subjects to make qualitative predictions about changes in voltage and current in simple electrical circuits. The circuits differed in the amount of resistors and batteries and the way they were connected (parallel or sequential).
Genter and Genter concluded that subjects who imagined electricity as a flowing fluid made good predictions about the problems with batteries. Subjects who imagined electricity as a moving crowd made good predictions about resistor-problems. The interaction between mental image and the type of combination (batteries or resistors) was significant. No other effects were significant. Kieras & Bovair (1984) studied how a mental model of a device influenced the learning of procedures to operate this device and if a mental model was responsible for inferring operating procedures. By a mental model they mean some kind of understanding of how the device works in terms of its internal structure and processes. They do not claim any visual or propositional representation.
In simple cases it is clearly unnecessary to know how a device works. A familiar example is a telephone. Many people use it easily, without knowing anything about the underlying system. Even a complex telephone system requires only how-to -do -it knowledge. However in more complex cases the situation may be different. Kieras & Bovair designed a control panel for a device based on the science fiction series Star Wars. It contained switches and indicators for useful contrivances as shipboard power, energy boosters and a phaser firing indicator.
In their first experiment they tested the influence of a mental model on the learning rate of procedures. The model group (n=20) received instruction about the internal working of the device. Both model en control (n=20) group received the same procedure training. The learned 10 procedures. Four of these procedures where designed to be inefficient. Then a retention test was taken and in a second test subjects were asked to look for short-cuts. One week later a second retention test was taken.

Table 1. Summary of results on learning procedures with and without a device model.

Group

Rote
Model
Improvement
mean device model training time (s) ---
1141

mean procedure training time (s)
270
194
28%
mean correct procedure retention (over all tests) 67%
80%
19%
mean correct retention after 1 week
71%
78%
11%
proportion of short-cuts(more efficient procedures when possible)
8%
40%
400%
mean execution time of retained instructed procedures (s)
20.1
16.8
17%

This experiment shows that understanding a device leads to better performances. The results are similar to those found in experiments with more meaningful learning material. However, the procedures itself are not more meaningful. The hypothesis Kieras & Bovair suggest is that the understanding of the device makes it possible to infer procedures. They test the hypothesis with a second experiment. In this experiment the conditions are the same, but the test is different. Subjects were asked to find two efficient procedures to make the PF-indicator flash. The number of actions was measured.

Table 2. Mean number of actions tried while inferring procedures

Attempt
Group
1
2
3
Rote
21.8
9.8
8.1
Model
8.6
7.9
---

The conclusion of this experiment is that a mental model of a device, makes it possible to infer procedures in a efficient way. Thus, the learning of procedures may be made more efficient if the underlying model is known to the student.
This result is consistent with a study of Shih & Alessi (1993). They used graphics and animation to teach programming skills. Programming is a field in which misconceptions are one of the biggest obstacles for students in learning to program. In this experiment seventy nonprogrammers practiced either code evaluation, code evaluation with the aid of conceptual models, or code generation. Conceptual models were provided by a computer based training that used a warehouse analogy and different animations to explain arrays, loops and the sequential evaluation of expressions. In the first sessions (of four) the group that practiced code evaluation with the help of conceptual models (Group EM) took more time to complete the sessions. In the later sessions they were able to solve the problems faster, and spent on the entire course an equal amount of time. The explanation lies in the assumption that group EM spent more time to form mental models, and benefited from these models because the time they needed to find a solution became shorter.
Shih & Alessi found that practicing code evaluation with the help of conceptual models promoted conceptual understanding and facilitated the learning of evaluation skill and transfer to code generation skill. Also the ability to solve transfer problems was highly correlated with the quality of the subjectsí mental models. Should these implications for programming instruction be generalized to other domains? Many things we teach require cognitive skills similar to those used in computer programming. We believe that students being taught any content that requires cognitive skills will benefit from conceptual instruction, including the use of conceptual models to help students develop appropriate mental models. We recommend the development of instructional design models that include the use of conceptual models in teaching cognitive skills. (Shih & Alessi, 1993)
The last experiment that is mentioned is the one Mayer (1989) conducted to investigate benefits of mental models.
Mayer (1989) tested whether the presentation of a conceptual model would influence conceptual recall, verbatim retention and creative problem solving. The conceptual model was presented to a learner before he heard a text about a topic. The conceptual model consisted of a diagram. The topics varied from Ohmís law to brakes and databases. Mayer suggested that a good conceptual model can provide a assimilative context to build useful mental models. The test results of various experiments led to the overall conclusion that transfer and conceptual recall increase. The verbatim retention decreases.

Table 3. Mayer (1989) observed positive effects offering a conceptual mode before instruction..

Type of test
Number of tests
Median percentage increase or decrease
Conceptual recall
10
+ 57 %
Verbatim retention
5
- 14 %
Problem solving transfer
1 6
+ 64%

2.5 Conclusion
A good mental model is a reason or a consequence of understanding. Understanding may even be the same as having a accurate mental model. Because a philosophical discussion of meaning, understanding and thinking is outside the scope of this paper, I will assume that a good mental model is beneficial in terms of better conceptual recall, better conceptual recall and the possibility to infer rules. It is worth teaching for understanding, or transfer a mental model to a learner if a subject matter lends itself for it. The subject matter must not be too easy, and must be meaningful. In the case of learning procedures about a device, the device must be made understandable.

2.6 Discussion
Until now, different aspects of mental models have been mentioned. Now it is time to provide a definition of a mental model in this paper. An essential characteristic is that a mental model is a depictive language of thought. This is what makes a mental model different from a production system. It is possible for people to infer rules from this model or to make predictions about the system they have in mind. Because a mental model is meaningful to people, it is remembered over a long period of time (compared to rote learning).
A basic conclusion is that nobody really knows how the language of thought works. Until we do, a part of cognition is called mental modeling. This may cover some relevant processes, but it is no more than a theory. Although mental model theory is not an explication of the language of thought, it seems to be a workable set of hypotheses. Before continuing, a definition of mental models is provided.
The definition of a mental model in this paper is: A mental model is a personal, meaningful, depictive representation in the brain of a system in reality, that allows a person to infer rules or make predictions about the behaviour of this system.


3. HOW IS A MENTAL MODEL FORMED?

Different authors suggest that people use different strategies to form a mental model. The process of mental model forming, can occur without any explicit instruction (paragraph 3.1). A goal of instruction may be the formation of an appropriate mental model. If this is the case, two approaches can be distinguished: direct instruction and instruction for generating mental models.

3.1 Without instruction
In the real world people form a lot of mental models without being instructed. Waern (1990) sees two principal different strategies for building a mental model: a top-down and a bottom-up learning approach. A learner who builds a mental model of a system solely on basis of the experiences of interactions with the system can be regarded to use a bottom-up approach. On the other hand a learner who builds a mental model of the system on basis of his expectations of the system, derived from his prior knowledge of similar tasks or systems can be regarded to use a top-down learning approach.
A change of a mental model takes place if:

1. a discrepancy between the expectancies and the observed effect of an action is detected
2. the discrepancy is serious enough to try to find a cause
3. the mental model is thought to be responsible for this discrepancy
4. a small number of alternative mental models is available

If too many alternative models are available a person may rather give up than get lost in a maze of alternative models. If no alternative model is available a person cannot change his mental model Waern (1990).
Shrager & Klahr (1986) investigated the way people are dealing with unfamiliar complex devices without being instructed. They chose a remote controlled, programmable toy-car called Big Trak as a device.
After a verbal protocol analysis of seven subjects who had to figure out Big Trak, they divided learning in two phases: (1)initial orientation and (2)systematic investigation. In the orientation phase people fail in their first attempts to control Big Trak. This first phase takes 30 seconds to 7 minutes.
In the systematic investigation phase, the second phase, Shrager and Klahr see three processes that guide behavior: (1)hypothesis formation, (2)experiment construction and (3)analysis of results. This may seem a scientific approach, but, ... in many regards instructionless learners are very poor scientists. They entertain mutually inconsistent hypotheses, they design primarily confirmatory and confounded experiments, they make some gross observational errors, and they form conclusions on the basis of inadequate evidence. Nevertheless, most subjects manage to master BigTrak in a reasonable amount of time. (Shrager & Klahr, 1986).
A more detailed description of the inferences people make is suggested by Lewis. His assumptions lack an experimental basis, but it is reasonable to expect that these or somewhat different lines of thinking are followed in human reasoning. Lewis (1986) concentrates on mental model construction in computer environments. He assumes a small number of simple heuristics is enough to infer a mental model of a computer program. He mentions: (1)identity heuristics, (2)loose-ends heuristics and (3)previous action heuristics.
Properties of causes are assumed to correspond to properties of their effects. People make causal inferences on the basis of these properties. This reasoning is based on identity heuristics.
The loose-ends heuristics assumes in an observation, every action contributes to accomplishing the goal. An unexplained action in the system can be tied to an unexplained response of the system.
The previous action heuristic attributes effects to the previous action. If an event follows an action immediately it is plausible that the action caused the event. The event that follows an action is seen as feedback of that action.

3.1.1 Conclusion
Lewis describes a natural approach of human reasoning which is not scientific, but works sufficiently well in everyday life. In the model of Waern it seems that human reasoning follows the lines of logic, but this is not necessarily true. Expectations of a system can be based on the heuristics of Lewis. People can hold the wrong expectations and form within the model of Waern a faulty mental model. Shrager and Klahr proved that the reasoning system is not scientific, but it works.

3.2 Instruction and mental models
Instruction consists of a set of events external to the learner, designed to support the internal process of learning (Gagnť, 1985). The design of these events may influence the forming of a mental model. The experiments of Shih & Alessi (1993), Kieras & Bovair (1984) and Mayer (1989) are not intended to discover an optimal instruction, but their design is meant to facilitate the forming of a mental model. Hegarty & Just (1993) varied the instruction in order to find the optimal design. Hegarty & Just (1993) used pictures and text to explain how a certain pulley system operated. They tried to find out how their subjects formed a mental model of the system. In an experiment 47 students studied diagrams alone (15), diagrams and text (15) or text alone (16). Each group was nearly equally divided in students with low mechanical ability and students with high mechanical ability.
Not surprisingly, the text-and-diagram group scored best on a posttest. Questions in the posttest about the configuration of the pulley system could not be answered directly from the text base. Questions about kinematics of the pulley system could not be answered from the diagrams alone. The score on the posttest indicated that the subjects had formed a mental model of the pulley system, because a significant amount of the questions was answered correctly.
The time subjects spent on the instruction showed another interesting result. The study times indicate that subjects spend less time studying a diagram alone than a text alone, although they attain equivalent levels of comprehension. This result is consistent with the view that a mental model of a pulley system is a representation in which the elementary units are representations of the basic components of the pulley system and these elementary units are organized spatially. (...) In contrast, the information about each component of the pulley system is distributed among several different clauses of the text so that subjects have to integrate the information in these different clauses in order to construct the representation of each component. Hegarty & Just (1993).
The direct instruction of expert models is not always an optimal one. Hong & OíNeil (1992) were able to detect intermediate mental models in the field of hypothesis testing. They claim that it is better to first teach a novice a mental model of an intermediate before teaching an expert mental model. To teach this intermediate model they looked at different instruction strategies: separate or combined presentation sequence and diagrammatic or descriptive presentation mode. In the separate presentation sequence conceptual instruction is provided prior to the procedural steps in hypothesis testing. In the combined sequence procedural and conceptual instruction are presented simultaneously. In the diagrammatic presentation, diagrams are presented, whereas in the in the descriptive presentation, instructional content is presented primarily verbally. The results of a posttest score are summed up in Table 4.

Table 4. Adjusted means and standard deviations of posttest scores of each educational level and of combined educational levels for treatment combinations.

Presentations sequence

Presentation mode
Separate
Combined
Mean of separate and
combined

M
SD
n
M
SD
n
M
SD
n
Diagrammatic
58.72
14.28
14
45.70
12.13
14
52.21
14.59
28
Descriptive
45.80
14.88
15
32.79
11.73
13
39.76
14.82
28

They conclude with two results. First, teaching conceptual instruction prior to procedural instruction in introductory statistical hypothesis testing significantly facilitated understanding the concepts and the procedures involved in hypothesis testing. (...) Second, instruction with extensive diagrammatic representation facilitated subjects development of representational ability for understanding the instruction and solving the problems by building diagrammatic mental models. Hong & OíNeil (1992).
These two result are consistent with the findings of Mayer (paragraph 2.4). The first result is also consistent with the conclusions of paragraph 2.5, on benefits of mental models.

3.2.1 Conclusion
Instruction designed to teach mental models is a fruitful approach. Using diagrammatic representations of the subject matter helps to facilitate understanding. A well designed instruction strategy will provide conceptual knowledge before procedural knowledge. An interesting question remains if it is better to instruct a expert mental model or a intermediate mental model. The answer depends probably on an analysis of the subject matter.

3.3 Discussion
An overall conclusion is that the use of diagrams is necessary in teaching a mental model. This is consistent with the ideas of Kosslyn, who described the depictive form of thinking. If people think in picture-like structures, it is efficient to provide these structures to them in instruction. A textual representation of knowledge will require a more difficult translation to a mental model.
Larkin & Simons (1987) theorize about two aspects of representing information: computation and information. Two forms of representation are informationally equivalent if they contain the same information. Information of one representation is also inferable from the other representation and vice versa. Two representations are computationally equivalent if they are informationally equivalent and if, in addition, any inference from them can be made with the same ease or difficulty. Larkins & Simon argue that textual and diagrammatic representations may be informationally equivalent, but that they differ in computational efficiency.
In general diagrams have the following advantages over textual representations (Larkins & Simon, 1987).

1. Diagrams can group relevant information, reducing the need for large amounts of search when trying to make problem-solving inferences.
2. Diagrams avoid the need to match symbolic labels by utilizing location information to group information about elements of the representation.
3. Diagrams support perceptual inferences which are extremely easy for human beings.


4. COMPUTER SIMULATION

This chapter deals with a specific medium for acquiring a mental model. Computer simulation makes it possible for a learner to build his own personal mental models, without being forced in an existing model of an expert.

4.1 Advantages
Computer simulation represents reality on a computer screen. Although this reality is always a bounded by the size of a computerscreen, it has a lot of advantages. Most lie in the fact that reality itself is not so easily studied. Reality may be too hot, too small, too far, too large, too wet or not observable; in short not easily accessible. A computer simulation can provide an relatively cheap and practical alternative.

In comparison with other media computer simulation provides the opportunity to be actively engaged in the learning process by influencing parameters. Computer simulation provides learner control and can be used within a personal learning strategy.

4.2 Classification
Computer simulation is a very broad term. In a traditional classification of courseware it differs from modeling and gaming. This classification is useful in practice, but the definitions are not mutually exclusive. The differences between the different classes are ill-defined, and it is accepted that one product may have characteristics of different types of courseware. A drill and practice may for example be a game. Products as simulation games or gaming simulations are also possible.

Table 5. Traditional classification of courseware.

drill and practice
tutorial
dialogue
inquiry
simulation
modeling
gaming
problem solving

For the purpose of this paper it is enough to distinguish between modeling and simulation (Min, 1995). Modeling is a process in which the student is the designer of a model. He determines the variables that are relevant and the way in which they influence each other. In a simulation this model is pre-determined. The student can influence some variables, but not the relations between them.

To classify simulations Gredler (1986) introduced a taxonomy based on (1) the use that is made of graphics and other visual displays and (2) the nature of the instructional process that the exercise represents. These are two important themes but their components are vaguely described. For example, the use of visuals in the lesson presentation are divided in : (1) graphic (or visual) demonstration of a specific item (2) visual presentation of a real- world problem (3) graphic or tabular presentation of a set of data. No distinction is made between different graphical representations. Based on her criteria she distinguishes four different types of computer simulation.

Table 6. Four types of computer simulations according to Gredler (1986).

1.structured questions and graphics simulation
2.variable assignment exercises
3.diagnostic simulations
4.group-interactive

The first simulation may also be a drill and practice, or tutorial. The student views simulated situations and answers specific questions. In the second simulation a student assigns values to a discrete number of variables; the same decisions are executed over and over. The third simulation presents a student with a realistic problem and engages in sequential decision making. The optimum strategy is typically derived from an established area of expertise. In the last simulation students are presented with a community or international situation and develop plans and strategies to solve problems and/or to meet goals (Gredler, 1988).
A last classification that is mentioned is from Alessi and Trollip. They distinguish four categories in computer simulation.

Table 7. Four main categories of computer simulations according to Alessi & Trollip (1985)

1.physical simulations
2.procedural simulations
3.situational simulations
4.process simulations

A physical simulation allows a student to interact with a complicated, difficult-to-obtain or expensive machine or piece of equipment. Examples are airplanes, a laboratory or a plain calculator. In a procedural simulation, a physical system is often simulated, but the aim of the simulation is to acquire skills and actions needed to operate it. An important subset of procedural simulations is diagnosis simulation. A student in this category of simulations has to diagnose a patient, an electronic circuit or an unknown substance. In a situational simulation is a role-playing situation, in which attitudes and behaviours of students are important. In a process simulation a student selects variables at the beginning of the simulation and then watches the process to occur without any intervention.

Different categorizations of simulation focus on different aspects of a simulation. Alessi & Trollip are looking primarily at the role of the learner in computer simulation. Gredler looks at characteristics of the program itself; how it uses visuals and interaction.

4.3 Model driven computer simulation
In this paper we are interested in computer simulation that has an underlying mathematical model. Min (1995) calls this "model driven computer simulation". Reigeluth & Schwartz distinguish three different aspects in this type of simulation: an underlying model, a scenario and an instructional overlay. These aspects will be shortly addressed.

4.3.1 The model
Any system (A) that is used to obtain information about another system (B) is a model (of system B). (Bertels & Nauta, 1976). Designing system (A) to obtain information about system (B) is what Min calls modeling. Joolingen & deJong (1991) see three different systems that can be modeled: physical, artificial and hypothetical or abstract systems.

A physical system is a system that exists in the natural world. Social and biological systems are also considered as physical systems. Because this are almost always very complex systems, it is impossible to model them completely. An other complicating factor is that physical systems have to be based on observation and observation is always biased. This is not a fundamental problem because it would be impossible to implement a complete model of a physical system on a computer (it would be too large). One should strive for a model with a high validity, which means that outcomes of the model correspond to reality. This is important for the student because he must be able to link the simulated content to his experiences.

An artificial system is a system that is created by humans. The difference with a physical system is that the system is completely known. Examples are machines, or artificially created situations (like a post-office). Of course every process is only known to a certain degree. If the system is considered at a deeper level with more details it will become unknown, but this level of consideration doesn't always make sense.
Hypothetical systems have no counterpart in the real world. It is therefore not possible to validate them. A good example is a frictionless world to demonstrate the impact of force. It is sometimes difficult to make a difference between an artificial system and a hypothetical system, for example in the case of economic system. But it is possible to validate models of economy and therefore they are artificial systems (Joolingen & deJong, 1991).

A model of a system is nothing more than a set of input and output quantities , with a certain input-output behaviour (Zeigler, 1976). If variables in the model are influenced by time, a model is called dynamic. If variables can be changed back and forth, independent of time a model is called static.

Another distinction is qualitative versus quantitative modeling. A quantitative model contains variables which can have different numerical values. A qualitative model describes input or output in some predetermined states.

Variables in a quantitative model can be discrete or continuous. Most of the time a model contains both types of variables. These models are called mixed models. Pure continuous or discrete models only occur as small models and, more importantly, as submodels of a larger system (Joolingen & deJong, 1991).

A qualitative model can be abstraction-based or quality-based (Fishwick, 1989). In a abstraction based model the causality of a system is modeled. An example of abstraction based models is an electronic circuit in which only properties as on and off and connected or not connected are taken into account.

In a quality-based model quantitative properties are qualitative modeled. For example: a function may have the property monotonically increasing, or the voltage is oscillating. A point of discussion are models with a quantitative input and a qualitative output. In this case a translation from quantitative to qualitative data is necessary. These models are still called quantitative models. (Joolingen & deJong).

Model-driven computer simulation is based on a qualitative model, with qualitative input and output. The system that is modeled can be artificial as well as physical or hypothetical.

4.3.2 The scenario
With a scenario Reigeluth and Schwartz (1989) mean the way in which the situation is represented to the learner. This can have an influence on the motivation of the learner. Alessi & Trollip (1985) clarify this point with a clear example.

For example, informing students that a lesson is about the Civil War is not likely to excite them. For may students this would conjure up memories of history books filled with dry facts about the war, and dates to be remembered. However, if the lesson states, You will play the role of adviser either to General Grant or General Lee. You will help make decisions about purchasing weapons, food, medical supplies, and about strategy, which will effect the outcome of the war, the student is likely to be more interested in pursuing the lesson. (Alessi & Trollip, 1985, p. 177).

4.3.3 Instructional overlay
With instructional overlay Reigeluth & Schwartz mean the element that are added to a simulation to improve the learning process. The actions they suggested can be categorized on two dimensions. The first is directive versus non-directive and the second is obligatory versus non-obligatory. (Njoo, 1994).
Directive measures take away some of the freedom of a learner and direct him in a certain way. Two examples are; providing hints and the implementation of progression in model complexity. Non-directive measures do not guide learners in a specific direction, but are meant to improve the thinking of the learner. An example is a hypothesis scratchpath, on which learners are able to keep track of their hypothesis.
The dimension obligatory versus non-obligatory deals with the question of learner-control versus program-control. The question is who (or what?) is going to decide what is going to happen next. A disadvantage of taking away learner control is that the it devaluates the character of an exploratory learning environment. A disadvantage of learning control is that a learner may not use it properly.
An important feature in the instructional overlay is feedback. Reigeluth & Schwartz distinguish natural an artificial feedback. Natural feedback is a real-life consequence of the actions a students takes in a simulation. Therefore it is not a part of the instructional overlay. Artificial feedback is an invented consequence which would not occur in real-life situations. Artificial feedback is necessary in complex situations. It may be motivational or informational.
A last feature that is mentioned is help. Help provides the learner with assistance during the simulation. Reigeluth & Schwartz mention three types of help. The first directs attention to some important aspect on the screen by using color, arrows, labels etc. The second provides commentary to link an example to a generality. The last one facilitates encoding by providing alternative representations such as a diagram, along with a definition.

4.4 The learning process
There are several approaches on how people learn when interacting with a computer simulation. Goodyear et. al. (1991) list: problem solving, discovery learning, inductive learning. These are all forms of exploratory learning. Goodyear et. al. provide a multi-
leveled description model of this form of learning, based on these approaches. Each level is a refinement of the previous one. Only processes that transform domain information into knowledge are described. This means that processes such as monitoring, planning, checking etc. are not included in the scheme. The model is not concerned with the instructional environment of a computer simulation, but only attends to the processes occurring when a learner interacts with the model of the computer simulation.

Table 8. Description scheme of simulation learning processes.

Level 1
Level 2
Level 3
Level 4

Orientation
Observation
Encoding

Retrieving

Analysis
Encoding

Retrieving

Model exploration

Hypothesis
generation

Test
Design experiment

Prediction

Manipulation

Interpretation

Evaluation
Observation

Generalization

The orientation phase is intended as an analysis in which a learner takes bearings on, amongst other things, the relations within the model or the learning goal(s). The hypothesis generation phase implies the formulation of a descriptive generalization or assumption about model relations. In the test phase the learner is able to test the hypothesis by the design of an experiment, prediction, direct manipulation of variables and parameters and interpretation of data. In the evaluation phase the learner has to give her/his understanding of the meaning of the output. For example, the learner can relate the outcome to the hypothesis or to the learning goal(s). This can result in a modification or alteration of the hypothesis. Goodyear et. al. (1991, p. 273)

A detailed description of the different phases is not provided by Goodyear et. al. It is obvious that the model is similar to that of Waern and Shrager & Klahr (paragraph 3.1). The learner is viewed as a scientist that conducts experiments to test hypothesis. This places high demands on the learner. He will need sufficient prior knowledge of the domain, skill in organizing and monitoring the learning process, attentional capacity to deal with information without getting overwhelmed, and motivation to see it through. Remembering the heuristics mentioned by Clark (paragraph 3.1), the learner must also apply a scientific way of hypothesis testing. If this is not the case, the use of common sense reasoning heuristics may lead to false beliefs about the model to be taught.

4.5 Learning obstacles
The expectations for learning with computer simulation have been high. Computer simulations would be fun, inexpensive, safe, realistic, facilitate better transfer, cause less threat and anxiety, teach critical thinking as well as content, encourage socialization and collaboration, and use learning time efficiently (Willis et. al., 1987, p50). Sometimes, some of these positive claims received evidence from empirical research. On the whole, the evidence for these beliefs is mixed (Woodward et. al., 1988). Some authors are even more negative.

Why have the results of research on simulation-based learning been disappointing? Why has the impressive list of claims not been corroborated by an equally impressive list of results? (Goodyear et. al., 1991)

In a discussion on learner control in learning environments Kinzie (1990) indicates three possible reasons why learning environments with much learner control show inconsistent learning results. These are:

* behaviorist versus cognitivist orientation
* individual student differences
* discomfort in exercising control options

In a behaviorist orientation instruction is highly directed, divided in short steps, a lot of practice and feedback, and nearly identical elements in both instruction and testing situations. With such a design, learning is more near transfer. Instruction based on a cognitive design allows student control over direction and monitoring of the learning process. This instruction is presented in larger chunks and the learning outcome is long- range achievement and continuing motivation to learn.

If a behaviorist test is used in a cognitive design, learning outcomes as motivation and far transfer will not be detected. Second, students who always receive very structured instruction may not be able to develop the management capabilities for their own learning, and thus fail to learn in a free exploration situation. The last point Kinzie makes is that an emphasis on achievement alone may damage students continuing motivation to learn.
A second reason for inconsistent learning results may be individual student differences. Research (Steinberg, 1989) indicates that students with more prior knowledge and the ability to direct their own learning benefit more from learner control than students with less prior knowledge and no management capabilities.

The last reason is discomfort in exercising control options. The interface of a learning environment may be confusing or create a negative attitude, because it is slow or childish. This may interfere with the learning results.

According to Goodyear (1991) the main reason for dissapointing results are the high demands placed on the student. A student needs sufficient prior knowledge, skill in organizing and monitoring the learning process, attentional capacity to deal with the simulation without getting overwhelmed and motivation to see it through. This view is support with the results of an experiment of Doerner.

Doerner (1980) studied the difficulties people had with a simulation of a city, called Smithtown. Subjects had to decide on taxes, city administration, shops, etc. The variables on which there competence was judged were variables like the number of unemployed people, the contentment of the inhabitants and the production in the city. They subjects were studied during 8 trials, but Doerner doesnít mention the number of subjects.

Almost all participants made three primary mistakes. The first is that they were more interested in the status quo, than in development over time. For example, they wanted to know how much money the city owned at the moment, and didnít care about the development in recent years. A second mistake was the inability to deal with exponential functions. Subjects didnít have any intuitive feeling for processes which developed exponentially. They were astonished when they saw results that they could have foreseen if they would understand exponentially growing curves. The last general mistake is the tendency to think in causal series instead of causal nets. In a complex system as Smithtown, an action has almost always more than one effect. People tend to oversee the effects they didnít intend. Some were able to adjust their thinking, but others didnít. Doerner (1980) is particularly interested in this last group of bad subjects. Unfortunately Doerner doesnít describe any characteristics of these subjects.

Bad subjects show an interesting behaviour, which is relevant to know for a designer of simulations. The cause of their behaviour is fear for a loss of control, and for failure, which leads to positive feedback loop in which failure, fear and loss of control increase each other.
Their behaviour consists of (a)thematic vagabonding, (b)encystment, (c)decreasing willingness to make decisions, (d)tendency to delegate and (e)exculpation tendency.

(a) Subjects jump from one topic to another. When they are having difficulties they try to escape, so they donít have to face their own helplessness more than necessary.
(b) Subjects show opposite behaviour as (a) thematic vagabonding. They feel comfortable in an area which is easy to manage. They stick to these areas, although these easy topics are the least important.
(c)Doerner counted the decisions subjects made and found that good subjects made more decisions in every trial, while bad subjects decreased the number of decisions after the fourth trial. Bad subjects make in every trial less decisions that good subjects.
(d)The responsibility for decisions is delegated. Subjects claim for example that The head of the housing department should worry about that!, while resources for this head were not made available.
(e)The last finding is that subjects try to find external reasons for their incompetence. They blame the designer for an impossible design, or the (simulated) management abilities.

If bad subjects face a situation in which they feel out of control they show a reduction in the number of (f)self reflections, (g)a reduction in the number of plans, (h)increasing stereo typing and (i)decreasing control over the realization of plans. This is what Doerner (1980) calls an intellectual emergency reaction. In an emergency situation there is no time to think (cf. a near car accident). Bad subjects show the same reactions, now their feeling of competence is threatened.
(f)Bad subjects do not often recapitulate and analyze their own past actions and thoughts. (This makes improvement difficult.) Good subjects have an average of 6.5 phases of self reflection, bad subjects have an average of 2.5 of these phases per trial.
(g)Making plans is an important factor for systematically investigating a simulation. Bad subjects have an average of 1 plan per trial, good subjects make on the average 3.8 plans.
(h)Subjects keep on thinking along the same lines over and over again, without noticing. This is a indication of reduced self-control.
(i)Finally bad subjects do not check whether their plans are realized or not.The intellectual emergency reaction has a consequences for the behaviour of subjects. They (j) increase risky behaviour, (k) they increase violation against the rules and there is (l) an increase in the tendency to escape.
(j)Doerner finds that subjects want to master the situation at any price. The are prepared to increase the level of risk taken, if they previously have experienced failure.
(k)Subjects are willing to break rules and regulations. They do not care whether their behaviour is allowed of forbidden.
(l)Subjects take longer breaks and try to talk to the experimentor about topics that have nothing to do with the simulation.
A last research finding is the way of hypothesis formation of bad subjects. (m)Hypothesis are more global, (n)subjects donít search for falsification and (o)goals become less concrete. This is not due to the intellectual emergency reaction, but because one cannot handle the amount of information.
(m)The hypothesis contain fewer and fewer causes for more and more effects. Subjects do not form a net of causality, but try to find one cause. Reductive hypothesis are attractive because they encourage the feeling of understanding.
(n)Subjects do not notice evidence against their hypothesis and are therefore able to keep it as long as they want.
(o)The goals subjects state become more global and hence difficult to measure. It is a last mechanism to protect themselves from a feeling of incompetence.

The mechanisms Doerner (1980) describe are arguments against computer simulation. It is a pity that he doesnít describe characteristics as education, age and profession of his subjects. It is however clear that optimistic ideas about computer simulation as a medium are quite naÔve. If students are bad subjects there is no way they will benefit from computer simulation. The only outcome is that they feel frustrated and incompetent.

The mechanisms that Doerner described are important to consider. It is not plausible that comfort with learner control, or a cognitive orientation will change these mechanisms. It is clear that computer simulation should only be used with learners who are can face very high demands. Doerner also provides an answer to the question why claims about computer simulation have not always come true.

5. COMPUTER SIMULATION AND MENTAL MODELS

In this chapter two topics will be related; computer simulation and mental models. In computer simulation research different items have been addressed. In this chapter each item will be discussed in the light of mental models.

5.1 Prerequisites
Model driven computer simulation is not commendatory for every learner. Learners must be able to manage their learning and to make proper inferences. They have to have enough relevant prior knowledge. If this is not the case, other forms of instruction should be considered. A feature of computer simulation is that it places a high demand on the learner. Mental model theory does not influence this finding.

5.2 Learning goals
Mental model theory makes it possible to define the goal of learning with computer simulation: computer simulation should facilitate the creation of a mental model. Then computer simulation aims at far transfer and understanding, rather than near transfer and remembering. The model a learner acquires, can be compared with the one of an expert.
A problem may be to explicate the mental model to be learned. The explication of a mental model leads to a conceptual model (paragraph 2.1). Sometimes this model is implemented in a computer program. It is obvious that we don't expect a student to think like a computer and follow hundreds lines of code or compute several equations. The mathematical model underlying a simulation is not appropriate to serve as a learning goal. We rather expect a student to form some kind of qualitative model. It is difficult to explicate this model:
"However I have not been able to find an explicit description of the type of models that the student should build. Apparently the student is not expected to build the same mathematical model that is implemented on the computer when this model consists of, say, 20 or even 200 differential equations." (Hartog, 1989).
Hartog supports the view that the model a learner has to form should be the same as the model implemented on a computer.
"But if the student is supposed to infer from a few runs of such a complicated model a much simpler model, it is unclear why this simpler model is not offered to the student in a more direct way." (Hartog, 1989).
In my view this is a simplification of the way in which people think. The model implemented in a computer differs from the model in the human brain. It is clear that people think fundamentally different than computers "think". But we do not exactly know how the brain works. Therefore, it is not possible to implement a mental model on a computer, because no one knows how a mental model "works" or can be formalized (besides, a mental model can differ from person to person). It is at this point that the difference between proposistional and depictive thinking becomes important. If people think as production systems simulate, there is no reason to abandon the belief that the model to be thought should be the same as the model in a computer.
The problem of explicating learning goals as mental models stays problematic. But the inferences one can make of a mental model are measurable. It is in this property that should be central in testing situations, and thus correspond with learning goals.

5.3 Tests
If the goal of learning with computer simulation is to acquire a mental model of a particular system, a test should measure this goal. Sasse (1991) used five different approaches for measuring a mental model of a word processing program:

1. observing users
2. asking users to explain the program
3. asking users to predict program behaviour
4. asking users to describe the program
5. observing users learning a program with a co-learner

Her conclusion is that approaches based on a teach-back, constructive interactive technique (methods 2 and 5) are most effective. Her research was not meant for designing tests. Swaak (1995) tries to find a way for testing knowledge that is implicit. She designed a number of WHAT-IF tests. Students had to predict which of the state representations (diagrams as well as texts) was the correct one after a certain event had happened. An important measure was the speed in which they were able to perform this task. After three pilot-tests she concludes:

... the three tests constructed with WHAT-IF items were able to tap an improvement in learning in all three pilot studies in all variations of discovery learning environments. (Swaak, 1995).

In the experiment of Kieras & Bovair (1984) students with a correct mental model were able to come up with short-cuts for controlling a device.
Concluding, all these alternatives have one common factor: insight. The acquisition of a mental model should be measured by tests that determine whether the student understands the system, if he is able to imagine behaviour, to explain it to others and to find better approaches to the subject matter. Insight and understanding are difficult concepts, which will not be discussed in this paper. 5.4 Learner control
An important design decision is the amount of control a learner gets over a simulation. At one end of the continuum, simulation becomes almost drill and practice, at the other end simulation is practical modeling. Learner control makes computer simulation different from media as video or texts.

Giving a student much control seems to have a lot of advantages. The student can follow his own learning strategies and interests. This can lead to ongoing motivation and optimal use of a simulation. It is in line with the view that a mental model differs from person to person. In the reviewed research, however, giving a students much control is not beneficial at all. They don't follow any strategies and they exit the simulation before they have learned all about it.

Reducing learner control has the disadvantage that the unique qualities of computer simulation are not fully used. Why not make a drill and practice in the first place? In mental model theories there are no arguments in favor or against computer simulation. Mental model theory is medium-independent. It is possible to form a mental model from text and diagrams as well as from animation or experience. Arguments for computer simulations (and learner control) have to be found in different areas of research.

If computer simulation is chosen, mental model theory does provide a simple model of learning processes a designer has to consider, especially when a lot of learner control is provided. Learners tend to follow the heuristics of Lewis (1986). The heuristics describe when students assume a causal relation between different events (paragraph 3.1). It is clear that this reasoning process may lead to misconceptions. The experiments of Shrager & Klahr with Big Track, lead to more positive results. The every-day heuristics of people are good enough to figure out the working of a device.

5.5 Progression of complexity
A simulation is usually quite complex. To confront a learner with this complexity can cause a cognitive overload and demotivation. The best design is usually one which begins with only one or two variables in the model and progresses by levels to include all important variables in the simulation at the end of the instruction (Reigeluth & Schwartz, 1989).
Hong & O'Neil (1992) have found an intermediate mental model in the understanding of statistics. It is reasonable that a novice is not ready to capture a mental model an expert possess. Therefore it may de appropriate to build a computer simulation that teaches an intermediate mental model, before it teaches an expert mental model. A guideline for design would be to take in account the mental models students have.
The progression of complexity in a computer simulation can be realized in a number of ways. Four different approaches (which can be combined) are according to Towne (1995):

top-down decomposition
bottom-up construction
function-driven exposition
qualitative to quantitative

In a top-down decomposition the student is presented with a total view of a system. The different parts can be reviewed in detail, until a level is reached that satisfies the learning objectives. A bottom-up construction the student starts with the parts and builds up a total view of the system. This approach might not be successful to show the ultimate system design and functionality, but the amount of information a student acquires is controlled. Some combination of top-down and bottom-up progression schemes are probably the best.
A function-driven exposition initially only displays those elements that are necessary to model the primary functions of the system. The elaboration continues with the adding of elements that are not used in primary functions and elements that are only used in less typical conditions.
The last model progression is from qualitative to quantitative. In this exposition scheme a learner first acquires a general idea of the model, before it can be explored in detail.
In important outcome of the experiments of Mayer (2.4) is that presenting conceptual knowledge before presenting procedural knowledge improves learning results. In the classification of Alessi and Trollip (1985) this means that a physical simulation has to be presented before a procedural simulation. This simulation can be the same simulation only the goal of learning between these two differs. In the classification of Towne this would mean a transition from qualitative to quantitative. However, Mayer conducted his experiments with text and diagrams.

5.6 Visualizing data
The problem of choosing the right visualization for information, may be a little clarified with the distinction of Larkins and Simon (1987) about informationally and computationally equivalent representations. Although they only apply this distinction to diagrams and texts, it is also possible to see these two aspects in animation, video or graphs. Informational equivalence is a rather objective measure. It may be possible to determine whether two representations differ in information. Computational equivalence, however, is a subjective measure that depends on the target audience. For some highly motivated college students a text and an animation might be computational equivalent, while the same text and animation may make a lot of computational difference to people who are not used to read. Experiences with the medium may account for this difference. The influence of videoclips, for example, may decrease the computational power highschool students need to understand video-
images.
From experiments of Mayer (1989) it is clear that the use of diagrams is beneficial for the building of a mental model. In computer simulations, a graphical representation has also a temporal dimension: it can change in time. A mental model can also change in time, so one can argue that the correspondence between mental models and computer models can be increased by using the temporal dimension of diagrams.
In an experiment of Williamson and Abraham (1995), 124 college students received the same text presentations with pictures, verbal explanation and statistic diagrams. One part of the group received an additional animation of the behaviour of molecules, and another part also participated in a scheduled discussion session. With two standardized test (for two different lesson contents) the differences between the three groups was measured:

Table 9. Results on two tests (PNMET 5 & PNMET 7) from three different groups, results indicate a beneficial effect of animation. (Williams & Abraham, 1995)

TEST

PNMET 5
PNMET 7
GROUP
count
mean
st. dev.
count
mean
st. dev.
control
48
6.438
2.268
54
3.481
1.788
lecture/animation
41
7.707
2.205
38
4.421
1.588
lecture/animation/discussion
35
7.657
2.127
32
4.531
1.796

Williams and Abraham conclude:
1. Treatment with animations may increase conceptual understanding by prompting the formation of dynamic mental models of the phenomena. The dynamic quality of animations may promote deeper encoding of information than that of static pictures. (...) Animations, which could be viewed as dynamic pictures may trigger the formation of deeper coding, thus more expert-like mental models of the phenomena.
2. Students who viewed static visuals such as transparencies or chalk diagrams may have (a) formed static mental models that failed o provide adequate understanding of the phenomena or (b) failed to form a mental model of the particulate nature of matter, and instead, were left with macroscopic views of the phenomena. (Williamson and Abraham, 1995)


6. EPILOGUE


The main conclusion of this paper is that there are no easy answers in designing computer simulations. Decisions about visualization, model complexity, and learner control will depend largely on the talents of the designer. With the concept of a mental model in mind, a designer may be able to improve his solutions. The mental model research does not provide clear heuristics that can directly be applied to computer simulation. Experiments described in the literature only apply to a small group with a lot of prerequisites. Important variables are learner characteristics and the structure of the subject matter.
Mental model research cannot provide answers on the question of the language of thought. Mental model research does not explain, but only indicates some processes in cognition. It is an approach that is followed in many areas of research.
In the literature on computer simulation the early enthusiasm and grotesque claims, are replaced by the concern about the demands that are placed on the learner. Learners are not used to think and learn on their own. Maybe this is an outcome of the current educational systems. My thesis is: The success of computer simulation in an educational system is a measure for the academic skills and student motivation."
A solution for the poor learning results is the addition of instruction. In my view more attention for motivating factors and progression of model complexity is a better solution.
Finally, in writing this paper I have encountered many interesting topics of research such as imagery, understanding, computer interfaces, and gaming. It is possible to connect all these themes with the concept of mental models or computer simulation. It was of course not possible to investigate all these areas in depth. I hope that the selection of these topics in this paper, will inspire designers of computer simulation to improve their simulations in a motivational manner.


7. ACKNOWLEDGMENTS.

Several drafts of this paper were reviewed by Marcel Gmelich Meijling and Jeroen Koffijberg. This improved the quality of the paper significantly. Also the enjoyable conversations with dr. Rik Min sharpened my view on computer simulation. Finally Origin/IMCC supported me with a fast computer and a pleasant work environment. I would like to thank all these people for their support.


8. REFERENCES


Anderson, J.R.(1987). Skill acquisition: compilation of weak-method problem solutions. Psychological Review 94(2), pp192-210.

Alessi, S.M. & Trollip, S.R. (1985). Computer-based instruction, methods and development. Englewood Cliffs, NJ: Prentice Hall.

Bainbridge, L. (1992). Mental models and cognitive skill: the example of industrial process. In Y.Rogers, A. Rutherford & P.A. Bibby (Eds.), Models in the mind. Theory, perspective and application (pp 119-144). London [etc.]: Academic Press

Bertels, K. & Nauta, D. (1976) Inleiding in het modelbegrip [introduction to the model concept] . Bussum: De Haan.

Craik, K. (1943). The nature of explanation. Cambridge: Cambridge University Press.

Fishwick, P.A. (1989) A study of terminology and issues in qualitative simulation. Simulation, 52, pp 5-9.

Gagnť, R.M. (1985). The conditions of learning. New York: Holt, Rinehart and Winstion.

Gentner, D.,& Gentner, D.R. (1983) Flowing waters or teeming crowds: mental models of electricity. In D.Gentner & A.L. Stevens (Eds.), Mental Models. Hillsdale, NJ: Lawrence Erlbaum Associates.

Goei, S.L. (1994) Mental models and problem solving in the domain of Computer Numerically Controlled programming. Wageningen: Ponsen & Looijen.

Gourgopoulos, A.P. (1995) Visual perception. Science vol 270 .pp 320-321.

Gredler, M.B. (1986) A taxonomy of computer simulations. Educational Technology. vol 5, p7 -12

Hartog R (1989). Qualitative simulations and knowledge representation for intelligent tutoring. In H. Maurer (Ed.) Computer Assisted Learning Proceedings of the 2nd International conference ICCAL pp 193-213 Berlin: Springer Verlag

Hegarty, M. & Just, M.A.(1993). Constructing mental models of machines from text an diagrams. Journal of Memory and Language 32 , pp717-742.

Hong, E. S. & OíNeil, H.F.(1992). Instructional Strategies to Help Learners Build Relevant Mental Models in Inferential Statistics. Journal of Educational Psychology, vol 84 (2) pp150-159.

Jonson-Laird, P.N. (1980). Mental Models in Cognitive Science. Cognitive Science 4, pp 75 -115.

Joolingen van, W.R. & Jong de, T.(1991) Characteristics of simulations for instructional settings. Education & Computing, vol 6, pp 241-262

Kieras, D.E., & Bovair, S. (1984) The role of a mental model in learning to operate a device. Cognitive Science 8, 255-273.

Kinzie, M.B. (1990) Requirements and benefits of effective interactive instruction: learner control, self-regulation, and continuing motivation. Educational Technology, Research and Development vol 38 (1)

Kosslyn, S.M. (1973). Scanning visual images: some structural implications. Perception and Psychophysics 14: 90-94.

Kosslyn, S.M. (1994). Image and brain, the resolution of the imagery debate. MIT Press: Cambridge, Massachusetts.

Lewis, C.A. (1986). A model of mental model construction. In M. Mantei & P. Orbeton (Eds), Human factors in computing systems: CHI Ď86 proceedings. New York : Association for Computing Machinery.

Mayer, R.E.(1989). Models for understanding. Review of Educational Research vol 59, no 1 pp 43-64.

Newel, A. & Simon, H. A. (1972). Human problem solving. Prentice-Hall, Englewood Cliffs.

Njoo, M. & DeJong, T. (1993). Supporting exploratory learning by offering structured overviews of hypotheses. In D.M. Towne, T. de Jong, H. Spada (Eds), Simulation-based Experiential Learning. NATO ASI Series, Series F: Computer and systems sciences vol. 122. Berlin: Springer-Verlag.

Njoo, M. K.H. (1994).Exploratory learning with computer simulation: learning processes and instructional support. Doctoral dissertation. Eindhoven: Technisch Universiteit Eindhoven.

Norman, D.A. (1983). Some observations on mental models. In D. Genter & A.L. Stevens (Eds.), Mental Models. Hillsdale, NJ: Lawrence Erlbaum Associates.

Payne, S.J.(1988). Methods and Mental Models in Theories of Cognitive Skill. In Artificial Intelligence and Human Learning. J. Self (ed.), New York ,NY: Chapman and Hall.

PPhylyshyn, Z. (1994). Mental pictures in the brain. Nature, vol 372. pp 289-290.

PReigeluth, C.M. & Schwartz, E. (1989) An instructional theory for the design of computer-based simulations. Journal of Computer-Based Instruction, vol 16 (1) pp1-10,

Schaubel, L., Glaser, R., Raghavan, K., Reiner, M. (1991). Causal models and experimentation in scientific reasoning. Journal of the Learning Sciences 1 (2). 201-238.

Shih, Y.F. & Alessi, S.M. (1993) Mental models and transfer of learning in computer programming. Journal of Research on Computing in Education v 26 2 (pp155-
175).

Shrager, J. & Klahr, D.(1986). Instructionless learning about a complex device: the paradigm and observations. International Journal of Man-Machine studies v 25 153-189.

Steinberg, E.R. (1989) Cognition and learner control: a literature review, 1977-
1988. Journal of Computer-Based Instruction, vol 16, (4) pp 117-121

Stillings, A.N., M.H. Feinstein, J/L. Garfield, E.L. Rissland, D.A. Rosenbaum, S.E. Weisler, L. Baker-Ward (1987) Cognitive science: an introduction. MIT Press: Cambridge, Massachusetts.

Streitz, N.A.(1988). Mental Models and metaphors: implications for the design of adaptive user-system interfaces. In Artificial Intelligence and Human Learning. J. Self (ed.), New York ,NY: Chapman and Hall.

Swaak, J. (1995). Development of tests for intuitive knowledge in formal domains: the WHAT-IF test. IST-memo. Internal paper. University of Twente.


Waern, Y (1990). On the dynamics of mental models. In Mental models and human-computer interaction 1. Ackermann, D & Tauber, M.J. (Eds). Elsevier Science Publishers. Amsterdam.

Willis, J., Hovey, L., Hovey, K.G. (1987). Computer simulations, a source book to learning in an electronic environment. Garland Publishing: New York.

Young, R.M. (1983). Surrogates and mappings: two kinds of conceptual models for interactive devices. In D. Genter & A.L. Stevens (Eds.), Mental Models. Lawrence Erlbaum Associates: Hillsdale, NJ.

For reasons of style only, a masculine form of denotation is used throughout this paper. Big Trak is a product of the Milton Bradley Corporation