Instructional design for computer simulations

By Wytze Koopal, Enschede, March 1993 (gereviseerd 14 okt. 1997). Supervision: R. Min & J. Moonen

Literature review for the Faculty of Educational Science and Technology of the University of Twente

Preface

To graduate in Instrumentational Technology at the Faculty of Educational Science and Technology I have conducted three activities: an assignment in the design and development of educational software, an internship in a company which produces educational software and a literature survey on the subject of educational computer simulation and instruction. The result of this last activity can be found in this report.

1. Introduction

This report describes and synthesizes literature that was surveyed on two main topics: instructional design theory and educational computer simulations. The literature survey has been done with a problem definition and a search method. These subjects will described in this section.

1.1. Problem definition

The literature survey which was done in order to write this report started with the following problem definition:
This problem definition was split up in several smaller units that focus on one aspect of the problem definition. By using a framework that split up the problem definition in several units it was hoped for that the survey would become more manageable.
These smaller units have been formulated as questions. The answers to these questions constitute the sections that make up this report. The questions were:
To these four questions answers have been sought in the literature. They are presented in the following sections of this report.
Once it was clear what constituded the different aspects of the problem definition, an analysis of the problem definition could be done. So, in the last section of this report, an answer will be formulated to the problem definition.

1.2. Search method

The search began with several talks with the supervisor of this survey. He gave several hints in which directions to search. Also some key publications were made available. With the problem definition in mind a list of possible key words was made.
Literature has been sought with the following key words: Computer Simulation, Simulation, Simulation Based Learning, Computer Assisted Learning, Computer Based Instruction, Instructional Strategies, Instructional Design, Instructional Design Theory, Educational Software, Instructional Software, Learner Centered Methods, Learning, Learning Technology and Learning Theories.
The literature search was done in May 1992. It was conducted along several lines.
The first way of searching was a search through different collections of abstracts with all of the above mentioned keywords. The goal of this search was to find recent publications in the field of interest. Literature was sought in the British Education Index of 1991 and 1992, in Contents Pages in Education of 1991 and Educational Technology Abstracts of 1991 and 1992. This gave 86 possible useful publications. After a quick scan this was brought back to less then 20 publications. These were all articles that were published in journals or magazines.
Another search was done with TODOK (the automated online catalog of the library of the Faculty of Educational Science and Technology). This was done to scan if there were sourcebooks or textbooks on the subject of this survey that were published some time ago. A few books (8) seemed interesting.
A third search was performed in the online catalog of the library of the University of Twente. A search with 'Computers' as the title and 'Simulation' as keyword already gave 66 hits. A search with 'Computer Simulation' as title gave 14 hits. The useful publications were few. Overlap with the previous search was also noticed.
A last search was performed with the ERIC CD-ROM. This was done with the keywords Computer Simulation, Instruction, Learning and Instructional Design. This gave 24 possibly useful publications. Four of them were considered useful. Here the search was stopped.

Because the writing of this report took place over several months, some effort was devoted to continue the search on a lower level during this time. So during the writing of this report several magazines and journals (e.g. Educational Technology, Journal of Computer-Based Instruction, Instructional Science) were periodically scanned to see if there were any interesting new publications. Also magazines in the Dutch language were scanned. This last activity was useful as it turned out: two articles were found.

2. What instruction is

To answer the question 'What is a chair?' someone can have two distinct approaches. One can give a straightforward answer naming all the necessary attributes and characteristics ('if this and this..... is available, then this is a chair'). Or one can give an answer in a not so straightforward way in which attributes and characteristics are named that are considered to have some links with the concept under consideration.
The last phenomenon can have several reasons, of which two are most likely and which are also closely linked to one another: an exact definition is not exactly known and/or it is thought not to be necessary to define exactly what is meant by the word or phrase. It is assumed that the readers know what is dealt with.
In the literature the predominant way in which 'instruction' is characterized and defined is not the straightforward way. To find literature dealing mainly (let alone solely) with the subject of instruction is difficult. But a lot of publications have something to say about instruction.
As the goal of this section is to present a workable definition of the word 'instruction' both ways of defining 'instruction' have been analyzed to get the best view possible of what instruction is.

2.1. The not so straightforward way

For this paragraph publications have been analysed that deal with other subjects than instruction, but in one way or another eventually also have something to say about instruction. This is literature on the subject of learning with or through new technical possibilities, but also literature on the subject of instruction(al) theory and models.
In no way has this analysis been complete, because the literature dealing in one way or another with instruction is very large. The selected writings can only serve as examples showing what the literature which is not directly concerned with instruction, is like.

2.1.1. About learning and new technical possibilities
The literature on 'learning' is large and has a long history. Most of these publications deal solely with learning. But some also deal with learning in the context of new technical possibilities. Some of these writings have been closely analysed.
In these writings the word instruction is used without explicitly stating what is meant by it. The writer assumes that the reader knows what instruction means. Or the writer implicitly lets the reader know what instruction in his or her opinion is or is not.

Duchastel, Brien and Imbeau (1988-1989) write about models of learning in the context of Intelligent Computer-Assisted Instruction (ICAI). They point out that instruction is the counterpart of learning. There is no further specification what is meant by this statement.
In analyzing ICAI systems they have found three instructional processes or tutorial strategies in which an Intelligent Tutoring System can be engaged in:

So, these authors consider motivating and diagnosing not to be part of instruction. They are called instructional processes. The question remains what exactly is meant by instruction. It entails at the least presenting information and correcting student errors. Furthermore it can be concluded that instruction is always is some way provided: there has to be someone or something that takes care of instruction.

Hannafin and Rieber (1989a) give an overview of the two main psychological orientations instructional research has had (behavioural and cognitive) and the impact these orientations have had on computer-based instructional technologies. The authors find that this research has had little impact on the design of computer based instruction. They state:
This would, according to these authors, mean that instruction can be presented, manipulated and managed. This is a somewhat other view than Duchastel et al. (1988-1989) have. They see instruction only as being presented/provided.

Eylon and Linn (1988) give an extensive overview of four perspectives on learning that are held and used in research in science education. Indirectly they give their opinion of what instruction is. As they say it:
Two things can be concluded from these statements: instruction leads to a change in the learner and instruction has features. It is however not clear how (features of) instruction lead to a change in the learner.

2.1.2. About instruction(al) theories and models
A lot of instructional theories and models exist. In Reigeluth (1983a) some of them are presented. The variety among them is enormous. But, as Warries (1987) puts it, they 'often concentrate on problems of learning and thinking, disregarding other instructional phenomena' (p. 105). Examples, which give evidence that this statement could be true, will follow.

Reigeluth (1983b, p. 6) contrasts curriculum with instruction: curriculum is concerned with what to teach and instruction is concerned with how to teach it. Reigeluth also regards instruction as a process. He calls the different ways (processes) of 'how to teach' methods of instruction. But what is involved in these processes, what instruction really is, is not clear.

Veenman (1992) writes about the a certain method of instruction: the direct instruction-model. Direct instruction is one of the seven forms of 'learning environments' in which learning, but also instruction, takes place. Direct instruction is (in its narrow sense) a certain way a teacher uses several strategies which together form the instruction. The way the teacher does this facilitates learning.
The author is quite clear of what constitutes the direct instruction-model. But he is not so clear in what is meant by instruction and what is meant by learning. For this author it is clear that in a learning environment learning takes place, but also instruction takes place. And this instruction is made up of strategies that a teacher uses. But what actually happens is not clear.

Landa (1983, p. 171), in writing about learning and instruction, states that his theory of learning and performance deals primarely with understanding and describing processes (operations) that turn knowledge into skills and abilities. His theory of instruction deals with the problem of how to use the information about these operations to develop them in the course of instruction.
From this it is clear what Landa considers to happen when learning takes place: knowledge external to the learner is internalized in skills and abilities. Landa uses the word instruction for that situation in which a learning theory is used to develop skills and abilities in the learner.

2.2. The straightforward way

In this paragraph straightforward definitions of instruction will be presented. Some other related terms will be presented, too.

To Bruner (1966) 'Instruction is, after all, an effort to assist or to shape growth.' (p. 1). He is not clearly defining what instruction is, he is defining instruction in terms of the goal.

To Gagne (1985) 'instruction' constitutes of the application by a mediator of the so-called instructional events, of which there are nine. At the same time as this instructional event is taking place a corresponding internal process is triggered. These internal processes are called learning processes:
Some years before the last mentioned publication, Gagne (1974) defined instruction as

So instruction is made up of the following parts:
Gagne suggests that a teacher, as the one who arranges the learning events, must have an idea of what learning is and how it occurs. This knowledge can be provided by learning theories.

Warries (1986) defines instruction as:
This description introduces the term 'instructional system'. This traces back to systems thinking, a way to look at some problem and trying to solve it. The function of the instructional system is to mould the learner so that learning takes place. The same author (Warries, 1991) later made a very clear distinction between an instructional system and a 'situation in which a human teacher teaches something'. An instructional system is a situation in which materials that are largely pre-planned and pre-manufactured are used to deliver instruction in a relative short time-period (say at the most 10 hours). The teacher certainly can be part of this instructional system, but is not the central part of this system.

Romiszowski (1981) is also a proponent of systems thinking. He proposes the following:
Instructional systems are thus characterized by the presence of precise goals or objectives and the presence of careful pre-planning and testing out. This makes up three parts of instruction:
Romiszowksi states that a system is a whole that a person sees as such. A system has a goal, a function, can adapt to the environment and has an internal structure. Once you have defined what is part of the system, you also know what is part of its environment. The thing to do is to find out what the system is about by finding the inputs and the outputs of the system.


Pieters (1992) makes a very useful distinction between first-order instruction and second-order instruction. The first-order instruction is instruction that directs the learner, the second-order instruction is instruction that supports the learner. This is closely related to the distinction that is made between explicit learning and implicit learning. Nowadays it is known that instruction not only leads to knowledge that is observable and measurable (explicit), but also to knowledge of which one is not aware (implicit). Second-order instruction can be used to make the implicit knowledge more flexible and therefore more (re-)usable.

2.3. Summary and concluding remarks

Publications have been analyzed concerning a definition of the word instruction. Two ways of defining instruction have been identified.
From an analysis of these publications it can be concluded that the word 'instruction' stands for a process, something that happens within a period of time (Romiszowski, 1981). Different processes are possible, which are called methods of instruction (Reigeluth, 1983b). This process is taking place in a so-called 'instructional system' when pre-manufactured materials are used in a largely pre-planned way (Warries, 1991).
Most authors agree that essential for something to be considered instruction is the presenting of new knowledge and/or information by or through some mediator (Duchastel et al. , 1988-1989; Hannafin et al., 1989; Gagne, 1974, 1985).
Another aspect of the concept of instruction is that as an effect of instruction a change will occur within the learner (Eylon et al., 1988; Landa, 1983; Bruner, 1966). This can be on the part of more of better skills, knowledge, abilities of capabilities of the learner. When this effect has been measured, 'learning' has happened (Romiszowski, 1981; Eylon et al., 1988; Veenman, 1992; Gagne, 1974, 1985). Pieters (1992) has argued that this statement cannot be entirely true because also implicit learning can take place, which often cannot be measured.
An important aspect is that a (learning) goal has been identified in advance (Warries, 1986; Gagne, 1974; Romiszowski, 1981). The manner of fullfilling a goal is more or less pre-planned (Romiszowski, 1981).
Knowing this the following definition of 'instruction' is proposed:

3. What instructional design is

This section deals with 'instructional design'. The goal is not to find or propose a pure definition of the term. The goal is to get an overview of what the field of instructional design is like.
In the first paragraph the proposed definition of instruction from the previous page will be used to define instructional design.
In the second paragraph several descriptions of 'instructional design' is, will be presented.
The last paragraph in this section gives a summary of this section.

3.1. An easy definition

An easy definition of instructional design can be derived from the definition of instruction given in section 2:
It seems that instructional design is merely the making of a plan. Instructional design is no more than the careful planning of what will happen when in the instruction and the desirable changes.
Gagne (1974, p. 97-146) already proposed a similar view, although he doesn't use the word 'instructional design' He sees two important aspects of instruction:
The first aspect of instruction in this sense is the instructional planning, that what happens before a learner is available. This could be called 'instructional design' following the easy definition. The second aspect of instruction is the instructional delivery, that what happens when a learner is available. This is the real 'instruction', as defined in the previous section.

3.2. Several descriptions

Several descriptions have been found of 'instructional design'. They will be presented here. They will also be contrasted with each other.

Warries (1991) proposes that instructional design is concerned with the design of 'instructional systems', not with the design of instruction. These systems are characterized by the fact that they are designed and developed by a professional who has knowledge of (the characteristics of) the system to be designed. This professional certainly is not a specialist on the didactics of the classroom. He or she is a specialist on the methods, outcomes and conditions of instruction.

In a publication (Russell, Burton, Jordan, Jensen, Rogers & Cohen, 1990) proposing a computer-based system to help the instructional designer and developer in his work a description of instructional design has been found. Russell et al., in introducing their Instructional Design Environment (IDE) describe instructional design as follows:
These authors simply state that instructional design is the transformation of knowledge of the domain, of the students and of the tasks an instructional designer has to do. Instructional design extends from analysis, via development, to delivery.
The amount of knowledge can be huge, the objective of the described IDE is to make the management of this knowledge easier.

Romiszowski (1981) proposes to use the systems approach to solve problems that have something to do with instruction.The systems approach loops through five main stages: (1) problem definition, (2) analysis, (3) selection of optimal solution, (4) implementation and (5) evaluation and possible revision.
Instructional design is seen as the overall term for the process in solving a problem which has something to do with instruction. This process is seen as a heuristic process. The person involved is the instructional (systems) designer. Instructional design in this sense is often called Instructional Systems Design (ISD).
Russell et al. (1990) offer, compared to Romiszowski's analysis, a narrower view of instructional design. They consider the problem definition stage and the evaluation and revision stage not to be part of instructional design. Warries (1991) is quite in line with the view of Romiszowski, although Warries is much clearer in the delineation of what instructional systems are and are not.

Reigeluth (1983b) sees instructional design as the 'linking science' between learning theory and educational practice. He states the following:
This author makes clear distinctions between what he calls 'areas of inquiry within education'. The field of education is comprised of knowledge about curriculum, counseling, administration, evaluation and instruction.
The area of instruction can be viewed (just as the other areas) as being comprised of five professional activities: design, development, implementation, management and evaluation of instruction. A (scientific) discipline is associated with each activity.
Instructional design in this sense is concerned (1) with the decision what methods of instruction are best for bringing about changes in a given situation (professional activity) and (2) with producing knowledge about methods of instruction (scientific discipline). Knowledge about methods of instruction mostly comes in the form of the so-called 'principles of instruction'. Principles of instruction exist naturally and are discovered. They show change relationships. They show how one change (or action) is related to another change (or action).
The principles of instruction are the knowledge an instructional designer uses to make instruction, containing the optimal methods, that effectively brings about changes. In measuring the effects of the instruction, sometimes new principles can be discovered.
Generally speaking this means that the output of the scientific discipline naturally is the input for the professional activity. And sometimes the output of the professional activity can be input for the scientific discipline.
The views of Romiszowski (1981) and of Russell et al. (1990) are not entirely congruent with this view of instructional design. These authors are only concerned with the professional activities. They are mainly concerned with the usage of the knowledge of the scientific discipline (about the methods of instruction) in the professional activities.

3.3. Summary and concluding remarks

The definition of 'instructional design' presented in paragraph 3.1 is a description focussing on the professional activities, on what a professional does when he or she designs instruction (or instructional systems). The definition is considerably narrower than the conception of Romiszowski (1981) of instructional design. Romiszowski considers instructional design to be the process of solving a problem which has something to do with instruction. This process is made up of 5 stages. The presented definition only takes care of stage 3 'selection of optimal solution'.
Russell et al. (1990) and Romiszowski (1981), when describing what the field of instructional design is like, forget that the knowledge that is used in the instructional design process has to come from somewhere. They take the available knowledge about methods of instruction for granted. But instructional design in a broader sense is not just the application of knowledge about methods of instruction, it is also the discovery of this knowledge, according to Reigeluth (1983b).
The next section on 'instructional design theory' will deal in more detail with the distinction between discovery and application of knowledge.


4. What instructional design theory is

This section zooms in on aspects which have been dealt with in the previous section. What is it that the people in instructional design use? What kind of theories are there regarding instructional design? These are the questions which will be dealt with in this section.
In the first paragraph of this section the question will be adressed what constitutes a instructional design theory and what not. This paragraph also gives a framework for evaluating instructional design theories.
In the second paragraph of this section a short historical overview of the field of instructional design theory and the psychological influences on this field are presented, after which the state of the field nowadays is described.

4.1. What is a instructional design theory? A framework

Reigeluth (1983b) has proposed a framework for instructional design. With this framework it is possible to pin down the characteristics of every so-called instructional design theory. This author makes on the basis of this framework a distinction between an instructional design theory and an instructional design model.

Every theory of instruction should have three components: methods, conditions, and outcomes (Reigeluth, 1983b):
Conditions and methods are not fixed categories. If a method can't be changed anymore in a given situation it becomes a condition. If a condition, on the other hand, can be changed it becomes a method.
This framework has been extended to make it more usable. Instructional methods are described as having three variables: organizational-strategy variables, delivery-strategy variables, and management-strategy variables. Instructional outcomes can be classified under three different headings: the effectiveness of the instruction, the efficiency of the instruction and the appeal of the instruction. Two broad classes of instructional conditions are defined: subject-matter characteristics and student characteristics.
Reigeluth defines two categories theories of instruction: 'prescriptive' and 'descriptive' theories. Descriptive theories take sets of conditions and methods as givens, they describe the outcomes. Prescriptive theories use the conditions and the desired outcomes as givens and prescribe the best methods. Reigeluth states that 'Instructional design is a prescriptive science because its primary purpose is to prescribe optimal methods of instruction.' (p. 22-23). With this statement he is narrowing instructional design down to the use of prescriptive theories.
On the basis of this framework Reigeluth defines a 'principle of instruction' as every statement that is in the conditions-method-outcomes format.
An 'instructional design model' is defined as a complete description of a method, with all strategy variables filled in. It only is concerned with the method variables.
A 'instructional design theory' should be a set of principles of instruction. It takes the form conditions-model-outcomes. With such a theory instructional phenomena can be explained, but also predicted.
The author indicates that learning theory is often confused with instructional design. An instructional design theory focuses on methods of instruction. A learning theory focuses on the learning process. Learning theory is difficult to apply in the classroom because it does not spell out methods of instruction.

4.2. Historical overview

Romiszowski (1981) gives an extensive overview of the history of instructional (design) theory and of learning theory. Short historical descriptions of instructional design theory have been found in Reigeluth (1983b), Merrill, Li and Jones (1990a) and in Hannafin and Rieber (1989a). These last two publications also describe new developments in the field. The following subparagraphs are based on these four sources.

4.2.1. Different views
Several different views that are held on (the history of) instructional design theory have been found. They will be presented here.

Reigeluth (1983b) concludes that most instructional design theories have developed out of two areas. The major contributions have been developed out of psychology (the learning theory). Isolated strategies and principles have come from the area of media and communications.
The major contributors to the establishment of the science of instructional design have been Skinner, Bruner and Ausubel, according to Reigeluth. Skinner was one of the first to see that instruction was different from learning. He had a behavioural orientation to instructional design. Bruner and Ausubel developed cognitive orientations.

Romiszowski (1981, p. 165-186) doesn't make a clear distinction between learning theory and instructional theory in his overview. Furthermore he makes no clear distinctions between theories of instructional design, development, implementation, management or evaluation as Reigeluth (1983b) suggests. But for the purpose of an overview of instructional design theory his account is still useful.
He sees the history of instructional design theory in the light of the 'educational psychology battleground'. He thinks that

There is no unifying idea in the field of learning theory with which every player in the field agrees, according to Romiszowski (1981). So there are quite a few different learning theories, which all have their counterparts in the field of instructional design. On the other hand these different viewpoints can be traced back to two approaches in psychology: the behavioural and the cognitive.
These approaches are fundamentally different. In the behavioural approach (favoured, among others, by Skinner and Gagne) learning has everything to do with observable changes in behaviour. The internal processes are of minor importance. Instruction is seen as a chain of stimuli in which the right responses of the learner are reinforced.
In the cognitive approach (Ausubel, Bruner, Piaget) learning is an information-processing task. Instruction is primarily meant to augment these internal processes.

Merrill et al. (1990a) conclude that the experimental learning psychology has been the predominant force on instructional design theory.
Merrill et al. refer to all instructional design theories based on experimental learning psychology as First Generation Instructional Design (ID1). The theory with the largest influence has been Gagne's (1985). Other theories of the first generation are proposed by Landa (1983) and Merrill (1983). All these theories suffer from limitations, according to Merrill et al. As a result of these limitations these theories have not provided the hoped for increase of instructional effectiveness of 'interactive, technology-based delivery systems'. Their main argument is that ID1 provides little guidance for developing instruction for these interactive systems because at the time of development of ID1 these interactive systems didn't exist, so they couldn't be accounted for.

Hannafin et al. (1989a) contend, on writing on influences of psychology on the design of computer-based instruction (CBI), that
They conclude that instructional research is more and more becoming cognitively oriented. But the design of CBI is still behaviourally oriented.
These authors have seen no improvement in instruction because of new technological capabilities (Hannafin & Rieber, 1989b). They use the same argument as Merrill et al. (1990a): many models were developed prior to the emergence of new 'contemporary technologies' and therefore provide little insight into instructional strategies for these contemporary technologies.

4.2.2. State of the art
New developments are emerging in the field of instructional design theory. Two new 'theories' have been found in the literature. They are the state of the art in the field of instructional design theory.
Merrill et al. (1990a) saw the state of the art of instructional design theory as having serious limitations. These limitations were:

To overcome the limitations and to increase the instructional effectiveness of 'interactive, technology-based delivery systems' Second Generation Instructional Design (ID2) has been proposed (Merrill, Li & Jones, 1990b). The capabilities and components of ID2 have been incorporated in the Instructional Transaction Theory (Merrill, Li & Jones, 1991).

Hannafin et al. (1989a) indicate that, although many differences exist between the cognitive and the behavioural approach,
These authors see the ideal solution for instructional design theory as a mix of behavioural and cognitive influences. Both approaches have been incorporated in the ROPES+ model (Hannafin & Rieber, 1989b). ROPES stands for Retrieving, Orienting, Presenting, Encoding and Sequencing. This model is based on 'applied cognitivism, emphasizing relationships among external capabilities of technologies, task and performance requirements, known causal relationships in learning, and the processing capabilities of individual learners' (p.102-103).

4.2.3. Synthesis
Some remarks can be given regarding the surveyed material.
Reigeluth (1983b) finds that instructional design theory primarily developed out of learning theory. Romiszowski (1981) even sees no other way than to develop an instructional design theory out of learning theory.
Hannafin et al. (1989a) find the field of instructional research cognitively oriented nowadays. Reigeluth (1983b) and Merrill et al. (1990a) find the field of instructional design theory still behaviourally oriented. Merrill et al. propose a new generation of instructional design, including the best of both worlds (Merrill et al., 1990b). Hannafin et al. (1989b) do the same. Merrill et al. (1990b) also include insights out of the media & communications world as Kember and Murphy (1990) have described.
Hannafin et al. (1989a) and Merrill et al. (1990a) state that new, cognitively oriented, research findings had not been incorporated in instructional design theory (limitation 4, Merrill et al.). In their new 'theories' they try to do just this.
Hannafin et al. (1989a) regard the instructional design theory of Gagne (1985) as part of the cognitive orientation. Romiszowski classifies Gagne's contribution in the behavioural approach. Merrill et al. (1990a) use an even different heading to classify the instructional design theory of Gagne: experimental learning psychology.

4.3. Summary and concluding remarks

The major influence on instructional design theory has come from psychology, more specifically learning theory (Reigeluth, 1983b) or experimental learning psychology (Merrill et al., 1990a). These two classifications essentially are the same, the influential persons are the same: Skinner, Ausubel, Bruner and Gagne.
An instructional design theory is only then an instructional design theory if it takes into account instructional conditions, methods and outcomes (Reigeluth, 1983b). An instructional design theory is preferably of the prescriptive kind. An instructional design model is only concerned with methods of instruction.
In the literature the terms 'model' and 'theory' are not always used in accordance with these definitions. To make it even more confusing the borderline between 'instructional design' and 'instructional design theory' is not very clear.
But it is clear that the field of instruction is twofold. From the discovery of knowledge concerning instruction runs a line to the application of this knowledge and vice versa.

A useful distinction could be the following:
'(prescriptive) instructional design theory' is concerned with knowledge of methods of instruction - given conditions and outcomes - and with the prescriptions that follow out of this knowledge,
'(descriptive) instructional theory' is concerned with the knowledge concerning outcomes of instruction - given conditions and methods- and
'instructional design' is concerned with the application of prescriptions for methods of instruction for the design of new materials.


5. What computer simulation is

This section will focus on computer simulation and specifically the use of computer simulation in the context of education. Questions to be answered are 'What are the characteristics of computer simulation?' and 'How does computer simulation relate to learning?'. In the first and second paragraph an analysis of these questions will be presented. The last paragraph summarizes the findings of this section.

5.1. Characteristics

In this paragraph first some aspects and characteristics of 'simulation' in general will be treated here. Then the focus will be narrowed to 'computer simulation'.

5.1.1. Simulation
Ellington, Addinall and Percival (1981) refer to simulation as a form of 'exercise' or a 'teaching method'.
Simulation is defined as 'an operating representation of central features of reality' (p. 16). This identifies two essential features that must exist before some exercise could be called a 'simulation': (1) it must represent a real situation and (2) it must be operational, that is it must constitute an ongoing process.
Simulation is contrasted with the concept of a 'game', which involves (1) rules and (2) competition against others involved or a system. An exercise that has the characteristics of a game and of a simulation should be called a 'simulation game'.

Romiszowski (1984, p. 173-214) gives the following two characteristics of simulation:

This stresses the importance of manipulation by the learner in order for learning to take place.

Willis, Hovey and Hovey (1987, p. 40) compare simulation to on-the-job training. Simulation is not entirely the same thing as on-the-job training, 'it is one of several types of training procedures that comes close. Role playing and simulations both put the student in an active role within a context that is not 'real' but duplicates some aspects of reality.' Important here is that the learner is actively involved.


These publications taken together give the following impression of what simulation is like:
1. Simulation is considered to be a technique used by a a teacher (a teaching method, exercise, training procedure) whereby
2. the learner exercises some behaviour (is actively involved), and
3. in an environment that duplicates aspects of the real world (the model).
The concepts of 'teaching method' and 'training procedure' are similar to the concept of 'method of instruction' (Reigeluth, 1983b).

5.1.2. Computer simulation
Willis et al. (1987, p. 32-33) state four advantages of use of computer simulation in education compared to other methods of simulation:

These authors have had difficulty defining what computer simulation is. They have identified six characteristics of computer simulations (p. 36):
It seems as if these authors conceptualize a simulation still as a sort of game. They use the word 'player' and also 'rules'. These terms seem not so appropriate for a simulation. They seem more appropriate for a 'game' or maybe a 'simulation game' (Ellington et al., 1982).

Van Schaick Zillesen (1990) gives a description of educational computer simulation with characteristics that are also named by the abovementioned authors (there is a representation of reality which the learner can manipulate and feedback is given that represents reality):
Van Schaick Zillesen (1990) and Min (1987) both use the word 'system' for the part of reality that is to be studied.
Systems exist in three categories (Van Schaick Zillesen, 1990): natural system (all systems that exist in nature), artificial systems (existing man-made systems) and imaginary systems (systems that exist in the imagination of man).
To construct a computer simulation a structured description of the simulated system is required, this is called a 'model'. A model describes (1) the state of the system and (2) the possible transitions of the state of the system in the form of rules or equations. Computer processing is only possible if a model is described in this way. The model is then called a 'formal model'.
Two global types of formal models exist: qualitative models based on logical and/or conditional relations between variables and quantative models based on (mathematical) equations of the relations between variables. In quantative models parameters may also be present. They represent the properties of the system. The values of parameters are fixed, in contrast to variables.
Quantitive models can be further divided. One group of models is the group of dynamic deterministic models. In these models the values of at least some of the entities change as a function of time and the equations in the model are based on causal relationships. Most computer simulations are based on these models.

Min (1987) has had difficulties finding a good definition of 'educational computer simulation'. He defines three characteristics of computer simulation:
1. an activity or process, that happens in reality, is simulated,
2. a predefined mathematical model is implemented in a computer program and
3. visualized output is used.
This represents more a technical view of the way an individual computer simulation program should look like.
Min also contrasts simulation with 'gaming' and with 'modelling'. In modelling the main purpose is that a learner develops a model of (a part of) reality. Simulation on the other hand is a situation where the main purpose is to experiment with the simulated reality. Computer simulation programs are 'discovery environments', this is due to the fact that a mathematical model has an undefined number of states. Every state is new.

Van Berkum, Hijne, De Jong, Van Joolingen and Njoo (1991) state that
These authors find the foremost characteristic that the model is not shown to the learner. These authors also distinguish three classes of systems on which a model is based.

Van Joolingen and De Jong (1991) reviewed the literature on definitions of 'simulation'. In their synthesis of the definitions they found they come to the following definition:
Here, again, simulation is seen as a way of experimenting. In doing this information (knowledge) is retrieved. This is a definition presented from the viewpoint of the user/learner.
These authors also make clear that a model must have a formal representation otherwise it can't be processed by a computer. They also use the qualifications 'quantative' and 'qualitative' to make distinctions between formal models used for simulation.

If these different descriptions and definitions are compared, it seems that each author has another conception of computer simulation. Characteristics that are most common are

5.2. Computer simulation for learning

The focus of this paragraph is on the application of computer simulation for learning purposes. This area of interest is tackled from the viewpoint of advantages that computer simulations in general are claimed to have and from the viewpoint of learning with computer simulation

5.2.1. Advantages of computer simulation
Several publications listing advantages of computer simulation have been found. The most often quoted advantages will are listed here.

The advantages associated with computer simulation are headed under

The advantages listed here are mainly based on one consideration: to bring the real world into the classroom in such a way that is can be used for education. This is the primary reason to choose computer simulation in favour of some other method of instruction.

5.2.2. Learning with computer simulations
Reasons to use computer simulation having to do with the learning process will be presented here.

Willis et al. (1987) compared the objectives that can be adressed by different forms of educational software. Their opinion is that computer simulations can adress objectives in the affective domain and in the cognitive domain very good.
These authors state that simulation is compatible with several theories of learning and instruction and therefore has many proponents and a few opponents.

Other authors essentially view learning with a computer simulation as learning in a so-called exploratory environment (Van Berkum et al., 1991; Goodyear, Njoo, Hijne and Van Berkum, 1991; Njoo and De Jong, 1992). Exploratory environments stimulate the application of active, constructive and goal-oriented processes (so-called learning processes). The idea is that because the learner is more or less forced to take an active, constructive attitude the meaningful incorporation of information into the learner's cognitive structure becomes easier (learning takes place easier).
Three different learning processes specific to computer simulation are distinguished by Goodyear et al. (1991). These are 'problem solving', 'discovery learning' and 'inductive learning'.

Min (1987) sees computer simulation primarily as a tool or a resource that is used in the instructional process to promote learning in a more effective way (meaning that one or more of the advantages of computer simulation are claimed to exist and therefore the computer simulation is used).
Min states that a computer simulation program is a kind of Computer Assisted Learning. This is contrasted with Computer Assisted Instruction, programs in which the computer takes the role of instructor or tutor.
Min presents five learning methods that can be applied in learning with computer simulations. These methods are 'discovery learning', 'learning by doing exercises', 'guided discovery learning', 'problem oriented discovery learning' (similar to 'problem solving') and 'learning by doing experiments'.

5.3. Summary and concluding remarks

This section presented a global overview of the field of computer simulation in education. This is an area of interest for a lot of different researchers. For this concept a lot of different names are used (e.g. educational computer simulation (Min, 1992), computer-based simulation (Reigeluth & Schwartz, 1989), instructional simulation (Alessi, 1988; Duchastel, 1990-1991), instructional computer-based simulation (Alessi, 1988), simulation-based instruction (Duchastel, 1990-1991)), but essentially they share the same characteristics. These are:
A computer simulation is mainly seen as an exercise or an experiment in which the learner is involved.
The concept of (computer) simulation is not the same as the concept of a (computer) game: games always involve competition and rules.
Computer simulations are claimed to have a lot of advantages. These advantages are probably the reason why a lot of researchers are investigating computer simulation. The advantages are based on the assumption that bringing the real world into the classroom promotes learning. Investigations are mainly done on the basis of this assumption.
Most researchers agree that computer simulation is in itself an exploratory (learning) environment (e.g. Goodyear et al., 1991). Several learning processes (Min, 1987a; Goodyear et al., 1991) have been identified that are claimed to happen when a learner is using a computer simulation. Although computer simulation is an exploratory environment, (learning) objectives should be stated (Willis et al., 1987). This characteristic is not always explicitly mentioned in the literature.


6. Instructional design theory for computer simulation

Previous sections provided insights into the concepts of 'instruction', 'computer simulation' and the like. These insights constitute the framework on which this section will be based. This framework is described in the first paragraph. In the second paragraph proposals for the instructional design of computer simulation will be described. The evaluation of these proposals on the basis of the framework is presented in the third paragraph.

6.1. The framework

The framework which will be used is constituted of the following parts.
An instructional design theory as well as an instructional theory should take into account methods of instruction, outcomes of instruction and conditions of instruction (Reigeluth, 1983b).
Computer simulation can be considered as a method of instruction. The most important characteristics of this method are that (aspects of the) real world are presented, that the learner is actively involved and that the learner explores the domain while manipulating the simulation.

6.2. The design of computer simulations

Several different publications have been found concerning prescriptions for the design of computer simulations. They will be described here.

Alessi (1988) has written about the relationship between fidelity (how close the simulation imitates reality) of a computer simulation program and (transfer of) learning.
It has always been assumed that increasing fidelity would increase transfer (linear relationship). Evidence regarding this hypothesis is not consistent. The reason why is twofold: (1) high fidelity means higher complexity which taxes memory and other cognitive abilities and (2) proven instructional techniques which improve initial learning also tend to lower fidelity.
These reasons lead to the hypothesis that the relationship between learning and fidelity is non-lineair and depends on the instructional level of the learner. The novice learner learns best from a relatively low-fidelity simulation. The experienced learner learns best from a intermediate-fidelity simulation. The expert learns best from a high-fidelity simulation.
Transfer is a complicating factor in this. Increasing fidelity, which theoretically should increase transfer, may inhibit initial learning which in turn will inhibit transfer. For the novice learner, this should be taken into account. This desciption already gives some prescriptions for the design of simulations. But Alessi goes even further.
He proposes a taxonomy of factors to consider in determining the fidelity of simulations. Presciptions should be on the level of these specific factors that can be influenced in the design-phase. There are four groups of factors that are of importance: factors concerning the underlying model, factors concerning the presentation, factors concerning the user actions and factors concerning the system feedback.
For physical and process simulations, that have as objective to learn about something, the most important factors are thost concerning the underlying model and those concerning the presentation. For procedural and situational simulations, that have as objective to learn how to do something, the most important factors are those concerning user actions and system feedback.
Alessi concludes that further research should adress which factors to vary under which conditions.

Duchastel (1990-1991) contends that objectives should be defined for a computer simulation. Objectives are categorized in two broad categories of the knowledge that is to be acquired with the simulation: process knowledge ('how it occurs') and procedural task knowledge ('how to do it'). These types of knowledge have their counterparts in two kinds of simulation: 'process simulations' and 'procedural simulations'.
The learning objectives are the main context for the instructional strategies that one can use in a computer simulation. Duchastel distinguishes 'broad instructional strategies' and 'specific instructional strategies' (specific for the two types of simulations).
The broad instructional strategies are 'simplification' and 'support'. Simplification is the process of eliminating many of the details in a simulation. The learner gets focussed on the main aspects of the simulation. He or she is presented with the 'big picture' first. Details that will be learned later will fit in this big picture: '(...) the progression is from a simplified model to more complex versions of the model, all the way to the desired level of specificity needed for the instructional goals being pursued.' (p. 269).
With support an instructional layer on top of the simulation is meant. In working with simulations the learner is exploring and discovering new knowledge. Support in all sorts (demonstrations, help, feedback) can help in acquiring this new knowledge. Support can be gradually diminished as the learner is learning more about the simulation.
The specific instructional strategies are grouped in three categories: demonstrating, tasking and explaining. Demonstrating involves showing the learner the simulation so that he or she figures out how it works. Tasking means that the learner is given a task so that he or she can explore how the simulation works. Explaining entails clarifying things for the learner.
The two groups of instructional strategies should be taken into account when designing computer simulations. The prescriptions regarding broad instructional strategies are of most importance in the design-phase. Once a simulation program is ready, adding simplification-facilities and support-facilities is not an easy task.

When designing a computer-based simulation one has to consider three major design aspects, according two Reigeluth and Schwartz (1989): a 'scenario', a 'model' and an 'instructional overlay'. The scenario is that what recreates to a greater or lesser degree the real life situation. The scenario determines what happens and how it takes place, it also determines the role of the learner and how he or she will interface with the simulation. The model (mostly mathematical) reflects the causal relationships that govern the situation. The instructional overlay is that part of the program that optimizes learning and motivation.
The authors have found that the nature of the content or behaviour being taught and learned is the major influence on the instructional features a simulation should have. They have focused on two types of content: principles and procedures. These two types of content are very different. Mastery of procedures is acquired gradually over time, mastery of principles is instantaneous. Methods of instruction must be very different for these two kinds of learning, just as the design of the simulation.
With regard to simulation and the two types of content, the authors identify three major types of simulations: those that teach procedures (procedural simulations), those that teach 'process' principles (process simulation) and those that teach 'causal' principles (causal simulation).
These authors further have identified three phases in the learning process:
So, these authors propose an instructional model for simulations. The authors have identified 6 simulation features that act as 'vehicles' for achieving acquisition, application and assessment. These features must be seen as the aspects of simulation that answer the question 'what to consider when designing simulations?'.
The features are:
On the basis of these features the authors propose a general model geared at the three learning phases and three additional issues, which make up the issues with which the model is concerned. This leads to a model for the design of the instructional overlay of simulations for teaching principles and procedures.
The design starts with the selection of the appropriate complexity of the underlying model. If this is comprised of only a few variables, then this model should be used as a whole. If there are more variables then the model should be simplified.
The simulation is preceded by a introduction. The introduction should present to the learner the scenario (prescription 1), goals and objectives (prescription 2) and directions and rules regarding the use of the program (prescription 3).
During the acquisition stage the learner should develop an understanding of the content. In an expository approach the generality along with a prototypical example should be provided (prescription 1). In a discovery approach, the learner should be required to 'figure out' the generality by exploring the prototypical example (prescription 2).
During the application stage the learner should develop the ability to use the principle or procedure. The primary elements of this stage are practice and feedback, as described above. The authors provide 10 prescriptions regarding the design for this stage.
In the assessment stage a criterion test should be administered. Five prescriptions for the design are given.
The last component of the general model is the issue of control. This influences the whole simulation. Six prescriptions are given for this issue.
In further refining the general model five additional issues are discussed that influence the design of the simulation. These issues are nature of content, complexity of content, learner participation, form of changes and motivational requirements.
Regarding nature of content the distinction in three types of simulations is used. For the acquisition stage two prescriptions are given for each type of simulation. For the application stage three general prescriptions are given, one specific for procedural simulations and two for causal simulations. The refinement goes even further to prescribe the practice form for 5 different behaviours that can be distinguished for procedures and principles.
Regarding complexity of content prescriptions are provided for breaking down content of procedural form and breaking down content of principle form.
For learner participation six prescriptions are given: three for the acquisition stage and three for the application stage.
For form of changes, the form the changes being taught have (physical or non-physical), is the major factor in determining the representation form. Four presctiptions are given.
Regarding motivational requirements five prescriptions are given.
The authors conclude with stating that the prescriptions are considered to be the first step in an attempt to construct a validated presciptive theory for the design of computer-based simulations. Empirical evidence for the prescriptions is not available.

Van Berkum and De Jong (1991) find the nature of simulation-based learning essentially exploratory. They have focussed on the instructional design for the so-called 'directive support':
The opinion of these authors is that the disadvantages of the completely free exploratory nature of a computer simulation can be eliminated by imposing 'instructional constraint' on the learning situation. The challenge is to strike the right balance between exploratory freedom and instructional constraint.
The instructional constraint is divided into non-directive support and directive support. The non-directive support is defined by the 'interface' of the computer program. The interface is the component of an instructional system that mediates between a learner and the system (De Hoog, De Jong & De Vries, 1991). The non-directive support has nothing to do with the instructional process, in contrast to the directive support. Both forms of support are primarily intented for second-order instruction (Pieters, 1992), with which the learner is guided and supported.
Van Berkum and De Jong (1991) state that the form of directive instructional support depends on characteristics of the simulation domain, on characteristics of the learner and on characteristics of the learning goals. Learning goals are classified along three dimensions. These dimensions are the kind of knowledge to be learned (conceptual or operational), the way the knowledge must be encoded (compiled or declarative) and the scope of the knowledge (domain-specific or generic). This gives a total of six different types of learning goals. One additional class of learning goals is distinguished: knowledge acquisition goals, the goals that learn the learner about it's own cognitive processes and products. The authors are convinced that any choices made during the design of a 'simulation-based learning environment' must be grounded in an assessment of the learning goal(s) at which the environment is to aim.
The authors have elaborated on the concept of instructional support for simulations. They have tried to derive from cognitive theory, from instructional design theory and from existing exploratory learning environments instructional features (strategies, actions, approaches) that could be relevant to the design of simulations.
From cognitive theory stems the distinction between conceptual knowledge and operational knowledge. For conceptual knowledge learning is seen as a process of mental model transformation, in which the models get better and more accurate. This results in several 'instructional principles' to guide learning. These refer to sequence and choice of models and problems, providing explanations and minimization of error. For operational knowledge learning is seen the transformation of declarative knowledge to specific procedures. The instructional principles associated with this is that this transformation can only take place in a problem solving context. Other instructional principles refer to model tracing, immediate feedback an minimization of working memory load. For knowledge acquisition skills four instructional principles have been found. These refer to strategies for monitoring comprehension, the teacher as model and coach, shared responsibility for the task, teaching and supporting multiple learning strategies.
With regard to instructional design theory only the more cognitive oriented theories have been analyzed. Four guidelines follow from this analysis: (1) interaction with simulations is only part of a more comprehensive instructional strategy, (2) interaction with the simulation may be accompanied by informative feedback and elaborations or more static presentations of information (expository or inquisitory), (3) learner can provoked to perform specific learning processes and activities by a tutor and (4) complex models and procedures can be taught by starting with simple elements of the model and subsequently presenting more complex models.
The last analysis by these authors has been done on intelligent tutoring systems (ITS), which are referred to as the existing exploratory learning environments. Research in this area mainly has been concerned with properly representing the (expert) domain knowledge to the learner. There is little evidence on the effectiveness of the individual instructional design features that have been used in the design of these systems. The instructional design considerations that have been used are the following: (1) multiple views of the same domain model (different perspectives, different diagrammatic ways, quantative as well as qualitative), (2) progressive model implementation (increasing complexity with more variables), (3) offering explanations of the behaviour of the model, (4) use the cognitive apprenticeship approach for teaching operational knowledge, (5) when learners work in pairs and comment on each other's idea's the learning of knowledge acquisition skills takes place better (but the learners have to be proficient in the domain chosen), (6) stimulate the learning process (use of assignments, offering a fault diagnosis task, give a ready made hypothesis).

The concept of non-directive support (the 'learner interface') also has been investigated (De Hoog et al., 1991).
Two metaphors exist in interface design for computers. The conversational methaphor, which essentially is an interface in which user and computer are talking partners. The user has to learn the complicated syntax and semantics of the computer language. Developments are that the computer language is getting more and more akin to natural language. The direct manipulation methaphor doesn't use language. Computer and user use objects and actions to communicate. The user doesn't have to know any of the syntax rules of the internal computer language. The authors have, on the basis of this fundamental distinction, made a classification of interfaces. This comes down to 16 classes of interfaces.
The authors further have defined three generic entities of a user interface for learning with simulations. Every interface will contain these entities. The entities are the model entity, the learning entity and the control entity. The model entity contains everything related to the domain model. It has an output part, used to communicate with the learner about the model, and an input part, used to give the learner the opportunity to change elements of the model. The learning entity contains everything that is related to the learning process of the learner. Most important aspect of this entity is the directive support. The control entity gives the learner possibilities of influencing the way the program is running.
No principles that should be taken into account in the design of computer simulations have been found by these authors:
Min (1987) and Van Schaick Zillesen & Min (1987) has described a design system for educational computer simulation programs. The simulation programs designed with this system have several distinct characteristics:
The design system itself contains two separate shells and procedure-libraries. The design method, simply stated, is no more than filling in gaps and replacing parts in the shells. The designer can follow guidelines ('steps') in designing the simulation program (Min, 1991).
Research with resulting simulation programs showed that the additional material were of utmost importance in the instructional effects on the learner (Min, 1992). In the light of this Min proposed the notion of the 'Parallel Instruction Theory', stating that learners are best served when the simulation as well as additional material in a simulation-based learning environment are within reach. If the additional material is paper-based, it has been noticed that they are not used. Therefore, computer-based materials are preferred, because they are even more in reach. This led to an additional guideline for designing simulation-based learning materials: when designing materials the designer should take notice that all material should be within reach when the material is presented to the learner.

6.3. Evaluation of proposals for design of computer simulations

In this paragraph first some general remarks will be made. Then the proposed design theories, principles and/or prescriptions will be evaluated with the framework presented in paragraph 6.1.

6.3.1. Some general remarks
The use of 'directive support' by Van Berkum and De Jong (1991) is somewhat broader than the way Duchastel (1990-1991) uses it. Directive support here probably entails all five instructional strategies that Duchastel proposes.
In the contribution by Duchastel (1990-1991) it is assumed that the support facilities should be computer-based. Van Berkum & De Jong (1991) essentially agree with this opinion with regard to 'directive support'. Min (1992) essentially states the same.
The design system proposed by Min (1987) seems to entail the concept of 'non-directive support' (De Hoog et al., 1991).
Alessi's (1988) 'fidelity' seems to be more or less the same as the concepts of simplification and support that Duchastel (1990-1991) proposes, taken together.
All authors agree that in one way or another the simulation program as a stand alone program has no future whatsoever. Components have to be added that make it to a simulation-based learning environment.

6.3.2. Evaluation with the framework
Two claims have been made that there are instructional design theories for computer simulations (Min, 1992; Reigeluth et al., 1989). In the framework these claims cannot be called instructional design theories. Both proposed 'theories' are not entailing all three aspects of an instructional design theory.
Reigeluth et al. (1989) present the most detailed proposals for the design of computer simulations. They can be called prescriptions.
Duchastel (1990-1991) and Alessi (1988) come close to these prescriptions. But there proposals have to be translated to prescriptions on the basis of empirical research. Moreover, Alessi has only concentrated on the fidelity aspects of a simulation.
Min (1987) gives prescriptions for the design of the simulation program only. The Parallel Instruction Theory (1992) only gives very general prescriptions, which have to be further investigated.
The thorough analyses by De Hoog et al. (1991) and Van Berkum & De Jong (1991) give some general proposals for the design of simulations. But these also have to be translated into usable prescriptions.

7. Conluding Remarks

In this literature survey the field of instructional design for computer simulations has been monitored. It should be quite clear that this field is in development, which is perhaps the main reason that no comprehensive theory exists. Therefore the information provided in this survey some day will become useless of even wrong.
Computer simulation is a very difficult subject. Of course, there are many advantages that are associated with computer simulation. But sometimes an advantage can become a disadvantage. When a computer simulation is simplistic and not so realistic, this could be a disadvantage. Maybe there is a border associated with every advantage which should not be crossed, because this advantage then will become a disadvantage. These borders are not yet known, maybe some effort should be directed to the search of these borders. We then could build a theory regarding the instructional design of computer simulations which is grounded in the advantages of computer simulations and not in the claims regarding the enhancement of learning or any other claim that is not directly associated with computer simulation.
Duchastel (1990-1991) is more or less doing this with some advantages, when building his proposals on the factors of fidelity, interactivity and artificiality of computer simulations. Alessi (1988) builds on the general claimed advantage of fidelity, which encompasses more then just 'likeness' or 'realism'. Reigeluth et al. (1989) take a quite different approach. They build on the knowledge of instructional design theory. Maybe that is not so good an approach because it assumes that computer simulation only involves instruction. There is certainly more to computer simulation then that. Min (1992) builds on both the claimed advantages of computer simulation, e.g. visualization, and on some of the notions generally associated with learning theory and/or instructional theory, e.g. the discovery learning approach.
The notion that a computer simulation must be seen as an exploratory environment (Van Berkum & De Jong, 1991) is a very useful one. In this environment support must be available, which should be in the form of second-order instruction (Pieters, 1992). Only then the computer simulation is an environment in which the learner will learn, explicitly but also implicitly. Further investigation should be done in the field of implicit learning and exploratory environments, because maybe the knowledge that is acquired in this way is much more useful in the long run than the knowledge acquired in first-order instruction by means of explicit learning.

References

Alessi, S.M. (1988). Fidelity in the design of instructional simulations. Journal of computer-based instruction, 15(2), 40-47.

Berkum, J.J.A. van, Hijne, H., de Jong, T., van Joolingen, W.R., Njoo, M., (1991). Learning processes, learner attributes and simulations. Education & Computing, 6, 231-239.

Berkum, J.J.A. van, & de Jong, T. (1991). Instructional environments for simulations. Education & Computing, 6, 305-358.

Bruner, J.S. (1966). Toward a theory of instruction. Cambridge, Massachusetts: The Belknap Press of Harvard University Press.

Duchastel, P. (1990-1991). Instructional strategies fo simulation-based learning. Journal of Educational Technology Systems, 19, 265-276.

Duchastel, P., Brien, R., Imbeau, J. (1988-1989). Models of learning in ICAI. Journal of Educational Technology Systems, 17, 165-172.

Ellington, H.I., Addinall, E., Percival, R. (1981). Games and simulations in science education. London: Kogan Page.

Gagne, R.M. (1974). Essentials of learning for instruction. Hillsdale, IL: Dryden Press.

Gagne, R.M. (1985). The conditions of learning and theory of instruction (4th rev. ed.). New York: Holt-Saunders International Editions.

Goodyear, P., Njoo, M., Hijne, H., van Berkum, J.J.A. (1991). Learning processes, learner attributes and simulations. Education & Computing, 6, 263-304.

Hannafin, M.J., Rieber, L.P. (1989a). Psychological foundations of instructional design for emerging computer-based instructional technologies: Part 1. Educational Technology Research & Development, 37, 91-101.

Hannafin, M.J., Rieber, L.P. (1989b). Psychological foundations of instructional design for emerging computer-based instructional technologies: Part 2. Educational Technology Research & Development, 37, 102-114.

Hoog, R. de, Jong, T. de & Vries, F. de (1991). Interfaces for instructional use of simulations. Education & Computing, 6, 359-385.

Jong, T. de (1991). Learning and instruction with computer simulations. Education & Computing, 6, 217-229.

Joolingen, W.R. van & de Jong, T. (1991). Characteristics of simulations for instructional settings. Education & Computing, 6, 241-262.

Kember, D. & Murphy, D. (1990). Alternative directions for instructional design. Educational Technology, 30(8), 42-47.

Landa L.N. (1983). The algo-heuristic theory of instruction. In C.M. Reigeluth (Ed.), Instructional-design theories and models: an overview of their current status (pp. 163-211). Hillsdale, NJ: Lawrence Erlbaum.

Merrill , M.D. (1983). Component Display Theory. In C.M. Reigeluth (Ed.), Instructional-design theories and models: an overview of their current status (pp. 279-333). Hillsdale, NJ: Lawrence Erlbaum.

Merrill, M.D., Li, Z. & Jones, M.K. (1990a). Limitations of first generation instructional design. Educational Technology, 30(1), 7-11.

Merrill, M.D., Li, Z. & Jones, M.K. (1990b). Second generation instructional design (ID2). Educational Technology, 30(2), 7-14.

Merrill, M.D., Li, Z. & Jones, M.K. (1991). Instructional transaction theory: an introduction. Educational Technology, 31(6), 7-12.

Min, F.B.M (1987). Computersimulatie als leermiddel. Een inleiding in methoden en technieken [Computer simulation as a learning resource: an introduction in methods and techniques]. Schoonhoven, Academic Service.

Min, F.B.M (1991) (ed.). Practicumhandleiding MacTHESIS, version 4.3 [Guide to MacTHESIS, version 4.3].. Internal publication. Faculty of Educational Science and Technology, University of Twente (Enschede, The Netherlands).

Min, F.B.M (1992). Parallel instruction, a theory for educational computer simulation. Interactive Learning International, 8(3), 177-183

Njoo, M., Jong, T. de (1992, October). Supporting exploratory learning by offering structured overviews of hypotheses. Paper presented at the NATO Advanced Research Workshop The use of computer models for explication, analysis and experiential learning', Bonas, France.

Pieters, J.M. (1992). Het ongekende talent [The unknown talent]. Oration. Enschede: University of Twente.

Reigeluth, C.M. (Ed.). (1983a). Instructional-design theories and models: an overview of their current status. Hillsdale, NJ: Lawrence Erlbaum.

Reigeluth, C.M.. (1983b). Instructional design: what is it and why is it? In C.M. Reigeluth (Ed.), Instructional-design theories and models: an overview of their current status (pp. 3-36). Hillsdale, NJ: Lawrence Erlbaum.

Reigeluth, C.M. & Schwartz, E. (1989). An instructional theory for the design of computer-based simulations. Journal of computer-based instruction, 16(1), 1-10.

Romiszowski, A.J. (1981). Designing instructional systems. London: Kogan Page.

Romiszowski, A.J. (1984). Producing instructional systems. London: Kogan Page.

Russell, D.M., Burton, R.R., Jordan, D.S., Jensen, A.M., Rogers, R.A., Cohen, J.R. (1990). Creating instruction with IDE: tools for instructional designers. Intelligent Tutoring Media, 1, 3-15.

Schaick Zillesen, P.G. van (1990). Methods and techniques for the design of educational computer simulation programs and their validation by means of empirical research. Meppel: Krips Repro.

Schaick Zillesen, P.G. van & Min, F.B.M (1987). MacTHESIS: a design system for educational computer simulation programs. Wheels for the mind of Europe, 2, 23-33.

Veenman, S. (1992). Effectieve instructie volgens het directe instructiemodel [Effective instruction according to the direct instruction-model]. Pedagogische Studi‘n, 69, 242-269.

Warries, E. (1986). De kennisbasis voor instructie [The knowledge base for instruction]. In S. Dijkstra, P. Span (Eds.), Bijdragen aan de onderwijsresearch. No. 1. Leerprocessen en instruktie (pp. 69-76). Lisse: Swets & Zeitlinger.

Warries, E. (1987). The knowledge base for instructional design. Instructional Science, 16, 105-108.

Warries, E. (1991). De grenzen van de instructietheorie [The borders of instructional theory]. Enschede: University of Twente.

Willis, J., Hovey, L., Hovey, K.G. (1987). Computer simulations. A sourcebook to learning in an electronic environment. New York: Garland.