1.1 Man-machine interaction
Until now, much research has been done about human-machine interaction. The nature of this interaction depends on both characteristics of the system and characteristics of its users. The system must meet certain requirements with respect to human cognitive abilities and weaknesses. Knowledge about human information processing must be used to fully adjust a system to its users. For example, with the development of learning tools, it must be considered that humans can only store 7 plus/minus 2 information chunks of information in memory at once (Miller, 1956). Besides research about restrictions of human memory, other relevant research has been done about human knowledge (Collins & Quillian, 1969; Collins & Loftus, 1975), human performance (Rasmussen, 1983), problem solving (Rasmussen, 1982; Rouse, 1981; Reason & Embrey, 1985), and so on.
There are numerous ways in which human can interact with machines (systems). Examples are working with a word processor, playing computer games, driving a car, etc. In order to drive, navigate, use or operate a system people have to own skills and knowledge (capabilities) that suit the skills and knowledge that are needed to perform these tasks successfully. The ideal way to match required capabilities for newly encountered tasks to user capabilities is to provide these users with a self-explaining system. Self-explaining systems are built so that needed skills and knowledge to perform the task are adapted to actual skills and knowledge of the target group of users. Moreover, they take into account compatibility with human information processing characteristics. They suit the system to natural sensitivities and weaknesses of human information processing. In principal people should be able to use these systems without previous task specific education. A good example of a self-explaining system would be a cash machine. Figure 1 shows that a self-explaining system fills the gap between skills & knowledge that are needed to perform the task and users skills & knowledge.
Figure 1: Model of man-machine interaction when a self-explaining system is involved
Although, it is always important to have machines as adapted as possible to human cognitive capabilities, certain tasks with or on a system are too complicated to be executed without education.
1.2 Education
Education is defined as systematically transferring knowledge and skills (van Dale, 1997) from a particular learning domain. Flying an airplane is an example of a task that can not be done without proper education. In this case skills and knowledge that are needed to perform the task are made accessible to users through education. Figure 2 shows that through education, users are given access to skills and knowledge that are needed to perform the task.
Figure 2: Model of man-machine interaction when education is involved.
Nowadays, a great number of education facilities have become available. A major cause is the growing variety of available ICT applications for education. There is the opportunity to use computer assisted education, but there is also the opportunity to provide education with the help of virtual reality. Figure 3 shows that with the help of ICT applications users are given access to skills and knowledge that are needed to perform the task.
Figure 3: Model of man-machine interaction when ICT applications are involved.
Just as with a self- explaining system, there are ICT applications that are fully self supported, such as a Computer Based Training package of a typing course (reference x). The basic principles, such as where to put ones fingers, is explained with pictures and a computer voice, the user is provided with lessons to practice the needed skills, and one can easily look into the progress that is made. Not all tasks or learning domains can be so easily taught without personal guidance of a teacher. Usually, such a high degree of similarity between the task to be learned and the training environment is not as easily attained as in the typing course example. Even though a flight simulator uses state of the art technology for creating a realistic cockpit, still some characteristics of a real flying environment are difficult to simulate such as the presence of extensive g-forces. However, there may be certain parts of such tasks that can be learned or trained with (the help of) ICT applications.
An arising question is to what extent ICT applications, on their own or in combination with other aids, can lead to an efficient and effective task performance. It has to be taken into account that each situation has its own task-, system-, and user characteristics. The context in which ICT applications are used will influence all the previously mentioned variables. Here, context is defined as the environment in which ICT applications are used. This varies from the kind of company to the type of workplace (car, control room of a ship, etc.). For example, a companys availability of (financial) resources can result in completely different design solutions. On the other hand criteria for displaying "In Vehicle Information" while driving a car differ from those applied to computer screens in the control room of a ship. An IVIS (In Vehicle Information System) is not allowed to force road users to pay visual attention to its display (Kaptein, Claessens & Janssen, 1998; van Winsum & Claessens, 1998), while with a vigilance task in a control room active interference of the system is needed to keep operators alert. Thus, design of ICT applications has not only to be adapted to different tasks, systems, users and performance goals, but also to the context it is placed in. Because of these aspects it is hard to generalize research results concerning particular man-machine interaction. Information gained from these studies only applies to man-machine interaction under specific circumstances. Figure 4 shows that context influences the design of ICT applications.
Figure 4: Model of the relation between man-machine interaction and its context when ICT applications are involved.
As previously said ICT application design depends on involved tasks, systems, users, and the context in which they are situated. In this study ICT design was investigated within the context of a Dutch company that produces naval combat systems and teaches people to operate or maintain these systems. "Hollandse Signaalapparaten B.V." possesses a large scale of different tasks and different ICT applications, such as a virtual reality room. The interest of Holland Signaal was to have investigated whether "new" ICT applications would yield profit for them and next how they could be implemented in training. They were especially interested in Computer Based Training (CBT) and Virtual reality (VR). With the arrival of their VR-room it was necessary for them to have investigated whether Virtual Reality could be used just as successfully for the concerning division Training as for several other divisions. Computer Based Training appeared to be an important topic because of the arising supply of CBT packages on the market.
The main question is how the ICT applications, Computer assisted education and Virtual reality, must be designed in order to lead to an effective and efficient task performance. To be able to generalize research results obtained from this study well defined tasks and ICT designs were tested in different task design combinations with a fixed group of target users and a fitted context (see Chapter x). A comparison of these task design combinations should bring about suiting design specifications for each particular task.
The research question is:
|
What kinds of design specifications are needed for the ICT applications, Computer assisted education and Virtual reality, in order to lead to an effective and efficient task performance, given a specific task? |
Thus, a specific ICT design should be matched to specific tasks in order to lead to a high task performance. Task performance will be based on the learning and training time that is spent compared to the amount of effect a design has on the goodness of the performance. Hereby, the effectiveness is measured by the amount of failures during performance, while the efficiency is based upon the relative amount of failures compared to the amount of learning time spent. With pressure of time design 1 that elicits a reasonable performance with one hour of study will be preferred to design 2 that causes faultless performance after six hours of study. Thus, a perfect balance has to be found between the learning time spent and the quality of the performance.
In the next chapter theoretical background information is given about the nature of different (skill-, rule-, and knowledge based) tasks, and about theories that generate a basis for alternative ICT design criteria. First, the conception of ICT especially virtual reality is further explained. Subsequently, relevant background information about the company and its context is provided, by means of documentation studies and interviews (Chapter 3). Finally, design requirements derived from the data obtained in the first two chapters were implemented and compared in several experiments to test certain task-design combinations (Chapter 4, etc.).
Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of semantic processing. Psychological review, 82, 407-428)
Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of verbal learning and verbal behavior, 8, 240-247.
Kaptein, N. A., Claessens F. M. M., & Janssen, W. H. (1998). Safety evaluation of in-vehicle devices that provide real time traffic information. (Report TM 98-C047). Soesterberg, The Netherlands: TNO Human Factors Research Institute.
Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97.
Rasmussen, J. (1983). Skills, rules and knowledge: Signals signs and symbols, and other distinctions in human performance models. IEEE transactions on systems, man and cybernetics. Vol SMC-3, 3, 257-268.
Rasmussen, J. (1982). Human errors. A taxonomy for describing human malfunction in industrial installations. Journal of occupational accidents, 4, 311- 335.
Reason, J. T., & Embrey, D. E. (1985). Human factors principles relevant to the modeling of human errors in abnormal conditions of nuclear and major hazardous installations. Dalton, England: Human Reliability Associates.
Rouse, W. B. (1981). Models of human problem solving: Detection, diagnosis, and compensation for system failures. Proceedings of IFAC conference on analysis, design and evaluation of man-machine systems. Baden-Baden, Germany.
Van Winsum, W. & Claessens F. M. M. (1998). Effecten van mogelijke vormen van informatiepresentatie van de OBU op rijgedrag en verkeersveiligheid. Effects of possible forms of information presentation of the OBU on driving behavior and traffic safety. (Report TM 98-C052). Soesterberg, The Netherlands: TNO Human Factors Research Institute.
Van Dale (1997). Van Dale Hedendaags Nederlands, Groot electronisch Nederlands woordenboek. [Van Dale contemporary Dutch, Great electronic Dutch dictionary]. Van Dale lexicografie, Utrecht: The Netherlands.
Information and communication technology refers to technologies that are used for collecting, storing, editing and passing on information electronically in divergent forms (data, text, images, and sound) (reference, 1999). In this study the possibilities of computer assisted education and virtual reality are investigated. Computer assisted education will be defined here as a piece of education that is meant to teach students a specific task on the computer without interference of the teacher. The definition of virtual reality was less obvious to determine. Originally, the term virtual reality (VR) referred to an artificial, three-dimensional (3D) world that was completely generated by a computer and that elicited a feeling of immersion. This immersive character was considered inextricable from VR. In real VR people had to get totally absorbed by the environment as if it was real.
Literally, a virtual reality is an only apparently existing reality. This can include anything that is anyhow an imitation of the real world. Not only a world but also separate environments, such as control rooms of a navy ship, and single objects, such as a molecules or DNA structures. Not only computer generated VR, but also (partly) physical VR such as flight simulators and the mock-up of a car. In addition, the three-dimensional component of VR is very complicated. Irrespective of the fact that some researchers state that people can see no better than 2.5D (Marr, 1982), there is a difference in seeing monocular or binocular 3D (respectively with one eye/without VR-glasses and with two eyes/with VR-glasses). These aspects lead to it that nowadays the term VR is used in a great variety of ways, what leads to confusions.
To make a clear distinction out of this variety of VR it must be investigated what main properties a virtual reality consists of. In this study Virtual Reality is considered to vary along the following variables:
Perceptual cues
One important aspect in which VR varies is the amount in which users are provisioned with the right perceptual cues. To understand how VR is simulated it must be clear how people sense actual reality. There are several cues that make people sense the world the way they do (see Coren, Ward & Enns, 1994). Cues are stimuli that one is not often consciously aware of and that function to shape our perceptual responses. In a virtual reality maybe some and otherwise many of these cues lack. The amount and composition of these cues determines also what level of reality a virtual world, environment or object contains. To obtain a perfect virtual reality the cues in a simulated situation have to resemble cues as they exist in reality in such a situation. Cues can almost be divided according to modality. There are visual cues, auditory cues, tactual cues and cues for kinesthesis (perception of weight and force).
Till now the easiest part to trigger in virtual reality (VR) is the visual perception of three-dimensional arrangement in space. This is because some very important visual cues for depth can easily be added to make for example computer images become more realistic. Visual cues for depth provide us with information about absolute distance and relative distance. Absolute distance (absolute depth perception), also known as egocentric localization, is the perceived distance from observer to objects. Relative distance (relative depth perception), which is also known as object-relative localization refers to the relative distance between objects or to the (relative) depth of (parts of an) object. Visual depth cues are divided into pictorial (monocular) cues, binocular cues, physiological cues and motion cues.
As previously said, there are some visual cues for depth that can very easily be implemented in VR. These visual cues are called pictorial depth cues, because they can simulate depth on flat pictures. They are also known as monocular cues. This is because they refer to depth perception as perceived through only one eye. Pictorial depth cues are for example shadowing (some parts of an object are supposed to be darker than other parts according to where the light source comes from) and image size (objects sizes diminish when farther away).
Cues that can only be perceived by two eyes are called cues for stereopsis or binocular cues. These cues make the difference between a flat picture of for example a building and the building itself. Because eyes are horizontally separated they have different directions of view, thus different images of the world. These differences between the images of each eye are called binocular disparity. There are different ways in which these different visions (binocular cues) are simulated in VR. These cues can be implemented (with the help of VR-glasses or head-mounted displays). Binocular cues are diplopia, uncrossed and crossed disparity. Diplopia is the failure to merge two eyes views completely (double vision). Uncrossed disparity is the phenomenon that the unfused image on the right eye appears on the left eye and the unfused image on the left eye appears on the right eye. Crossed disparity is the phenomenon that the unfused image on the right eye appears on the right eye and the unfused image on the left eye appears on the left eye.
There are also some physiological cues that inform about distances from the observer to objects. These physiological cues lack in many VR environments because they can not be triggered by images on a flat screen. With computer images the camera imitates these physiological functions. The physiological cues for depth are accommodation, convergence, and divergence. Accommodation is the change of the lenses curvature or shape in order to focus the retinal images of objects at different distances (Dalziel & Egan, 1982). Convergence and divergence are respectively, rotation of the eyes towards the temples when fixated on objects far away and rotation of the eyes towards the nose when fixated on close objects.
Another depth cue that is often involved in VR is the motion cue (for example, in driving/fly simulators). It provides information about distance from the observer to objects and about the depth of an object. The motion cue for depth is motion parallax. This is the phenomenon that objects proximity to you and your fixation point rotate faster that objects farther away and move in the opposite direction to your movement, objects farther away appear to move in the same direction. There is also a relative pattern of movement of parts of a rotating object (= kinetic depth effect; Gibson, 1966; for examples see, Carpenter & Dugan, 1983; Doner, Lappin & Perfetto, 1984).
Finally, in real settings there always is a combination of depth cues. The accuracy of perception of distance often depends on the interaction of a number of cues.
These interactions of depth cues are:
There are not only visual cues for depth, but there are also cues for direction. These are also very important in the perception of space in VR. To determine the location of objects in space or to orient we integrate two types of directional judgements to sense up, down, right and left (Howard, 1982). These judgements are about body-centric direction (imaginary vertical line parallel to spinal cord) and about head centric direction (imaginary horizontal line through the nose).
Besides vision, sounds can be of considerable contribution to the immersive character of VR. To orient one self or determine the location of objects people use several auditory cues. To create realistic VR these cues must be taken into account. Auditory localization cues provide us with information about direction and distance of objects.
Auditory cues are for example, sound shadow (sound intensity at one ear differs from sound intensity at the other ear dependent on the angle of the sound source).
Kinesthetic cues, weight, pressure etc. can make VR even more immersive.
The triggering of olfactory senses will not be discussed here because it can hardly or not be simulated by cues. It can only be triggered by real scents. Note that smells can be very useful in VR. For example, when fire fighters have to learn how to deal with stressful situations during a rescue mission, it can be imagined that the scent of fire would be of considerable help.
Interaction versus demonstration
Besides the possession of the right cues there are other aspects that determine to what extent a virtual environment is realistic. The environment in which a reality is simulated can differ with respect to its level of (physical) interaction. There is VR where users are only able to observe, like architects that evaluate the construction of a building. There are also virtual environments were people can actually do something. Sometimes they can manipulate, like a surgeon that is operating in a virtual body and sometimes they can even interact in a virtual world, for example communicate with avatars (computer generated people in VR). They can do this with a mouse or even by walking. In very rare cases people can actually pick up things, extract them or rotate them with a mouse or gloves.
Worlds versus environments versus objects
The subjects that are realized in Virtual Reality can be divided into worlds (flying through the area in a simulator), separate environments (a control room of a ship; a virtual theatre on the web), and single objects (a complex machine, molecules).
Different combinations of perceptual cues, the amount of interaction and subjects of virtual reality can bring about all kinds of virtual realities. Current examples are physical mock-ups, projection of images, or combinations of these two. An example of a mock-up without computer generated images is the sinking ship that was used by TNO Human Factors Research institute to investigate the effect of different kinds of sign-posting aboard of a ship. It was a slanting wooden hallway with cabins, accompanied by the sound of a running motor. People were blindfolded and brought to a cabin on the ship. The assignment was to find the way from their cabin to the assembly station. Although the mock-up did not move, some people experienced movement. However, mostly mock-ups are supported by computer generated images, such as driving simulators and flight simulators.
The projection of images can be either projected on screens or inside of a head mounted display. When it is projected on a screen it can be on a big screen in front of you (VR-room, Holland Signaal) or around you (CAVE, SARA) or on a computer screen.
With respect to the available technical possibilities at Holland Signaal and the types of tasks that were chosen to experiment with (Chapter 3) it was decided to use virtual objects on a computer screen with which students can interact. Perceptual cues were restricted to monocular depth cues (and sound?).
2.2 TasksChapter 2. Theoretical background
2.1 ICT
As mentioned earlier differences in tasks can ask for other design characteristics from ICT. There are several ways in which tasks can be classified. Sternberg (1985) considers tasks to be a continuum between strategy-free tasks and strategy-intensive tasks. Strategy-free tasks are simple tasks, for example recognition tasks that have little person to person variation with respect to performance. They are so easy to do that people with different levels of preliminary knowledge can perform them all but equally well. Strategy containing tasks are somewhat more complex, like identification and comparison tasks. Because, each personals task strategy, like the use of mnemonic procedures, can be more or less effective, personal differences in task performance begin to obtain here. Strategy-intensive tasks involve assignments of cognitive complexity that require strategic intervention. These tasks elicit a great variety of individual performance levels. This classification especially refers to laboratory tasks such as recall or recognition tasks (For example, from Shiffrin, 1993).
When the interaction with a machine, such as a car or a radar system is involved a distinction of tasks can more specifically be derived from Rasmussens levels of cognitive processing. Rasmussen (1983) distinguished three levels of problem-solving behavior: skill-based behavior, rule-based behavior and knowledge-based behavior. According to these levels tasks could be divided into tasks that are done automatically (skill-based), tasks that are done on the basis of general rules that can be applied in different situations (rule-based) and tasks that are each time performed in whole new situations (knowledge-based).
Performance of skills is always automatically, such as changing gears. The acquiring of skills needed to perform tasks varies from extensive to less extensive learning processes. Skill-based tasks that need extensive learning processes demand for formal instruction and systematically training. Those that can be performed successfully with less extensive processes only need minimal formal instruction.
Rule-based tasks require former knowledge and can be made explicit, for example in the form of abstract symbols. An example of a rule-based task is giving right of way.
Knowledge-based tasks emerge there were problems cannot be solved by skills or rules, such as fixing unfamiliar car defects. (Van Leyden, 1993)
Table 1 shows the task characteristics that result in skill-, rule-, and knowledge-based behavior.
Skill-based behavior Rule-based behavior Knowledge-based behavior Task characteristics
Reason and Embrey (1985) put the relation between these tasks in a Generic Error Modeling System (GEMS). Figure 5 shows the relation and interaction between skill-, rule-, and knowledge based tasks.
Knowledge-based
No
No
Yes Yes
Skill-based (Automatic progress, interrupted by conscious checks)
Figure 5: The relation between skill-, rule-, and knowledge-based tasks; GEMS model
The GEMS model shows that people in principle rather try to solve a problem on an automatically (skill-based) sub optimal level that suffices, than that they search endlessly for the best possible solution (bounded rationality principle of Simon, 1957). Only when skill-based behavior is not working one switches over to respectively rule- or knowledge-based behavior.
These three kinds of tasks each had to be matched with a specific ICT design. Table 2 shows task performance of different kinds of tasks with different kinds of ICT designs.
Task performance ICT design 1 ICT design 2 ICT design 3 Task 1 (Skill) Task 2 (Rule) Task 3 (Knowledge) In the next paragraph possible ICT designs will be discussed.
2.3 ICT design
Systems get more sophisticated every day. Their functionalitys increase and thereby the amount of information that can be retrieved from them. Because available information rises, more and more information units are presented simultaneously, for example in different windows behind different icons. The big question is how all this information can be organized in order to maintain or create an efficient man-machine interaction. The same is true for the ICT applications that have to reflect these loads of data. An adequate selection must be made out of possible ways to represent all this in a proper manner.
Min claims it is effective to work with parallel instruction (PI-Theory, 1992). Hereby, he made a distinction of information that is continuously presented simultaneously and information that is absent sometimes, but is continuously direct available. These are respectively referred to as first order (no overlap) and second order (overlap) parallel presentation (Min, 1994, 1997). Figure 6 shows first and second order parallelism.
1st order 2nd order
Parallel presentation Parallel presentation
Figure 6: First and second order parallelism (Min, 1994, 1997).
There are several underlying reasons to base this theory on. Simultaneously presented information can serve as an external memory (Benshoof and Hooper, 1993). This decreases the chance of forgetting relevant information. Parallel information can also provide users with continuously available context information, such as spatial and informational cues (Tombaugh, Lickorisch, & Wright, 1987). An example of spatial information is the online lay out - option in Word that continuously presents a documents lay out while one is working with this document. This helps users to orient. An informational cue could for example be the time clock that stays active during working in a document. Another advantage of the direct availability of parallel information is that it decreases the search for needed information, because it is already present or directly available. Moreover, it increases the possibility to compare and associate this close at hand information.
A study of ter Burg and Groenewoud (1996) investigated the effect of information that was simultaneously presented on multiple screens (condition 1), compared to information that was presented on one screen with multiple windows (condition 2). On the screen with multiple windows the information was not continuously presented parallel, but was continuously direct available (second order parallelism).
Subjects had to perform a task with a simulation program, CARDIO. In this program medicines had to be administered so that the blood pressure changed. These changes had to be registered in tables. Results showed that task performance (percentage correct answers) was better in the first condition with multiple screens. A questionnaire showed that subjects (students at the University of Twente) also slightly preferred information that was presented on two screens. In addition, although there were no differences found in speed of working and perceived usability comfort, subjects thought they had worked faster in the multiple screen condition. Thus, for this particular task the more first order kind of presentation elicited better task performance than second order parallel presentation.
Figure 7 shows the experimental conditions of the study of ter Burg and Groenewoud.
Condition 1
Condition 2
Figure 7: The experimental conditions of the study of ter Burg and Groenewoud (1996)
As earlier mentioned these research results cannot be generalized to tasks in general. This is proved by research from Benshoof and Hooper (1993). Their experiments showed that second order information presentation elicited better task performance from high ability students than first order parallelism. Low ability students performed equally for single- (1st order), and multiple (2nd order) window treatments. In this case the task was more memory intensive than performing actions like registering gained information (Burg & Groenewoud). Here, subjects had to apply a formula in different situations (rule-based). The formula consisted of ciphers that were represented by symbols. To calculate sums subjects had to remember or look up which cipher was represented by a certain symbol. In the multiple window case subjects had to rely more on their short term memory than in the condition where the meaning of symbols was available on the same page as the sums. Benshoof and Hooper presumed that the former manner of presentation was more demanding with respect to students memory strategies. It forced students to process information more deeply and accurately resulting in better task performance. This explained why high ability students that used short-term memory processing strategies more effectively (Case, 1985) performed better than low ability students when they were forced to use their own learning strategies. In addition, high ability students could process more information at one time in working memory, because their memory strategies were more efficient (Case, 1985, Chi, 1978).
Figure 8 shows the task performance of high versus low ability students with single or multiple window presentations.
High ability students
Performance
Low ability students
Low
Single window Multiple windows
Figure 8: Task performance of high versus low ability students single with or multiple window presentations.
Thus, the external memory simultaneously presented information provides, increases the chance of using shallower information processing, what increases forgetting.
Another negative aspect that parallel presented information can trigger is that users may get confused when there is too much information available at one time (information overload).
Table 3 shows benefits and losses of presenting visual information in a parallel manner.
Parallel presentation of visual information Benefits Losses
Besides the simultaneous presentation of information in different windows (visually), information can be presented parallel via different modalities. The dual coding theory of Paivio (1986, 1991) deals with the processing of verbal (auditory) and pictorial (visual) information. He states that long-term memory contains two independent but interacting subsystems. One that presents the linguistic information (e.g. spoken information), and the other that represents non-linguistic information (e.g. pictures or object manipulation). The implication of the subsystems independence is that when the same information is coded both verbal and nonverbal it will have greater mnemonic power than only coded one way. In addition, according to this theory, if nonverbal information is processed in more than one sensimotor modality (e.g. visual and proprioceptive = receiving and processing stimuli that originate from the central nervous system), this will have additive effects in recall. Thus, the more codes that represent the information in memory, the better that information will be remembered. Several study results were consistent with this point of view. Cohen (1981) found that when subjects were read a series of instructions for mini tasks and then performed them, they were better able to recall them than when the instructions were just read to them. Salz and Donnenwerth-Nolan (1981) also found prove for the increase of recall when motoric enactment was involved. They compared sentence retention when sentences were repeated out loud with sentence retention when sentences were carried out ("The chef flipped the pancake").
An alternative theory, the text representation theory of van Dijk and Kintsch (1983), makes a distinction between good remembrance or recall and succesfully performing a task. It claims that for keeping information in memory people have to from a good propositional representation. This means that they have to have a good knowledge about the meaning of the text. For performing tasks, such as using a washing machine, people have to posses a good situational representation. Situational representations also represent the situation or context described by the text. Reading a text about how to use a washing machine would provide users with a propositional representation, whereas a picture of a washing machine would help creating a situational presentation. Unlike the dual coding theory the text representation theory predicts that bare text will give people the best basis for a good recall. There is no interference with other information so users can purely concentrate on the contents of the text (propositional representation). However, when one has to perform a task it is considered ideal to have more context information (situational representation). A study of Perrig and Kintsch showed that propositional representations indeed lead to better recall, while situational one's lead to more accurate inferencing (= derivation, deduction), recognition and problem solving.
To clear up the contradiction between the dual coding theory and the text representation theory Diehl and Mills (1995) set up experiments that would prove either one or the other theory. They asked subjects to read a procedural text about how to use a timer clock or about how to make and use a spool device. Afterwards they were respectively asked to recall the instructions, to answer true/false questions and to perform the task. Subjects were assigned to several different conditions, namely: Only read the text, read the text while looking at the device (clock or spool), read the text while performing the task, read the text while seeing the experimenter do the task and read the text while imagining the device. Results showed that according to the theory of van Dijk and Kintsch the read only and read seeing the device conditions elicited better recall and better performance on true/false questions. This research also proved that motoric enactment was only of additive value when task performance was the end goal. Doing the task, watching the experimenter do the task, and seeing the device described by the text while reading lead to a better situational representation and therefore in better task performances.
These results were also not in accordance with the results of the Cohens experiments (1981). This may be found in the fact that Cohen compared verbal information (spoken text) with verbal plus proprioceptive (doing the task) information, while Diehl & Mills compared visual information (read text) with visual plus proprioceptive (doing the task) information. Doing the task could have more additive value on difficult to remember spoken (auditory) information that is only effectively to be used in the case of short simple messages, than on easier to process visual information (ceiling effect). Unfortunately, for Cohen this could not be the case when task performance was involved. Other than with the recall results doing the task did add value to visual information (task performance was better in the see-do condition). In Table 4 situations are shown in which the use of visual or auditory presentations are preferred with respect to successful information processing (Deatherage, 1972 in Sanders & McCormick, 1992).
Rule-based
Yes
No Yes
|
Auditory presentations |
Visual presentations |
|
|
Other aspect that are expected to better task performance, but that was not mentioned up till now is imitation. Children learn by participating in a context. They are very good in associating. You have to make them understand the function of things by means of participation not just by pointing out external things. For example, teaching them words related to feeding while you feed them. Tomasello (x) found this for language acquisition in children. He divided processes of learning language into:
Imitation may also have its positive effects on teaching adults.
Conclusions & Discussion
There are several design variables that influence task performance. First, information can be visually presented in a parallel or in a serial manner. Second, information is presented through different modalities. Third, information can be text based or more context based (from pictures to motoric enactment, such as imitation).
The value of each variable should be matched to each specific task (skill-, rule-, and knowledge-based tasks). In the next chapter specific tasks will be selected. After that predictions can be made about the outcome of task-design combinations.
Benshoof, L. A. & Hooper, S. (1993). The effects of single- and multiple window presentation on achievement during computer-based instruction. Journal of computer-based instruction, 20 (4), 113-117.
Carpenter, D. L. & Dugan, M. P. (1983). Motion parallax information for direction of rotation in depth: Order and direction components. Perception, 12, 559-569.
Case, R. (1985). Intellectual development: Birth to adulthood. New York: Academic Press Inc.
Chi, M. T. H. (1978). Knowledge structures and memory development. In R. S. Siegler (Ed.), Children's thinking: What develops?, 73-96. Hillsdale, NJ: Erlbaum.
Cohen, R. L. (1981). On the generality of some memory laws. Scandinavian journal of psychology, 22, 267-281.
Cohen, S., Ward, L. M., Enns, J. T. (1994). Sensation and perception. Florida: Harcourt Brace College Publishers.
Dalziel, C. C. & Egan, D. J. (1982). Crystalline lens thickness changes as observed by tachometry. American Journal of optometry and physiological optics, 59, 442-447.
Sanders, M. S. & Mc Cormick, E. J. (1992) Human Factors in engineering and design. Singapore: McGraw-Hill Editions.
Diehl, V, Mills, C.B. (1995). The effects of interaction with the device described by procedural text on recall, true/false and task performance. Memory and cognition, 23, 6, 675-688.
Doner,J., Lappin, J. S. & Perfetto, G. (1984). Detection of three-dimensional structure in moving optical patterns. Journal of experimental psychology: Human perception and performance, 10, 1-11.
Gibson, J. J. (1950). Perception of the visual world. Boston: Houghton Mifflin.
Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin.
Howard, I. P. (1982). Human visual orientation. Chichester: Wiley.
Marr, D. (1982). Vision. San Francisco: W. H. Freeman.
Min, F. B. M. (1997). Nieuwe ontwikkelingen op het gebied van computersimulaties en multimediale leermiddelen in het hoger onderwijs: Het World Wide Web en Java. New developments in the discipline of computer simulations and multimedial learning tools in higher education. Tijdschrift voor hoger onderwijs & management. Themanummer hoger onderwijs online, 4.
Min, R. (1992). Parallel instruction, A theory for educational computer simulation. Interactive learning international, 6 (3), 177-183.
Min, F. B. M. (1994). Parallelism in open learning and working environments. British Journal of Educational Technology, 25, 2, 108-112.
Rasmussen, J. (1983). Skills, rules and knowledge: Signals signs and symbols, and other distinctions in human performance models. IEEE transactions on systems, man and cybernetics. Vol SMC-3, 3, 257-268.
Reason, J. T., & Embrey, D. E. (1985). Human Factors principles relevant to the modeling of human errors in abnormal conditions of nuclear and major hazardous installations. Dalton, England: Human reliability Associates.
Regan, D. M., & Beverley, K. (1973). Disparity detectors in human depth perception: Evidence for directional selectivity. Science, 181, 877-879.
Regan, D. M., & Beverley, K. (1979). The visual perception of motion in depth. Scientific American, 241, 136-151.
Regan, D. M., Frisby, J. P., Poggio, G. F., Schor, C. M., & Tyler, C. W. (1990). The perception of stereodepth and stereomotion: Cortical mechanisms. In L. Spillman & J.S. Werner (Eds.) Visual perception, 317-347. New-York: Academic press.
Salz, E., & Donnerwerth-Nolan, S. (1981). Does motoric imagery facilitate memory for sentences? A selective interference test. Journal of verbal learning & verbal behavior, 20, 322-332.
Shiffrin, R. (1993). TODAM and the list length and list strength effects: Comment on Murdock and Kahana. Journal of experimental psychology, 19, 6, 1445-1448.
Simon, H. A. (1957). Models of man. New York: Wiley.
Sternberg, R. J. (1985). Human Abilities; an information-processing approach. W. H. Freeman and company: New York.
Tombaugh, J., Lickorisch, A., & Wright, P. (1987). Multi-window displays for readers of lengthy texts. International Journal of Man-Machine Studies, 26, 597-615.
Paivio, A. (1986). Mental representations: Dual coding approach. NewYork: Oxford University Press.
Paivio, A. (1991). Dual coding theory: Restrospect and current status. Canadian Journal of Psychology, 45, 255-287.
Perrig, W. & Kintsch, W. (1985). Propositional and situational representations of text. Journal of memory & Language, 24, 503-518.
Van Dijk, T. A. & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.
Van Leyden, J. (1993). Psychologische Functieleer, theorie, techniek, toepassing; Psychonomy, theory, technology, use. Bohn Stafleu Van Loghum: Houten/Zaventem
Hollandse signaal apparaten B.V. is a Dutch defense electronics company that develops and produces naval combat-, and battlefield detection systems. The main production line consists of surveillance-, weapon control-, or combat management systems. Systems such as APAR, SMART-L and MIRRADOR are made to detect, track and / or identify (hostile) objects. Besides the Dutch navy, Holland Signaal, serves clients all over the world within the political boundaries. Figure 9 shows the structure of the organization. Because Holland Signaal is a big organization, the figure only shows the important (sub) divisions (ILS, Training) that were relevant for this study.
Figure 9: The structure of the organization Hollandse Signaal Apparaten B.V.
The division Integrated Logistic Support (ILS) is responsible for all support concerning delivered systems. Logistic support is the responsible department for the provisioning of appropriate equipment and services in order to support the system during its lifecycle in a way that it performs optimally against acceptable minimal costs. This support lasts as long as a systems lifecycle, and is seen as integrated because it merges the expertise of the different subdivisions of ILS. They have to work and depend on each other. There are several subdivisions operating closely together in order to tune the integrated activities to efficient and effective customer support. They are the subdivisions: Documentation, Training, Provisioning, Management & Engineering (PME) and Support.
Engineering starts to make up an inventory of all possible functionalitys and defaults of the system in relation to its purpose. Thorough analysis will be made about: Mission and Operational Availability, Functionality and Modeling, Failure Mode Effects and Criticalitys, Reliability concerning Maintenance, Maintenance Tasks, Levels of Repair, Spares and Life Cycle Costs. The data for these analyses are gained with the co-operation of the other Signaal departments.
A Management team will study the analysis reports of engineering and negotiates with the sales managers and customers. Then they assign appropriate tasks to each sub discipline. In planning and dividing all the work, the division Support supports the management team. Provisioning will take care of the storage and supply of tools, spare parts and test equipment, while Documentation will provide technical manuals.
The tasks of Training are to develop adequate education and to see to it that customers can support themselves during missions with the help of manuals, purchased spares, tools and test equipment. When a radar system is sold to the customer one of the purposes of ILS is to see to it that in the end the personnel of a navy ship can control a complete SEWACO (Sensor-, WeApon-, and Command) system. Here, the department training fulfils a major role. People with different preliminaries have to be trained to deal with the system. Figure 10 shows the structure of a SEWACO system. The ship has a platform with equipment, an organization, a combat system and there are procedures according to which will be worked. A complete combat system consists of radar systems (sensors on top of a ship), a command and control center (control room on the lower deck of a ship) and effectors (weapons on top of a ship).
Figure 10: The SEWACO (Sensor, WeApon and Command) system on a warship
There are courses for officers, operators and maintainers. These people all have their specific tasks. In Figure 11 it is shown that in a control center, officers and operators each have their own role within the operational process.
In addition, there are different kinds of officers and operators that each take care of another part of the operational process.
Picture Compiler Operators guard the situation via the sensors. They register and map all information on the screen. For example, when they spot an object on the screen, they try to identify whether this object is hostile or not. Surface Picture Compilers (SSPC) keep an eye on targets under water, Surface Picture Compilers watch objects on the water, Link Control operators (LINCO) keep in contact with colleagues on friendly ships to gain information about their picture of the situation, and Air Picture Compilers (APC) watch the sky for objects.
The information of the Picture Compilers will be sent to the Warfare Officers and on the basis of this image of the situation they evaluate and make tactical decisions. There are officers that are specialized in Anti Surface Warfare (ASW, on the water), Anti Sub Surface Warfare (AsuW, under water), Electronic Warfare (EW) and Anti Air Warfare (AAW). In the end the officers determine the actions (tracking or firing) weapon operators have to perform.
Weapon Operators are all specialized in a specific effector. There are Weapon Operators specialized in STIR (x), in WCO (x) and in missiles. They have to act according to the instructions of the officers, what changes the situation and asks for a new establishment of the situation, new tactical decisions, new actions etc.
Situation
Establish situation Action
Evaluation and decision
Figure 11: Operational process in a control room
Thus, officers are trained to manage the activities of operators. Therefore they have to know related performance of the whole or partly related SEWACO system. They are the ones to make the important tactical decisions.
Apart from officers and operators, maintainers have their own procedures to follow. They have to perform preventive maintenance and when appealed to by officers or operators they have to perform corrective maintenance.
In the next paragraph it will be shown what kind of systems operators and maintainers got to deal with and more information will be given about the specific tasks they have to perform. Finally, a decision based upon the literature review from chapter 2 and upon the requirements of Holland Signaal will be made about what tasks from what system will be used to experiment with.
There is a large scale of different sensor systems and tasks available at "Hollandse Signaal". A radar system and accompanying tasks had to be selected to experiment with. A necessary criterion is that the tasks can be defined according to Rasmussens levels of information processing (Chapter 2, paragraph 2.2). For the company Holland Signaal it had to be tasks from an actual system.
References
Chapter 3. The company Hollandse Signaalapparaten B.V.
To find an ideal design for CBT and VR as support of Signaal courses, involved tasks, systems, users, and the context in which they will be used were inventoried. The organizational structure of the company (§ 3.1), its systems and tasks (§ 3.2) were described. Because some important background information was not available in (Signaal) documentation, this information was gained by means of interviews with teachers from Holland Signaal (§ 3.3). These interviews revealed the companys specific teaching circumstances and bottlenecks. Finally, design requirements that had arisen from the obtained information were given in paragraph 3.4.
3.2 Systems and tasks
The systems that have to be supported and maintained are situated on naval vessels. There are a number of radar systems on the upper deck, there are several operator consoles (MOCs) in the control room between-decks and on the lower deck there are cabinets that contain the hardware of the radar systems. A complete defense system always exists of at least two kinds of radars: Surveillance radar systems and Tracking radar systems. Surveillance systems guard the area for hostile objects and tracking systems are built to follow spotted objects and measure all its data (Signaal, 1997). Figure 12 shows a selection of systems that are placed on a frigate.
Insert specific radar system chosen to do experiments with, the reason for this choice and a description of the system.
Figure 12. Holland Signaal" systems on a naval vessel
Operator tasks
The task of the operators is to assess and monitor the tactical situation and to control subsystems such as fine control equipment and radars. During a mission the operators sit behind their MOCs (Multifunction Operator Consoles) on which the application TACTICOS (Tactical Information and Command System) is active. All functions in the combat information center on board of a modern warship can be controlled and monitored from the multifunction operator consoles. The MOCs modularity allows a great variety of console configurations and console arrangements. Via the MOC the operators can select one of the task related capabilities of TACTICOS, such as: picture compilation, tactical command support or fire control based tasks (threat evaluation, weapon assignment and sensor assignment = TEWASA). TACTICOS will also provide status reports of the subsystems. As soon as (sub) system failures are detected the maintainers will be warned.
Operators do not have to have a technical background. Although the tasks they do are based on rules and facts, they have to be performed automatically / without thinking. It can be concluded that many of their tasks are most likely to be performed on a skill-based level (Chapter 2, paragraph 2.2).
Insert operator task chosen to do experiments with, the reason for this choice and a description of this task
Maintainer tasks
Maintainers can be educated in On board Level Maintenance (OLM), in Intermediate Level Maintenance (ILM) or in Depot Level Maintenance (DLM). The maintenance that is executed by the ships crew with materials available aboard is called On board Level Maintenance. The maintenance that is executed by people ashore is called Intermediate Level Maintenance. This maintenance is performed for example when a ship rests in a dock. ILM-maintainers work with materials available ashore, such as measuring instruments. The other type of maintenance that is performed by people ashore is Depot Level Maintenance. This work requires more specific knowledge. DLM-maintainers are expert in a small domain. They can do for example the complete overhaul of a radar system.
These three kinds of maintenance can include preventive and corrective maintenance. To perform preventive maintenance they use Maintenance Requirement Cards. These cards show time schedules of when and how to change units or perform other small jobs to prevent the system from decay. Preventive maintenance for example is the change of a component that has reached the end of its lifetime or regular checks and adjustments. Corrective maintenance deals with technical problems that can occur at any moment. Corrective maintenance is done on the basis of strict procedures with the help of BITE (build in test). This means that the system checks itself for defects. When a fault report comes in from an operator or an officer this fault has to be detected and identified ((isolated and diagnosed as type of fault). Then it has to be decided which task is needed to putt this fault right (identify task) and on the basis of available information (documentation about the tools, tests and skills that are needed) the task has to be executed (test). In the end a report has to be written. When faults cannot be detected with built in tests alternatives have to be found to solve these problems. Figure 13 shows the faultfinding process called Fault diagnosis. Fault diagnosis is a process to find and isolate the cause of malfunctions. Once isolated the correct actions (such as task identification) are to be executed in order to restore the function.
Thus, maintainers have to learn to keep a system in shape and to occasionally commit corrective maintenance. To do this they use technical manuals, tools and related test equipment that support them in performing their tasks and function as guides.
No
No No
Figure 13: Process of fault diagnosis.
The tools that maintainers have to be used differ per kind of job and apparatus. Nowadays, the use of measuring equipment becomes less because hardware is more and more operated by software. An example of equipment that was already mentioned the Maintenance Requirement Card. This is the schema that shows information about how and when to carry out preventive maintenance. Other tools are procedure books, Software installation manuals, laptops and measuring equipment. In addition, personnel can be trained in order to act as one team during operations (sea operability team training courses).
Courses normally contain both a theoretical and a practical part.
As previously said there are several levels (OLM, ILM, DLM) in which maintainers can be trained. These levels do not always give much information about the difficulty of a course. They are more bound to the location the maintenance takes place. The depth or difficulty level of a course will be determined in consultation with clients and depends mostly on the following two aspects:
Courses are divided into four difficulty levels: 1, 2, 3, and 3+. All levels include looking whether the system works or not. The first two levels include the transfer of very simple knowledge or skills. For example, cleaning or changing the system units due for replacement (preventive maintenance tasks). There is no technical education needed to learn this. Some teachers consider that the first two levels are insufficient to give students the feeling that they are capable of performing their maintenance tasks. Course 3 and 3+ both involve corrective maintenance (solving technical problems that can occur) with the help of measuring equipment. Students of level 3+ are also given enough background knowledge of the system to be able to solve problems that emerge unexpectedly and can not be solved with the available procedures. Reasoning is required and detailed schemes will be used. Table 5 shows the different difficulty levels of maintenance.
|
Difficulty levels of maintenance |
||
|
Level 1 / 2 |
Level 3 |
Level 3+ |
|
|
|
The first three levels all involve rule-based behavior. Procedures have to be followed in different situations (Chapter 2, paragraph 2.2). However, level 3+ deals with problems that did not occur yet. This situation, were no fixed rules / procedures are available demands an appeal on maintainers reasoning capabilities and can therefore be described as knowledge-based behavior (Chapter 2, paragraph 2.2).
A more detailed selection of tasks that maintainers have to learn was inventoried and shown in table 6.
|
Tasks |
|
Insert operator task chosen to do experiments with, the reason for this choice and a description of this task
Besides this selection out of the systems Holland Signaal produces and the scale of tasks they have to teach, specific design requirements have to be extracted adjusted to the specific situation at Holland Signaal. The next paragraph contains an exploratory investigation about potential users of Signaal ICT and the context within which it will be used.
Some contextual aspects, such as circumstances under which teachers have to give courses, and aspects such as teachers opinion about Virtual reality, were not available in (Signaal) documentation. Therefore, it was decided to explore the expert knowledge of teachers about certain topics by means of interviews.
3.3.1 Methods
A qualitative method of investigation was maintained, because the goal was to explore attitudes, opinions and eventual problems with respect to the teaching circumstances (Swanborn, 1987). Thus, not only information about the daily routine of teaching, but also details about present bottlenecks was collected. More specific it was aimed to gain information about the course in general, teaching methods and the students in order to extract eventual criteria that would elicit specific design requirements. Some informal conversations with several teachers were held to discover points of interest that seemed to be of important influence on the implementation of ICT in training. On the basis of these conversations a checklist was created for interviews with teachers. At first a questionnaire was made, but after a small pilot is seemed better to interview the teachers in person. Teachers had problems with providing concise answers to the strict systematic questions without spending a considerable amount of their time to write their complex answers down as neatly arranged surveys. The questionnaire was now used as a topic list. During the interview the questions from the questionnaire were discussed.
The scientist that had these conversations and interviewed the teachers participated in the organization as a staff member. The interviewer and interviewee knew each other from the shop floor and it was common knowledge that she was doing research about the use of ICT applications for training Signaal clients.
During the conversations the interviewer brought up subjects like the nature of the courses, the type of students and the learning material, without expressing her own opinion about these matters. Table x shows the topics that were treated in the interviews.
Subjects
The interviews were taken from teachers from the subdivision Training. All the available teachers (some of them were sent on secondment or were abroad to give a course) were interviewed. Seven teachers participated. They all had a background of high technical education. Their ages ranged from 40 to 56 years of age, except for one of them who was 26 years old. Their teaching experience ranged from two years (one teacher) to 26 years.
Procedure
The subjects were interviewed in a private room about the topics of the questionnaire. All respondents were given the same introduction to read in advance and during the interview the questions were formulated according to questions that were drawn up beforehand to increase the intern validity (Swanborn, 1987) (Appendix 1). The interviewer adhered to the interview techniques as described in Emans (Emans, 1990). Respondents answers were continually summarized to check mutual understanding. In addition, subjects were encouraged to explain their answers.
Data analysis
Answers to the questions were summarized, recorded and then classified on the basis of their contents (Appendix 1). The questions about the maintainer levels, systems, tasks and tools (respectively questions 1, 2, 3, 6 and 13b) were not mentioned in table 7 and not incorporated in the results, because they did not yield information additive to that provided in paragraph 3.2.
|
Course |
Teaching methods |
Students |
|
|
Problems |
|
|
|
|
Attitudes & Opinions |
|
|
|
3.3.2 Results
During the informal conversations it already became clear that things as the potential marked and wishes from the customer determine the kind of products or systems that Holland Signaal produces. A major subject that was stressed was the flexibility that was demanded from teachers, among others because of the varying type of students and the restricted availability of systems and documentation.
In the following paragraphs the answers of the interviews are arranged in tables according to their contents. In addition the frequency with which they were mentioned was rated.
3.3.2.1 Problems
Students entry level (7) and their inability to comprehend or speak the official course language (English or Dutch) (5) were the biggest problems. An important complaint of teachers about the course was that the available time was not enough (4). Courses are too short to deal with all the subject matter in the difficulty level that was planned. The remark that the courses are too difficult for many students is related to this lack of course (preparation) time (3). The more difficulty students have with the subject matter the more time is needed for extra training, further explanation etc. Another aspect that may interfere with a smoothly way of teaching is the hierarchy student groups may have (3). Most groups exist of people with different ranks. This often complicates the way of teaching. Teachers have to be both cautious what to tell or ask students with a higher rank (to prevent them from loss of sight) and what to tell or ask students with a lower rank (to prevent them from sanctions). Problems with the availability of the system were generally caused because it was not ready or defect (3). The bad availability of information consisted mostly of documentation or procedures that were not ready yet (2). Some student groups were trained in different subjects and because each teacher has its own specialty these students are brought into contact with different teachers. This may cause problems because each teacher at Holland Signal has his own way of teaching. A problem that was mentioned was that sometimes students became unmotivated during classes of previous teachers, and that this is very difficult to reverse (1).
Table 8 shows mentioned problems with the course in general, the teachers and the students. In addition, they were ranked on the basis of the frequency with which they were mentioned.
|
Problems with course, teachers and students |
# problems mentioned |
|
7
5
4
3
2
1
|
There were also some questions about specific problems with students: Problems with the age of students, problems with their lack of computer experience/knowledge and with their cultural background.
Three teachers stated that age is not of influence on the successful performance of students. In addition, it seemed that the advantages of being young were advantages of being old. For example, were young students had less practical experience, older students usually had.
Younger students were said to be less motivated (2), have less practical experience (2), have a shorter history of education, have more problems with hardware technique and have only knowledge about their own discipline. Older students were expected to have less experience with computers (3), need more time to learn (3) and to have more problems with newer systems and techniques.
Experience with computers was also found of small relevance to the performance of students. However it was mentioned as a convenient characteristic with surplus value (4). "It is easier to understand when a student has experience with the computer". In addition, it was said to become more important nowadays. At the same time the number of students who do not know much about computers is decreasing.
Culture was already at the basis of some previously mentioned problems, such as problems with language and group hierarchy. The specific question about problems with cultural differences also elicited some other aspects. Students learning methods differed. From one culture students were used to learn everything by heart while students from another culture were used to reason and improvise (2). Students customs could also elicit problems. For example, normal behavior in one culture can be considered as rude in another culture. Just as the entry level differed per culture the quality of their military and civil personal did too. Some country placed more importance upon their military personal and others on their civil personal. Finally, affinity with technology and motivation were also seen as typical traits for students from some particular countries. It cannot be claimed that the problems that were mentioned are always necessarily caused by a cultural difference. Another fact is that problems cannot be totally generalized to all the students from one culture. Thus, problems mentioned were considered to be typical in general for one particular culture.
Ideal situation
Although it is reasonable to assume that an ideal course should contain all the characteristics opposite to the ones that were mentioned in the previous paragraph, there were also other aspects that were considered to be important for a good course, teacher or student.
For example, enough time with the same student group. Sometimes more teachers give the same courses because all teachers have their own special expertise. Another aspect mentioned is that it is ideal to work with small student groups. This is because with rehearsing on the system only a maximum of three persons can practice at the same time.
The question what characteristics a good teacher should possess, was answered more enthusiastically. The characteristics could be roughly divided into didactic skills, social skills and expert knowledge.
Table 9 shows characteristics that were considered to determine the quality of a course, teacher, student and the frequency with which they were mentioned
|
Ideal course |
# characteristics mentioned |
|
12
7
5
4
2
1
|
Teaching methods
Questions related to teaching methods were about the goals teachers intended to reach at the end of a course, the way they structured their lessons into theory and practice, the teaching tools they used and their opinion about Computer Based Training and Virtual Reality. First questions were asked about teachers.
Goals: In general the goal of teachers is to teach students the most important tasks (skills) and knowledge. These two are rated about equally important. Tasks that were mentioned are recognize whether system works, faultfinding and interpreting fault reports, exchanging procedures, tuning procedures and learning to install. With respect to knowledge teachers demanded comprehension and overview, knowing technical details and knowing the meaning of buttons.
Table 10 shows the goals that where mentioned by teachers and how many times they were referred to.
|
Teaching goal |
# goals mentioned |
|
5
4
1
|
To test whether the aimed goal was reached at the end of a course teachers did not use formal oral or written tests. This was because it would be an invasion of students privacy. It could for example endanger a students career. As a consequence teachers use alternative manners to measure students capabilities. These informal tests can briefly be named observation judgements. Table 11 shows the nature of these observations.
|
Tests to measure whether the teaching goal was reached |
# tests mentioned |
|
4
3
2
1 |
Theory & practice: Officially a maintainer course should contain 50% of theory lessons and 50% of practice lessons. The time that teachers get to prepare a course is 1,5 times the length of the course. In advance there is contact with the client about what they want their personal to be trained in. The current situation is that these times vary now and than (see Table x). Some teachers came up with ratios specific for certain tasks: Faultfinding: Theory: 20% - Practice: 80%; Fine-tuning Theory: 0% - Practice: 100%, and Operator course: Theory: 10% - Practice: 90%.
Table 12 shows the ratio of theory and practice that teachers use with maintainer courses
|
Ratio theory-practice maintained by teachers |
# ratios mentioned |
|
3
3
1 |
|
Reason for ratio theory-practice |
# reasons mentioned |
|
3
2
1 |
The amount of theory and practice teachers used in their lessons did vary from time to time according to the given circumstances. For example, the system should be available for 50 %, but in reality this is not always the case (3). The entry level of students is also a determinant (3). When students have much experience there is less need for practice and when they have much knowledge there is less need for theory lessons and vice versa. In addition, the more difficult tasks are the more theory lessons are needed. Finally, there is only room to practice with maximally three students at the system. So with large student groups less time can be spend with the system. Table x shows the reasons teachers had to maintain a certain ratio of theory and practice.
There were teachers that explicitly judged theoretical knowledge to be more important than practical skills (3). Reasons were that skills are easy to learn, so not many time had to be spend on this and the fact that build in tests (BITE, system checks himself on failures) from the systems are not perfect just as the procedures from procedure books. Other teachers considered knowledge as a basis (3). Students have at least to understand a bit of the system as a whole, what is in it, the location of all system parts/ units and how it functions (for example as with a car). They have to gain insight in the system. On teacher stated that theory and practice are equally important. Table 14 shows reasons for the need for theoretical knowledge.
|
The need for theoretical knowledge |
# knowledge mentioned |
|
|
3
1
|
An equal amount of teachers (3) stated that practice is more important than skills. "Practice is the final goal". In the end procedures have to be done without thinking (automatically) The reasons given were that practically dealing with the system was the final goal and that in the end procedures had to be done without thinking (automatically). Table 15 shows reasons given for the need for practical skills.
|
The need for practical skills |
# skills mentioned |
|
3
1
|
Teaching tools: The materials and apparatus teachers used in their lessons were mostly the same for all teachers. For example, all the interviewed teachers used handouts (with block diagrams and service schemes or schemes to adjust), blackboard and overhead projectors. On teacher said to use demonstration materials such as transmitters. Table 16 shows the teaching tools that were used for the maintainer courses.
|
Teaching tools that are used |
# teaching tools mentioned |
|
7
4
2
1 |
ICT applications: Teachers were also questioned about more modern tools, namely ICT applications such as Virtual reality and Computer Based Training. Teachers agreed with the use of VR and CBT on the condition that they would not replace the teacher. Students need social contact and personal guidance. One teacher stated that possibly later students will get used to education without teachers, but now it would be too radical. Table 17 shows the mentioned advantages and disadvantages of Virtual reality.
|
Advantages of Virtual Reality |
# advantages mentioned |
Disadvantages of Virtual Reality |
# disadvanta-ges mentioned |
|
1
|
|
3
1
|
It did not become very clear how teachers would implement good educational Virtual Reality.
They mentioned both passive VR (On a large screen, for example the Signaal VR room with moving 3D images) (5) and active VR (with head-mounted display and glasses, possibilities to walk around objects and enter them plus possibilities to perform actions) (5). However, many of them did not regard active VR as a realistic option in the near future. One teacher mentioned the importance of feedback by means of alarm noises that warn for illegal operations. And another advised to keep the sequence of actions intact.
Almost all teachers put forward that Virtual Reality could not replace the real system (6). This is because VR is never the same as the real thing. One teacher suggested using VR as training tool before working on the real system, whereas another suggested using it as replacement for practice lessons.
Table 18 shows what subjects are appropriate to be learned with the help of Virtual Reality according to the teachers.
|
Ideal subjects to teach with the help of Virtual reality |
# subjects mentioned |
|
5
2
1
|
Table 19 shows the mentioned advantages and disadvantages of Computer Based Training.
|
Advantages of Computer Based Training |
# advantages mentioned |
Disadvantages of Computer Based Training |
# disadvanta-ges mentioned |
|
3
2
|
|
4
1 |
According to the teachers there were several learning subjects that are appropriate to be learned with the help of Computer based training. Table 20 shows these subjects.
|
Ideal subjects to teach with the help of Computer Based Training |
# subjects mentioned |
|
5
2 |
Teachers would provide good educational Computer based training with possibilities to interact ("pushing buttons resulting in actions") (2) and animations (2). In addition they would build in the possibility to record every act in enormous databases.
CBT is not seen as an application that will replace teachers. Teachers could refer to CBT so students could access the material at their own time. It can also be seen as a "tool" to practice maintenance in advance when the system is not available. Half of the students could use CBT while the other half practices with CBT. The amount of CBT can be adapted to the students affinity and / or experience with the subject matter. Another possibility is that CBT will serve / be sold as renewal course abroad. One teacher stressed the importance of instruction from a book.
3.2.3 Conclusions & Discussion
During the informal conversations it already became clear that in the first place things as the potential marked and wishes from the customer determine the kind of products or systems that Holland Signaal produces. As a consequence it is already certain what will be the learning domain (subject matter) of the courses that are involved in these systems. Second, regardless of certain requirements from Holland Signaal as to preliminary knowledge and skills, in practice customers decide themselves what kind of people they will send for a course. This was endorsed by the interview result that students entry levels in general appeared to be too low. Thus another unchangeable factor (and as was proved by the interview results also a largely varying factor) here are the students. Finally, interview results confirmed the importance of teachers flexibility.
What we can conclude from these interviews with respect to the development of ICT applications for training maintenance tasks is particularly that these applications cannot be used independently from the real system or teachers. First, rehearsal with the real system during practice lessons is inevitable. Till now CBT or VR cannot simulate the real system completely (actual size of system, weight of units, etc.). Second, lessons should be adapted to each specific student group. Potential CBT lessons should be selectively assigned to different students with respect to their contents and difficulty level. In addition social contact can be quite necessary to motivate students or make them feel at ease. Teachers were also skeptical about the costs (money + effort) of ICT applications. However, ICT applications combined with teachers may offer extra help. They are easier to edit than paper manuals. They also could decrease the lack of rehearsal with the system (no system available, system broke, only small amount of students can rehearse with the system at the same time) by means of simulations of the system with which students can interact. Especially tasks that occur often (recycling of ICT) and that are difficult (extra practice with ICT) or time consuming (ICT unburdens costly time with system) are profitable too put into an ICT application. Finally, the individual character of ICT applications can partly take care of the group hierarchy problem. Students who are afraid to ask questions because of the presence of their superiors are now more easily able to look up information any time they like in private.
Table 21 shows the advantages and disadvantages of ICT for maintainer courses.
|
Advantages ICT applications |
Disadvantages ICT applications |
|
|
To be used within the context of maintainer courses at Holland Signaal, ICT should answer to certain design requirements.
The main teaching goals are to teach students the most important tasks (5) and to transfer necessary understanding and knowledge about the system (4). Theory and practice are both seen as important, but the emphasis on them varies. Almost half of the teachers emphasize theory (3) and the other half stresses the importance of practice (3) (see also Table x). One of the teachers rated theory and practice equally important. In the end both knowledge and skills have to lead to correct actions of maintainers. As a consequence ICT applications should contain information needed to come to a successful task performance. This needs practice and because of the earlier mentioned bad availability of the system practice with ICT could be useful. There could be made a simulation of the system and / or task procedures. Hereby, the interactive aspect should be taken into account in order to guarantee the value of reality.
This possibility of extra rehearsal (with simulations) is also useful because intended users (students) of Signaal ICT usually have a low entry level and a bad control of the English / Dutch language. As a consequence ICT applications should not contain large pieces of (complicated) text, but visualizations to get rid of the abstractions from difficult concepts. Because of the varying entry levels and control of the course language ICT should be flexible with respect to speed and difficulty. Just as teachers have to adapt the speed of lessons to (the entry level of) students, ICT design should allow the student to maintain his / her own learning speed. In addition, as previously mentioned the teacher should assign students to the right kind of lessons or difficulty level, unless the student is able to judge this for him/herself.
Registration can, in addition to the now used observation methods, be used to get insight in the success of a students participation to the course.
Table 22 shows design requirements of ICT to be used within the context of Holland Signaal.
|
Situation |
Design requirements |
|
End goal: successfully performing tasks
Adapted to bad availability of the system
Adapted to low entry level of users + bad control of official course language
End goal reached? |
|
The design requirements made in this and in the previous chapter will be processed to a couple of experimental designs. In the next chapter these will all be assigned to several tasks and tested.
Emans, B. (1990). Interviewen; Theorie, techniek en training. Interviewing, technique and training. Wolters-Noordhoff, Groningen: The Netherlands.
Signaal (1997). Van oorschelp tot radar; de geschiedenis van een bijzondere onderneming. From auricle till radar; the history from a special company. Drukkerij Twente Hengelo, Hengelo: The Netherlands.
Swanborn, P. G. (1987). Methoden van sociaal-wetenschappelijk onderzoek. Methods of social scientific research. Boom, Amsterdam: The Netherlands.
Enschede, op web gezet dd. sept. 9, 2002
A Introduction interview
Beste docenten,
In de loop der jaren is het onderwijs steeds veranderd. Er wordt veel onderzoek gedaan naar methoden die moeten leiden tot een effectieve en efficiënte manier van opleiden of trainen. De vraag is in hoeverre de afdeling ILS hiervan zou kunnen profiteren.
Om hier achter te komen is het belangrijk om eerst stil te staan bij hoe de zaken er nu voor staan.
Ik zou graag gebruik maken van uw expertise als docent bij Holland Signaal om meer inzicht in deze kwestie te krijgen. Bij deze wil ik uw mening vragen over de volgende onderwerpen:
Er zijn geen goede of foute antwoorden mogelijk. Het gaat om uw mening.
Dit interview gaat alleen over de Maintainer cursus.
Alvast bedankt voor uw medewerking.
B Topic list interview
Naam:
Datum:
Doel
4a Wat is (zijn) uw doel(en) bij het geven van een maintainer cursus?
4b Lukt het u vaak dit (deze) doel(en) te bereiken?
4c Waaraan merkt u of uw doel(en) bereikt is (zijn)?
Methoden
Leermaterialen = Al het materiaal dat gebruikt wordt in de cursussen voor het aanleren van de onderhoudstaken (boeken, sheets, elektronische handboeken, enz.)
Hulpmiddelen = Materiaal dat gebruikt wordt ter ondersteuning van het uitvoeren van de onderhoudstaken (bijv. corrective maintenance card)
5a Wat voor soort leermaterialen gebruikt u bij de lessen?
5b Waarom hebt u voor deze leermaterialen gekozen?
6 Wat voor soort hulpmiddelen worden gebruikt in de maintainer cursussen?
7a Hoe denkt u over het gebruik van Virtual reality bij Maintainer cursussen?
7b Hoe vindt u dat Virtual reality moet worden ingericht om het succesvol te kunnen gebruiken bij de verschillende niveaus Maintainer cursussen?
8a Hoe denkt u over het gebruik van Computer Based Training bij Maintainer cursussen?
8b Hoe vindt u dat Computer Based Training moet worden ingericht om het succesvol te kunnen gebruiken bij de verschillende niveaus Maintainer cursussen?
9a Uit hoeveel procent theorie en praktijk bestaat een maintainer cursus bij u?
Praktijk: .. %
9b Waarom houdt u deze theorie praktijk verdeling aan?
10 In hoeverre vindt u het leren volgen van procedures belangrijk? Leg uit?
11 In hoeverre vindt u het leren begrijpen van het systeem belangrijk (weten waarom wat gebeurt)? Leg uit?
Problemen
12a Wat zijn veelvoorkomende problemen met het geven van Maintainer cursussen?
12b Wat voor eigenschappen zou een goede cursus volgens u moeten hebben?
13a Wat zijn veelvoorkomende problemen met gebruik van leermaterialen?
13b Wat zijn veelvoorkomende problemen met het gebruik van hulpmiddelen?
14 Heeft u over het algemeen tijd genoeg om al de leerstof te kunnen behandelen? Leg uit?
Problemen
15a Wat zijn veelvoorkomende problemen met cursisten?
15b Wat voor eigenschappen zou een goede cursist volgens u moeten hebben?
16a Wat zijn veelvoorkomende problemen die cursisten hebben met docenten?
16b Wat voor eigenschappen zou een goede docent volgens u moeten hebben?
17a In hoeverre is de leeftijd van cursisten van invloed op het succesvol volgen en afsluiten van een cursus? Leg uit?
17b Komt het vaak voor dat de leeftijd van cursisten problemen oplevert? (Als u vindt dat leeftijd geen invloed heeft, kunt u deze vraag overslaan)
18a In hoeverre is de ervaring van cursisten met computers van invloed op het succesvol volgen en afsluiten van een cursus? Leg uit?
18b Komt het vaak voor dat de mate van ervaring met computers problemen oplevert? (Als u vindt dat ervaring met computers geen invloed heeft, kunt u deze vraag overslaan)
19a In hoeverre is relevante theoretische kennis van cursisten van invloed op het succesvol volgen en afsluiten van een cursus? Leg uit?
19b Komt het vaak voor dat de mate van theoretisch kennis problemen oplevert? (Als u vindt dat theoretische kennis geen invloed heeft, kunt u deze vraag overslaan)
20a In hoeverre zijn relevante praktische vaardigheden van cursisten van invloed op het succesvol volgen en afsluiten van een cursus? Leg uit?
20b Komt het vaak voor dat de mate van praktische vaardigheden problemen oplevert? (Als u vindt dat praktische vaardigheden geen invloed hebben, kunt u deze vraag overslaan)
21a In hoeverre is het land van herkomst van cursisten van invloed op het succesvol volgen en afsluiten van een cursus? Leg uit?
21b Komt het vaak voor dat het land van herkomst problemen oplevert? (Als u vindt dat het land van herkomst geen invloedheeft, kunt u deze vraag overslaan)
OPMERKINGEN:
C Answers interview
|
What are problems with giving a maintainer course? (Question 12a) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
Do you have, in general, enough time to deal with all the subject matter of the course? Explain? (Question 14) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 7
|
|
What characteristics does a good course have to posses? (Question 12b) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
Teaching Methods
|
What are recurrent problems from students with teachers? (Question 16a) |
|
Teacher 1 - Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What characteristics does a good teacher have to posses? (Question 16b) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What is / are your goal(s) with giving a maintainer course? (Question 4a) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
Do you often manage to reach your goal? How can you tell whether you reached the goal you strive for? (Question 4b, c) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What is the theory -practice ratio of your lessons? Why? (Question 9) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What is the importance of skills or procedures? Why? (Question 10, 20) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What is the importance of knowledge? Why? (Question 11, 19) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What kind of learning materials do you use in you lessons? Why? What are recurrent problems with learning materials? (Question 5, 13) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What do you think of the use of Virtual Reality for maintenance courses? How would you implement this? (Question 7) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What do you think of the use of Computer Based Training? How would you implement this? (Question 8) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What are recurrent problems with students? (Question 15a) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
To what extend does the age of students influence the success with which they attend and conclude a course? Why? (Question 17) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
To what extend does the experience of students with computers influence the success with which they attend and conclude a course? Why? (Question 18) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
To what extend does the cultural background of students influence the success with which they attend and conclude a course? Why? (Question 21) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
What would an ideal student be like? (Question 15b) |
|
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
|
|
Remarks |
|
Teacher 4
Teacher 5
|
Enschede, sept., 10, 2002