5. Experience and Evaluation

5.1 Experience

With a meaning of "learning by doing" in the process of design and redesign the experiment, the empirical study has another aspect that is "studying in working". The process of experiencing the experiment is such kind of strategy. By experience, the knowledge will be enhanced and the comprehension will be improved. In the following sections, the process of experience will be described.

5.1.1 Goal of experience

With a careful design and redesign of the experiment, the main purposes can be described as follows:

1. Experiencing the experiment by myself to make a empirical observation of the procedure and process of the experiment and make a necessary note and records of the responses of the subjects;
2. Working as a coach in the experiment to deepen the effect of "learning by doing" on design and redsign, at the same time to deal with the troubles occationaly happened in the experiment;
3. Making necessary interviews after each case to get the accurate information in time.
4. Monitoring the concrete actions of each subject to help to collect data correctly.

5.1.2 Contex and Instrument for the experiment

The experiment was conducted in the computer room of the Faculty of Educational Science and Technology (TO in Dutch), University of Twente, the Netherlands. In the computer room the students can use Internet 24 hours for free. The following table 13 shows the general conditions of the instrument.

5.13 Procedure and Responses

As mentioned in section 4.2 in the chapter 4, the experiment contains five respective tasks guided by the PI theory and different types of Parallelism and they represent the five basic styles of the web pages. The procedure of the experiment is such designed: Working as a coach I first make an appointment with the randomly selected subjects, give them a brief introduction about the production, motivate them to join in in a proper time. The sites of the experiment are always set in the computer room of TO faculty and the web-based learning environment is the TeleTOP environment. Among the five tasks, the first four are the regulative tasks, while the fifth one is an extra task which aims to give an examination of the importance of the distance ‘x’ in web-based learning environment with Parallelism. Table 14 describes the concrete requirement of the experiment.

Table 13. General conditions of instrument

Context		Computer room of TO faculty
Web-based learning environment		TeleTOP management system
Instrument		Computers on-line
CPU		Pentium III850MHz
Mem, HDU	128 MB, 20.5GB
Video	onboard Intel 815
Ntw	onboard 3Com 3C920
Operational system	Window NT
Monitor	Philips 107
Model	10

Table 14. The concrete requestment of each task:

Task one (A)	A regulative task designed with parallel instruction on a sheet of paper in English, from which the subjects can get the direction what to do and how to do in a scrolling windows on computer screen. See Figure19 in Chapter 4.
Task two (B)	A regulative task designed again with parallel instruction on a sheet of paper in English, from which the subjects can learn the concrete order to do the excises on a parallel window on computer screen. See Figure 22 in Chapter 4.
Task three (C)	Also a regulative task designed with parallel instruction on a sheet of paper in English, from which the subjects can know the details of how to do the excises in a new-opened window on computer screen. See Figure 23 in Chapter 4.
Task four (D)	A regulative task designed with parallel instruction on computer screen in English, from which the subjects can follow the directions to do the excises in a scrolling window on computer. See Figure 21 in Chapter 4.
Task five (E)	An extra task designed with parallel instruction on a sheet of paper in English, form which the subjects have to do the excise on the computer screen two meters away.

The features of the experiment are that the experience time is flexible antd all the subjects are randomly seletcted and motivated to join in this interesting epxerience . Twelve subjects (60% of total amount) are highly motivated both by a small gift to the encourage and my eathusiasm introduction ( a small gift is promised in advance), Two subjects (10%) have low responses and the other 30% are medium.

A overview of the process of the experience is showed in following tables: table 15 shows the general information of the subjects and table 16 shows their responses in the experiment.

Table 15. The general information of the subjects

sequence		Name	Sex	Academic background	Nationality
#1		Vivien	Female	MSc Master Candidate	China
#2		Hsin-yi	Female	MSc Master Candidate	Taiwan, China
#3		Yongshen	Male	Bachelor of Science	China
#4	Wang- Chunrong		Female	MSc Master Candidate	China
#5	Yangxiong		Male	MSc Master Candidate	China
#6	Huang - Fengmin		Female	MSc Master Candidate	China
#7	Ruslan		Male	Bachelor of Science	Uzbek
#8	Herry		Male	MSc Master Candidate	Indonesia
#9	Tran		Female	MSc Master Candidate	Vietnam
#10	Nguyen Thuy		Female	MSc Master Candidate	Vietnam
#11	Vredemetu		Female	MSc Master Candidate	Indonesia
#12	Vanessa		Female	MSc Master Candidate	China
#13	Nanang		Male	MSc Master Candidate	Indonesia
#14	Chen Wei		Female	MSc Master Candidate	China
#15	Meina Jang		Female	MSc Master Candidate	China
#16	Annie Lu		Female	MSc Master Candidate	China
#17	Anna		Female	MSc Master Candidate	Indonesia
#18	Rabab		Female	MSc Master Candidate	Egypt
#19	Tulkin		Male	Bachelor of Science	Uzbek
#20	Tau		Male	MSc Master Candidate	China

Table 16 shows the observation of the reponses of the subjects. In this table, the word "Best" means that the subject thinks that this kind of style of task is his first choice of web page in a web-based learning environment, while the word "Last" means that the subject thinks that this tyle is the worst design among the regulative four tasks, which he (she) thinks should be avoided in web paged design. The frame without words means the subject thinks that task is neither the best nor the worst for him. As to the fifth task, except for one subject who liked the design of the style for especial reason (she thought it is a good way to practise memory), all the subjects addressed that they did not like it at all. Since it is an extra task, we do not take it into account when comparing the effect with the first four regulative tasks. I just recorded the times between the sheet and the computer which subjects have to spend to check the figure because of their bad short-term memory. More intereseting records are the figures showed in the first four tasks, for instance "(2)" in task three means that the subject checked twice when he dealt with task three. This record, mainly refers to task five, is very useful to explain that Parallelism and the PI theory as a design and information management theory can decrease the extraneous cognitive load and help users to improve their learning effectiveness. The sequence of the tasks is strictly followed the design and redesign decision on the designed scheme implemented in real experiment in section 3.2.3. (See table 4. The designed scheme implemented in real experiment.).

Table 16. shows the general information of the responses of the subjects

No.	Date	Task One	Task Two	Task Three	Task Four	Task Five
#1	26/05/02		Best	Last		One Time
#2	27/05/02		Best	Last		One Time
#3	29/05/02	Last	Best			One Time
#4	29/05/02		Best	Last		One Time
#5	30/05/02		Best	Last (2)		One Time
#6	30/05/02	(2)	Best	Last	(2)	One Time
#7	31/05/02		Best	(2)	Last	One Time
#8	31/05/02		Best	Last	(2)	One Time
#9	01/06/02		Best		Last	Two Times
#10	04/05/02		Best		Last	Two Times
#11	07/06/02		Best		Last	One Time
#12	08/06/02		Best		Last	One Time
#13	11/06/02		Best	Last		One Time
#14	11/06/02		Best	Last		Two Time
#15	11/06/02		Best	Last		One Time
#16	12/06/02		Best		Last	Two Time
#17	16/06/02		Best	Last		One Time
#18	19/06/02		Best		Last	One Time
#19	24/06/02		Best		Last	Three Times
#20	24/06/02		Best		Last	One Time

This information is the raw information gotten through observation and records of the experience, the accurate data is collected by automatic logging system, which will be illustrated by examples in section 5.2.3 or refer to the appendices about raw data.

5.2 Evaluation

Evaluation is the important segment of the chain of the project. The evaluation of the products enables designers and instructors to test whether the objectives are achieved or not. Evaluation in some sense means the systematic collection of information for the purpose of informing decisions to design and improve the product.

5.2.1 Goal and Types of Evaluation

"The purpose of evaluation is to delineate, obtain, and provide information for making educational decisions. This information is …highly particularistic and specific to a decision situation, rather than generalizable to many or all settings. Thus, evaluation methodology is not necessarily designed to produce universally valid information, but information that is valid and useful within the decision-making context." (Stufflebeam et al., 1971, p. 140) In practice, evaluators often use the tools and methods of research. To the extent allowed by practicality, utility, and cost, evaluation meets the research criteria of objectivity, and validity (Flagg, 1990). As far as to my project, the evaluation has the same functions of providing information for making design and redesign decisions as well as to improve the quality of the final product.

There are two kinds of evaluations. The first one is to evaluate with experts the products during and after the process of design, redesign and development, which can be named formative evaluation. Formative evaluation helps the designers of a product, during the early development stages, to increase the likelihood that the final product will achieve its goals. The second kind of evaluation is called summative evaluation, which is applied to evaluate the products after implementing products in real situations. This type of evaluation is intended for "evaluating the materials in terms of their effectiveness in order to provide data for decision-makers who may adopt or continue to use the materials" (Smith & Ragan, 1999,p.338). In my project, these two kinds of evaluation are both involved. The formative evaluation of the project is an unsystematically evaluation without statistic calculation, only with a list of questions. The summative evaluation is a systematically evaluation with subjects by observation and with SPSS. Apart form these, other methods related with evaluation are also introduced such as realization, testing and researching and development.

5.2.2 Instrument of the Evaluation

Both informative evaluation and summative evaluation are used in the development process of my project. In the early phase of the product such as needs analysis, design and redesign of the experiment and the development phase of the product, formative evaluation works as the main methods. "The term formative indicates that information is collected during the formation of the product so revisions might be made cost-effective." (Flagg, 1990) The instruments used for formative evaluation are checklists and interviews and so on. As to the instrument for the summative evaluation, I mainly depend on the observation and record notes during experiencing the experiment, interviewing with the subjects after their finishing the experiment tasks and the well-designed automatic logging system to collect detailed data.

5.2.3 Procedures: Formative Evaluation and Summative Evaluation

Formative Evaluation

According to the design and redesign model adopted in my project (See section 4.2.2 the design model and figure14 in chapter 3) the process of the evaluaion runs through the whole process of the project. For the part of design and redesign of the experiment, formative evaluations were conducted frequently. The major methods are arranging discussion with expert and asking feedback from him; using checklists to modify and revise the content of the design and redesign and so on.

During the period of my project, one of the major ways of communication with my mentor is to arrange a face to face talks about the concrete content of the project, including the goals of the project, the principles and procedure of the design and redesign as well as the sequence and steps on the experiment and so forth. Another way of communication with my mentor is through the e-mail to have close contact and discussion. I always can find some smart suggestions and surprising feedbacks from him in the e-mail box.

Another common used tool in formative evaluation is the checklist. During the time of design and redesign the old version of the experiment, we used checklist a lot to correct the mistakes and modify the insufficient part. The following table 17 is the example of the checklist by which we ever used to improve the quality of the design and redesign.

With various kinds of formative evaluation during the analysis phase, designa and redesign phase and the development phase, my project developed successfully step by step. From my experience with the process of the project, I do think that formative evalution is indispensable during an instructional design (redesign), so are the conditions in software programming and other processes of problem-solving.

Summative Evaluation

In my project, summative evaluation is adopted during the implementation phase of the product. As mentioned in section 5.2.2, the main instrument implemented in summative evaluation include observation and record notes during the experience, the interviews after the subject finishing their expeiment tasks and the well-designed automatic logging system for data collection.

During the period of experience, I always arranges the experiment context properly so that I can make a overall observation to gain the first hand data about the responses of the subjects. As a coach, I did my best to keep an objective attitude during the experiment and record the true process of excises.

Interview is an effective method to collect necessary data. During the interview, most questions were prepared in advance to guarantee the high effectiveness and efficiecy of the interview. Some relevant questions might be asked during the interview to keep the flexiblity and activity. Table 18 is the main questions prepared for the interviews

Table 17. Example of checklist

Table 18. Questions for interviews

Questions for the interviews: Have you heard of the concept of Parallelism and the PI theory before? What are your opinions about this experiment? The strong point and weak point. Can you see the principles of Parallelism in the experiment? Which segment do you think is the best one for us among the five task? Why? Which segment do you think is the worst one for us among the five task? Why? Apart from the extra task- task five, which task do you prefer in a web-based learning environment? Which one is your last choice? What are the reasons? What is your comment and suggestion to this instructional design (redesign) under the web-based learning environment? Do you think the distance between the instruction and the target object is a problem for the short-term memory? What is your ideal output of the information in web-based environment?

Data collection by automatic logging system is another critical and crucial procedure and method to make a reliable summative evaluation. More details can be found in section 5.2.4.

5.2.4 Automatic Logging System (ALS) for Data Collection

In this project conducting a scientific analysis of the colllected data by well-designed automatic logging system is a key point, not only because of its well-designed function for the collection of detailed information, but also because of its quick-feedback system and its accuracy and correctness. Before the analysis it is necessary to have an overview of the working principle of the automatic logging system, which is based on the recording the value of the experiment.

• Composition and Structure of Computer Screen

A close review of the terminology and structure of the computer screen can make it clear. Figure 30 illustrates the terminlolgies about the computer screen, which is composed of ‘Browser: Navigator’, ‘Window’, ‘Document’, ‘Frame’, ‘Form’, ‘Field’ and ‘Value’. Here ‘Value’ refer to the figures filled in the "field’.

In program language, the "frame", "form" and "field" are represented by characters such as "a", "v" and "y" and so forth. In our design, the "fields" on the left "form" which is represented by "A" are "a1", "a2" to "aN". The schematic structure of computer screen shows in figure 30 and the schematic structure of Task 2 is given in figure 31. A well designed example of interface of the structure of Task 2 is illustrated in figure 32.

This Figure means that ‘Field’ in the left "Form" will be represent by ‘a1’ , ‘a2’…’aN’, while ‘Field’ in the right ‘Form’ will be represent by ‘v1’, ‘v2’ etc. Thus the automatic tracking system can record the ‘Value’ in ‘Field aN’ and compare the ‘Value’ with the ‘Value’ in the right ‘Field vN’.

• Procedure of the Automatic Logging System (ALS)

According to our design and redesign, the experiment contains four regulative tasks and one extra tasks, and at the same time each task includes four excises. The automatic logging system will focus on two crucial points: the exact time during each excise and the input outcomes of the subjects.

The time caculation can be recorded automatically depending on eight clicks for the four excises within one task.

1. Starting Click for Excise One: After looking and reading the instruction, the subjects will click the button as directed to find the "Value" in the "Field ‘yN’" and remember it.
2. End Click for Excise One: After filling the "Value" learned form the "Field ‘yN’" in the "Field ‘aN’", the subject will click the end button under the "Field ‘aN’".
3. Repteating the above process until finishing the Task.

Figure 33. FlowChart of the procedure of finishing one task

Table. 19 Example of the data feedback collected by ALS, it is the second part of ALS

5.3 Results of the experience and the evaluation

After experiencing the experiment guided in my research by Parallelism and the PI theory, the research results are highly consistent with the basic tenets of the PI theory and our assumption that the PI theory does work as a design theory in web-based learning environment. The evaluation of the collected data also shows the same result.

5.3.1 Observation and recording in experiment

Through the overview and analysis of the process of the experience, a brief conclusion about the result of the experience can be got as follows:

1. Parallelism and the PI theory also works well in the web-based learning environment. The result shows that 100% subjects prefer to the Task 2 (the parallel excise) as their open learning, working, and doing environment.
2. Distance does influence the effectiveness and the efficiency of learning and influence the motivation of learners. 95% (19 subjects) chose the extra task (Task 5) as their learning environment neither on the web-based, computer-based nor other learning environment even the physical distance is not so far away, say two meters. One expcetion is that one of the subjects (5%) likes the design of the task five because she thought it was a good idea to motivate and practise the memory. That is no paradox to Parallelism and the PI theory, in contrary, it proves from the other hand that Parallelism and the PI theory can reduce the cognition load during learning and working.
3. Among the first four regulative tasks, the choices of the subjects show a different individual favor. 50% (10 subjects) said Task 3 (the task with a top window) was the last choice for them while 45% chose Task 4, which has a long window on the screen and the instruction is digital one. The problem for the Task 3 is that when it is closed, the top window will hide in the clipboard which makes the user spend lots of time to find it and click again. It shows clearly that users do not like any work which is at risk to increase extraneous cognitive load. While the problem for the Task 4 is that the long screen also spend users extra energy to find necessary information. A interesting phenomenon is if the roller in the mouse of computer works well, more users will prefer Task 4 to Task 3 because the easy use of the roller will save their time. Only 5% (one subject) thought Task 1 was the worst learning environment for them. Actually the biggest difference between Task 1 and Task 4 is that the instruction materials is on paper intead of on computer in Task 1, while the length of the window is a problem in Task 4. This result seems to prove that users and learners still like to use paper materials, manuals and instructional sheets as their parallel instruction materials.
4. Coaching and teaching are still necessary even if offering literature instruction. Part of the subjects can not understand the terminology and special nouns when they face a new learning environment or strange instruction. It will take long time and energy for them to get a entire comprehension. Taking this experiment as an example, 60% of the subjects (12 out of 20 subjects) asked me to explain what meaning the ‘last value’ was or what was the ‘frame’ and ‘field’ and so on.
5. Distance really makes some problems for the people who have a poor short-term memory. According to the observation and records, there were four subjects (25%) who had to check the instruction and the excise twice during doing Task 5, in which there is a distance of two meters defaulted. Apart form this one subject (5%) of them had to check three times because he is more older than others. So a hypothesis arises here that the PI theory maybe work well for some elder people who have a poor memory. Table 20 lists the outcome of the observation.

Table 20. Outcome of the observation

100% subjects prefer to parallel excise (task 2) than other styles 95% subjects dislike the extra task (Task 5, two meters distance) 20% subjects have to check their excise twice, 5% subjects three times in Task 5 50% subjects think the Top Window is their last choice (Task 3) 45% subjects think the vertical window is their last choice (Task 4) 60% subjects ask for help during the experiment

5.3.2 Result of the Data Collection

The analysis about the data collection can be taken as the firt step to summative evaluation of the experiment. The outcome the analysis and evaluation also gives a positive result about the application of the PI theory in a web-based learning environment.

According to the design and redesign schedule, there are totally 20 subjects joined in the experiment. The automatic feedbacks from the automatic logging system show that almost all data about the subjects dealing with the experiment are collected successfully except for one task data missing. The reason may be that subject forgot his last click of the automatic logging system for that data. Since the misssing feedback is about Task 5, which in fact has no contribution to the data analysis, so the result still have a high vadility.

From the data collection, the information is showing as follows.

1. Comparison between the defaulted value and the results of excises.

According to the design, the automatic logging system only recorded the result of the first four regulative tasks. Among the total 20 subjects, 60% (12 subjects) finished their tasks 100% correctly, while 40% subjects made some mistakes. 30% (6 subjects) only made one error in their four tasks, 5% (1 subject) had two errors and another 5% (1 subject) had three errors. This means that the experiment is a proper design for the examination.

The statistic figure shows that 5 errors (6.25%) of value input appeared in Task 1, 3 errors (3.75%) appeared in Task 3 and 2 errors (2.5%) was in Task 4. Only one error (1.25%) was found in Task 2, which is a task of typical parallel excise. This result, therefore, shows that Parallelism and the PI theroy work well in the problem-solving environment. The details can be found in appendix 5.

2. Time calculation:

Time calculation is very important in this experiment, because at the same condition, more time means more cogitive load. Through the analysis of the collected data by the automatic logging machine, the results illustrate that cognitive load and the PI theory play critical role in instructional design.

Four mistakes (5%) of task-end clicking have been maden by the subjects in Task 1, while seven errors (8.75%) have been found in Task 4. No subject made mistakes in Task 2 and only one error in Task 3. Here the mistakes and errors mean the invalidty of the records of the time by the automatic logging system. The reasons for these mistakes mainly due to the facts that the subjects were not sure if they had click the timer or they forgot that they had done it so they repeated again. They must have cognitive overload at that time.

In the design and redesign of the experiment, we set 6 figures in Task 1(the figure is 36.3562), 5 figures in Task 2 (the figure is 986.73), but 4 figures in Task 3 and Task 4 (they are 17.36and 682.7). The results are quite logic: subjects spent the longest time in Task 1, the middle time in Task 2, the shorter and shortest time in Task 4 and Task 3 (the shortest time appeared in Task 3, see figure 34 as an expample). Table 21 gives a overview of the outcome of data analysis.

Table 21. Overview of the outcome of data analysis

Items	Data analysis
About the subjects	60% (12 subjects) finished their tasks correctly 30% (6 subjects) made one error in their four tasks, 5% (1 subject) had two errors and another 5% (1 subject) had three errors
Value comparison	5 errors (6.25%) of value input appeared in Task 1, 3 errors (3.75%) appeared in Task 3 ...one error (1.25%) was found in Task 2, the parallel excise No subject made mistakes on task-end-clicking when they finished Task 2 while seven errors (8.75%) have been found in Task 4

Finally by statistic analysis, we got the mean time used by the subjects in the first four regulative tasks: the mean time in Task 1 is 59.73 second/ person (s/p), the mean time in Task 2 is 43.49 s/p, the mean times in Task 3 and Task 4 are 47.99/per and 43.92 s/p. The shortest time appeared in Task 2 again. This result shows clearly that the parallelism works well in a instructional design under the web-based environment, which is consistant with the assumption we made. Table 23 gives a clear explanation.

We also found that there is not much difference between most of the subjects in the same task, which illustrates that no big difference exixts among most the subjects with different backgrounds. More details see appendix 1 to appendix 4.

Figure 34. The example of data analysis for Task 3

In this chapter, I introduced the procedure and the process of my experiencing the experiment in the first section. In the second part I offered the detailed information about the evaluation including the formative evaluation, summative evaluation. Results were offered finally. In the last section of the thesis- chapter 6, a conclusion and reflection will be given.

Chapter 6. Conclusion & Reflection

6.1 Conclusion

Generally speaking my practice on the final project "Empirical study to Cognitive Load and the PI theory with well-designed products for procedure skills and parallel instructions" is an interesting and attractive experience, I enjoyed it very much and think the whole process is successful.

First from the perspective of empirical study for myself, it is a successful study and research. The unique strategy of "learning by doing and working" proved to be a good solution for the acquisition of the knowledge. Literature reading is only the first step to understand the original thing. Doing, working and experiencing is more important to enhance the knowledge, furthermore to improve the overall comprehension and to achieve proficiency. In this experience of empirical study, literature reading exactly enlighten me and widen my sight, but the process of design and redesign as well as the sequent experiment deepen my understanding, which play more crucial role.

Secondly, from the perspective of the design and redesign an instruction for web-based learning environment, the project can also be taken as a successful project. Through the design and redesign, the quality of the experiment has been improved to fit the application in web-based learning environment, and it now can be used for international students. During the experience of design and redesign, I learnt a lot from my mentor Min, at the same time, I had a good opportunity to review and enhance the technology and skills for web-based learning environment. For instance, I need to use HTML and JavaScript to design and redesign the products, which is obviously the best excise for me to improve my program language. Simultaneously, during the process of the design and redesign, I learnt a lot about the procedure of instructional design such as the needs analysis, the model selection, the decomposition of goal into objectives and so on and so forth.

General principles and assumptions that underlie instructional design (ID) we chose in my final project are as follows:

• The design and redesign is a systematic process.
• The project has a problem-solving orientation (needs assessment leads to activities directed at improvement of instruction, which in turn lead to evaluation).

• The design and redesign has as a goal efficient, effective, and appealing instruction.

• The project insists on congruence between objectives, design, and evaluation.

• The project is both theoretic and empirical (as opposed to "intuitive").

• The design and redesign of instruction has been directed by needs and shaped to fit the learning environment.

• The design and redesign included consideration of the learner characteristics.

• Appropriate technology has been reflected.

• Evaluation has been conducted as part of the design and redesign as well as development process.

Last from the perspective of the experiment experience. Although this is the first time to design and redesign the experiment and experience it by myself, with the help of my mentor, I should say it is a successful experiment. Following the detailed plan and careful arrangement, the experiment conducted very well. Three methods were used to collect data: observation and records, interviews and data collecting through automatic logging system. All these make the validity and credit of the experiment high. The results are positive and encouraging. Parallelism and the PI theory as a design theory proved to be welcome among the users. To the problems such small screen of the computers, cognitive overload of information, linear output of the information and so forth, Parallelism and the PI theory can be a good solution. In fact, Parallelism does not mean that the more information the better, but mean that the more necessary and dependable information the better; maybe the name of ‘the parallel information theory’ is more suitable. More and more cases show that the PI theory can be applied to many domains including in military applications.

Thus a positive conclusion can be gained due to the above statement. Certainly there are some problems still existing. In the next section of reflection, the problems exposed in the project will be mentioned and could make a valuable reference for the other design and redesign with Parallelism and the PI theory either for an open-doing and working environment or a web-based learning environment.

6.2 Reflection

Although our design and redesign proved the concept of Parallelism and the PI theory do work to open learning environment, especially to web-based learning environment, there is still a long way to go, either to this finished experiment or the coming ones. More topics and directions can be considered during the futures’ design. For instance, the concept of lifelong learning is accepted by more and more people, but what kind of effects to the old aged people will occur when Parallelism and the PI theory are applied in the open learning environment as well as web-based learning environment? What is the difference between the old aged learners and young people? etc. Here I just give my thoughts on this experiment, I expect the other experiments concerned with Parallelism and the PI theory could be carried out earlier and eager to hear the further encouraged information.

6.2.1 Better design and redesign could be achieved for this experiment

This experiment we designed and redesigned just focus on the theoretical and abstract environment, more concrete and practical environment with micro-world, for instance the simulation environment could be involved in following experiments. For this experiment, my reflection is as follows:

1. More detailed instructions could be offered in advance. For instance, the terminology and special nouns could be explained clearly before the experiment so that the subjects can go on smoothly.
2. New methods could be implement to improve the credit and validity of the data collection. For example to the task five, I suggest to use two separate computers as materials to conduct the experiment, because the automatic result track system can work well during that environment. The scheme of the design is as following: two parallel computers can be introduced here. One computer works for the digital instruction, the other works as the work place. A distance of two meters is defaulted between these two computers. Subject can learn the instruction on one computer and finish his task on the other one. When he finish all his work, the computer system can follow his action and record what he has done. The result could be more accurate and credit.
3. To make this design and redesign better to suit for the web-based learning environment, some web-site guidelines are still working such as the bookmark function could be used to prevent the lost of users during the experiment. The users often lose in the complicated web pages and give up navigation. At the same time the animation and simulation technology could be taken in account to motivate the users and increase the effect of the instruction.

6.2.2 For the empirical study and the PI theory

The empirical study proved to be a successful strategy for our theoretical knowledge acquisition, especially to the technology and skill learning in a computer-based environment. Certainly, theoretical support is necessary before the excise and practice. As far as to my project, I think it is more important to build association and relationship between the PI theory and other learning theories, information processing theories as well as relative philosophies so that the PI theory can be used in a larger scope and a wider environment. In addition, Parallelism and the PI theory in English version could be defined and authorized more accurate and scientific so that the novice and international learners can understand it easier and clearer. Obviously more designs or redesigns are needed to continue. Min has offered more experiences and experiments for the research on the concept of Parallelism the PI theory, which are different, complex even challenged but more effective and efficient. The more experiments will be done, the more credit and validity can be get. On the base of my project, it is reasonable to believe that future experiments could be more successful.

The world is changing, technologies are developing and theories are improving. Facing the new challenge in educational domain of the new era, I hope the PI theory could serve well to more and more educators, instructors and designers to improve the effectiveness and efficiency of teaching, and to create as much easy learning environments as possible.

Enschede, sept. 4, 2002

References

Baddeley, A. D. (1986). Working memory. Oxford: Clarendon Press

Baddeley, A. D. (1992, January 31). Working Memory. Science, 225, 556-559.

Benshoof, L.A., en S. Hooper (1993). The Effectes of Single- and Multiple-Window Presentation on Archievements During Computer-Based Instruction'. J. of Comp. Based Inst. vol. 20, no. 4, p. 113-117

Berenfeld, B. (1996). Linking students to the infosphere. T.H.E. Journal, 4(96), 76-83.

Berge, Z. L. (1999). "Conceptual Frameworks in Distance Training and Education." In D.A. Schreiber and Z.L. Berge, (Eds). Distance Training: How innovative organizations are using technology to maximize learning and meet business objectives. San Francisco: Jossey-Bass, 19-36.

Bower, G.H. (1975). Cognitive psychology: an introduction. In W. K. Estes (Ed.), Handbook of learning and cognitive processes (Vol.1) Introduction to concepts and issues (pp.25-80). Hillsdale, NJ: Erlbaum

Burg, B., and Groenewoud, U. A. (1996). Effect van mate van parallelisme op taakuitvoering en gepercipieerd gebruikersgemak. [Effect on measure of parallelism on task performance and perceived user comfort]. Proceedings of the symposium within the framework of the subject 'Research Assignment' (196037). Internal paper. Twente University, Faculty of educational science and technology (The Netherlands).

Chandler, P., and Sweller, J. (1996). Cognitive Load while learning to use a computer program. Appl. Cogn. Psychol. 10: 151-170

Chi, M., Claser, R., & Rees, E. (1982). Expertise in problem solving. In R. Sternberg (Ed.), Advances in the psychology of human intelligence (pp. 7-75). Hillsdale, NJ: Erlbaum

Clark, J. M. & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149-170.

Claessens, (1999) The effect of different ICT-designs on learning specific tasks; First year report of PhD Study.

Cobb, P. (1996) Constructivism and learning. In T. Plomp and D.P. Ely (eds.), International encyclopedia of educaional technology (2^nd ed.). (pp. 56-59) Tarrytown, NY: Elsevier Science.

Collis, B., & Moonen, J. (2001). Flexible learning in a digital world: Experiences and expectations. London: Kogan Page.

David Mioduser, Rafi Nachmias, (2000). WWW in Education, To be published in: H. Adelsberger, B. Collis, & J. Pawlowski (Eds.), (2000), Handbook of Information Technology in Education and Training.Berlin: Springer: Verlag.

DINKEL Institute. (2001). TeleTOP. Available via the World Wide Web at http://www.teletop.nl/ Date accessed July 4, 2002.

Driscoll, M. P. (1994). Psychology of learning for instruction, Boston: Allyn and Bacon.

Barbara N. Flagg, (1990). Formative evaluation for educational technologies. Lawrence Erlbaum Associates, Inc., Publishers Hillsdale, New Jersey 07642.

Fenrich, P. (1997). Practical guidelines for creating instructional multimedia applications. Orlando: The Dryden Press.

Kandogan. E, Shneiderman. (1996) Elastic Windows: Improved Spatial Layout and Rapid Multiple Window Operations, ACM AVI'96 Advanced Visual Interfaces, Gubbio, Italy, pp. 29-38.

Kandogan. E & Shneiderman. B (1997). Elastic Windows: A Hierarchical Multi-Window World Wide Web Browser, to appear in ACM UIST'97 Symposium on User Interface Software and Technology.

David R. Krathwohl. (1998). Methods of educational and social science research: an integrated approach-Second edition. Longman.

Laurillard, Diana, (1993) Rethinking University Teaching: a framework for the effective use of educational technology, London. New York: Routledge.

Mayer, R. E. and R. Moreno (2000). A split attention Effect in multimedia Learning: Evidence for Dual Processing Systems in Working Memory. J. of Educ. Psych., Vol. 90, no. 2, p. 312-320.

Miller, G.A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97

Min, F.B.M., (1992) Parallel Instruction, a theory for Educational Computer Simulation. Article. Interactive Learning Intern., Vol. 8, no. 3, 177-183.

Min, R. (1994). Parallelism in open learning and working environments. Britsh Journal of Educational Technology, Vol. 25, No. 2, pp. 108-112. ISSN 0007-1013. [Online] Available via the World Wide Web at: http://projects.edte.utwente.nl/pi/papers/Parallelism.html Date accessed May 27, 2002

Min, R. (1996). Parallelism and the Parallel Instruction Theory. [Online] Available via the World Wide Web at http://users.edte.utwente.nl/min/home/Theory2.html Date accessed May 27, 2002

Min, R. Vos, H. Kommers, P.,& Dijkum, C. van (2000). A concept model for learning. Journal of Interactive Learning Research Vol.11, no. 3/4, p.485-506. [Online] Available via the World Wide Web at http://projects.edte.utwente.nl/pi/papers/Learning.htm Date accessed May 27, 2002

Min, R. (2001). Simulation and discovery learning in an age of zapping and searching; a treatise about the educational strength and availability of digital learning tools and simulation on the World Wide Web. [Online] Available on the World Wide Web at http://projects.edte.utwente.nl/pi/Papers/DiscLearning.html Date accessed July 13, 2002

Miller, G.A., Galanter, E., & Pribram, K.H. (1960). Plans and the Structure of Behavior. New York: Holt, Rinehart & Winston.

Moreno, R., and R.E. Mayer (2000). A coherence Effect in multimedia learning: the case for minimizing irrelevant sounds in the design of multimedia instructional messages. J. of Educ. Psychology, vol. 92, no. 1, pp. 117-125.

Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, vol. 87, pp. 319-334

Nasseh, B (1998). "Training and Support programs, and Faculty’s New Roles In Computer-Based Distance Education in Higer Education Institutions." [Online] Available on the World Wide World at http:// www.bsu.edu/classes/nasseh/study/res98.html Date accessed June 29, 2002

Paul A. Kirschner. Cognitive load theory: implications of cognitive load theory on the design of learning. In Learning and Instruction: the Journal of the European Association for Research on Learning and Instruction. Pergamon.

Paivio, A., (1986). Mental representation: A dual coding approach. Oxford England. Oxford University Press

Plomp, Tj. (1982). Onderwijskundige technologie: enige verkenning. [Educational science and technology: Some explorations]. Inaugurele rede. Enschede, University of Twente, Faculty of Educational Science and Technology.

Sherry, L. (2000). The nature and purpose of online discourse: A brief synthesis of current research related to the Web Project. International Journal of Educational Telecommunications, 6(1), 19-52

Schlager M., Fusco J, and Schank P. (1998), "Cornerstones for an On-line Community of Educational Professionals." IEEE Technology and Society, 17(4), 15-21, 40.

Schneider, W. and Shiffrin, R.M. (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84, 1-66

Shiffrin, R. M. and Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual Learning, automatic attending, and a general theory. Psychological Review, 84, 127-190

Smith, P. L. & Ragan, T.J. (1999) Instructional Design the second edition. John Wiley & Sons, Inc.

Spiro, R.J., Feltovich, P.J., Jacobson, M.J., & Coulson, R.L. (1992). Cognitive flexibility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. In T. Duffy & D. Jonassen (Eds.), Constructivism and the Technology of Instruction. Hillsdale, NJ: Erlbaum

Spiro, R.J. & Jehng, J. (1990). Cognitive flexibility and hypertext: Theory and technology for the non-linear and multidimensional traversal of complex subject matter. D. Nix & R. Spiro (eds.), Cognition, Education, and Multimedia. Hillsdale, NJ: Erlbaum

Sweller,J. (1988). Cognitive load during problem solving: Effects on learning.

Cognitive Science, 12, 257-285.

Sweller, J., van Merrienboer, J. J. G., and Paas, F. G. W. C. (1998). Cognitive Architecture and Instructional Design. Educational Psychological Review, Vol. 10, No. 3, pp. 251-296.

Stufflebeam, D.L., Foley, W.J., Gephart, W.J., Guba, E.G., Hammond, R.L., Merriman, H.o., & Provus, M.M. (1971). Educational evaluation and decision-making. Bloomington, IN: Phi Delta Kappan National Study Committee on Education.

Schaick Zillesen, P.G van. (1990) Methods and techniques for the design of educational computer simulation programs and their validation by means of empirical research. PhD. thesis, University of Twente, Enschede, Holland (promotors: E. Warries and F.B.M. Min).

Web documents:

Min, R (2001) Online available at http://projects.edte.utwente.nl/pi/Papers/DiscLearning.html

Min (2000) Online available at http://projects.edte.utwente.nl/pi/papers/Learning.htm

Min (2002) Online available at http://projects.edte.utwente.nl/pi/Papers/Parallelism.htm

Greg Kearsley (2001) Online available at http://tip.psychology.org/sweller.html

Greg Kearsley (2001) Online available at http://tip.psychology.org/paivio.html

Greg Kearsley (2001) Online available at http://tip.psychology.org/spiro.html

Greg Kearsley (2001) Online available at http://tip.psychology.org/miller.html

Schroeder (2002) Online available at www.uis.edu/~schroede/sources.htm)

Dave Kristula (2001) Online available at http://www.davesite.com/webstation/html

University of Maryland (20xx) Online available at http://www.cs.umd.edu/hcil/elastic-windows/

Maricopa Center for Learning and Instruction (MCLI) (20xx), Online available at http://www.mcli.dist.maricopa.edu/tut/tut0.html

Refsnes Data (2002) Online available at http://www.w3schools.com/html/html_elements.asp

Heidi Brumbaugh (2001) Online available at http://www.webdeveloper.com/javascript/javascript_js_tutorial.html

Aaron Weiss (1998) Online available at http://www.wdvl.com/Authoring/JavaScript/Tutorial/versions.html

Aaron Weiss (1998) Online available at http://www.wdvl.com/Authoring/JavaScript/Tutorial/grammar.html

Appendix 1

Appendix 2

Appendix 3

Appendix 4

Appendix 5

During the experiment, because the subjects do not familiar with the operational procedure of the experiment, I have to help them between the break of two excises, so the time of reading instruction can not be reflect correctly in the records of the automatic logging system (ALS). Therefore the summative evaluation refers only to the excising time to the four regulative Tasks. (Task 1 to Task 4). As to the Task 5, ALS does not work to it, so I just collect data by observation and recording in Task 5.

Enschede, sept 2002