In the past few years, the quality of higher education outcomes has become one of the most important topics, which received a great deal of discourse and research. The main purpose was assessing the learning outcomes of the various educational institutions. Determining the level of an outcome is considered as the first step of any evaluation process. Hence, every institution of the higher education that has different academic departments must determine earlier on the required characteristics, descriptions, skills, and knowledge that should be available to students graduating from the respective programs. Due to this fact the idea of preparing standardized tests to assess the level of the quality of the learning outcomes inside the educational institutions of higher education was established as a part of academic accreditation requirements for evaluating educational institutions. Many higher education institutions use standardized tests to appraise voluntarily the quality of their programs as part of a self-assessment process.

1. Introduction

   
   In the past few years, the quality of higher education outcomes has become one of the most important topics, which received a great deal of discourse and research. The main purpose was assessing the learning outcomes of the various educational institutions. Determining the level of an outcome is considered as the first step of any evaluation process. Hence, every institution of the higher education that has different academic departments must determine earlier on the required characteristics, descriptions, skills, and knowledge that should be available to students graduating from the respective programs.
   
   The next step is the implementation of a measurement process to assess the level of success attained by an academic department (or educational institution) in providing their graduating students in general terms with all the required skills and knowledge. These required skills and knowledge gained by the graduating students is the key indicator in judging the level to which a certain academic department has achieved its goals. Otherwise if these required skills and knowledge are not achieved this means that the academic department has failed to fulfill its objectives.
   
   Indeed, there are a number of measurement tools that can be used to assess the outcomes of the academic department. These measurement tools are either direct or indirect. The direct measurement tools are closely related to objectives of the academic department. The indirect measurement tools, on the other hand, are somehow related to the level of the skills and knowledge of the graduates of the academic department. For example, one of the indirect measurement tools is the duration it takes for the graduates to get jobs appropriate to their skills and knowledge. If the period of time taken graduates for acquiring employment in their major areas is not too long this is a clear indication of the success of the academic department. Another indirect measurement tool is the follow-up of the graduating students after being hired to find out whether or not their employers are satisfied with their work.
   
   Nevertheless, there is a notable problem with the indirect measurement tools. This problem is due to a number of random factors other than the effectiveness of the academic department that can influence the results, and as a result could underestimate the effectiveness of the academic department. Therefore, in addition to the indirect measurement tools it is absolutely essential that we employ the direct measurement tools to properly assess the effectiveness of the academic department. One of the most common direct measurement tools is testing. Testing can help us measure the effectiveness of an academic department by assessing the performance of the students as a group.
   
   Due to this fact the idea of preparing standardized tests to assess the level of the quality of the learning outcomes inside the educational institutions of higher education was established as a part of academic accreditation requirements for evaluating educational institutions. Many higher education institutions use standardized tests to appraise voluntarily the quality of their programs as part of a self-assessment process.
    

2. Program Objectives
   1. Raising faculty members awareness about the importance of standardized tests in assessing the effectiveness of academic programs;
   2. Exposing faculty members to the characteristics of standardized tests of high quality, which have been developed based on international standards;
   3. Training faculty members to design standardized tests for each academic program based on the objectives of the program;
   4. Training faculty members to build item banking so that they can refer to it when needed;
   5. Generating standardized tests to assess the extent to which the academic programs had achieved the intended goals by measuring the knowledge and skills of the graduating students.
       
3. International Experience

   
   With respect to the importance of the direct measurement tools in assessing the outcomes of academic programs in higher education, many international universities have started applying standardized tests to judge the quality of their educational outcomes. For instance, some of the universities such as Michigan State University and Austin Peay State University have sought the expertise of international organizations, like ETS (Educational Testing Services), to prepare these tests. On the other hand, other universities such as the University of Missouri have decided to build these tests using their own experience. On the contrary, King Fahd University of Petroleum & Minerals (KFUPM) would like to combine these two experiences in such a way that they want to seek the help of the international expertise to conduct training for faculty members but  design, prepare, and analyze these tests within KFUPM.
    

4. Who should attend?

   
   The faculty members of King Fahd University of Petroleum & Minerals.
    

5. Program Content

   
   The content of the program can be summarized as follows:

   1. Training Workshop: The intent of this workshop is to provide the faculty members with the necessary skills to produce high-quality standardized tests that are in line with the requirements of the international accreditation bodies;

   2. Discussion Forums: The purpose of these forums is to review and assess the items authored and submitted by faculty members;

   3. Analyzing Experimental & Final Test Results: The aim of these analyses is to ensure the merit of the test items and decide on which of the submitted items to keep in the final version of the test.

6. Program Duration

   
   The program basically runs for eight (8) days; with an average of four (4) hours of training per day. Afterwards there will be discussion forums as needed (from 3-5 forums), to discuss the psychometric analysis results, which will be conducted by the Testing & Evaluation Center. The discussion forums and the psychometric analysis will be done at the end of the second semester, as outlined in the Time Table.
    

7. Expected skills achieved from the Program

   
   After completing the training program, each trainee should have the following skills:

   1. Perfectly grasping the psychometric characteristics of a high-quality standardized test;

   2. Acquiring the ability to prepare and design standardized tests as done by international organizations such as ETS;

   3. Mastering the ability to write high-quality multiple-choice items;

   4. Having the ability to write high-quality essay items;

   5. Grasping the ways of correcting essay items;

   6. Grasping the basic steps for designing an item banking;

   7. Ability to produce and design standardized tests to assess the educational outcomes.

8. Mechanism for assessing the program

   
   The program will be assessed using multiple mechanisms:

   1. Surveys or questionnaires will be conducted after finishing every activity of the program to assess the level of satisfaction towards the positive and negative aspects of the activities. Also these surveys will be used to gather opinions about the skills that have been gained through the activities and suggestions will collected for improving all the activities;

   2. Assessing the work of the participants through evaluation of the quality of the items that they have written during the exercises and meetings;

   3. Evaluating the quality of the final version of the tests based on the concepts of validity and reliability that can be done through psychometric analysis (reliability coefficient, factor analysis, etc.).

9. Assessing the Impact of the Program

   
   The impact of the program can be assessed through two stages:

   1. Short Term Stage: This stage can be assessed by evaluating the level of the quality of the final version of the test through measuring the reliability and validity of the test. The reliability can be accomplished by calculating the internal consistency and the validity by obtaining the statistical correlation between the test and other equivalent tests. Or through other indicators such as the Grade Point Average (GPA) of the students.

   2. Long Term Stage: This stage can be assessed by evaluating the impact of the test on the future policies of the academic program and specifying the strengths and weaknesses of the academic program outcomes based on such test results.

10. Frequency of Program Implementation

    
    In the current academic year, it was suggested that the program will be implemented only once at KFUPM.
     

11. Standards for selecting participants

    
    The various academic programs of the university will nominate the participants of the program.
     

12. Capacity of the Program

    
    Due to the need of an intensive training, it is important that the number of participants is limited; therefore, the suggested number of participants is 30 only.
     

13. Program Team and Consultants

    
    The Testing & Evaluation Center at the Deanship of Academic Development will coordinate all activities related to this program. The Director of the Testing & Evaluation Center will lead a working team that will provide administrative and technical support to the program. A group of experts and research assistants who have wide expertise in the subject areas will participate in the project. The following Table presents the project members:

    #	Name	Current Job
    Trainer
    1	Dr. Eid R. Al-Harby	Director, Testing & Evaluation Center
    Consultant
    2	Dr. Neil Kingston	Facilitator for training workshop and discussion forums

News & Announcements

The opening ceremony will be held on January 05, 2008 in Building 20 (Conference Center).

A training workshop on "Peer Consultation in Teaching (PCT)" will be held from December 29, 2007  - January 9, 2008.