Dear Associate Partner of the Erasmus+ project OP4RE,
Following the first transnational meeting of October 25 and 26, the OP4RE Project Management Group is progressing in the development of the project. For each of the 5 Intellectual Outputs (IO), more detailed plans are under construction.
The project members that are concerned with privacy and security aspects of remote examination are working on setting up a framework for institutional impact assessment procedures and agreements regarding security, risk acceptance and privacy. More information on this IO will follow in later Newsletters.
Identifying and getting all conditions in place in order to conduct experiments takes quite some leeway and members of the PMG has started to initiate this already. Below we present three examples of experiments in preparation.
Example 1: Selection test for entrance to Bachelor programs with numerus fixus
Experiments for remote proctoring are now set-up for Bachelor study programs that require selection of students (Numerus Fixus programs). In normal conditions Dutch candidates needed to take a digital selection test in the Digital Exam Hall of the Vrije Universiteit Amsterdam for that purpose. Dutch institutions for higher education however are obliged to offer students living in the so-called special overseas municipalities (Bonaire, Saba and Sint Eustatius) the possibility to take these selection tests under the same conditions as Dutch students. For these selection tests, the OP4RE project will provide the services to take the test for these overseas candidates remote at their home. The selection test that will take place in February 2017 to April 2017 will be used as a pilot for future selection test (2018). In the pilot, the procedures, the survey questions, the technical set-up of the online tests etc. are being detailed and tested. Extensive discussions are conducted with regards to these procedures and the possible problems that could arise and their mitigation.
- The problem of preventing fraud or the illegal recording of the test itself by candidates;
- The procedures in which students are made aware of their consent for privacy and security of online remote testing;
- The equivalence of the different forms of the test. For example, in the Digital Exam Hall, students get a case-based questions handed out on paper, while overseas students need to read the case information from the screen;
- How to deal with the issue of going to the toilet for a 3-hour test. In the face-to-face setting, candidates are allowed to go to the toilet after finishing part of the test and before entering the successive part: how to deal with this in a remote situation.
Example 2: Experimental study with a large cohort of students
The second pilot study is concerned with an experiment in which the equivalence and comparability of exam results for on-campus proctoring and remote proctoring of digital tests is studied in a scientific experimental setting for the domain of statistics for a large cohort (300+) of students for December 2017. For this experiment, teachers and the board of examiners need to be involved heavily in the decision making process. In particular for this experiment, the set-up is chosen to split the final grade in two parts. The first part will be the online test, the second part will be a regular final exam. In order to enable this experiment, changes need to be approved for the official course as laid down in the previously established regulations of the course.
Example 3: Mathematics proficiency tests for international students
The third example is concerned with offering remote examinations for international students who want to enter an International bachelor study program in the Netherlands. A number of students need to provide evidence of sufficient mathematical skills. The current methods of providing this evidence is in the form of supplying known diplomas or certificates based on:
- International Baccalaureate, Mathematics Higher level
- UK: Mathematics A levels
- Germany: Zeugnis der allgemeinen Hochschulreife, including Mathematics LK/eA.
- Romania: Diploma de Bacalaureat, including a Mathematics and Computer Science Profile, or Natural Sciences profile.
- Courses and exams at international locations associated with the International Baccalaureate Mathematics Higher Level/Cambridge International A-levels or in the Netherlands itself at a few specialized institutes.
The leeway up to a possible experiment with remote testing involves discussions with a lot of stakeholders in the University concerned with international admissions, subject matter experts, as well as setting up a business case. Conducting an experiment could provide more empirical evidence to support such a business case. In such a business case, the possible outcomes for remote examination (in terms of increase of student enrollments for specific countries and chances of successful study careers) are weighted against the costs of designing, maintaining and administering high quality home-made mathematics tests.
Reach-out session 1 SAVE THE DATE: April 4 2017
The OP4RE project will produce a Start Report in which the current state-of-the-art with respect to online remote examination will be described. Given the progress of our project, the concept of this report will be shared with the Associate Partners on March 24. Discussions of the Start Report with the Associate Partners will be conducted on April 4 in an online session.