Newsletter 8

Category : Dissemination

June 28th, 2018

The project is continuing in a steady pace. More and more proctored exams are being organised by the partners. We are gaining more experience which contributes to the development of guidelines, rulebooks etc.

In this regard, Fontys University of Applied Sciences developed an instruction video for students, to inform them about what it is like to take an online proctored exam, how it works, which data are recorded,… You’ll find this videos on our website. They conducted several small scale pilots (with a large scale pilot coming up, with over 100 participating students). One thing they have learned, is that it is essential to properly prepare students regarding what to expect and how the system operates. They have seen that this prevents a lot of possible issues and makes the experience much more relaxed for students, given that basically almost all their questions regarding the system have been answered before they start the ProctorExam procedure.

On Thursday April 12th, Silvester Draaijer and Robin van der Wiel presented the OP4RE project and the concept of online proctored exams to the VU University’s student panel. As an introduction to the topic, the students watched a video made by ProctorExam in order to understand how the system works. Students quickly started thinking critically about online proctoring and asked questions such as “is someone going to watch all of the video footage?”, “how is my video footage processed?” and “why are toilet breaks not allowed during the online proctored exam?”. Read more about this on our website.

At Hasselt University some new proctored exams were set up. Three students of Rehabilitation Sciences and Physiotherapy, on foreign internship, took one of their exams of the master programme via ProctorExam. For these students, taking a ProctorExam abroad provided both time- and financial savings. In addition, the master of Statistics organised a try-out exam via ProctorExam for one of their courses, both for on-campus as distance students. In this way they already gained a first experience with distance exams. Both groups were enthusiastic about the opportunities ProctorExam offers.

In previous Newsletters the OP4RE project announced to develop an online mathematics proficiency (eligibility) test in partnership with the Dutch e-learning company SOWISO. SOWISO developed the platform PassYourMath for this purpose. The offering of this test as a pilot has started at the beginning of May 2018. The development of this initiative shows that online proctoring serves the goal of increasing access to higher education well and shows the feasibility of the applied procedures and technologies. In the upcoming year the pilot will be scaled up. Interested in more?

The project partners from Risbo had a good meeting with the NVAO about online proctoring and accreditation. Continuing the previous conversations, in May the colleagues from Risbo met with NVAO (Accreditation Organisation of the Netherlands and Flanders) at the Erasmus University Rotterdam, the VU attended via Skype. The NVAO is interested in the developments in online education especially as the current shifts from within the field of education are challenging the existing approaches of the establishment principle and macro efficiency principles. Read more about this meeting on our website. 

In March 2018 the University of Hertfordshire project team, in the person of Dr. Stilianos Vidalis and his associates, set up the means to test the potential for illegal penetration of the ProctorExam website. During a 24 hour period, intense and persistent attempts were then made to access the ProctorExam servers illegally but no unauthorised access was possible, the denial of service attack was not successful and no information leakage was detected. Want to know more about this? Check our website.


Online proctoring at Hasselt University

Category : Dissemination

In the UHasselt magazine of June 2018, a short article was dedicated to online proctoring at Hasselt University.
This article describes the OP4RE project and briefly presents the state of affairs at Hasselt University.

Article only available in Dutch.

UHasselt Magazine, June 2018 (3).


OP4RE talks with NVAO about accreditation          

Category : Dissemination

Continuing our previous conversations, in May we met with NVAO (Accreditation Organisation of the Netherlands and Flanders) at the Erasmus University Rotterdam, the VU attended via Skype. The NVAO is interested in the developments in online education especially as the current shifts from within the field of education are challenging the existing approaches of the establishment principle and macro efficiency principles. Online education is still seen as an addition to campus-based education, this is legally defined by law. Fully online offered educational programs can in the meanwhile not be accredited. Even an online offered master program via MOOC the testing takes place on campus.

Creating change in the current approach to education is possible. In case there is sufficient support from educational institutes who are developing innovative initiatives, changes in the field of education can take place. New programs and courses are approved by commissions. The NVAO is involved in the case of the program, whereas with courses external commissions are involved such as Quanu. It is advisable to check the expertise of the commission connects the course content.

Further, there are conventions about developments in education on European level. In the course of this year a report from the ENQUA is likely to be published. For now, the NVAO suggests to check the TeSLA project for current developments in Europe.

Both Risbo and NVAO want to continue the pleasant cooperation continue these conversations about the current developments that are emerging form the educational field. Thus, OP4RE is invited for an e-learning day in November during which developments in online education will be discussed, the Dutch ministery is invited to attend.


Testing the hackability of the ProctorExam assessments website

Category : Dissemination

In March 2018 the University of Hertfordshire project team, in the person of Dr Stilianos Vidalis and his associates, set up the means to test the potential for illegal penetration of the ProctorExam website.

In the original Erasmus+ project proposal submitted in March 2016 we had identified this activity as an important area to explore in line with identifying future guidelines to present in the final report for Intellectual Output 2 Security. Thus the aim of this exercise was to demonstrate whether a typical online student would be able to access an unprotected area of the ProctorExam servers and illegally penetrate the security of any aspect of the online assessment systems. The Proctor Exam system does of course already operate with its own secure system designed to counteract typical professional attacks and other random attempts to hack into their site.

The Penetration Testing Execution Standard (PTES) was used together with Open Web Application Security Project (OWASP) methodologies as a baseline for undertaking the tests by Stilianos.

The process requires the initial agreement of typical use cases and scenarios to identify example personas of those likely to want to attack or hack into the online assessment system, and their motivation, to either steal or change data stored on the site or severely impede legitimate activity. During a 24 hour period, intense and persistent attempts were then made to access the ProctorExam servers illegally but no unauthorised access was possible, the denial of service attack was not successful and no information leakage was detected.

It will be important to note in the final project report that all organisations which set themselves up as providers of secure online assessments should be able to withstand a similar simulated attack to assure future users and partners that assessments are undertaken in a secure environment where personal student data and performance details are protected and inaccessible to unauthorised users.


Meeting with VU University’s student panel

Category : Dissemination

On Thursday April 12th, Silvester Draaijer and Robin van der Wiel presented the OP4RE project and the concept of online proctored exams to the VU University’s student panel. As an introduction to the topic, the students watched a video made by ProctorExam in order to understand how the system works. Students quickly started thinking critically about online proctoring and asked questions such as “is someone going to watch all of the video footage?”, “how is my video footage processed?” and “why are toilet breaks not allowed during the online proctored exam?”. After Silvester had explained some more about the system, the floor was opened for a discussion of privacy matters concerning the system.

Most of the students noted that they would not feel comfortable with someone watching their every move. At a university exam location, there is one proctor for every 20 or so students; being watched one on one feels extreme. Students also recognized the positive aspects of the system. It allows students that are abroad or not able to come to a university location to make exams safely from a remote location. It can be concluded from the student panel that it is very important to inform students. They want to know how the system works and how their video footage is processed. The remarks of the student panel will be taken into consideration during the remainder of the OP4RE project.


Preparing students for an online proctored exam

Category : Dissemination

As the project progresses and more assessments take place via online proctoring, we -the project team- have gained more experience. Communication with our students appears to be a critical issue in the process of organising and setting up the online proctoring of exams. As a result Fontys have developed a short video in which they explain in ‘student language’ what online proctoring is, how such assessments work, which videos are recorded and the overall process for online assessment  at Fontys. Tips and tricks, and do’s and don’ts are given. In addition, they have also developed a video to guide students through the start-up procedure of ProctorExam.

Watch the videos here (Dutch version):


OP4RE – ProctorExam and the Mathematics online Eligibility test 2018

Category : Dissemination

Managing a diverse stream of incoming domestic or international students can be a real challenge for universities. That is why, in partnership with several Dutch universities, SOWISO will start offering upcoming first-year students an online mathematics test to show that their mastering of mathematics is sufficient to be eligible for enrollment.

To make sure that there’s no cheating, the test will be proctored with the help of ProctorExam and OP4RE; this means students will be monitored during the test.

The content of the test was constructed by a small group of university mathematics teaching experts to converge to a test that has broad acceptance with respect to validity and reliability in the Netherlands. The offering of this test as pilot has started at the beginning of may 2018. More details: take a look at our website.

Currently, the first candidates have taken this online test. Expressed in numbers we now see the following:

  • About 80 students of the Vrije Universiteit were offered the option to take either a mathematics eligibility test in the Netherlands or online.
  • About 35 students expressed their interest to take this test on the online platform of PassYourMath
  • About 20 students registered to take the online preparation modules for the test to become accustomed to the platform (content and level of difficulty of questions, method of entering answers).
  • About 10 students registered to take the online proctored test so far. About half of them had some kind of difficulty in setting up the test correctly. It turned out that because of some internet restriction imposed by the country in which the candidate resided, it was not always possible to use the webcam.
  • Not a single case of suspected fraud was identified. All students were issued a Certificate of Achievement by PassYourMath.
  • About 7 of these students achieved a sufficient score set as threshold for eligibility.
  • The first Certificate has now been verified as means to grant a student access to the Vrije Universiteit.

The development of this initiative shows that online proctoring serves well the goal of increasing access to higher education. This is exactly the main goal of the Erasmus funding. No processes have yet been automated with respect to registering students and have student sign-on the PassYourMath and ProctorExam with one account. This pilot however shows the demand by prospective students for online proctoring. The development also shows the feasibility of the applied procedures and technologies. In the upcoming year the pilot will be scaled up.

Read the blogpost 


Jury prize @ the FIED International colloquium

Category : Dissemination

At the FIED international colloquium held in Paris on 13 and 14 December 2017 the OP4RE project was presented at a round table discussion attended by Silvester Draaijer, Daniel Haven, Sabine Bottin-Rousseau and Pierre Beust.
It should also be noted that a video competition entitled “My innovation in distance learning” was organised prior to the colloquium, the prizes of which were awarded during the colloquium.
The experimentation of remote monitoring of home examinations carried out in Caen within the framework of OP4RE’s IO3 won the jury prize of this competition.
Watch the video here.


Update on IO 5: Privacy

Category : Dissemination

Outline of IO5 – Privacy

In order to be compliant with respect to privacy in international online proctored remote examinations, it is needed to have ‘lawyer proofed’, EU compliant contract templates, all kind of agreement forms and guidelines for examinees and tester. These templates are intended to be part of the Intellectual outputs of the OP4RE project, but also to convince all future relevant stakeholders, such as accreditation agencies and data protection officers of the value and legitimacy of online proctoring.

Current situation

The first step of the team responsible for IO5, which info (inside which documents) we need in order to fulfill our goals. Therefore two privacy impact assessments have been performed. The identified documents are risk acceptance forms, data classification and processing agreements, privacy policies. Furthermore guidelines and templates for legal notices, consent texts, and privacy statements are required.

The next step was to collect information from different sources, such as data protection experts, EU regulations, and already existing material from all OP4RE partners as best practice templates. Now, we are in the middle of the info consolidation phase.

Planning of year 2

The finalization of documents for review is followed by a fine tuning by all OP4RE partners. In parallel, the experiments with online proctoring at the various partners of the OP4RE project will be expanded.


Update on year 1 of IO3: Bring your own device/ off-campus examination protocol

Category : Dissemination

Pierre Beust & Sabine Bottin-Rousseau, of FIED (Fédération Interuniversitaire de l’Enseignement à Distance), wrote a report on the progress and the activities on IO3 about bring your own device and off-campus examination protocols.