• 0

An update on IO 4

Category : Dissemination

Last Wednesday 6 December 2017, the first large-scale experiment for an online proctored exam was conducted at the Vrije Universiteit Amsterdam. The goal of the study was to investigate to what extent students would successfully go through the stages of setting up the student computers at home and take a trial exam in light of optimizing the student experience.

The target group consisted of about 600 first year Bachelor students for a course in Statistics. The group was divided at random in two groups. Group one took the trial exam at the central exam hall of the university. Group two took the trial exam at home. The preliminary results of this experiment show that students prepare their hardware (computer, webcam, screen recording, microphone, mobile phone) at very late stages before the actual exam. For a number of students, this leads to problems during the log-in process of the proctoring session. As this number was quite high, it turned out that assisting these students by the VU proctors using the integrated chat-possibilities of the proctoring platform was difficult. Especially because in the tense and short period in which the exam could be started, communicating clearly, understanding the problem and provide solutions was not satisfactory for both students and proctors. The experiment showed that students need to be guided and urged in earlier stages to check all their equipment. Maybe guidelines should be developed in which the responsibility for a technical successful exam is transferred to the students at a certain stage before an actual exam. This experiment provided the OP4RE project already with important learning experiences  that will enable the design of fit-for-purpose guidelines and rulebooks.

  • 0

Update on IO 1

Category : Dissemination

Imagine Bachelor and Master degrees being completed online. Envision students enrolled in online degree programs, taking validated assignments and exams from home or any other location of choice. Increasingly mobile and flexible online offered education asks for more flexible and off-campus examination procedures in the near future.

In the past months, Risbo has taken  steps towards this goal, by starting with collecting of information on current laws and regulations about (inter)national privacy, fraud and security. Meetings with experts in the field of education accreditation like the Dutch-Flemish Accreditation Organisation (NVAO), and the Privacy Officer at the Erasmus University Rotterdam- who is an expert on privacy, fraud and security – have well informed our research.

Early 2018 will start off with the actual testing of online assessment. Students will take proctored assignments and exams within MOOCs and within Educational Programmes. For these pilots, Risbo is excited to collaborate with the IHS and RSM who have both developed  MOOCs. Other departments at de Erasmus University have shown interest to collaborate as well. Eventually, these pilots contribute to the collective development of a guideline for educational institutions who are interested in online proctoring in the near future.

Risbo/Erasmus University Rotterdam is looking forward to the first outcomes and will keep you posted in the upcoming year!

  • 0

IO2: An update on security and anti-fraud legislation protocols

Category : Dissemination

The work at the University of Hertfordshire has continued from the activities outlined in the annual report written for 2016-17 for IO2, with a focus on developing and refining a risk assessment document for online proctoring.

Currently the team is researching with a group of campus-based and trained proctors employed by our university with the aim of developing a set of guidelines for training online proctors. The team has designed a variety of methods for investigating proctoring approaches for invigilating online examinations. These are:

  1. Showing a ‘Record and Review’ scenario to a group of current proctors, followed by a guided discussion in a focus group to discuss the points they would raise as potentially suspicious behaviour from students. This was a largely reflective and qualitative study undertaken in August/September 2017
  2. A blended mixed methods approach using a campus-taught module with an online assessment is planned for early 2018. This will use a live student authentication session followed by a Record and Review mode for the assessment, with the same group of proctors retrospectively reviewing the process and identifying any potential ‘pinch points’.
  3. Thirdly, a complete ‘Live proctored’ session will be reviewed by our group of campus-trained proctors, this is planned for later in 2018.

Using recorded ‘live scenarios’ helps us to further understand what the trained and experienced invigilators/proctors in a campus-based situation are looking for when invigilating an assessment and thus where the threats to secure online assessment which is free of cheating might be.

  • 0

White Paper: Privacy in Online Proctoring

Category : Dissemination

Youssef Fouad, legal adviser of ProctorExam on personal data protection, published a white paper about Privacy in Online Proctoring.

  • 0

LERU 2017, Milan

Category : Dissemination

From 7-10 November 2017 Sabine Bottin-Rousseau attended the working group on e-learning of the LERU conference. She gave a presentation about the project and experiences at UPMC with online proctoring.

  • 0

Newsletter 6

Category : Dissemination

Newsletter 6 – November 6th 2017

On October 2nd 2017 we organized our 2nd Transnational meeting at Wilhelm Büchner University of Applied Science (Darmstadt, Germany). It was a constructive meeting on which we discussed the past project year and our plan for the next year. In particular, we have seen that the first experiments with online proctoring were conducted with various degrees of success and that these results enable us to set-up more large scale experiments in the upcoming 2 years.

Some of our project members also went to conferences to present and talk about the OP4RE project. Silvester Draaijer presented on 9 October at the TEA 2017 Conference in Barcelona the paper “Online Proctoring for Remote Examination: A State of Play in Higher Education in the EU”. Sabine Bottin-Rousseau and Pierre Beust went from October 16-19 to ICDE in Toronto to present their paper about “Online Proctoring: Experimentation of Remote Examination in E-learning programs”. They also published an article about France and the Pajama-Friendly Exam. Amanda Jefferies of the University of Hertfordshire presented a paper with the title ‘Trialling Online Proctoring for E-assessments: Early Outcomes from the Erasmus+ OP4RE Project’ at the ECEL 2017 of 26-27 October 2017 in Porto, Portugal. Silvester Draaijer will present the first findings of the OP4RE project at the upcoming Dutch SURF Onderwijsdagen on 7 and 8 November. For more information about these papers, we refer to the OP4RE website.

After the presentation at the TEA Conference, Silvester Draaijer, together with representatives of the Erasmus TeSLA project and Smowltech entered into a round table discussion. In particular, themes were discussed concerning the question of whether online proctoring should be more secure than face-to-face proctoring. How secure is face-to-face proctoring? The conference attendants agreed with the identified target groups of candidates (exchange students, sport students, students with disabilities, prospective international students) that are in line with the long term intended outcomes of the Erasmus programme to increase access to higher education.

Also, issues were discussed regarding technologies for integration and how TeSLA functionalities could work in combination with online proctoring. Also, discussions focussed on the differences in cultures in South America and the United States in contrast to Europe. In particular, it seems that in in South America, because of the wider geographical spread of the student population and less stringent requirements regarding data protection and privacy, digital identification mechanisms such as facial recognition and storing video footage can be much easier applied.

Step by step, we are adding articles to our website. Recently we added some articles about proctoring. A first article gives us a look behind the scenes of the team of proctors of ProctorU. A second interesting article gives some tips and tricks, a number of things regarding online proctoring, such as technologies, procedures, advantages, disadvantages and the like. Take a look at our website for more information.

At this moment we are planning a date for a 2nd Reach out session. We will keep you posted.

  • 0

OP4RE @ ECEL 2017

Category : Dissemination

On the 16th European Conference on eLearning (ECEL 2017), held in Portugal on the 26-27 October 2017 our partners from the University of Hertfordshire presented a paper about “Trialling Online Proctoring for e-Assessments: Early Outcomes From the Erasmus+ OP4RE Project”.

Paper: Jefferies, A., Barton, K., Meere, J., Peramungama, S., Pyper, A., & Yip, A. (2017) ‘Trialling Online Proctoring for E-assessments: Early Outcomes from the Erasmus+ OP4RE Project’ in Peres, P. and Mesquita,  A. (eds)  Proceedings of 16th European Conference for E-Learning (ECEL), Porto, ACPI pp. 221-228

  • 0

Reflections TEA Conference 2017

Category : Dissemination

Session Silvester Draaijer, Amanda Jefferies and Gwendoline Somers

Online Proctoring for Remote Examination: A State of Play in Higher Education in the EU

Silvester Draaijer, project leader of the OP4RE project, presented the paper ‘Online Proctoring for Remote Examination: A State of Play in Higher Education in the EU’ during the international Technology Enhanced Assessment conference of 8 and 9 oktober 2017.

After the presentation, Silvester Draaijer, together with representatives of the Erasmus TeSLA project and Smowltech entered into a round table discussion. The discussion were conducted in a number of small groups in which several themes were discussed. In particular, themes were discussed concerning the question of whether online proctoring should be more secure than face-to-face proctoring. How secure is face-to-face proctoring? Further, conference attendants agreed very much with the identified target groups of candidates that are in line with the long term intended outcomes of the Erasmus programme to increase access to higher education.

Also, issues were discussed regarding technologies for integration and how TeSLA functionalities could work in combination with online proctoring. Also, discussions focussed on the differences in cultures in the Southern European Countries and the United States in contrast to Europe.

– Silvester Draaijer

The pre-conference proceedings with the paper included can be found here: http://www.teaconference.org/en/contributions-submissions/pre-conference-proceedings/

The Powerpoint presentation can be found here: https://drive.google.com/file/d/0B0rrO_BjDmhjbXFmMUZEdDZ5b1E/view?usp=sharing

  • 0

Online proctoring at Caen University

Category : Dissemination

On Wednesday 18th of October 2017 Isabelle Duchatelle, Pierre Beust and Valérie Cauchard attended the “Journée thématique Examens et Certifications en ligne et/ou à distance” about “A quel niveau de maturité en sommes-nous ?” at the CNAM in Paris. They presented their experiences with online proctoring at Caen University: Expérimentation sur les examens télé-surveillés à distance. Click here for the programme.

In June 2017 they also went to the Colloque Questions de Pédagogie dans l’Enseignement Supérieur (QPES), Grenoble, to talk about “Examens télé-surveillés par webcam au domicile de l’étudiant”.

  • Isabelle DUCHATELLE – VP Déléguée aux Transformations Pédagogiques et Numériques à l’Université de CAEN
  • Pierre BEUST – Directeur du CEMU (Centre d’Enseignement Multimédia Universitaire) Détaché auprès de la MiPNES sur les missions d’Enseignement à distance
  • Valérie CAUCHARD – Ingénieure de Recherche au CEMU (Centre d’Enseignement Multimédia Universitaire)

  • 0

OP4RE @ ICDE, Toronto

Category : Dissemination

Sabine Bottin-Rousseau and Pierre Beust are giving a presentation at ICDE, Toronto, 16-19th of October 2017. ICDE is The World Conference on Online Learning: Teaching in The Digital Age – Re-Thinking Teaching & Learning. Regarding changing models of assessment, they will be presenting some experiments remote examination in e-learning programs.

Online Proctoring : Experimentation of Remote Examination in E-learning programs

Our paper will deal with the change in examination conditions for students registered in remote learning courses. The remote learning evaluation on e-learning platforms (LMS) can carry out individual and collective activities but these activities aren’t supervised. For exams, examination conditions and identity verification must be as reliable as if the examination were taking place at the institution. Today, in most of the cases, students registered in remote learning course must always travel to the university to sit their exams. In some cases, the universities have foreign partners who can organize for the exams to take place near the student’s location (which can still be costly in terms of the workload for the organizations and not being able to offer it to a larger number of students, and also in terms of fees for the students). Students can’t understand no longer the incoherence between educational activities and home support on the online platforms and the exams that must be sat at the institution. We observed that many remote students only chose to sit their exams in the end of the second semester in the “second chance” exams session which, consequently, ends up being their only exam period, breaks the rules of fairness for the students. This is easy to understand for students who live far from the university and particularly for foreign students and/or students who work. To have to travel to an institute can be expensive (taking leave from work, transport, hotel costs, accommodation…). For some students, this constraint can even be the reason why they don’t register for the course, even when it is offered online. This problem makes remote learning unattractive to potential students.

Different ways of providing remote exams are now available and can solve this problem with educational alignment. These methods must be tested and evaluated in terms of their cost, their security, their resistance to fraud, the induced stress for the students and the conditions that are put in place. We must explore alternative methods for carrying out exams remotely. We are currently experimenting with examination conditions supervised by a webcam at the student’s house and screen sharing for students studying in two universities, members of the French Association of E-learning (Fédération Interuniversitaire de l’Enseignement à Distance – FIED). These are : – The Caen University in Normandy. – The Pierre & Marie Curie University of Paris The Caen’s experiment deals with students in a master degree in a health management program. The Paris’s experiment deals with all distant programs, in mathematics and physics, from bachelor to master, for specific categories of students only for this first year of experiment.

These experiments take place in the Erasmus+ European Project “Online Proctoring For Remote Examination” (OP4RE). We will give in the paper the first results obtained from our experiments and their outcomes. We will present the results of survey that were done before and after taking the exams to evaluate the impact of the stress, and the encountered difficulties. We will then discuss about how succeed in generalizing our experiments.