Author Archives: admin

  • 0

Desk research relevant articles

Category : Uncategorized

The deskresearch phase unveiled both interesting scientific, professional as well as general information and documents. We will present a few of them here.

Behind the Webcam’s Watchful Eye

The Chronicle of Higher Education published an interesting article about the proctors at ProctorU: Behind the Webcam’s Watchful Eye, Online Proctoring Takes Hold. Author Kolowich provides a lively description of what is going on in rooms full of proctors, what they do and how proctors are proctored to do a good job.

See more at: http://www.chronicle.com/article/Behind-the-Webcams-Watchful/138505#sthash.KtyW57ZK.dpuf


  • 0

Newsletter 5

Category : Newsletter

On May 9th 2017 we organized our first online reach-out session. It was a rich learning experience, both on content level as on organizational level. The technology let us down, but we’ll take this as a working point to our next reach-out session. We hope to welcome more of our associated partners the next time.

The reach-out started with an explanation of the design of the OP4RE project and of the content of the Start Report. After this explanation of the OP4RE by project coordinator Silvester Draaijer, Daniel Haven, from ProctorExam, showed their proctoring tool. After this introduction, 3 partners explained about their first experiences with online proctoring. First Michiel van Diesen (Fontys, the Netherlands) shared their experience with online proctoring. Then Pierre Beust (FIED, France) highlighted the experiments of home proctor exams at the University of Caen. From Belgium we invited VIVES University College, department of Distance Education. They have a long tradition of distance learning and they work with ProctorExam to deliver some 1000 of online exams annually. In order to support this amount of online tests to be taken, VIVES has dedicated staff to plan, administer and proctor the exams. For more information and presentations, we refer to the OP4RE website.

In the coming months we will continue to work on intellectual outputs and more experiments will be set up. As we go on, we experience that we differ from our planning, due to the fact that the IOs regarding Accreditation and Privacy do not lend themselves very well for strict linear project planning. Maybe some shifts will be made during the second Transnational Meeting on October in Darmstadt. Concerning the previously mentioned issues with privacy and data protection, we continue our quest. The exact rules and practices to comply with high sensitive personal digital data are far from clear yet. Further, ProctorExam has made a new partnership with the “Associatie voor Examinering”, which was already an Associate partner of the OP4RE project. We start more intensive discussions with them for collaboration.

 


  • 0

Online Proctoring Systems Compared

Category : Security

In March 2013, David Foster Caveon of Caveon and  Harry Layman of The College Board published an article comparing in quite some depth various online proctoring systems. On the last pages they present some overall findings in which they distinghuis what system would be needed for the height of the stakes of the exams. Very insightful.

Further, they report about the percentage of recorded proctoring misbehavior incidences that are typically encountered during proctoring sessions. In the OP4RE project this will be taken as a benchmark.

You can read the complete report here:

 

https://ivetriedthat.com/wp-content/uploads/2014/07/Caveon-Test-Security.pdf 


  • 0

Reach out session 1 – 9th of May 2017

Category : Reach out sessions

On May 9th 2017 we organized our first online reach-out session. We invited our associated partners. It was a rich learning experience, both on the content as on the organizational level. The technology let us down, but we’ll take this as a working point to our next reach-out session.

We started with an explanation of the design of the OP4RE project and of the content of the Start-report. This report is the official“start” of our project. We looked into the current state of affairs regarding the online proctoring of remote examinations in participating countries. From this desk research a lot of questions have emerged, which we are currently taking on in the project. After this explanation of the OP4RE project coordinator Silvester Draaijer, Daniel Haven, from Proctor Exam, showed us their proctoring tool. Proctor Exam joins the OP4RE project.

Then we got an explanation of 3 partners about their first experiences with online proctoring. First Michiel van Diesen (Fontys, the Netherlands) shared their experience with online proctoring. Then Pierre Beust (FIED, France) highlighted the experiments of home proctor exams at the University of Caen. For the proctoring they use Proctor U. In year 1 (2015-2016): 55 students participated. Blended learning and home proctoring were proposed in addition to classical examinations. In year 2 (2016-2017) 50 students in a professional Master degree in the area of health with 2 or 3 exams participated. They were full distance learning students, spread all over the world. Home proctoring was organized instead of classical exams. From Belgium we invited VIVES. In Belgium, they were the first (and as far as we know the only) institution for higher education, which started with the proctoring of exams. VIVES has a long tradition of distance learning. About 15% of the students follow a distance learning programme (on a total of approximately 13.000 students). Students in distance programmes can take their exams in the exam center in Kortrijk or online via proctoring. For the proctoring, VIVES works with ProctorExam, one of the partners of the OP4RE project.

Attachments:


  • 0

E-Assessment: commissioning, implementing and delivering

Category : Dissemination

Daniel haven and amanda Jefferies represented the OP4RE project today at the 15th annual E-Assessment conference and exhibition. The focus of this conference was on hwo computer based assessment have grown significantly since the early days of automatic test publishing and certification. We have seen developments like paperless examination projects, e-proctoring, open badges, social media, BYOD, the growth of e-portfolios, the proliferation of question types, the rise of government initiatives and policies following FELTAG, Sainsbury and Curriculum changes, the expectations of digital natives and rapid changes in technology.

The path has been slow and inexorable but hardly smooth, the need to manage risk and costs as important as drive for innovation and the expectations of the users.

The focus for the 15th annual international e-Assessment Question was on the practical issues of commissioning, implementing and delivering e-Assessment solutions.

For more information visit:

http://www.e-assessment-question.com/


  • 0

Newsletter 4

Category : Newsletter

The anticipated complexity of privacy and data protection is surfacing as is shown in the flow chart below. For any institution wanting to start to use online proctoring one should consider the concept of Privacy by Design. This flow chart can be of assistance in designing processes and agreements with that goal. So, after identifying opportunities for online proctoring, each institution will have to develop and implement privacy and data protection policies, regardless of any proctoring system being used. The relevant officers need to be identified and the relevant procedures and agreements should be drawn up and agreed. This Privacy by Design approach needs to be conducted along other aspects of online proctoring that are of importance such as Security, Fraud detection, Fraud regulations etc. So, multiple streams of policies and technical studies need to be executed when an institution want to start using online proctoring.

Yet again, some general – and relative easy and obvious – guidelines can be identified already when doing any kind of online proctoring. We will provide a few examples:

  • When performing an online exam, candidates need to be informed in advance about the nature of the exam and their consent to use the data is needed. Furthermore, candidates need to be made explicitely aware of what is going to happen with the data (ownership, privacy,…). In some institutions this kind of experiments (with students) needs to be submitted to an ethical commission.
  • When conducting online remote examination, some Privacy Policy arrangements need to be provided, including protocols, procedures and rules of conduct. Both for the data owner (institution of higher education) as well as the data processor party (the proctoring solution provider). Privacy and security impact assessments need to be performed by institutional Privacy and Security Officers. In addition processor agreements should be drawn up and agreed upon between the proctor service provider and the institution as data owner. Additionally, it may be that also processor agreements are needed between the test supplier and the data owner.
  • Issues concerning Privacy Regulations when multiple and/or foreign countries are involved should be cleared out as well. For example: which regulations need to be complied with when an institution for higher education in the Netherlands is organizing online proctoring for remote examination of Russian students, and the video data is stored in Germany. How do international regulations (i.e. foreign laws, local laws) and institutional procedures match? Which specific regulations are applicable? Not at least because institutions for HE as responsible parties can face large fines if the act illegally or do not comply fully.

 

 


  • 0

VU education day

Category : Dissemination

Today Silvester Draaijer and Daniel Haven presented the OP4RE project at the VU education day. The topic of this mini-conference was the Gross Academic Value. How do the three central concepts of social connectedness, competence and sense of autonomy play a role  in motivation and in education? What are these concepts and how can they be used to design curricula and courses?

To address this topic Daniel Haven and Silvester Draaijer presented the added value that online proctoring can offer both the traditional and modern eudcational setting.

See the link below for more information:

http://knowvu.nl/en/events/education-day-vrije-universiteit-2017/


  • 0

Newsletter 3

Category : Newsletter

Dear Associate Partner of the Erasmus+ project OP4RE,

Following the first transnational meeting of October 25 and 26, the OP4RE Project Management Group is progressing in the development of the project. For each of the 5 Intellectual Outputs (IO), more detailed plans are under construction.

The project members that are concerned with privacy and security aspects of remote examination are working on setting up a framework for institutional impact assessment procedures and agreements regarding security, risk acceptance and privacy. More information on this IO will follow in later Newsletters.

Identifying and getting all conditions in place in order to conduct experiments takes quite some leeway and members of the PMG has started to initiate this already. Below we present three examples of experiments in preparation.

Example 1: Selection test for entrance to Bachelor programs with numerus fixus

Experiments for remote proctoring are now set-up for Bachelor study programs that require selection of students (Numerus Fixus programs). In normal conditions Dutch candidates needed to take a digital selection test in the Digital Exam Hall of the Vrije Universiteit Amsterdam for that purpose. Dutch institutions for higher education however are obliged to offer students living in the so-called special overseas municipalities (Bonaire, Saba and Sint Eustatius) the possibility to take these selection tests under the same conditions as Dutch students. For these selection tests, the OP4RE project will provide the services to take the test for these overseas candidates remote at their home. The selection test that will take place in February 2017 to April 2017 will be used as a pilot for future selection test (2018). In the pilot, the procedures, the survey questions, the technical set-up of the online tests etc. are being detailed and tested. Extensive discussions are conducted with regards to these procedures and the possible problems that could arise and their mitigation.

  • The problem of preventing fraud or the illegal recording of the test itself by candidates;
  • The procedures in which students are made aware of their consent for privacy and security of online remote testing;
  • The equivalence of the different forms of the test. For example, in the Digital Exam Hall, students get a case-based questions handed out on paper, while overseas students need to read the case information from the screen;
  • How to deal with the issue of going to the toilet for a 3-hour test. In the face-to-face setting, candidates are allowed to go to the toilet after finishing part of the test and before entering the successive part: how to deal with this in a remote situation.

Example 2: Experimental study with a large cohort of students

The second pilot study is concerned with an experiment in which the equivalence and comparability of exam results for on-campus proctoring and remote proctoring of digital tests is studied in a scientific experimental setting for the domain of statistics for a large cohort (300+) of students for December 2017. For this experiment, teachers and the board of examiners need to be involved heavily in the decision making process. In particular for this experiment, the set-up is chosen to split the final grade in two parts. The first part will be the online test, the second part will be a regular final exam. In order to enable this experiment, changes need to be approved for the official course as laid down in the previously established regulations of the course.

Example 3: Mathematics proficiency tests for international students

The third example is concerned with offering remote examinations for international students who want to enter an International bachelor study program in the Netherlands. A number of students need to provide evidence of sufficient mathematical skills. The current methods of providing this evidence is in the form of supplying known diplomas or certificates based on:

  • International Baccalaureate, Mathematics Higher level
  • UK: Mathematics A levels
  • Germany: Zeugnis der allgemeinen Hochschulreife, including Mathematics LK/eA.
  • Romania: Diploma de Bacalaureat, including a Mathematics and Computer Science Profile, or Natural Sciences profile.
  • Courses and exams at international locations associated with the International Baccalaureate Mathematics Higher Level/Cambridge International A-levels or in the Netherlands itself at a few specialized institutes.

The leeway up to a possible experiment with remote testing involves discussions with a lot of stakeholders in the University concerned with international admissions, subject matter experts, as well as setting up a business case. Conducting an experiment could provide more empirical evidence to support such a business case. In such a business case, the possible outcomes for remote examination (in terms of increase of student enrollments for specific countries and chances of successful study careers) are weighted against the costs of designing, maintaining and administering high quality home-made mathematics tests.

Reach-out session 1 SAVE THE DATE: April 4 2017

The OP4RE project will produce a Start Report in which the current state-of-the-art with respect to online remote examination will be described. Given the progress of our project, the concept of this report will be shared with the Associate Partners on March 24. Discussions of the Start Report with the Associate Partners will be conducted on April 4 in an online session.


  • 0

Presentation of the OP4RE project

Category : Dissemination

Today Silvester Draaijer presented the setup of the OP4RE project for the Special Interest Group Digital Assessment (SIGDT) and the SURF Foundation. During this boar meeting many of the aspect, goals, and aspirations of the project were discussed. Especially with regard to the future use for EU digital assessment.


  • 0

Conference: The Online, Open and Flexible Higher Education Conference

Category : Dissemination

At this conference Sabine Bottin-Rousseau and Daniel Haven presented the OP4RE project. Information about their presentation and the rest of the conference on creating online higher education can be found:

https://tinyurl.com/ya3nttna

https://conference.eadtu.eu/images/Proceedings/Conference_proceedings_2016_defcompressed2.pdf