Update 25 September 2021
In 2021/2022 education and assessment will preferably take place on-campus. In Q1, some programmes may still choose to offer students an alternative for on-campus exams, in case they need to self-quarantine. Possible alternatives are:
- Schedule a third exam on-campus (students can only take part in two exams per academic year).
- Offer a hybrid exam, i.e. synchronous on-campus and online exam. See for practical information on hybrid exams.
It depends on the nature of your assessment which one is more practical for you and your students. In case students need to pass your course to get access to a follow-up course that starts before your regular retake, a hybrid exam is preferable. In other cases, scheduling an extra exam on-campus might be less work than taking fraud prevention measures in a remote exam.
As of 25 September, exam halls can be used for any numbers of students without 1.5 meter distance, as long as the ventilation norms are met. See the latest Service Documents of the Ministry of Education.
Depending on the choices of your faculty, some assessments might still need take place fully remotely.
This page contains advice on how to create remote assessments/the remote part of hybrid exams. Follow the steps in the reading list below.
Table of contents and links to related pages
- Hybrid written exams: choose the optimal combination of on-campus and remote assessments for your course.
- Read the eight guidelines for remote assessment
- Follow the decision tree: How to create a remote assessment?
-
Choose your remote assessment type
Educational how-to guides per assessment type:- Projects/assignments:
- Written exams:
- Oral exams:
- Remote oral exams (by Gillian Saunders, lecturer at TU Delft)
- Security and privacy of remote oral examination
-
Choose your remote assessment tools
And find out how to use the selected tool: - Choose your fraud prevention and detection measures
- Follow the procedure for switching your assessment to remote
2. Read the eight guidelines for remote assessment
The guiding principle for developing quality remote assessment is that during assessments, we should enable our students to demonstrate how well they master the learning objectives. This requires constructive alignment and fairness. The eight guidelines in the table below ensure that. For more detailed information, see the reader of UTQ ASSESS.
Table 1: Eight guidelines for developing good quality remote assessment
The underlying assessment quality requirements are validity and reliability.
Validity
The assessment should cover the learning objectives, and nothing more.
- If one learning objective cannot be assessed properly in a remote situation, dispensation may be given if that objective is tested elsewhere in the programme. However, this has to be approved by the Board of Examiners.
Figure 1: Constructive alignment of learning objectives, learning activities and assessment to stimulate (active) learning
Validity matches the concept of constructive alignment (Biggs, 1996, see Figure 1). This implies that the assessment of your course should match a) the learning objectives and b) the learning activities during the course. As a result, students should be able to practice with what they will be assessed on.
Reliability
The grade should reliably (for background information, see reader UTQ ASSESS) represent how well a student masters the learning objectives.
For a reliable grade, instructions in assignments and questions should be clear for students (e.g. they should know the required level of detail of the answer in order to get a full score).
- Grading should have enough precision. This implies that large questions that are automatically graded as ‘correct’ (all points) or ‘incorrect’ (no points) should be prevented or mitigated, for example by having students explain their calculation method and granting partial points for partially correct answers.
- Assessors should receive clear grading instructions that should ensure objective assessment, minimize bias and align the grade with the learning objectives and assignment instructions. Other grading-related measures include anonymous grading and grading per question (instead of per student).
- While assessing, grading, or determining fraud, the lecturer should strive to be unbiased. If possible, grading should be anonymous and oral authenticity checks should be based on an unbiased sample of students.
See the subpage on fraud prevention for exams for more information.
Take sufficient fraud-prevention measures to reduce the risk, while minimizing the downsides of these measures: our students should still be able to demonstrate how well they master the learning objectives, without being hindered by the fraud prevention measures. Furthermore, students should not feel the need to commit fraud because they feel well-prepared for the assessment. Furthermore, students should not feel the need to commit fraud because they feel well-prepared for the assessment.
Be available during the assessment to resolve possible unclarities in questions/instructions, and for technical issues. You could use email, telephone, Brightspace Discussions (which requires refreshing the screen to be able to see changes, both for students and you), or conference/chat tools. Ask for help from TAs or colleagues if needed. This will reduce the inclination to contact their peers and therefore commit fraud.
Students should be able to practice with the assessment setting, questions and tool(s), ample time before the assessment day. Do a practice exam 1-2 weeks before the real exam. Goal: to test the procedure as well as students’ devices and apps. Furthermore, it will familiarize students (and you) with the procedure, which will diminish stress and improve performance.
Study delay should be prevented.
- Check for conflicts with deadlines and assessment slots in other courses (5).
- The assessment should be feasible for the lecturers, examiners and assessors (during assessment construction, during the actual assessment and during grading), (5).
This is based on the underlying assessment quality requirement of inclusiveness.
- Some students are entitled to extra time (10 extra minutes per hour), as is stated in the TER (article 25). You will need to adjust the settings of the assessment tool.
Click for step-by-step guides on how to organize extra time in Brightspace Quizzes and Brightspace Assignments, using ‘Special Access’. Detailed instructions for quizzes can be found here and for assignments here.
In case of a Möbius exam, Digital Exams will take care of the extra time settings. When starting the exam, students will see two options which they can select: the exam with the regular duration and one with extra time. - Students with special needs should have enough time to request and discuss special accommodations. Adjustment can involve form, timing, aids permitted during testing, and location. In case your students are entitled to other types of adjustments, like audio support, it might be useful to create separate exams. Special needs provisions are given by the Board of Examiners. Students can apply for these provisions via the student counsellors. The Boards of Examiners (via the study counsellors) should inform you which provisions need to be available for which student. Lecturers themselves are not allowed to grant these provisions.
- Use the approved tools from the TU Delft tool list that are compliant with privacy rules and regulations. In case you use other tools while there are suitable tools available on the TU Delft tool list, students have the possibility to refuse to use the tool, in which case the lecturer should offer the assessment without the use of this tool to the student However, this implies extra work for the lecturer, so the use of incompliant tools should be prevented.
- In case recordings are made, this should be considered privacy-sensitive data, and the recordings should be kept in a safe place and deleted as soon as their existence is not necessary anymore. Click here for guidelines for security and privacy of (recordings of) oral exams.
- Students need to be informed as soon as possible of any changes in assessment and procedure compared to the study guide description.
- Communicate details of the exam procedure. That way, they know what to expect and can focus on the content.
- It should be clear on what criteria students will be assessed in assignments, and what level of detail is expected in exams and assignments.
- The grading of the assessment and weighing of assessment parts (or criteria) should be clear before, during, and after the assessment.
- One way of creating clear criteria for assignments and projects is a rubric. For more information on rubrics, go to Rubrics to Grade Assignments
3. How to create a remote assessment?
Use the following scheme to determine how large the changes are that you need to change. Click on the image below for a pdf with links. It answers the following questions:
- Do I need a (partially) closed-book assessment?
- If so, how do I set up a closed-book remote assessment?
- In general, do I need to change the set-up of my assessment (or can I use the remote version of the on-campus assessment)?
In case your conclusion is that you need to make large changes that need approval, read the section below.
4. Choose your remote assessment types
Here is a rough schedule that can help you to determine your remote assessment type. The general advice is to keep the changes small, by preferably switching to the remote version of the original assessment.
Figure 2: Frequently used changes of assessment type. For *written exams, there are multiple possibilities, see running text.
The schedule above demonstrates frequently chosen changes from on-campus assessment type to remote assessment type. A combination of remote assessment types is also possible. In that case, strive for a spread of deadlines and exams for your course and the other courses that students take.
Remote written exams can be administered in two ways:
- as downloadable assignments: During the exam part’s time slot, students download the exam and upload their answers before the deadline, for example, 90 minutes later. Exams can be split into 2-4 parts with each their own time slot. The answers can, for example, be typed word-documents or pdfs, or scanned handwritten answers in pdf format. Uploading and converting documents can take up to 10 minutes per test. Uploading videos is also possible (both for the student and the lecturer).
For a how-to in Brightspace Assignments, see here. - as a (partly) automatically graded digital exam: you and your team can either manually grade longer questions, which allows for partial grading (giving a student for example 2 out of 5 points for partially correct answers), or use automatically graded questions with short answers. In the latter case, you would have to review the incorrect answers for unforeseen answers which are nevertheless correct.
For a how-to in Möbius, see here.
For a how-to in Brightspace Quizzes, see here.
Ans Delft or Grasple is available in some faculties. For a comparison between the tools mentioned above, please click here.
Larger projects and assignments can have multiple deliverables and assessment items, like reports, presentations (either live in Virtual Classroom (Bongo) or MS Teams session, or by submitting a recording in Brightspace Assignments), code, or observed skills. Handing in content in Brightspace Assignments with plagiarism check is frequently part of the solution, as well as a (group) presentation via a conference tool. Make sure that you provide students with sufficient supervision and support.
Depending on your selected remote assessment type (see above), you can choose an assessment tool and fraud prevention measures. Open the next chapter to read more on the different assessment tools.
5. Choose your remote assessment tools
The available tools can be divided into two solutions:
- Downloading the exam and having students upload the results (in Brightspace Assignments).
- Using an assessment tool in which students fill in the answers per question, and which are (potentially) partially automatically graded (other tools).
The latter option gives more possibilities for automatic grading and for giving students unique exams. The first requires less effort from your side because you can use a similar workflow as for an on-campus exam.
An extensive comparison table of the available tools can be found here.
For oral exams, you can use the following tools:
- Virtual Classroom (Bongo): integrated in all Brightspace courses. The waiting room and break-out rooms are available to have TAs welcome students and help them test and set up their connections (handy in case of short exams or oral checks and large student numbers).
- Skype4Business: no recording available on Macs, no waiting room. Students cannot log in with NetID (which could help with the identity check)
- MS Teams: Break-out rooms have become available. Students log in using their netID.
-
- Brightspace assignments: students upload their assignments or report individually or as a group (Brightspace allows for group assignments), before the deadline. The uploaded work is typically a report (pdf or .docx), code, drawing or a recording of a presentation (movie). You can find more information on how to create good assignments and how to create rubrics.
- Brightspace assignments can also be used formatively, to give students larger assignments or projects and have them upload intermediate products for you to provide feedback.
- For peer feedback on intermediate products, use FeedbackFruits Peer review within Brightspace. Contact Teaching & Learning Support to enable this in your Brightspace course.
6. Choose your fraud-prevention and detection measures in remote assessment
You can find an overview of all fraud prevention options on the subpage on fraud prevention, including more detailed information on online proctoring.
7. Follow the procedure for changing your assessment to remote
Depending on the faculty, lecturers adapt the assessment description to the remote situation in the study guide either:
- Before the start of the academic year: Lecturers described both on-campus and remote assessment in the study guide (which is an elaboration of the Teaching and Education Regulations, TER) before the start of the academic year.
- Shortly before the start of the semester/quartile: In some faculties, lecturers can propose changes to the study guide shortly before the start of the semester of quartile, to adjust the assessment (and education method) to the current COVID-19 measures. The proposed changes in the study guide may need to be approved by the dean, after consent of the Faculty Student Council and advice of the Board of Studies.
However, if you need to change your assessment later and need to deviate from the study guide text due to unforeseen circumstances (for example an unforeseen extremely low/high number of students, or a sudden change in COVID-19 measures), you need to ask permission from your Board of Examiners, depending on their rules and guidelines.
Contact your programme coordinator or programme director for information about the procedures in your faculty.
Request permission for online proctoring
Online proctoring during digital exams as an anti-fraud measure is only allowed as a last resort. Be aware that you need to ask permission to use online proctoring as an anti-fraud measure, usually from the Board of Examiners and sometimes from the faculty. Contact your programme coordinator or programme director for information about the procedures in your faculty.
Relevant regulations
In case of suspicion of large-scale fraud, the Board of Examiners can declare all announced grades invalid, if it is impossible to determine who committed fraud and who did not. Any kind of participation in large-scale fraud is considered a case of serious fraud. In case of serious fraud, ‘the Board of Examiners is entitled to propose to the Executive Board that the student’s enrolment on the degree programme be permanently terminated’. Click here for more information on fraud and consequences.
In case the reliability of the grade is not sufficient to express the results as a grade (1-10), the Board of Examiners can allow the examiner to change the grade to a pass/fail decision (fail-pass). Variations on pass/fail are not allowed.
Pass/fails do not count for the student’s average grade (GPA).
If a student passes a course with a 6.0 and wishes to retake the exam with a pass/fail exam, the grade will not be replaced by a ‘pass’, because the (new) Rules & Guidelines of the Boards of Examiners state that when having received both a grade and a pass/fail after a retake, any pass grade is considered to be higher than a ‘pass’ and will be kept
Students with special needs can be entitled to extra time (10 extra minutes per hour), as is stated in the TER (article 25). Other adjustments can involve form, aids permitted during testing and location.
There are two practical ways to organize extra time:
- Give ‘Special Access’ to groups of students with disabilities. Set their end-time of the test 10 minutes later per available 60 minutes. Detailed instructions for Brightspace Assignments can be found here and for Brightspace Quizzes here.
- You could also create two identical exams that differ only in duration, split the students into a normal-time group and an extra-time group, and make each quiz available to either of the groups. Make sure they can only access the applicable relevant exam.
Page history
- Added information on Q1 assessment, included a link to the hybrid assessment page and basic information on scheduling two exam resits
- Added the new assessment tools comparison page.
- Link to the new Möbius Support Overview page of the Teaching lab.
- Updated the fraud prevention page with a comparison of fraud prevention measures.
- Updated this page, added the decision tree for remote assessment
- Incorporated the tips from Q3 lecturers in this page, fraud prevention page and the step-by-step guides on how to set up examinations in Brightspace Assignments and Brightspace Quizzes.
- Merged guidelines from Boards of Examiners into general assessment guidelines. Added the guiding principle that in assessments, students should be able to demonstrate how well they master the course to the guidelines for assessment. Focused the guidelines on this guiding principle. Improved the key elements for good quality remote assessment.
- Changed order of page (start with the guidelines)
- Improved overview on content and subpages at start of page.
- Adapted the procedure to new Rules and Guidelines and Teaching and Education Regulations 20-21.
The following guidelines were added and changes made:
- Small changes because the page was checked for 2020/21.
- Maple TA is replaced by the new version called Möbius, see Written exam tools.
The following guidelines were added and changes made:
- The subsite How to create fraud resistant exams was improved. Overlap with this page was reduced and overlapping parts were moved to the fraud resistant exam page.
- Added links to the Online Proctored Examinations Regulation. Updated content on online proctoring here and at the fraud resistant exam page to match the content of the exam.
- Updated links, changed minor parts of the page, mostly textual.
- Added a link to a recording of an introductory webinar on MapleTA by Kasja de Jong.
- Added two step-by-step manuals (under construction) on how to create exams in Brightspace Quizzes and Brightspace Assignments.
The following guidelines were added and changes made:
- Updated links, changed minor parts of the page, mostly textual.
- Guidelines on the following topics were processed on this page:
- Do’s, don’ts and tips based on experience of teachers with remote assessment in Q3
- Postponement of assessment is no longer an option: deleted text that discussed the options to postpone assessments.
Guidelines on the following topics were processed on this page:
- Update Taskforce Assessment (6 April 2020) on studying with a disability, security and privacy guidelines for oral exams, and complementary oral checks.
- Advice on ‘poor man’s proctoring’ (30 March 2020)
The following manuals were added:
The previous guidelines on the Brightspace support website are quite strict, because Boards of Examiners were initially quite reluctant for remote assessment. Given the current situation, they are more open to remote assessment, if the quality of assessment is assured. Part of the quality of assessment is to minimize the risk of fraud. Therefore, we have adapted the homepage to reflect more lenient guidelines for remote assessment.
The first version of this page was published on 24 March 2020