The purpose of this guidance is to outline ORMS’ approach to the preparation and scrutiny of summative instruments of assessment (for example, examination papers and coursework/assessment briefs).
The guidance applies to all full-time and part-time taught provision and applies to the early stages of courses as well as the final award bearing stages.
3. Key Principles
The detailed processes adopted within ORMS may vary according to local circumstances; however, it is essential that the following three key functions of the system are efficiently coordinated and implemented:
- preparation of summative instruments of assessment and associated model answers/grading schemes;
- internal scrutiny of summative instruments of assessment and associated model answers/grading schemes;
- communication with External Examiner(s), where relevant.
4. Preparation of Summative Instruments of Assessment
Each Course Leader / Course Director, or their representative, should establish timescales for the production of summative instruments of assessment in line with the assessment schedule for the course.
4.1 Assessment Schedule
At the beginning of each stage of study, students should be provided with an assessment schedule. This should provide them with information about:
- when the various summative assessments will take place across the whole year;
- brief information of the format of the assessment i.e. coursework/exam/presentation;
- word count or duration as appropriate;
- when results can be expected.
Assessment schedules may be produced as a stand-alone document, as part of the Student Handbook or posted on iLearn if this is available for the module. Consideration should be given to whether formative assessment tasks are also documented on the assessment schedule.
Timescales for the production of summative instruments of assessment will be influenced by:
- the assessment tool;
- the marking criteria/grading criteria;
- model answers.
The timescales must ensure adequate time is allowed for module and, where appropriate, Course/Programme Management Team scrutiny of the assessment instrument and also for review and comment by the External Examiner.
The Module Leader, who will be a member of academic staff teaching on the course/programme, has responsibility for the development and sign-off of accurate instruments of summative assessment and related documentation. ORMS’ D1 Assessment Policy outlines the relevant responsibilities. Considerations that may be useful for producing the assessment instrument include:
- What worked well with regard to the assessment instrument?
- What needed to be modified?
- How well did the marking criteria work for the task?
- Has there been any modification to the module learning outcomes for the next academic year?
- Has the module content been changed for the coming academic year?
- What impact do these considerations have on the summative assessment instrument that was used last year?
- Has there been any feedback from the external examiner?
- Keep ‘as is’, modify or change completely?
- If implementing changes, does this have implications on the Module Descriptor and/or Module Performance Descriptor and what process does this need to go through for approval (if any) e.g. Academic Development Committee?
After considering these issues the Module Leader should produce the summative instrument of assessment(s), model answer(s) and grading criteria. Before being sent for external scrutiny, if appropriate, internal scrutiny should be performed by the module and or Course/Programme Management Team.
5. Internal Scrutiny of Summative Instruments of Assessment
This process is essential to ensure that the instrument(s) of assessment are unbiased, reflect the learning outcomes, are clear to students and in a standard format. It may be useful to use a proforma for this process (see the Checklist for Scrutiny of Summative Instruments of Assessment in paragraph 7).
This process is essential to ensure that the instrument(s) of assessment are unbiased, reflect the learning outcomes, are clear to students and in a standard format. It may be useful to use a proforma for this process (see E3b5: Checklist for Scrutiny of Summative Instruments of Assessment).
Key considerations are:
- Does the summative instrument of assessment ensure constructive alignment with the module learning outcomes?
- Are the questions/case study/situations used in tasks this year sufficiently different to those used last year?
- If not, is there a clear rationale why?
- Are the questions/tasks clearly and unambiguously written?
- Do the marking criteria map to the assessment instrument?
- Is the assessment instrument and marking criteria at an appropriate level?
- Are there any typing errors?
- Is the model answer appropriate and sufficiently detailed?
- Have you ensured that the answer(s) to the assessment instrument are not provided in the assessment information to be provided to students?
- Is the task achievable in the time/word limit?
- Are any important additional instructions clear, e.g. additional materials permitted in an examination?
Where the same module is undertaken by students on various courses/programmes internal scrutiny may be required by all teams involved. In such circumstances a shared scrutiny meeting may be beneficial so that all views can be gathered at one time.
Once the Course/Programme Management Team is satisfied with the instrument of assessment it should be sent to External Examiners for review. ORMS’ Assessment Policy indicates this must be undertaken for all instruments of assessment in award bearing stages. However, some External Examiners are willing to review instruments of assessment for all stages and this should be considered good practice and encouraged wherever possible.
6. Communication with External Examiners
There is a clear procedure regarding who should undertake this communication. ORMS’ D1 Assessment Policy indicates that:
- the scrutinised summative instrument of assessment and its model answer/grading scheme to be sent to the External Examiner at least one month before the date of the assessment;
- The checklist E3b5: Scrutiny of Summative Instruments of Assessment should be completed and accompany the assessment.
- comments made by External Examiner to be considered by the Module Leader for incorporation into the final drafts of summative instrument of assessment and its model answer/grading scheme;
- the External Examiner to be notified as to what action has been taken on his/her comments;
- the External Examiner to be informed, as appropriate, of any changes made to the summative instrument of assessment and its model answer/grading scheme after his/her approval;
The person responsible for this process should also be responsible for forwarding comments made by External Examiner(s) to the module coordinator for consideration for possible incorporation into the final drafts of summative instrument(s) of assessment and their model answer(s)/grading scheme(s).
Prior to signing off the final version of the instrument of assessment the Module Leader should ensure:
- There are no typographical or spelling errors.
- The answers are not included on the material to be given to students.
- All material to be given to students is accurate and complete.
6.1 Specific consideration for coursework
Have clear instructions about the following been included in the information provided to students:
- Word count and any associated penalties?
- Submission date, time and method?
- Referencing format to be used?
- When marks and feedback can be expected?
- Format of feedback to be expected?
6.2 Specific consideration for practical examinations
Have students been provided with clear information about:
- The time and place of their practical examination?
- Whether the examination will be video, or audio recorded and the purpose of the recording?
- When marks and feedback can be expected?
- What format the marking and feedback will take?
- If a safety fail is possible?
- Appropriate clothing for the examination e.g. uniform?
- Any equipment students are expected to bring with them?