Oxford’s Memorandum for Examiners for the Degree of Doctor of Philosophy (GSO5) sets out the examiners' responsibilities and includes guidance regarding the timetable for examinations, regulations governing viva voce examinations, and the recommended outcomes available to examiners. As an examiner you will receive a copy of this document by email when you are invited to serve and again when you are notified that the hard copy of the thesis has been despatched. You will also receive, attached to the despatch email, a guidance document about fees and expenses. With the exception of the thesis itself, of course, hard copies of documents are available on request only.
Copies of the thesis may not be dispatched until the Research Degrees Team has received your acceptance of the invitation and confirmation of the address to which the thesis should be sent.
In addition, Oxford's internal examiners are expected to undertake a number of vital administrative responsibilities. See DPhil Examination arrangements.
The internal examiner must:
- Advertise details of the viva and notify the Research Degrees Team. The simplest way of doing this is by emailing a completed GSO9 form (supplied in the despatch email) to the University Gazette with a copy to the Research Degrees Team. The notice should reach the Gazette Office not later than 12 noon on Wednesday of the week before publication.
- Book a room for the viva.
- Communicate the viva arrangements to the candidate.
- Arrange hospitality for the external examiner.
Advice from experienced examiners to colleagues about to examine their first doctoral dissertation suggests the following:
- read as many examiners' reports as possible (being on a Higher Degrees committee is helpful here);
- talk with experienced examiners to find out how they approach the examination process;
- invite an experienced colleague to work through parts of the first thesis that you have been asked to examine to compare your views (while most universities require confidentiality, most would also agree to a request from a novice examiner to discuss particular issues, in confidence, with a colleague);
- contact the Examination Schools Research Degrees Team if you have any concerns about procedures and outcomes.
- it may be useful to review how others understand originality, significance and quality. See Lovitts on Originality, Lovitts on Significance and Lovitts on Quality.
In some fields a monograph thesis is more common and in others a series of published or publishable papers is the norm.
Paper-based theses can take a range of forms: a) papers alone, b) hybrid theses with papers inserted as chapters, and c) theses with publications appended. Generally, experienced examiners feel able to apply the institutional criteria (usually defined in relation to a monograph thesis), but they report defining their own standards for the following:
- the coherence of the thesis,
- the intellectual input of the candidate in any multi-authored publication, and
- the ranking of the journal in which the paper is published.
In the case of a monograph thesis, many experienced examiners approach the task in the following manner:
- Read the Abstract, Introduction and Conclusion to gauge the scope of the work and see whether what the candidate says is going to be done is actually done.
- Look at the references to see what sources have been used and whether there is a need to follow up on any of them.
- Read from cover to cover, taking detailed notes.
- Go back over the thesis to check whether questions noted while reading have been answered or whether criticisms are justified.
An important feature of the thesis is the use of the literature. Research suggests that examiners use the following criteria to assess this.
- Coverage: relevant material.
- Working understanding: a familiarity that demonstrates a reasonable level of analysis and synthesis.
- Critical appraisal: weighing up a body of knowledge.
- Connection to findings: demonstrating a disciplinary perspective.
Coverage alone is not sufficient to achieve a pass. There also needs to be some use of the literature that signifies a working understanding and some critical appraisal. Use of the literature that is considered outstanding includes extensive critical appraisal and excellent integration of findings.
Questions to ask yourself, or to discuss with a colleague, about examining the written thesis (From Carter, 2008, pp366-367)
- When you are asked to examine a thesis, do you feel dread at the work or interest? What either puts you off or encourages you within a few pages of reading?
- How short would be too short? If a thesis was original, competent and valuable, how short might it be?
- If an idea is brilliant, how important is the standard of presentation?
- To what extent do you check the literature review material? Would you look up works that you do not know?
- How do you cope with personal bias, if, say, your own work or a friend’s work is not listed in the bibliography?
- How do you cope with personal bias if you disagree with the approach, the paradigm or the conclusion of the thesis?
- What about when a thesis is found to be radically unconventional in its approach or organisation? How can you ensure that you examine it fairly?
- Can you suggest ways that anyone whose thesis departs from the conventional (by crossing disciplines for example) can ensure that the marker finds it easy to pass their work?
- Do you have any pet dislikes or preferences that you suspect might be common in doctoral examiners?
If you would like to assess your knowledge of the Oxford regulations, try the quiz in the following section. Click on each question to check the answer.
1. Suggested by the supervisor, in consultation with the student;
2. Generally sounded out informally in advance;
3. Expected to notify the university if there is any conflict of interest;
4. Appointed formally by the research degrees team on behalf of the Graduate Studies Committee or Board; or
5. All of the above.
The correct answer is 5 - All of the above.
The choice of examiners is an important part of the role of the supervisor. Experienced examiners generally take into account topic/methodology fit; understanding of the system from which the dissertation is coming; access to useful networks for the student; experience in supervising and examining; availability; high but fair standards; intellectual courtesy and generosity; reliability. It is essential that examinations be set up through the proper, university channels.
The correct answer is 2 - Within 3 months of receiving the thesis.
The process should be completed as soon as possible and, at least, within 3 months of receiving the thesis.
1. Should not communicate about the candidate or the thesis to the examiners;
2. May communicate about difficulties with source material or publication of material too late to be taken into account in the thesis;
3. May communicate about factual matters which are directly relevant to the exam and which examiners may legitimately take into account;
4. May communicate about illnesses or personal circumstances relevant to the examination;
5. Both 2 and 3; or
6. Both 3 and 4.
The correct answer is 5 - Both 2 and 3.
According to the University of Oxford Examinations Schools’ Notice of Appointment of Examiners letter, the supervisor may communicate directly with examiners only on "matters of fact which are of direct relevance and assistance in connection with the examination of candidate and which the examiners may legitimately take into account in conduct of the examination." Several illustrative examples are included, including difficulties with source materials and publication of materials affecting the content of the thesis. Issues related to merit of the thesis are expressly forbidden. Issues related to illness or personal circumstances are also forbidden; those should be "forwarded to the Proctors, who will pass on to examiners as deemed necessary."
Much of the research in this area is based on monograph-type theses. Examiners report that they undertake the examination of theses for several reasons including, firstly, a sense of duty regarding:
- maintaining standards within the discipline,
- the belief that one of the roles of an academic is to examine theses,
- a duty to one's students, or the 'quid pro quo' concept: "It's...a reciprocal obligation from having one's own students examined" and "I have eight students at the moment which means I need 16 examiners soon...so I need to reciprocate" (Mullins and Kiley, 2002).
Secondly, they examine because of the excitement and interest involved, particularly where there is the opportunity to read at a level of detail not included in the examiners' day-to-day professional reading. Research notes that many examiners spend a great deal of time examining a dissertation, and that they take this role very seriously.
Examiners tend to place considerable emphasis on use of the literature in making a decision about the quality of the thesis. Criteria used include:
- coverage (relevant materials);
- working understanding (a reasonable level of analysis and synthesis);
- critical appraisal (weighing up the body of knowledge); and
- connection to findings (demonstrating a disciplinary perspective).
Use of the literature that signifies a working understanding and some critical appraisal is necessary for a pass. Use of the literature that is considered outstanding includes extensive critical appraisal and excellent integration of findings.
While examiners are asked to make a recommendation on the dissertation, the research suggests that, other than inexperienced examiners, most expect that a substantial percentage of dissertations will be in the 'middle' range and with others in the 'pedestrian' range. As Tinkler and Jackson (2004) say of one examiner's report "It was clear this was a weak thesis, the question was, how weak can it be and still pass?" (p119)
What we know of the evaluation of paper-based theses is the following. Although most examiners use similar criteria for assessing these theses as they would for monograph theses, some also use their own criteria, for instance: student input and the role of co-authors; the quality of publication venue, with some even considering the time taken between submission and acceptance; the maintenance of coherence throughout the thesis, especially since there are different ways to demonstrate coherence.
The above text was based on the following research:
Carter, Susan. (2008) Examining the doctoral thesis: a discussion. Innovations in Education and Teaching International, 45(4), 365-374.
Holbrook, A., Bourke, S., Fairbairn, H. & Lovat, T. (2007) Examiner comment on the literature review in PhD theses. Studies in Higher Education, 32(3), 337-356.
Kiley, M. and Mullins, G. (2004) Examining the examiners: How inexperienced examiners approach the assessment of research theses. International Journal of Educational Research, 41(2), 121-135.
Mullins, G., & Kiley, M. (2002). 'It's a PhD, not a Nobel Prize': How experienced examiners assess research theses. Studies in Higher Education, 27(4), 369-386.
Sharmini, S., Spronken-Smith, R., Golding, C., & Harland, T. (2014). Assessing the doctoral thesis when it includes published work. Assessment and Evaluation in Higher Education (published online 24 February 2014).
Tinkler, P., & Jackson, C. (2004). The Doctoral Examination Process: A handbook for students, examiners and supervisors. Buckingham: SRHE and Open University Press.