REF FAQs

Information for researchers

Expand All

The REF is not intended to be used as an exercise to assess individuals. However, it is to the advantage of all researchers working here that Oxford’s REF submission is of the highest quality. The QR funding allocated as a direct result of the submission is key to ensuring research costs can be met, and the reputational boost of excellent REF results can help in winning future research grants and contracts.

In terms of showcasing an individual’s research, eligibility for submission to REF will demonstrate the staff member’s independence. It will be possible for an individual to calculate how many outputs selected for submission they were a co-author on, with potential to demonstrate a record of research quality for both submitted and non-eligible staff.

Results of the REF 2021 exercise will be published in Spring 2022, including a list of submitted outputs (without author details) and quality profile scores for the unit.

It may not be immediately evident which and how many outputs were attributed to which member of staff. Attribution of outputs to staff members will be done in order to maximise the quality profile of the unit, rather than the individual, and so a staff member’s ‘best’ output may not necessarily be attributed to them.

Where there are multiple authors on an output it will not be possible to work out which outputs were attributed to each staff member.

Staff members will be asked to nominate at least one output, and up to a maximum number outputs, typically 5 or more.

Depending on the level of co-authorship it may be appropriate to request that staff members nominate more outputs to be reviewed for inclusion in the submitted output pool.

If a staff member has circumstances which mean they are not able to nominate the maximum number of outputs requested they can nominate a reduced number. Eligible staff will be able to have a maximum of 5 outputs attributed to them and must have a minimum of one output attributed.

Outputs are decoupled from staff and will be reviewed for quality to allow us to attribute them to staff in order to maximise the overall quality profile of the unit.

Staff employed on research-only contracts must be independent researchers to meet the funding bodies’ definition of eligible staff.

Most PDRAs at Oxford meet the REF definition of research assistants, who are defined as academic staff on research-only contracts, who are employed to carry out another individual’s research programme rather than as an independent researcher in their own right and whose employment is usually funded from research grants or contracts. Research assistants are not eligible unless, exceptionally, they meet the definition of an independent researcher.

The determination of eligibility will be done though evaluation against the REF criteria using the indicators set out in the University’s code of practice. All PDRAs who are eligible will be submitted.

Staff eligibility

Expand All

Eligibility will be determined through a formal process run by the academic division with responsibility for the submission to the relevant main panel (ie Medical Sciences Division for Panel A, MPLS Division for Panel B, Social Sciences Division for Panel C, and Humanities Division for Panel D), and overseen by REF Project Board.

The principles for the code of practice outline the indicators that will be used to help make decisions about an individual researcher’s eligibility, and how the indicators will be applied. The funding bodies have also published a (non-exhaustive) list of fellowships the funders have confirmed demonstrate research independence.

Talk to your divisional REF project manager in the first instance.

REF contacts

The University is planning a submission which will include staff with college-only appointments as long as they meet the definitions of eligibility set out in the funders’ guidance and the University’s code of practice. The same criteria and indicators will apply to departmental and college staff. This is subject to the approval of the code of practice by the funding bodies.

The code of practice will cover the process for identifying eligible staff, including college staff. This is currently being developed by REF Project Board, on which Conference of Colleges is represented.

Eligibility will be determined for both college and departmental staff through a formal process run by the academic division with responsibility for the submission to the relevant main panel (ie Medical Sciences Division for Panel A, MPLS Division for Panel B, Social Sciences Division for Panel C, and Humanities Division for Panel D), and overseen by REF Project Board. We expect to ask college senior tutors to nominate college staff to be considered in spring 2019. Eligibility decisions will be communicated to the staff affected in due course.

No, the total FTE of submitted staff is made of those employed at the census date (31 July 2020). It is this total FTE of submitted staff that determines the volume for funding calculations, and the total number of outputs and case studies required.

For the first time in REF 2021, the outputs of former staff can be included as part of the output pool, as long as the staff to whom the output is attributed were eligible when the output first became publicly available.

Possible indicators for substantive connection given by the funding bodies include both connection to the unit’s research environment, and connection to the institution.

It is recognised that submission to the unit of assessment may not map neatly onto departmental or other administrative structures within higher education institutions. The contextualised indicators for Oxford are given in the University’s code of practice.

Outputs

Expand All

Output eligibility depends on the date when an output first became publicly available. To be eligible this must be between 1 January 2014 and 31 December 2020. The outputs attributed to submitted staff, employed by Oxford on 31 July 2020, can include outputs generated prior to their employment at the University, as long as they first became publicly available within the REF period.

Date of acceptance is only relevant to a small number of output types. It is a key date in determining compliance with the open access policy for journal articles and conference contributions.

The guidance addresses common issues around determining the timing of publication. In particular, when an output is first published in its final form during the REF 2021 publication period, but was ‘pre-published’ before the period (eg as a pre-print), it is eligible provided the ‘pre-published’ version was not submitted to REF 2014 by the same institution.

For REF 2021 outputs of staff employed on the census date will be portable. Therefore, an eligible member of staff who has moved institution can have their outputs submitted to the REF by both their previous institution (if they were employed on an eligible contract during the REF period) and their current institution if they are eligible on the census date.

If the staff member has left Oxford by the census date, their outputs may still be eligible to be submitted by Oxford if they were employed on an eligible contract at the time that the output was first made publicly available.

If a staff member produces an output at Oxford and deposits it in ORA, then moves to another institution by the census date, both Oxford and the new institution may submit the output to the REF. By ‘acting on acceptance’ and depositing an author accepted manuscript (AAM) in ORA, researchers are ensuring their research is freely available to all. Depositing an AAM in ORA will ensure ongoing open access compliance. The University is committed to managing the long-term preservation, continued access, and storage of material in ORA. Deposits will not be removed when a staff member leaves the University.

The total FTE of submitted staff determines the total number of outputs (2.5 per staff FTE) and case studies required. A part-time submitted staff member must still have a minimum of one, and up to a maximum of five, outputs attributed to them. Output selection will be based on quality, with no fixed relationship to FTE.

The REF guidance allows for the submission of a large range of research outputs, beyond the more traditional journal articles, monographs and books. The University’s procedures will follow the funding bodies’ principle of equity in assessment for all types of research output.

The funding bodies have made it clear that no output type will be preferred over any others. For non-traditional output types, for example, for an exhibition, the output submission would likely include the exhibition catalogue and may include digital or even physical material, and supporting information such as a statement describing the research process, content and contribution of the researcher. The material submitted should provide the panel with coherent evidence of the research process, the research insights and the dissemination.

The decision on which outputs to submit rests with the academic unit of assessment coordinator. Staff will be encouraged to consider all types of output when they nominate outputs, and to create records for them on Symplectic Elements.

The decision about which panel the research output is submitted to would be made in order to maximise the quality profile of Oxford’s submission.

If an output meets the definition of interdisciplinary research, institutions are required to flag it to the unit of assessment (UoA) sub-panel so that they can make use of additional criteria in the assessment. Interdisciplinary research will be neither advantaged nor disadvantaged in the assessment.

If, in the sub-panel’s opinion, the sub-panel and its assessors do not have the required expertise to assess parts of the submission (eg specific outputs), they can be cross-referred to other sub-panels for advice. The sub-panel for the UoA to which the output was originally submitted remains responsible for recommending the quality profile for the work submitted in its UoA.

Outputs of extended scale and scope can be double-weighted (count as two). This is most applicable in panel C and D, and most commonly used for long-form outputs such as books and monographs. The decision to request double-weighting will rest with the academic unit of assessment coordinator.

Impact case studies

Expand All

Impact case studies must be underpinned by research undertaken by staff within the remit of the submitting unit of assessment between 1 January 2000 and 31 December 2020.

This research will be evidenced by outputs referenced in the case study. This underpinning research as a whole must be of a quality at least equivalent to two stars: ‘quality that is recognised internationally in terms of originality, significant and rigour’. 'Underpinned by' means that the research must have made a distinct and material contribution to the impact claimed.

For REF 2021, impact case studies must demonstrate impact in the period from 1 August 2013 to 31 July 2020, from research undertaken by staff in the submitting unit from 1 January 2000 to 31 December 2020. 

The REF guidance defines a continued case study if it was based on the same research as one submitted to REF 2014, and the impact claims are of the same type.

All panels have agreed to assess continued case studies on an equal basis to new case studies. Panels B, C and D do not wish to receive information on which case studies are continued.

Underpinning research can be produced beyond the impact period. For the purposes of the REF the period for impact is 1 August 2013 to 31 July 2020. However, the underpinning research can be from a much longer period, beginning 1 January 2000 and extending beyond the impact period to 31 December 2020.

This may depend on the individual circumstances.

If the staff member was employed by the University at the time when the research was undertaken, the research should still eligible to be used as underpinning research.

Impact is not portable, however, so if the researcher was already retired at the time of undertaking the research, it may not be eligible as underpinning research.

If the research was undertaken by more than one researcher, the research may still be eligible.

It is worth checking with the REF team and divisional REF leads if you have concerns about eligibility of underpinning research.

There are many provisions in place for impact case studies that contain sensitive information.

Case studies can be redacted before publication or can be unpublished. It will also be possible to limit who can review the case study, eg for reasons of competition or even national security.

For case studies that require security clearance we are required to notify the funding bodies of our intention to submit the case study to allow time for recruitment of panel members with appropriate security clearance.

If you think your case study contains confidential information please contact your divisional REF manager for advice.

Impact case studies assessment is a peer review process. The case studies will be reviewed by academics and panel member reports from REF 2014 suggested that though clarity in the case study narrative was welcomed, the lack of academic language and emphasis in case studies that had been written by specialist writers hindered the panel’s ability to judge against the criteria.

Wherever possible, we recommend that case studies be developed with the researcher(s) responsible for the underpinning research, to ensure that the academic voice comes across.

Open access

Expand All

Appropriate embargo periods are a key element of meeting the access requirements of the open access policy for outputs in scope of this policy. If the output is compliant with the open access policy, or has a verified exception, being under embargo at the time of submission will not make a difference to the assessment of the output.

The funders’ REF team will attempt to source all submitted journal articles in electronic format directly from the publishers. The publisher’s version of record, rather than the author accepted manuscript held in ORA, will therefore normally be the version assessed.

REF process and results

Expand All

The decoupling of staff and outputs in REF 2021 is intended to provide increased flexibility to institutions in building the portfolio of outputs for submission. There are many reasons why an excellent researcher may have fewer or more outputs attributable to them in an assessment period. It is therefore not expected that all staff members would be returned with the same number of outputs attributed to them in the submission.

Where there are multiple authors, sub-panels undertaking the assessment will not know the staff member which the output is attributed to and will not know which institution has submitted the output where there are collaborating institutions.

Research income has to meet the HESA definition of research income from table 5 of the finance return. This has to be split by unit of assessment in the submission but must total the figure submitted to HESA.

This covers income in respect of externally funded research where the research project scope has been agreed with the funder. The research specified should be carried out by the University, and should conform to the conventions of the Frascati definition of research. The income returned should be that for which directly related expenditure has been incurred and should be stated at the full value, including any recovery of indirect costs. Where a research grant or contract is made for a number of different purposes including research (for example research, training and clinical work) only that portion of the grant or contract against which research has been conducted should be returned as research income.

The REF 2014 results are available on the REF website. The scores are available as quality profiles for each unit of assessment submission. The individual scores for outputs and impact case studies are not published.

List of site pages