3. Dr A complains UKRI refused to invalidate one of the three peer reviews of her research proposal.
4. She says this meant she has been unable to carry out her research.
5. Dr A would like UKRI to accept her proposal for funding and ‘invalidate’ the review that gave the low score.
6. In May 2021, Dr A applied to UKRI for funding for her research proposal. UKRI reviewed the proposal, and two out of three reviews commissioned (B and C) gave it a score of 3 (Satisfactory, on a scale from 1 (Poor) to 6 (Outstanding)). Dr A complained shortly afterwards that B and C were not fit for purpose. UKRI agreed with respect to B and replaced it with a new review.
7. The new review B gave the proposal a score of 5 (Excellent). UKRI found a suitably qualified reviewer had completed review C and had done so according to relevant policy. For this reason, it refused to replace this review.
8. Dr A then complained to UKRI about review C in December 2021, this time on slightly different grounds. UKRI investigated her concerns again. On 24 January 2022, it explained to her in writing it still considered review C was completed according to the relevant standards. It added it would not uphold her complaint or commission a new review.
11. Before we decide if we should investigate a complaint in more detail, we look at whether there are signs the organisation concerned has got something wrong. We do this by comparing what should have happened with what did happen. If what happened fell far short of what should have happened, we call this a failing.
12. Having done this, based on the evidence available, we believe UKRI was right not to invalidate the review or recommend her proposal for funding based solely on the two other favourable reviews.
13. Dr A complains review C was faulty because it:
1. discriminated against her as an ‘early career-stage’ applicant 2. was at odds with UKRI’s criteria for assessing applications 3. inaccurately assessed the level of support the project partner provided, and disrespected them 4. did not refer to key pieces of information.
14. Our Principles state, when making decisions, public bodies should follow their own policy and procedural guidance, take account of all relevant considerations, ignore irrelevant ones and balance the evidence appropriately.
15. In its guidance ‘Reasons for Return of Reviews’, 2021, UKRI states the following criteria point to a faulty or unusable review: • the comments and grades a reviewer gives throughout their review contradict the overall grade they give • a reviewer declares a low confidence level in their ability to provide a review • the tone and language a reviewer uses is confrontational or emotive • a reviewer identifies themselves within their comments (inadvertently or overtly) • the comments made are speculative or do not relate to the content or context of the proposal • a reviewer’s comments are too brief • a reviewer has not addressed the areas expected for effective peer review, and there is little or no critical examination of the proposal • a reviewer makes specific comments regarding Arts and Humanities Research Council processes and policies.
16. For us to recommend UKRI ‘invalidate’ the review, we would have to see clear evidence the review meets one or more of the above criteria.
17. In what follows, we consider in depth each of the specific points Dr A raised.
18. Dr A says the reviewer suggested she submit her research at a later stage. She says this is not in keeping with UKRI policy, because the scheme to which she submitted the proposal is specifically for early career applications. She notes UKRI does not allow resubmissions, and so it was wrong for the reviewer to suggest this. She adds the fact he did so implies a lack of familiarity with the principles of the scheme.
19. UKRI responded to this specific allegation and explained the reviewer was suggesting she submit her proposal to a different scheme, and not resubmitting to the same one.
20. We recognise this suggestion confused and frustrated Dr A. However, we do not agree this necessarily suggests the reviewer was unfamiliar with the relevant criteria for assessing her application. We think it at least equally plausible the reviewer simply thought her proposal was not developed enough to merit a higher score. This was probably why he suggested she work on and submit the proposal to a different scheme later on.
21. Even if we accept the reviewer misunderstood whether Dr A could resubmit, there is nothing in the relevant guidance to suggest UKRI should have invalidated the review as a result. It appears UKRI refused to do so because the relevant guidance does not count misunderstanding whether the proposal could be resubmitted as strong enough grounds to render the review unusable. UKRI therefore made its decision according to our Principles.
22. Dr A says the reviewer’s evident lack of familiarity with UKRI criteria for assessing her application undermined the review.
23. Dr A says her proposal should have been considered against the criteria of ‘Interdisciplinarity’, ‘Engagement’ and ‘Development’. She says a key criterion for a successful application is evidencing the research will move into new fields through clinical engagement supported by an appropriate mentor and collaborators in the field of medical humanities.
24. Dr A complains it was wrong for the reviewer to describe her approach as a ‘rather shaky interdisciplinary methodology and framework’. She believes these critical comments amounted to discrimination as they limited her from moving into new fields, one of the main supposed benefits of a successful application.
25. We have seen review C, and it is clear to us the reviewer was not convinced Dr A’s proposal should be prioritised for funding. The fact Dr A disagrees with the criticisms in the review does not mean the review is faulty. We can see the reviewer did consider how Dr A was moving into new fields, and engaging with collaborators and partners. On page 3, under ‘People’, he even gave her a score of ‘good’. He also commented favourably on the networks she is building in France and the UK. It therefore appears the reviewer has directly considered the criteria above.
26. Despite the above, the reviewer found the proposal ‘rather shaky’. He explicitly considered the degree of ‘Interdisciplinarity’ it contained and concluded this area was not developed enough for him to recommend the proposal for funding. This is a matter of academic opinion, and there is no standard available to us against which we can test this. There is also nothing in the relevant guidance to suggest a difference in academic opinion, however passionately held, points to an unusable or faulty review. UKRI therefore appears right to have refused to write off the review on this basis.
27. Dr A complains the reviewer inaccurately assessed the level of support the project partner provided, and disrespected them. She says the letter of support her prospective project partner sent was incredibly detailed and evidenced a high degree of support. She says the fact the author of review C described the support as ‘impressionistic’, and the other two reviewers disagreed, shows the author of review C did not properly consider the letter.
28. This allegation is much the same as the one addressed above. There is no standard available for us to determine whether or not the reviewer was correct to describe the project support as ‘impressionistic’. This is a difference of academic opinion. It does not seem reasonable to infer the review went wrong due solely to a difference of opinion. The evidence clearly shows the author of review C did consider the project support but reached a different conclusion on the proposal to that reached by the other reviewers.
29. We also do not share Dr A’s belief that in expressing this academic opinion, the reviewer was disrespecting the partner. As above, there is nothing in UKRI’s guidance to suggest the review is faulty or unusable just because Dr A contests the reviewer’s opinion. On this basis, we can see no signs UKRI failed to follow the relevant guidance (and our Principles) when it refused to invalidate the review.
30. Dr A also complains the review form was not detailed enough and did not reference key pieces of evidence at all. She believes this suggests the reviewer did not actually consider all the relevant evidence. Specifically, she says the review made no mention of the project collaborators, the early career mentor, the doctors on the advisory board at the project partner’s institution, or the data management plan.
31. UKRI guidance states a review can be considered unusable if the comments are too brief or the reviewer does not address the areas expected of effective peer review, and there is little or no critical examination of the proposal. Any significant omissions in the review might suggest UKRI did not follow relevant guidance when it refused to invalidate it.
32. We can see the review does not reference specific project collaborators, the early career mentor, or the doctors on the advisory board at the project partner’s institution. It does refer to the data management plan when it says it is clear the proposal includes plans to ‘properly anonymise data’. The question is whether we can reasonably say these omissions show the review falls short in the way outlined in UKRI’s guidance.
33. We recognise Dr A feels very strongly the reviewer is mistaken in his assessment of her proposal. We also accept her frustration he gave a different score to the other two reviewers. Yet we do not consider we could reasonably expect the reviewer to have made explicit reference to every piece of information in the proposal. Clearly, in such reviews the reviewer will reference the aspects they find most relevant to their decision.
34. Also, although the reviewer did not specifically reference individual collaborators, he did comment more generally on the collaborative aspects of the project. It does not seem proportionate to expect him to have commented exhaustively and explicitly on every piece of information he received. Therefore, we do not consider it reasonable to say the review was too brief or did not address the expected areas. On this basis, as above, we consider UKRI’s decision not to invalidate the review followed its own guidance and our Principles.
35. For this reason, we will take no further action. We recognise this will not be the outcome Dr A hoped for, and her strength of feeling about this is unlikely to change. We hope she is assured we only reached this conclusion after careful consideration.
1. The Parliamentary and Health Service Ombudsman has carefully considered Dr A’s complaint about UK Research and Innovation (UKRI). We recognise she feels very strongly UKRI should invalidate a less favourable review of her research proposal and accept it based on two others that gave it a higher score.
2. Based on the evidence available, we consider UKRI’s refusal to do so followed relevant standards and guidance. We have therefore decided not to consider Dr A’s complaint any further.