Anonymous #50

Name (Individual/Organisation)

Anonymous #50

Responses

Q1. How could the purpose in the ARC Act be revised to reflect the current and future role of the ARC?

For example, should the ARC Act be amended to specify in legislation:
(a) the scope of research funding supported by the ARC
(b) the balance of Discovery and Linkage research programs
(c) the role of the ARC in actively shaping the research landscape in Australia
(d) any other functions?

If so, what scope, functions and role?

If not, please suggest alternative ways to clarify and define these functions.

It would be beneficial to specify the balance of Discovery and Linkage research programs in legislation, to prevent future arbitrary shifting of funds from Discovery by the Minister to other schemes, including the Linkage research programs, to serve political purposes of the day. Also, Linkage funding should be complemented by other effective policies that will incentivise innovation in Australian industries and should not be an on-going solution to this persistent problem. The public should only play a limited role in footing the R&D costs for profit-oriented industries. Researchers conduct the basic research and Australian industry should be capable of taking the research outcomes and converting them to new products and services.

Q2. Do you consider the current ARC governance model is adequate for the ARC to perform its functions?

If not, how could governance of the ARC be improved? For example, should the ARC Act be amended to incorporate a new governance model that establishes a Board on the model outlined in the consultation paper, or another model.

Please expand on your reasoning and/or provide alternative suggestions to enhance the governance, if you consider this to be important.

No, it is not adequate and it would be important to do an overhaul of the ARC governance structure. A board model as suggested in the consultation paper would be suitable for this purpose, with the ARC CEO nominated by the board and not directly appointed by the Minister. The selection of board members should be done comprehensively, perhaps in consultation with the wider research community if it can be done in a non-burdensome way, with a holistic assessment of their research track record/contributions and less emphasis on bibliometrics.

Q4. Should the ARC Act be amended to consolidate the pre-eminence or importance of peer review?

Please provide any specific suggestions you may have for amendment of the Act, and/or for non-legislative measures.

Yes and the desired expectations of the peer-review should be stated, that is, it should state clearly the principles to be adhered to, such as transparency and fairness. The current peer-review process for ARC grant applications has serious flaws that need urgent addressing. Please see my extensive response to Q7.

Q5. Please provide suggestions on how the ARC, researchers and universities can better preserve and strengthen the social licence for public funding of research?

There is too much emphasis on the unquantifiable notion of “impact” and not enough acknowledgement of the intrinsic value of basic/curiosity-driven research on its own to understand the world around us, as knowledge base upon which future technologies can be developed, and for Australia to build a reputation as a “smart country” rather just a lucky country. There are always inherent risks with a particular research line, in that it may not pan out as expected despite the promise. Applicants have always been required to explain the value and potential benefits of their research and this provision is already inline with the expectation of the social licence.

Q6. What elements of ARC processes or practices create administrative burdens and/or duplication of effort for researchers, research offices and research partners?

The ARC grant applications are more onerous than they should be and very time consuming, many sections should be considerably simplified so that applications do not need to run up to possibly more than 100 pages. Considering the low success rate, this amounts to many hours wasted preparing the applications for the majority of unsuccessful ones. Overseas PIs who will be named on an application but do not receive direct monetary funding should not be required to complete lengthy sections of the application.

Q7. What improvements could be made:

(a) to ARC processes to promote excellence, improve agility, and better facilitate globally collaborative research and partnerships while maintaining rigour, excellence and peer review at an international standard?

(b) to the ARC Act to give effect to these process improvements, or do you suggest other means?

Please include examples of success or best practice from other countries or communities if you have direct experience of these.

Following from my response to Q4, the peer-review process lacks transparency, fairness and useful feedback for applicants, considering the amount of effort that goes into submitting an application.

Applicants are only given the comments of the detailed assessors and final ranking bands of their application, without knowing the scores that their application received from the detailed assessors and the carriages. Both of these scores should be provided to applicants not least for the sake of transparency. In the case of carriages, applicants have absolutely no information about their assessment of the application. Often the final ranking bands are contrary to the detailed assessors’ comments. Carriages should be able to moderate the ranking process in a transparent way. Arbitrary looking rankings reflects poorly on the carriages, especially since they do not read the application in detail. If there are substantial discrepancies between detailed assessors scores and the carriages’ scores then there should be some explanation provided to applicants to justify the discrepancy.

For comparison, in the NSF in the US, panels that discuss applications are drawn from a diverse set of researchers (rather than a specially selected “elite” panel from the College of Experts as with the ARC) with a program manager providing oversight in the panel discussions. Disagreements of the panel with the grant peer-reviewer reports are recorded for the decision making on an application, and applicants can seek feedback on their application from the program manager. In the ERC grants in Europe, applicants are also provided the scores assigned to their application by the assessors and the panel provides comments on the decision made on an application.

Also, there are real concerns that the current process is leading to concentration of research funding on a small number of select researchers, creating inequity in research funding, leaving many highly capable and talented early and mid-career researchers behind as they are unable to access the support needed to continue their research. This is creating a system that ends up “funding people/careers” rather than ideas, which is clearly contrary to the stated purpose of the ARC in the ARC Act, which is not to fund whole careers. For basic research, many directions must be explored as it is not known at the outset which of them may lead to future breakthroughs.

These problems are likely due to two issues: bias in the peer-review process (including the “Matthew effect”) and a growing culture of reliance on bibliometrics (publications counts, citation counts, journal impact factor). These issues should be acknowledged and steps taken to address them. Bibliometrics are especially problematic as the emphasis is placed on quantity over quality. There are also the now numerous widely-publicised concerns, including the trend of increasing number of authors named on papers, which is not normalised against in the citation counting process, gift authorships and manipulation of citations (by researchers and journals). Such a system only benefits groups of socially well-connected researchers many of whom are able to produce voluminous papers just for the sake of publishing and increasing their bibliometric counts, even if many of these publications are incremental in nature or minor variations of the same themes (yet still able to be published in reputable journals). In this scenario, high publication rates lead to more grants that then begets more incremental publications and more grants, perpetuating an endless cycle with no clear benefits other than propagating the status and careers of those who engage in it. The investigators’ contributions, experience and expertise with respect to the proposed project should not be overshadowed by metrics. Peer-review should adopt and enforce the principles outlined in declarations for research assessment such as DORA and the Leiden Manifesto. As an example, the ANR in France advises a more holistic evaluation of researchers and not to judge by the venue (journal) in which their publications appear.

A healthy funding success rate should be around 30-40% (according to Nobel laureate Professor Peter Doherty in an article published in the Sydney Morning Herald). To reduce bias and the Matthew effect, especially acute in the current regime of historically low funding levels, one approach worthy of being trialled is the introduction of properly designed randomisation in the selection process. A simplified peer-review process (still required to be transparent and provide useful feedback) could be used to determine all applications that are of sufficient quality for funding and can be entered into the selection lottery. This idea has been around for a while now, and there are peer-reviewed papers addressing this approach. Elements of randomisation are already being trialled or incorporated into their grant selection processes by some funding agencies abroad such as the SNSF in Switzerland, the British Academy in the UK and the Novo Nordisk Foundation in Denmark.

Q9. With respect to the ARC’s capability to evaluate research excellence and impact:

(a) How can the ARC best use its expertise and capability in evaluating the outcomes and benefits of research to demonstrate the ongoing value and excellence of Australian research in different disciplines and/or in response to perceived problems?

(b) What elements would be important so that such a capability could inform potential collaborators and end-users, share best practice, and identify national gaps and opportunities?

(c) Would a data-driven methodology assist in fulfilling this purpose?

A data-driven approach would not assist in meeting this purpose. Bibliometrics in most instances do not necessarily correlate with “impact”, “quality” and “value”. Perhaps the best approach would be to do a follow up survey to grant recipients 10 to 15 years after the conclusion of their grants to solicit verifiable information about the form of impacts from the research beyond just citations.

Q10. Having regard to the Review’s Terms of Reference, the ARC Act itself, the function, structure and operation of the ARC, and the current and potential role of the ARC in fostering excellent Australian research of global significance, do you have any other comments or suggestions?

There are many issues with research funding in Australia and the ARC. This has created a declining trust of the ARC in the wider Australian research community and reduced the participation in research. This review is important for the future of Australian research and I hope that all thoughts and ideas expressed in the submissions will be given due consideration

Submission received

14 December 2022

Publishing statement

Yes, I would like my submission to be published but my and/or the organisation's details kept anonymous. Your submission will need to meet government accessibility requirements.