Anonymous #01

Name (Individual/Organisation)

Anonymous #01

Responses

Q3. How could the Act be improved to ensure academic and research expertise is obtained and maintained to support the ARC?

How could this be done without the Act becoming overly prescriptive?

The ARC Act should be amended to ensure that the highest quality research is being funded in Australia and that the review process is fair and transparent.

The Act should call a periodic independent review of the ARC protocols to ensure that they are indeed operating in manner that channels funding to the highest quality research. The key is the review is conducted by an independent team that is familiar with other government grant-funding structures like the National Science Foundation (NSF), National Institutes of Health (NIH), ESRC, Swiss NSF, etc. The team should include academics from around the world. The review should conclude with a report that identifies problems in the governance and protocols and suggestions improvements and solutions. This report should be made public and changes should be made to improve the system based on the suggestions in the report.

Q4. Should the ARC Act be amended to consolidate the pre-eminence or importance of peer review?

Please provide any specific suggestions you may have for amendment of the Act, and/or for non-legislative measures.

The system of peer review ensures that the highest quality research receives funding. However, the quality and diversity of the reviewer pool is instrumental in ensuring this is achieved. If the reviewer pool is narrow and lacks expertise, the result will be in-group of academics that control the COE, the reviewer pool, and the awards in a way that is not transparent or meritocratic but smack of favouritism. Transparency of the process is also important.

There are apparent problems with the current system. For example, at the current moment, only previous ARC recipients serve as reviewers/assessors. While this need not necessarily be the case because non-recipients can sign up to serve as reviewers, the experience in my field from a number of people who have volunteered to review have never be invited, despite numerous submissions in their subfield.

People are aware of instances where people with obvious conflicts of interest were invited to review on a proposal. For example, the reviewer had submitted a proposal in the same call and same subfield as the proposal they were being asked to review. Upon notifying the ARC of the obvious conflict of interest, the ARC asked the reviewer to proceed with the review nonetheless!

Also, instead of asking for reviews from experts in the field of the proposal, the ARC continues to (by unnecessarily limiting the reviewer pool to those who have had funding) get non-expert, low quality reviews from individuals well outside the field of study for the proposal.

Finally, an academic submitting a grant only sees assess comments but not the associated numerical score (which are critical for the grant). The numerical scores should be released.

Q5. Please provide suggestions on how the ARC, researchers and universities can better preserve and strengthen the social licence for public funding of research?

The best way to preserve/strengthen the social licence for public funding of research is to make the process transparent and meritocratic. Fund high quality, impactful research. Don't operate like an all-boys club.

Q7. What improvements could be made:

(a) to ARC processes to promote excellence, improve agility, and better facilitate globally collaborative research and partnerships while maintaining rigour, excellence and peer review at an international standard?

(b) to the ARC Act to give effect to these process improvements, or do you suggest other means?

Please include examples of success or best practice from other countries or communities if you have direct experience of these.

My previous answers have addressed these questions.

Q8. With respect to ERA and EI:

(a) Do you believe there is a need for a highly rigorous, retrospective excellence and impact assessment exercise, particularly in the absence of a link to funding?

(b) What other evaluation measures or approaches (e.g. data driven approaches) could be deployed to inform research standards and future academic capability that are relevant to all disciplines, without increasing the administrative burden?

(c) Should the ARC Act be amended to reference a research quality, engagement and impact assessment function, however conducted?

(d) If so, should that reference include the function of developing new methods in research assessment and keeping up with best practice and global insights?

a. Not necessarily.
b. I'm not sure, but it would be worth seeing what other national funding bodies might be using/developing.
c. The ARC Act should certain reference research quality. Engagement and impact assessment, I'm not sure.
b. Yes, I think so.

Q9. With respect to the ARC’s capability to evaluate research excellence and impact:

(a) How can the ARC best use its expertise and capability in evaluating the outcomes and benefits of research to demonstrate the ongoing value and excellence of Australian research in different disciplines and/or in response to perceived problems?

(b) What elements would be important so that such a capability could inform potential collaborators and end-users, share best practice, and identify national gaps and opportunities?

(c) Would a data-driven methodology assist in fulfilling this purpose?

a. The ARC should measure outputs like academic publications (and their citations) based on projects funded by the ARC. They can also ask grant recipients to report of their research informed policy (internationally), and which policies. There are other metrics we would care about.
b. See above.
c. Yes, data-driven methodology is clearly important as I discussed metrics.

Submission received

12 November 2022

Publishing statement

Yes, I would like my submission to be published but my and/or the organisation's details kept anonymous. Your submission will need to meet government accessibility requirements.