Do metrics matter?
Long story short: In Australia we have a metricised assessment of research quality that universities are compelled to complete. QUT recently put in a submission that basically said they think the metrics ERA tracks are not a very good measure of research quality. QUT also contrasts the peer assessments in ERA (which have been largely stable over time) with the more metricised assessments (which have shown improvements).
”Indeed, the stability of peer review assessments may be read as a commentary on the assumptions about the fitness of metrical approaches as proxies for underlying quality, particularly over time. The divergence has now widened to the point that the fitness of the metrics as proxies for quality requires empirical validation.”
Whoa. Once more for those in the back:
“the fitness of the metrics as proxies for quality requires empirical validation”