Measuring public science by dollar value alone 'stupid', says top science adviser

Measuring taxpayer-funded investment in science using a simple dollar value approach is "stupid," says Motu economist Adam Jaffe, who is welcoming a new government big data project to allow richer measurements in the future.

The Ministry of Business (MBIE) was embarking on a process to create a "government-wide data infrastructure," which would make it easier to track the public science effort, Mr Jaffe said at the Future for Scientists in New Zealand conference at Wellington's Te Papa yesterday. Jaffe, a globally recognised expert on innovation policy, is a director at Motu and sits on MBIE's science board.

"The idea is the government is going to track, across all government-funded agencies for research, all research proposals, the people involved with them and basic data about each of those research investments," Mr Jaffe said. "They're going to create the fundamental baseline of data, which means that over time researchers can come along and say 'how would we measure the impact of that project?'"

There were many ways of measuring the impacts of such projects, varying from direct measures such as revenue to proxy measures such as investment in the same product from private firms, he said. However, as a researcher, Mr Jaffe said if he were looking to track a programme's impact years later, he would need to be able to find the people involved and MBIE's proposed infrastructure would enable this.

Impact shouldn't be reduced to a simple dollar value, Jaffe said.

"The goal should not be to say that the government's rate of return on the $1.5 billion they spent last year was 4.6% – or some other number, I've made that up. That's a stupid way to think about assessing the impact of research, because what we're trying to do is inherently and intrinsically multi-dimensional," he said.

"The only way to keep track of intrinsically multi-dimensional impacts is to recognise there are multiple impacts. You're not going to be able to add them together, you're not going to be able to say we did this much improvement of public policy and this much enhancement of capability," Jaffe said. "That shouldn't be the goal, and the fact you can't do that doesn't mean you can't evaluate the impact."

Earlier in the day, Science and Innovation Minister Steven Joyce said the government was developing "more easily accessible information" on science, including the people, institutions and funding involved.

"This is will enable us all to make more intelligent investments, continually raise the bar on performance, and make the best use of public money," Joyce said in notes for his speech.

(BusinessDesk)