Eminence vs Evidence In Fundraising – Part 2: What is “Proper” Research
Judging from readers’ Comments to Part 1 of this series there’s a truly felt need and desire for collecting — and sharing — proper fundraising research and findings.
Two questions arise. Just what is ‘proper’ research? And, ‘how’ can this information best be shared?
What Is ‘Proper’ Research?
The definition of what constitutes ‘proper’ research can get complicated — overly complicated — very fast. Since I’m neither a philosopher, theologian nor a scientist I’ll let others with that competence attempt a definition.
My criteria for ‘proper’ research includes ‘evidence-based’, ’empirical’ findings arrived at through appropriate testing methodologies. (For more on this, see my post The Curse of Testing Illiteracy)
Others will disagree with my criteria and substitute their own. That’s fine with me because you’re going to make decisions based on what you consider ‘proper’ or not. (The Stanford Encyclopedia of Philosophy contains pages and pages on what constitutes ‘evidence’ if you’re inclined to dive deeper into the definitional pond.)
Quite frankly, this discussion on proper evidence and proper research can go on and on; round and round until we all chase our mental tails to the point of collapsing into a puddle of paralysis and inaction.
An academic researcher will view what is proper evidence one way; a rushed fundraiser searching for an answer to a practical problem another.
I believe at this stage in our trade’s evolution from myth and habit to the greater use of evidence what is important is that we keep moving in the direction of ‘evidence’. In short, we should always be skeptical of how perfectly the ‘evidence’ was arrived at, but not paralyzed into inaction as we wait for the perfect to emerge.
Let me illustrate.
From Evidence to Action with a Pause for Nitpicking
Two years ago, at the height of the media meltdown in the UK caused by over-use of industrial solicitation tactics and punctuated by the death of Olive Cooke, two ’eminences’ of British fundraising swung into action to remedy or at least relieve a disastrous situation.
Ken Burnett and Giles Pegram, CBE set out to change the sector’s mindset and actions where the treatment of donors is concerned. They established The Commission on The Donor Experience, launching 28 collaborative projects consisting of a series of ‘best practice’ practical reports that focus on putting the donor, not the fundraising organization’s financial targets, at the heart of fundraising strategies.
Over the next 24 months thousands of volunteers were involved … the reports were prepared … then released for use free of charge. (You can see all the Commission’s project summaries here.)
THEN … no sooner had The Agitator proclaimed that the new Commission on the Donor Experience (CDE) is A REALLY BIG Deal then criticism of this significantly ambitious project sprang forth.
Ian MacQuillin, director of Rogare, the fundraising think tank at Plymouth University’s Hartsook Centre for Sustainable Philanthropy led the charge in a piece titled OPINION: THE COMMISSION ON THE DONOR EXPERIENCE—A GOOD THING IN ITSELF, BUT PHILOSOPHICALLY CONFUSED that you can read in its entirety here. [Disclosure note: I am a board member of the Hartsook Centre for Sustainable Philanthropy]
Ian wrote, “ The Agitator article unreservedly praises the CDE’s recommendations, which it says are “backed by a cornucopia of practical tips and recommended actions that lead to better experiences for donors”.
“But at no point do Roger and Tom recommend a critical reading of the overarching blueprint and the 28 projects to evaluate whether this “cornucopia of practical tips” and the theory on which they are based will actually deliver what they claim to. They simply take for granted that what the CDE has said is correct.
“And that’s a problem, because the Commission for the Donor Experience is flawed in places, and while I said that I would have loved to have done some of their projects at Rogare, there are others that Rogare would not have published at all, frankly, because the quality of the research isn’t up to standard.” [Emphasis by The Agitator.]
Here at The Agitator we don’t take much for granted, let alone recommending that our readers blindly “take for granted” what the Commission says. And we sure don’t accept on blind faith that a project with as much potential as the Commission is flawed because it’s academically imperfect.
As much as I respect and support academic research and the evangelical efforts of Ian to advance it, I fear the polarizing dangers to the sector that come from placing too much emphasis on elevating the quality of academic research over the experiential evidence of practitioners.
This passage from Ian’s post illustrates how many academics are likely to view the work of the Commission On the Donor Experience:
“The philosophical flaw with the Commission On the Donor Experience is that it begged the question that improving the donor experience was the right thing to do. Because of this, the CDE did not seek evidence for many of its claims, considering evidence not to be required because they were ‘self-evident’, and dismissing arguments that contradicted its pre-established general conclusions.
“This philosophical flaw leads to a methodological flaw. Some CDE research looks suspiciously like it was constructed in order to arrive at particular outcomes, rather than to genuinely test an hypothesis, thus leaving open the possibility that the hypothesis (i.e. the pre-established conclusion) could be falsified.”
On the other hand thoughtful practitioners understand that it’s too risky to stay stuck in the status quo while waiting for irrefutable evidence. As Mark Phillips put it in his response to Ian’s critique: .
‘I haven’t read all the reports in the CDE yet and I imagine I will disagree with some of what I see. I wholeheartedly support your drive for evidence, but I would suggest that the flat lining of charitable giving over the last decade might be evidence enough that we have hit the buffers on transaction based fundraising. Relationship Fundraising most definitely offers us an alternative that in absence of any other approach is perhaps the best hope we have.”
For now, fundraisers must become far more skilled at accessing and assessing the practicalities of academic research (evidence-based). At the same time, they must become equally skilled at accessing and assessing the recommendations of experienced practitioners (experience and eminence-based).
In short, we must turn to and learn from both if we are to make advances.
For now, I’ll give the last word in this ‘debate’ to Ian because he has apt advice to how we all should be approaching information — whether eminence/experience-based or evidence-based.
“It is now the responsibility of fundraisers to read these reports critically (and all of them, certainly not just the blueprint [summary]), to identify the questions we need to ask about what evidence either proves or falsifies the CDE’s claims, how we can improve on them, and what we need to know next and do next. And there is much here to build on.”
P.S. On Monday, in the final part of this series, I’ll turn to the question of ‘how’ we access and share information and share some thoughts on the part each of us should play.