Creating a Theory? Really?

At an early stage in my dissertation work, I was confronted with the problem of what it meant to do a theoretical dissertation. All the methods courses I had taken prior to my comprehensive exams focussed on the design and implementation of a quantitative (or qualitative) methodology aimed at testing a theory. In the case of my proposed dissertation work, my research design envisaged a controlled experiment in which test participants would be exposed to one form of briefing instrument (either text-based briefings or ICT-based briefings); following which their support for following the recommended course of action would be measured. If it could be demonstrated that the individuals in one group were persuaded to accept the recommendation more than in the other group, I could state that that form of briefing instrument was more effective than the other.

The committee’s reaction surprised me. Peggy Storey, an HCI expert from Computer Science, was the first to state that it appeared that I was attempting to test a theory that didn’t exist. The committee agreed, noting that the literature review as described in my comprehensive exam concluded that there was no theory of effective briefing in the policy sciences – electronic or otherwise.

Now this starting point (i.e., that, as with Gertrude Stein’s Oakland, there’s no there there) strikes me as the first dilemma in crafting a research question for one’s dissertation, and a central fundamental problem facing me in this particular case: if you are going to contribute something new, how do you prove the non-existence of something? Is there really no formal theory of the policy advice communication process? It would be embarrassing, to the point of being intellectually devastating, to argue that there is no theory of policy briefings before setting out to create one, only to find out that such a theory does exist and is widely accepted and working quite well, thank you. I’ll come to that other bit below, however – that even if it’s working well, it doesn’t mean that a better theory can’t be sought. But this is a bit of a problem – the non-existence of something – and one that is compounded in the realm of theory creation.

Anyway, for the time being, let me make the controversial statement that the policy sciences has no formal theory (what is a “formal theory” anyway? Is it “a set of interrelated constructs [variables], definitions and propositions that presents a systematic view of phenomena by specifying relations among variables, with the purpose of explaining natural phenomena”? [Kerlinger, 1979]. This is a bigger question I am grappling with.) for the communication of policy advice and the informal ones we operate with (i.e., that analysts write briefings that Ministers act upon) are known by everyone to be false. (I’ll pause again to consider this statement. Is this true? Would we call the theories we have now – theories that inform our practice, but not any research, as far as I can tell – “formal theories”? This strikes me as the smoking gun: we know the theories are false, but we still teach people to write briefing notes based on a false set of theories.) I find this remarkable, by the way (which is why part of me has trouble believing that there isn’t a vast body of theory and research out there just waiting for me to land in it). Schools of public administration – scores of schools – all oriented in part towards the question of policy analysis and what to do with it, and we don’t have a theory of policy advice communication? There have been various observations that policy analysis, once grounded in rigorous assessment, should be enhanced through effective presentation. (E.g., Meltsner, A.J. 1980. “Don’t Slight Communication: Some Problems of Analytical Practice” pp. 116 – 137 in G. Majone and E.S. Quade (eds.) Pitfalls of Analysis. Chichester, UK: John Wiley & Sons; and Lindquist, E. 1998. “Getting Results Right: Reforming Ottawa’s Estimates.” In L.A. Pal (ed.). 1998. How Ottawa Spends 1998-99 (Balancing Act: The Post Deficit Mandate). Toronto: Oxford University Press.)

But we have people teaching courses in how to write and communicate effective briefing notes (e.g., Former British Columbia deputy-of-all-things Philip Halkett teaches a course to civil servants on policy analysis and effective policy briefing. The Canada School of Public Service offers a one-day course on “Writing Briefing Notes” that professes (confesses?) to drawing heavily on the egregious “How to Create Superior Briefings”), and there’s no research to provide a basis for this? How can a system survive so long on impressionistic musings and idiosyncratic preferences? Quite a long time, it turns out. This issue has received such little attention in the policy analysis literature – indeed, the entire issue of effective communication in the briefing function appears to have languished in the policy sciences – and it is this gap toward which my interest is oriented. I’ll return to this in a moment, but for now I’ll just repeat: remarkable!

Well, the upshot of the committee’s reaction was that they wanted me to develop a dissertation proposal that aimed at creating a theory of policy briefing. Imagine my shock. That I was allowed to do this – i.e., create a theory – let alone be asked by my committee to do it, was frankly shocking to me. I will admit to being somewhat embarrassed at being shocked (I’m sure I played it cool in front of the committee), but I had no idea that this was allowed. I took two research methods courses during my coursework phase (both in sociology, because it seemed to involve the least painful cross-disciplinary acrobatics): a standard quantitative statistical analysis course, and a research methods course. It was in the research methods course that I was given the clear and distinct impression, if not directive, that a dissertation involved the development of a methodology for collecting data in order to answer a clear question. There was, of course, qualitative study: but this was something that was tolerated as a kind of social activism, not the kind of thing you’d do as a dissertation. You have to understand that the seminar leader had a long history of trying to get sociology students to focus on a researchable question that could lead to a defensible research methodology and reproducible results. So he can be excused for the lumps on his head.

That may not have been what was intended, but it’s what I heard. And I definitely never heard anything about going out and creating a theory. So to have my committee say that they wanted me to do just that was quite frankly revolutionary. It opened up a whole new world to me – a world of induction and theory development, of imagination and insight and discovery.

What I found strange as I embarked upon this journey was the dearth of guidance that awaited the would-be theorist. To begin to get a handle on how this was done – how a theory could be created in a dissertation – I went back to a prayer book of sorts: Rudestam and Newton’s “Surviving Your Dissertation”. This book has been enormously helpful over the past several years and hasn’t failed me yet. In a total of 298 pages, here’s the essence of what Rudestam and Newton have to say about theoretical dissertations:

Another possible approach to writing a dissertation is to write a theoretical dissertation and bypass the need for data collection entirely. This is by no means an easy alternative. Original theoretical contributions are a profound intellectual challenge….

If you know an area of inquiry inside out and are intimately familiar with the issues and controversies in the field, you have the chance to contribute a new theory …

If you do choose to pursue a theoretical dissertation, you will be expected to argue from the literature that there is a different way of understanding a phenomenon than has heretofore been presented. Some of the more viable theoretical dissertations in the social sciences are those that bring together or integrate two previously distinct areas.

Well I found that somewhat hopeful – as far as it goes (except for the bit about the “profound intellectual challenge”). I did like the bit about “a different way of understanding a phenomenon” and the effort to “bring together or integrate two previously distinct areas.”

Next up – how Henry Mintzberg came to the rescue (or how he at least provided some geeky comic relief).

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s