Case Study Writing Strategies

This is a tale of the two approaches I took to writing up case study research based on fieldwork and qualitative coding.

When I started writing up my dissertation case studies, I really had no idea how to do it. I’d read plenty of case studies but never tried to emulate them. I did, however, have a handy-dandy theoretical framework that needed to be worked into the findings.

I had three cases to report and more than enough data. Multiple case studies are typically used for comparative purposes, meaning that not only does this research design require writing up the individual cases, but also a cross-case comparison. I ended up writing four chapters to cover all of that material, with about 184 pages for the three cases and around 50 pages for the cross-case comparison.

I started off by writing up the case that had the most data – might as well get the big one out of the way, right? I wish I’d taken the reverse approach so that I would have saved some work when I found that my first try at writing up a case fell flat!

Method 1: Theoretical Framework Laundry List

I was told to be thorough in my dissertation writing. That may have been a mistake on my advisor’s part, as the final document was over 400 pages long, but I was determined to be as methodical and thorough as I could.

I started off by structuring my case description by the theoretical framework that I had developed. I went through every code in my framework and pulled out illustrative quotes that I organized under each heading, and then wrote up what I found for each concept in the framework. Even with rich and interesting empirical data to draw upon, however, it was deadly dull. It turned into a horrific laundry list in which readers became lost, much like one of those freaky hedge mazes you see in horror movies. It was ponderous and really soporific.

Repeating that two more times for the cases? No way. It was extremely slow and laborious writing, jerky and discordant, and there was no way I could meet my writing deadlines with that strategy. Fortunately, my writing group set me straight and offered suggestions of alternative structures. I listened, as one should when others are kind enough to read through drafts of heavy academic material and give thoughtful comment thereupon. Then I started over.

Method 2: Semi-Structured Thematic Template

I started over by cutting the chapter into strips and then physically coding and rearranging them into themes. Suddenly, there was a story and a flow to the material!

The first draft of the case study, cut into shreds and reassembled into a new structure.

It was done in a day. I remembered (just in time) to mark each strip of paper with the page number from which the material originated so that I could find it in the digital document to cut and paste. The process of cutting, pasting, and smoothing over transitions took another couple of days. I had every theoretical concept covered, and the material took on a much more palatable and interesting shape.

As I wrote the next two cases up, I started again with quotes, retrieving them systematically and writing up notes on the insights gleaned from them. Next, I organized them thematically rather than by conceptual framework constructs. It was easy to write the material that connected the quotes into a (mostly) coherent story, and much more interesting as the writing process generated more insights. I actually had fun with a lot of that writing!

I structured each case study chapter to start with sections providing the history and organizational setting of the case, an overview of the technologies and participation processes, and then continued from there with the thematic sections. At the end of each chapter, I included a summary with the main themes from each case and linked the highlights back to the research questions and constructs therein.

The overly-structured approach to writing a case study was painful and frustrating, but going with my intuition (while remaining steadfastly systematic) produced better results much faster. It also reduced repetition from linking concepts together and made those relationships much clearer. I expect every researcher will have to figure out an individual writing strategy, but it’s valuable to remember that the first approach may not be the best, and taking a different tack does not mean throwing out all the work you’ve already done.

The strategy for constructing the case comparison chapter, however, was a different matter entirely and a story for another day.

Qualitative Analysis Tools

In part three of my review of software that I use for my academic work, I’m covering that all-time favorite, qualitative analysis tools! I have never seen a topic that gets so many requests for help (mostly from doctoral students) with so few useful answers. So here are a handful of tools that I have found helpful for my dissertation work, which involves qualitative analysis of semi-structured interviews, field notes, and documents.

As always, my main caveat is that these are Mac OS X programs. In fact, almost exclusively. If you’re spending a lot of time with a piece of software, having it behave like an OS native application is not worth the compromise. And as usual, I tend to favor of open source, free, or low-cost options. For the work that I’ve done, the applicable categories include data capture, transcription, coding, and theorizing (which might also apply for some quantitative work, depending on the nature of the beast.)

Data Capture

Sometimes you need screen shots. For this, I just use the Mac OS X built-in tool, Grab (may be under “Utilities” in your Applications folder), which works with keyboard shortcuts – my favorite! However, it grabs tiffs, which aren’t the most friendly format, and no matter what tool you use, screen captures are almost always 72 dpi = not print quality. So I resize to 300 dpi with Photoshop, making sure not to exceed the original file size (interpolated bits look just as bad as low dpi).

Sometimes you need to record a whole session of computer-based interaction. For that, nothing rivals Silverback for functionality and cost. It’s pretty cheap, works like a dream, and is great for capturing your own experiences, or that of participants. It uses your Mac’s built-in camera and mics to pick up the image and sound of the person at the keyboard, while logging and displaying keyboarding and mouseclicks. And it doesn’t make you record your settings until the end, so that’s one less thing to screw up when you’re setting up your session. Brilliant! I have to thank the WattBot CHI 2009 student design competition finalists from Indiana State for this discovery, since I never would have though to look for something like this. I use Silverback to log my own use of online tools for participant observation. It’s really entertaining to watch myself a year ago as I was just starting to use eBird. OK, more like painful. But compared to now, it’s really valuable to have those records of what the experience used to be like.

Transcription

I record all my interviews with a little Olympus digital recorder. It’s probably no longer on the market, but it was about $80 in 2007 and well worth every penny, even though at that time I mistakenly thought that I’d never do qualitative research. It was the second-best product from Olympus at the time, and has a built-in USB to move the files to a computer. Great. Except that all the files are in WMA format. All2MP3 to the rescue – free software is hard to beat. For awhile, I used a different audio converter, but it stopped working with an OS update and then I found this one. It’s dead simple, and despite the warnings that it always gives me about suboptimal formats, it works like a charm, every time.

But once those interviews are translated into a playable format, I still have to transcribe them. It’s good data review, of course, besides being cheaper than hiring someone – depending on your calculations. MacSpeech Dictate (now called Dragon Dictate) is my tool of choice for this task; it’s the Mac equivalent of Dragon Naturally Speaking, for you Windows users out there. Both softwares are owned by the same company, and you basically shouldn’t waste your time with anything else, because they are the market leader for a reason.

I use the voice recognition software to listen to my audio recordings with earbuds, and use the included headset to dictate the interview. The separate audio and voice systems are truly necessary, because if I can hear myself talking, it distracts me from what I’m dictating. It’s not flawless, but once the software was trained and so was I, it has worked pretty well. The big drawback is that it costs about $200. The big plus is that I went from 4-5 hours of transcription time for each hour of recording to 2-3 hours, and that’s a nontrivial improvement! I have definitely saved enough hours to make it a good deal for the grant that paid for it.

If you’re using dictation software, you have to dictate into some other software. And something has to play your audio files, too. Surprisingly enough (or not?), I have found open source software from IBM that works pretty well: it’s called IBM Video Note Taking Utility. Although it was originally PC-native, I begged the developer to encode Mac keyboard shortcuts as well, which he did – awesome!

The software was created for video transcription, but I just use it for audio. It’s very simple: you load up an mp3, it makes a text file, and you can use keyboard shortcuts to skip forward, backward, pause, and speed up or slow down the recording (plus some other stuff I don’t use). There are a couple of quirks, but the price is right and it does exactly what I want without lots of extra confusing stuff going on. Most of my transcription happens at 0.6 times normal speed, so when you take into account some correction time, the fact that I’m transcribing an hour of transcript in 2-3 hours means it’s nearly real-time transcription and there’s very little additional overhead. It’s just not possible to do any transcription at normal speaking speed, because unless you’re a court reporter, you just can’t keep up with what people are saying!

Coding/Annotation

When I first started working on qualitative research, one of my initial tasks was finding coding software that I liked. If you’re not using software for this task, consider joining the digital age. There are better options out there than innumerable 3×5 cards or sticky notes, even if you have to pay for it and spend a little time learning how to use it; the time you save is worth much more than the software costs. After some fairly comprehensive web searching, I was kind of horrified at how bad the options were for Mac-native software. $200 for what? Not much, I’ll tell you that. And from what I’ve seen looking over others’ shoulders, I don’t think the PC stuff is a ton better.

But there was something better than the modernized HyperCard option that I found, and pretty much everything else. And it, too, is open source! TAMS Analyzer has got my back when it comes to qualitative data analysis. It’s super-flexible, has a lot of power for searching, matching, and even visualizing your code sets, and can produce all the same intercoder reliability stats as the pricey licensed software. There’s a bit of learning curve, but I expect that’s true of any fully-featured annotation software. Plus, there’s a server version that has check-in/check-out control, which is awesome if you have multiple coders working on the same texts, and it’s pretty easy to set up (all things considered, you do have to be able to set up a mySQL database.) I have barely scraped the surface in terms of using its full capabilities. I’m constantly finding yet another awesome thing it can do, and I learn the functionality as I need it – all the really powerful stuff it can do doesn’t interfere with using it out of the box, so to speak.

And after you’ve spent some quality time with your coding, the time will come to sort those codes. For this, I use OmniOutliner, another product from the awesome OmniGroup. Once you have a huge heap of codes, the drag-and-drop hierarchical outline functionality is a highly convenient, fairly scalable way to handle getting your codes in order. I’ve done this with note cards, and it’s a big mess, excessively time-consuming by comparison to using digital tools, and wastes a lot of paper that is then hard to store. I also like keeping an “audit trail” of my work, so having the digitally sorted codes (in versioned documents) is a great way to do it.

Theorizing

Ah, theory. That’s what we’re all doing this academic thing for, right? Well, that or fame and glory, and we all know which one of those is more likely.

Everyone has their own way of thinking about this. I draw diagrams. And when I draw diagrams, whether for a poster, paper, or to sort out my own thinking, I use OmniGraffle. I can’t begin to say how awesome this software is, and how much mileage I’ve gotten out of my license cost. Enough that I should pay for it again, that’s how good it is. My husband calls OmniGraffle my “theory software” because when I’m using it, he knows I’m probably working on theory. I find it really useful for diagramming relationships between concepts and thinking visually about abstractions. Depending on the way you approach theorizing, it might be worth a try (free trials!)

So that’s the end of my three-part series on software to support academic work. I hope someone out there finds it useful, and if you do, please give one of these posts a shout-out on your social network of choice. You’ll be doing your academic pals a favor, because we all know that’s how people find information these days. :)

Tools of the Trade: Quantitative Analysis

Following up on my last post about the tools that I prefer for organizing and writing in academic work, today I’m going to review my preferred software for quantitative analysis. Yep, there’s enough that falls under “analysis” to merit two posts. This will be the easier of the two posts to write on analysis tools, because I find that qualitative analysis takes a much more complex assembly of technical tools to support the work.

All of these tools are cross-platform (except the SNA software) so although the view on my Mac OS X screen may look a little different than it would on other platforms, the essential functionality is all the same. Isn’t that nice? So let’s begin with the tool that makes the research world go ’round: Excel.

Yes, Excel is a Microsoft product, which I usually avoid. But it’s so functional that it’s hard to use anything else, and I have extensive experience doing some very fancy tricks with Excel. You know, the “power user” kind of stuff, like PivotTables in linked workbooks with embedded ODBC lookups (yep, fancy!) The simple fact of the matter is that a lot of science is done with Excel, so almost no one doing quantatitive research can completely avoid it. However, the advice that I offer when working with a spreadsheet tool for research is:

  1. Keep a running list of the manipulations you’ve done on your data. Embed explanations on your worksheets. It’s way too easy for a worksheet to become decontextualized and then you have no idea how you got those results or why you have two sets of results and which one is the right one. This is a pain to do, but trust me, keeping a record like this will save your hide at some point.
  2. Take the time to learn how to use named ranges and linked worksheets. This dramatically improves your ability to do data manipulation in a separate worksheet without touching the original copy, meaning you always have the initial version to return to. This is more important than I can possibly emphasize. Don’t mess with your raw data in Excel unless you have another (preferably uneditable) copy elsewhere!
  3. Customize your toolbars for maximum utility if you’re a frequent user. For example, I have added a button on the toolbar for “paste values” because this is a really useful function that doesn’t have an adequate keyboard shortcut, even though I’ve tried to program one. And for that matter, programming custom keyboard shortcuts for commonly used commands is also a really good idea if you use Excel often.
  4. Install the Analysis Toolpak for grown-up statistics. Use the Formula Viewer to understand what the heck is supposed to go into the formulae. I’ve found this helpful for data interpretation on more than one occasion.
  5. VLOOKUP. Learn it. Love it.

R is my go-to tool for statistical analysis, including network analysis. If you don’t know R, it’s basically a robust, free answer to (very expensive and limited time licenses for) SAS or SPSS. It can do just about anything you want, and it has a core-and-package structure that lets you download and activate packages at will to do specialized kinds of analysis. R is well supported in the research community and you’re sure to find a package that does what you need. Like the other major statistical analysis tools, it has its own sort of syntax, but I suspect it’s no harder to learn than the other stuff. R is a great tool, and it hooks into other analysis tools very nicely.

Tools like Taverna, which is a scientific workflow tool. I’ve used this for replicable, self-documenting, complex data retrieval, manipulation, and analysis routines. I’ve written papers about it and spent time with the myGrid team in the UK helping them evaluate usability. I’m definitely a fan of Taverna and I found it really useful for the kind of complex secondary data analysis that I worked on for free/libre open source software research. I’ll even be teaching a course this fall on eScience workflow tools, including Taverna.

Protege is an ontology editor. Ontologies aren’t exactly quantitative analysis, but they can be really useful in doing quantitative analysis of large data sets with semantic properties. If for any reason you need to build an ontology, Protege is a really nice tool.

Finally, the ultimate irony – buying proprietary software to run open source software. I use VMWare Fusion to run Windows XP so I can use Pajek for social network analysis. VMWare Fusion is extremely satisfactory software for the purpose and doesn’t cost much; I have been very happy with it. Windows XP is, well, Windows.

Pajek is nothing but ugly, interface-wise, but don’t let that put you off because it does the job well and has a lot of really detailed options for SNA. It has the most insanely deep menus I’ve ever seen, but to be fair, there’s a lot of analytical complexity under the hood. It also does visualizations, but they aren’t the prettiest thing you’ve ever seen. There are a lot of tools that you can choose for SNA, and this software choice reflects the fact that what I usually need is statistics, not pretty pictures. There’s even a great book for learning how to use Pajek – it was worth every penny when I was learning SNA, because it not only shows you how to use the software, but explains the SNA concepts pretty effectively as well.