Writing an Effective Literature Review Quickly(ish)

Chances are that, at some point during your research career, you will be asked to write a literature review.

It’s a great way to get to know the existing research on a topic, so I suggest you agree. And then, perhaps you will sit down at your computer with a stack (or perhaps a Zotero folder) of papers and stare at your screen for a long time before you break down and Google “how to do a literature review.”

Every time I sit down to write a literature review (even if I am in the middle of the same project as last time) it seems like I forget how to do it! So when my advisor asked me a few weeks ago to write one, I googled (again). In the interest of not forgetting next time, I took notes on my process. In the interest of sharing the wealth, I am going to write it out here for you :)

What is a literature review? This seems like a silly question for someone to ask if they’ve already written one, but I think it’s worth addressing. I’ve heard of a few things referred to as literature reviews: annotated bibliographies, “related work” sections of papers, stand-alone documents, and published synthesis papers, for example.

Before you start your lit review project, it’s worth trying to assess what kind of document you need. Often, your finished project will need to be a cited piece of prose, which will stand on its own or lay out your “research space” in a larger paper. Make sure that, as you read, you are aware of the context into which this paper will fit– look for rhetorical links to the ideas you’ll present in other parts of your papers.

If you are trying to build familiarity with the research or gather relevant papers for multiple, related projects, your best bet might be an annotated bibliography– something flexible or modular that you can borrow pieces from without collapsing a rhetorical structure. In my case, that’s exactly what I did. I even used workflowy, so I could easily move things around and use an infinite number of levels in my outline (I’ve been known to get carried away).

Process Overview

I search for keywords in Google Scholar. I supplement GS with my University’s worldcat subscription for books or hard to find titles. I read abstracts to find relevant articles, and save those to Zotero using the browser plugins. During this stage, I keep an eye out for new keywords in those articles that I can use to search.

Once I have a handful of seed articles (really, just a few), I start reading the introductions. These aren’t the most important parts to get a sense of the article’s place in the literature, but the are great for finding more papers, related topics, and keywords.

Before I go back and grab all these new papers, I set up a draft outline based on what I’ve read so far and the format I need. Then, I go back and read the methods (if necessary), results, and conclusions of the seed papers and sort them into my outline with a brief description of what you think you will need out of this paper. I put each paper in multiple categories sometimes– it may have more than one contribution and interesting methods, for example. This outline is going to evolve once we get more papers, but it’s worth having some kind of basic model in your head as you keep searching.

Now, I go back and find those cited papers, search for my new keywords, and look at related topics. This is an iterative process, so don’t get caught up on things being in order. For each batch of new papers, read what you need, write yourself some notes, and stick them in your outline.

For any literature review, you’ll need to be looking for larger models that tie papers together, gaps in the research, and disagreements among authors that can become or support your argument: keep an eye out and keep notes.

Then, if you need to, write it up!

Here are some tips and specifics

First, give yourself more time than you think it’ll take. When I was planning the lit review beforehand, I titled the workflowy list I made to keep notes “Lit review in 1 week.” LOL no.

Searching for literature

  • Track your keywords. You may need them later! Also, you can avoid redundant searches.
  • Use the cited by feature. Google Scholar tells you how many people cited a paper, and if you click on that citation count, it will bring you to a list of papers that have cited it. These papers may be similar and relevant, or not! You can search within citing papers by clicking the checkbox underneath the search bar.

Reading

  • Read what you need. I said this before, but it’s worth repeating. Only read in depth what you need for your purposes. If you’ve already read 15 similar intros, start skimming and skipping them. Keep an eye out in the abstract for contributions and novelty: did they use a method you haven’t seen yet? Better read that section.
  • Take notes. These are just for you, so don’t spend too much stress on them. Especially when you are just getting into an area of literature, it’s not always obvious when someone messes up, when there’s a big gap, or when it’s you who is misunderstanding. Feel free to ask these questions and make these critiques in your personal notes; they will be really useful later when half of them turn out to be useless, but the other half can show you the gap your research can address, or just make you look like a great critical thinker 😉
  • Note themes especially early on. These will help you make your structure!

Structure

  • Map themes. When you’ve read a handful of papers, start mapping the themes if an outline structure isn’t already obvious (or dictated by your format). I used the pencil on dead tree format for that this time, and it worked well. I did find myself wishing I had a white board for this step, though. You can draw lines to represent connections, try to group the ideas into themes, or anything else that seems useful to you to start framing your paper. What you want out of this is an idea of what the argument of your paper will be and the skeleton of your outline.
  • Find a story. I hear this all the time in academic writing advice, and I honestly find it a little frustrating. What does that mean?? I could (and perhaps I will!) devote a whole blog post to this subject, but the TL;DR is that your paper should make an argument. Organize your ideas around your argument.
  • Sort papers into your argument or outline. Look for holes to search for more papers.

Writing

  • Create section headings in your document based on the structure you created
  • Re-read your notes for the papers in each section. If you need to fill in some details, go back to the abstract or appropriate section of the paper.
  • Write up the parts of the research that are important to your argument in your own words, and cite carefully.
  • Add your own synthesis or ideas that support your argument
  • Conclude with a summary, what it means, and why it is important.

Productivity tips: I used the Pomodoro technique this time around, and set goals for myself every day. For example, once I got into the reading groove it’d be something like “read, note, and outline 7 papers.” I’d work in intervals of 20 minutes with a 5 minute break in between for up to 5 hours. I had a hard time keeping track in Zotero of which papers I’d done and not, so I would copy and paste the notes I took for the paper into a Note in Zotero and sort the whole folder by how many notes each paper has. Not perhaps how it’s designed, but it worked :)

I hope this all helps! Do you have ideas about how I can improve my plan? Any particular road bumps you encounter in a lit review? Leave a note in the comments and we can try to make lit reviewing better :)

Continue Reading

ComSciCon Cornell 2016

cornellHave you ever gotten the sense that your journal articles, press releases, or blog posts are little more effective than shouting into a void? I feel you!

I’ve recently returned from Ithaca, New York, where I attended a ComSciCon conference: a workshop for graduate students about tactics, strategies, and even whole careers in effectively communicating about science to people who aren’t familiar with the field.

Writing has always been important to me, and one of the reasons research is an exciting career to me is because of how much writing it involves. I’ll finally have something worth writing about!

But graduate students often don’t have formal requirements or even available courses about how to reach non-scientists. In my program, we are lucky that one of our first-year courses (called “the Engaged Intellectual”) covered what it means to be a public figure as a scientist, and a few ways of doing so. I (predictably) got excited about the idea of writing for non-scientist audiences, and I started this blog after taking that course! I am so glad I was alerted to the options about public communication of science in my first semester so I can pounce on opportunities to develop the skills and experience I’ll need.

ComSciCon offered talks and panels with practical advice about writing, being interviewed by the press, and even interacting with policy-makers. I loved hearing about all kinds of science that I know nothing about: diseases in cattle, lasers, cancer treatments, and bees that live under ground, to name a few. We each wrote a piece intended for general audiences, got peer feedback, updated our drafts, and then got expert feedback. There were lots of interactive exercises and a chance to give and hear a 60 second “pop talk.” The food was good, too :)

A big takeaway from the conference for me is that I can do this. From where I sit, it feels like I only just now am seeing how much there is to learn and how little I know. Who am I to write as an expert? But ComSciCon helped me see that there’s a lot I can in fact offer, and perhaps I can bring some sense of the wonder and curiosity and creativity involved in science to my science writing. The idea that “I know so little!” is closely followed by, “Let’s go find out!”

So how will ComSciCon change my life?

First, I’m going to start thinking more about policy. I came into it looking for tips and feedback, but I left actually quite excited about policy. I care a lot about politics, but I’ve done very little (other than vote, of course) because I haven’t felt like I could. Knowing that my research can help inform policy and that politicians are actually interested in talking to scientists was really empowering and exciting!

Second, I am going to write more about science. Mostly I’ve been writing about what it’s like to be a graduate student because those are the kinds of blogs I read before heading off to school. But maybe that’s not all I can offer. It would be good practice for me to write up findings from studies I’m reading in Regular Human English, a good opportunity to think about its broader impacts, and perhaps interesting to the general Internet. I learned at the conference that 58% of online adults have a broad interest in science and technology, but it comprises only 2% of news coverage. I’m not going to fill that gap alone, but it’s encouraging to know that there are people who wouldn’t find it boring if they stumbled on to it :)  So look forward to more posts about the kinds of science I am reading and doing!

Third, I am going to talk about the excitement of doing science in this venue and elsewhere. Somehow, I made it all the way through elementary, middle, and high school, plus 5 years of college and a Master’s degree without noticing how exciting science is. As a kid, I definitely got the sense that adults were excited about science, but I could never figure out why. Science class meant memorizing old findings, writing super formal reports, and doing “experiments” which were in no way experimental– the adults in the room knew exactly what would happen. I never noticed that being a scientist would mean asking questions that no one knew the answer to and working out how to find the answer. Science class involved no creativity (except when we got to make a poster!) and actual science feels like all creativity, all the time (except when I have to make a poster 😉 Neither science class nor social studies clued me in to the fact that social science exists and is fascinating. I wish I would have noticed this sometime before my mid-twenties, so I’ll try to pass along my enthusiasm.

Continue Reading

Baby’s First Academic Conference

Hi!

So I’ve been gone– hopefully this doesn’t happen too much while I’m in grad school, but it seems somewhat inevitable, so I won’t start apologizing until there are readers here to complain 😉 Feel free to pipe up in the comments and redeem one groveling at no cost.

I’ve done a lot while I’ve been gone, but I thought I’d start with the most interesting event, the experience most relevant to new grad students, and the thing that went most differently than I expected: My First Conference.

Near the end of February and the beginning of March, I went to an ACM-SigCHI conference called Computer Supported Cooperative Work. Actually, no one calls it that: they call it CSCW and so will I. :)

Here’s what I knew going in: it’s a medium-sized conference (benchmarked using CHI and iConference, if that means anything to you), my advisor does a lot of work there, and I have a poster to present. But I’m surely not the first self-aware over-thinker to pass through the metaphorical halls of academia, and I was pretty worried about the whole thing.

Here’s were the challenges I identified:

  • the conference hotel was expensive. My travel award had a limit, and $300 a night gobbled that up pretty quickly, not to mention the cross-country flight and registration
  • I have to schedule travel and just hope that I don’t miss something important (SPOILER ALERT: I did.)
  • I have never designed, printed, or presented a poster before
  • Communities like this tend to have norms, history, status hierarchies, and all manner of other unseen mechanics that I would be completely unaware of. (what if I make an ass of myself?)

So here’s what I did.

First, the travel arrangements. I went on to airbnb and found that, even at half the price of the conference hotel, I’d still go over my travel award at the market rate there. This particular conference would have been worth it for me to spend a little money on, but I live on a graduate student stipend, so it was worth some extra hunting to see if I could get within my conference award.

I tweaked my search parameters a few times and found eventually found a residential hotel for half the market rate with a 7 day minimum stay. Even without the cheaper airfare for flying in and out on slow days, it was cheaper to stay longer! The guy had no reviews and he ignored my request the first time, but I was determined, so I sent a second request and I called him on the actual phone (!). (If you do this, I recommend that you make sure that the booking is through Airbnb. They have lots of protections for you if the host turns out to be sketchy, and if you book directly, you lose out on those.)

On missing things and gaining things. The conference itself was on the same days as my classes would have been, so no matter how I booked it, I’d miss one meeting of each class. But the rest of my week is generally free, so I had several days on each side to play with.  I picked the cheapest flight days, triple-checked my calendar, and booked!

So suddenly, I have a little mini-vacation on my calendar! 3 of those days are conference, and the rest are prep for conference and explore the city days! Certainly, not everyone will have the luxury of doing this, but perhaps this is the advantage that grad students (despite their limited funds) have over some faculty in attending conferences.

I took advantage of it and invited my LDR partner to join me there. He’s a 45 minute flight from the conference, and the trip fell just a couple days after our 2 year anniversary. Not exactly a luxury vacation, but we had an absolute blast. I don’t have time to go over all the fun stuff we did, but I’ll give you three of my favorite things I’ll be doing in future conference cities.

  • the work time: I’m putting this first because it sounds boring, but it really unlocked the fun of the whole trip for me. Since I was missing class and had all the same work obligations I do at home, from the first day, I had a lot of work on my mind. My partner also has autonomy over his time and work, so I think he had an easier time committing to the trip because it wouldn’t be a total loss in terms of productivity. So he suggested that we decide early on how many hours worth of work we wanted to do each (non-conference) day, and plan to do it so we could enjoy the rest of the time exploring. We decided on about half of our daily, non-vacation goal amount, and made sure we squeezed the most work out of it that we could. To be honest, I got a lot done because of that constraint, and it freed my mind to enjoy the city.
  • the backup charcuterie: on our first day, we went to Trader Joe’s and picked up a few kinds of cheese, a truffle pate, a baguette, and some beer and wine (I missed you California booze and booze regulations!) We had a mini fridge, so we were able to keep deliciousness in stock throughout our time there! It was great for relaxing and luxurious meals in or a quick snack. We couldn’t find (or travel with) a knife and didn’t want to buy 100 disposable plates or cups, so we used scissors to cut up our food, disassembled paper bags as a cheese board, and plastic coffee mugs for all drinks. No regrets– it was amazing, will do again.
  • the mysterious ferry adventure: You can accomplish this on any form of public transit, but we chose the ferry because we enjoy boats more than buses, BART is loud, and San Francisco has a lot of destinations on its ferry system.
    1.  find the transit vehicle of your choice
    2. Purchase a ticket to the next destination on the schedule
    3. Board the transit vehicle, and disembark when you arrive!
    4. Now, go explore the place you landed.

    We ended up in someplace in Marin County, found a nice walking trail, explored a community of marsh homes, and had dinner and beer at a local brewery! We like to read together, so we brought our books for the journey, and it was a real highlight of the trip. I’ll be doing something similar in future cities (time permitting :)

As the spoiler alert mentioned, though, I did miss something. After my travel arrangements had been booked, I got an email about a department event I was expected to participate in the day after the conference was over. I tried to rearrange my travel, but I wouldn’t have been able to do so without canceling the one affordable airbnb option. It was a bummer, but we worked out a way for me to contribute without attending. The bottom line, and the reason I mentioned it at all, was that I am really grateful for understanding faculty and the opportunity to go to the conference with their support and flexibility. And for all of us very-early-career academics, conferences are an important part of finding and building out your place in a research community, and it’s worth it to ask for what you need and make some compromises and sacrifices to get there.

Poster presentations. This is where I invested most of my anxiety, and where everything was honestly just fine.

Designing the poster was easily the hardest part. My first draft was a disaster. My best advice is to have other people look at it. Once you’ve spent that long staring at it, you’ve lost objectivity.Even though I massively reduced the amount of text on my poster, I wish I had done so even more. I also recommend that you do not use Illustrator to accomplish this. YUCK. I’ve heard people use powerpoint, but that sounds terrible as well. If anyone has software recommendations, please leave them in the comments!

Printing the poster was annoying. It was half the price to do so at Staples as it was at my university, but it was a pain. I went in, they told me to email it to them, I did, and then they told me I had to submit it through the website. Annoying, but it looked fine in the end, and, because I had a couple extra days, I could print it in the destination city instead of carrying it on the plane. WORTH IT. Someone else in my lab got theirs printed ahead of time on fabric, folded it into his suitcase, and that was that. That’s certainly a good option to consider if you have the lead time!

Presenting the poster was actually pretty fun. I can be quite shy in new social situations, and this conference certainly qualified as a new social situation. But the poster presentation gave me an excuse and topic to talk with others about, which helped me feel comfortable and gave me the space to get to know people. If I had something to recommend here, it’d be to keep your eye on your goal for the night. The easiest path is to use it as an opportunity to refine a “pitch” for the study, and this is really useful for that. Sort of like if you broke your arm, for the first couple of days you’d tell the story a different way, and by the end of the week, you’d have the most concise and effective story down pat. There’s a lot you can learn from this, so don’t underestimate it. But in doing this, it’s really easy to find yourself doing a lot of the talking. But you already know what you know– try to get the other people talking so you can find out what they know

Norms, etc. I made the debatable choice of using the conference as the site for an ethnography assignment for class. This meant that I was constantly taking notes about how people were behaving and how I was feeling. The pros were that I noticed a ton that I wouldn’t have otherwise and discovered that I really enjoy ethnography. The major con was that it made me very self-conscious. You are supposed to be reflexive, but yikes.

So I was really worried about violating some unspoken norm and making an ass of myself. This did not happen. (YAY!) People did try to get a sense of where you were in your career, so it was clear from the beginning of conversations that I was a lost little bunny. It sounds bad, but honestly, I asked questions like that, too, and I often told people unsolicited that it was my first conference. I thought it would be a disadvantage to be at the bottom of the proverbial totem pole, but people were kind and understanding and seemed to genuinely be interested in my lost-bunny experience.

But most of the reason I didn’t make an ass of myself is that I was too exhausted even by the end of the first day to participate in anything optional, especially if that thing were social. So as silly and obvious as this is, the most useful thing I learned is that conferences are exhausting. I was expecting something like a day full of meetings– tiring, but the kind of tiring where you’re up for a party later to unwind. Now, there were parties to go to (and I did feel obligated to go to them) but I elected not to. By the end of the first day, I knew I would have (and be) absolutely no fun, so I went to the (lovely!) hotel atruim/bar thing, had a drink with my partner to celebrate my survival, and went back to the hotel, presumably to eat some backup charcuterie and pass out.

Next year, I will be mentally prepared to work all day and party all night, but I don’t feel guilty that I didn’t take advantage of the social and networking time in my first year. I will be much more comfortable next year, and in other future conferences, now that I am confident that I have the lay of the social land. I know not everyone feels they need it, but I want to encourage you that if you do, you’re not alone.

My final thought is that it really helped me to have my partner there. Although I often gave up valuable networking opportunities because I wanted hang out with him, it was well worth the trade off for my first year to have someone you know you can reach out to when you are kicking yourself for saying something awkward or thrilled about something you learned, or proud of yourself for gathering the nerve to ask a question. I recommend finding someone you can text or talk to who will support you and help you keep perspective. It’s not a dangerous or scary event by any means, but I found it over-stimulating, to say the least, and it was really great to have someone to reach out to.

So this turned into quite the omnibus post, but hopefully it was helpful to get a sense of what your first academic conference might be like. Prepare for a lot of activity, a lot of people, and a lot of fun, but don’t feel bad if you say something silly or need a break. People understand, and they won’t remember by the time they see you next year. 😉

Continue Reading

Winter Break 1.0

Long time, no see! I just got back from winter break. You’d think that I’d have had time to write something, wouldn’t you? Well, that’s sort of what this post is about.

Before winter break started, I was really excited about all the work I’d be able to get done. It was a simple formula:

No class = more time

Right? Not exactly

During the last lab meeting before winter break, we went around the table and talked about what we would do before we met next. I talked with my cohort about what winter break would be like. I planned my trip. I knew it was a norm to joke about how difficult it will be to get work done, but I knew I had bitten off less than I could chew. I look back on my confidence with amusement.

Now I see that having no structure in my time is, for me, a disaster. I did get the main priorities for the break completed, and I’m happy with what I’ve done, but let’s just say that if I didn’t return 5 days ahead of time just before a huge snowstorm that gave me an excuse to not leave the apartment for days on end, I would be significantly less happy.

And those few days were like pulling teeth to get anything done, even though I clearly had the time. Like the total cliche I am, I found other things to do. I cleaned, I cooked– I even baked a loaf of bread! (That part was worth it. delicious.)

It’s important to me to learn from this experience for a few reasons:

First, I will have other breaks, including summer, when I will have to get work done.

Second, and hugely, assuming all goes well, I will have a lot of unstructured time during which to write my dissertation. During that time, I need to write my dissertation, not worry about writing my dissertation and then write it all at once at the last possible moment.

Thirdly, I need to have practice imposing and conforming to structure on my time to be a researcher who succeeds in accomplishing her own priorities, rather than what happens to be urgent at that moment.

After a year or two of arranging my life pretty haphazardly and years of historic failure to handle long term projects for school, I’ve been pleasantly surprised with my productive response to the structure of last semester. What that tells me is that, just like studies suggest, I am capable of improving and building new skills. Now it’s time to extend that to productive use of unstructured time. And I’ll have several, but not unlimited opportunities to practice and refine those habits before dissertation time.

I’ll report on my plans and progress in this area, and I hope you’ll leave your suggestions for being productive with unstructured time in the comments!

Continue Reading

Day in the “life”

For my first act as Blogger, I shall recount to you one graduate student day. Namely, today.

Today, I woke up, like any other day, at 7:34. Then, I shut off my alarm and went back to sleep for another 3 hours, because I could. Before you judge, know that I had great reasons for this:

  1. 7:34 am is early
  2. I do what I want!

Just kidding. I stayed up late skyping with my partner in California, and I was working on a project with Californians, so I figured it would work out.

I did NOT figure that the maintenance guy would show up with my apartment a mess and me in my PJs, but you know. Oh, well.

So I made some coffee, fed my cat, darted into my room to change while the nice man got some part from his truck, and sat down to work. (Of course, he fixed the thing in moments, so my switching from comfy pants to jeans was a debatable choice.)

Work today consisted of finishing and submitting a poster to a conference. If all goes as planned and our submission is accepted, I will get to fly to San Jose in March, stand next to a large poster, and talk to scientists and students who walk by and ask questions (presumably fascinated by my ability to cram a papers worth of motivation, methods, and findings on to one page).

From what I hear, this event will be just like a middle school science fair, but with fancier clothes, bigger words, and much(!) more wine.

The goal for presenting a paper at this conference is to get feedback on some work we are doing while it is still in progress so we can frame it differently, do more analysis, or even gather more data before we submit a full paper somewhere.

Our submission consisted of a draft of the poster itself, and a 4-page “extended abstract” about the work we were doing. The extended abstract was very challenging to write: I took an 11-page paper draft and tried to get across what was going on in less than half that length. Also, the abstract itself has an abstract– what? Writing the abstract, I really internalized the value of the Shitty First Draft: getting something complete on paper will be a great foundation (even if it is indeed shitty.)

The poster was also challenging: trying to fit the content of the 4 pages in an even further reduced area, and trying to make as much of it visual as possible. It was somewhat easier for me than the abstract, which was nice. (I wonder what software people use for this– I used Illustrator, but I think there’s got to be a better way.) I learned the value of starting over on a blank sheet, of starting with the easy stuff and of laying things out visually before you try to write about it. So I worked on these things all day, corresponding with my co-authors along the way.

At 7, my cohort was scheduled for our weekly hangout, and I was hosting. So after I had a passable draft of both in my collaborators email boxes, I cleaned like a maniac. Once my apartment was in shape, I read and incorporated most of the feedback I’d gotten, and went out to shop for snacks and wine. I met one of my cohort along the way and she helped me carried the spoils back to my apartment.

She and I chatted for a few hours before we realized it was well after 7. The other two had bailed, for good reasons I’m sure (such things happen when your cohort consists of 5 people :) So my friend and I talked and ate a bunch of snacks– worth it.

After my friend left, I pour myself a glass of wine, finished my final round of edits, and submitted the abstract and poster. I learned that it feels substantially better to file a submission hours before the deadline than minutes. What if I tried days?

Sadly, the world may never know.

Continue Reading