Writing an Effective Literature Review Quickly(ish)

Chances are that, at some point during your research career, you will be asked to write a literature review.

It’s a great way to get to know the existing research on a topic, so I suggest you agree. And then, perhaps you will sit down at your computer with a stack (or perhaps a Zotero folder) of papers and stare at your screen for a long time before you break down and Google “how to do a literature review.”

Every time I sit down to write a literature review (even if I am in the middle of the same project as last time) it seems like I forget how to do it! So when my advisor asked me a few weeks ago to write one, I googled (again). In the interest of not forgetting next time, I took notes on my process. In the interest of sharing the wealth, I am going to write it out here for you :)

What is a literature review? This seems like a silly question for someone to ask if they’ve already written one, but I think it’s worth addressing. I’ve heard of a few things referred to as literature reviews: annotated bibliographies, “related work” sections of papers, stand-alone documents, and published synthesis papers, for example.

Before you start your lit review project, it’s worth trying to assess what kind of document you need. Often, your finished project will need to be a cited piece of prose, which will stand on its own or lay out your “research space” in a larger paper. Make sure that, as you read, you are aware of the context into which this paper will fit– look for rhetorical links to the ideas you’ll present in other parts of your papers.

If you are trying to build familiarity with the research or gather relevant papers for multiple, related projects, your best bet might be an annotated bibliography– something flexible or modular that you can borrow pieces from without collapsing a rhetorical structure. In my case, that’s exactly what I did. I even used workflowy, so I could easily move things around and use an infinite number of levels in my outline (I’ve been known to get carried away).

Process Overview

I search for keywords in Google Scholar. I supplement GS with my University’s worldcat subscription for books or hard to find titles. I read abstracts to find relevant articles, and save those to Zotero using the browser plugins. During this stage, I keep an eye out for new keywords in those articles that I can use to search.

Once I have a handful of seed articles (really, just a few), I start reading the introductions. These aren’t the most important parts to get a sense of the article’s place in the literature, but the are great for finding more papers, related topics, and keywords.

Before I go back and grab all these new papers, I set up a draft outline based on what I’ve read so far and the format I need. Then, I go back and read the methods (if necessary), results, and conclusions of the seed papers and sort them into my outline with a brief description of what you think you will need out of this paper. I put each paper in multiple categories sometimes– it may have more than one contribution and interesting methods, for example. This outline is going to evolve once we get more papers, but it’s worth having some kind of basic model in your head as you keep searching.

Now, I go back and find those cited papers, search for my new keywords, and look at related topics. This is an iterative process, so don’t get caught up on things being in order. For each batch of new papers, read what you need, write yourself some notes, and stick them in your outline.

For any literature review, you’ll need to be looking for larger models that tie papers together, gaps in the research, and disagreements among authors that can become or support your argument: keep an eye out and keep notes.

Then, if you need to, write it up!

Here are some tips and specifics

First, give yourself more time than you think it’ll take. When I was planning the lit review beforehand, I titled the workflowy list I made to keep notes “Lit review in 1 week.” LOL no.

Searching for literature

  • Track your keywords. You may need them later! Also, you can avoid redundant searches.
  • Use the cited by feature. Google Scholar tells you how many people cited a paper, and if you click on that citation count, it will bring you to a list of papers that have cited it. These papers may be similar and relevant, or not! You can search within citing papers by clicking the checkbox underneath the search bar.

Reading

  • Read what you need. I said this before, but it’s worth repeating. Only read in depth what you need for your purposes. If you’ve already read 15 similar intros, start skimming and skipping them. Keep an eye out in the abstract for contributions and novelty: did they use a method you haven’t seen yet? Better read that section.
  • Take notes. These are just for you, so don’t spend too much stress on them. Especially when you are just getting into an area of literature, it’s not always obvious when someone messes up, when there’s a big gap, or when it’s you who is misunderstanding. Feel free to ask these questions and make these critiques in your personal notes; they will be really useful later when half of them turn out to be useless, but the other half can show you the gap your research can address, or just make you look like a great critical thinker 😉
  • Note themes especially early on. These will help you make your structure!

Structure

  • Map themes. When you’ve read a handful of papers, start mapping the themes if an outline structure isn’t already obvious (or dictated by your format). I used the pencil on dead tree format for that this time, and it worked well. I did find myself wishing I had a white board for this step, though. You can draw lines to represent connections, try to group the ideas into themes, or anything else that seems useful to you to start framing your paper. What you want out of this is an idea of what the argument of your paper will be and the skeleton of your outline.
  • Find a story. I hear this all the time in academic writing advice, and I honestly find it a little frustrating. What does that mean?? I could (and perhaps I will!) devote a whole blog post to this subject, but the TL;DR is that your paper should make an argument. Organize your ideas around your argument.
  • Sort papers into your argument or outline. Look for holes to search for more papers.

Writing

  • Create section headings in your document based on the structure you created
  • Re-read your notes for the papers in each section. If you need to fill in some details, go back to the abstract or appropriate section of the paper.
  • Write up the parts of the research that are important to your argument in your own words, and cite carefully.
  • Add your own synthesis or ideas that support your argument
  • Conclude with a summary, what it means, and why it is important.

Productivity tips: I used the Pomodoro technique this time around, and set goals for myself every day. For example, once I got into the reading groove it’d be something like “read, note, and outline 7 papers.” I’d work in intervals of 20 minutes with a 5 minute break in between for up to 5 hours. I had a hard time keeping track in Zotero of which papers I’d done and not, so I would copy and paste the notes I took for the paper into a Note in Zotero and sort the whole folder by how many notes each paper has. Not perhaps how it’s designed, but it worked :)

I hope this all helps! Do you have ideas about how I can improve my plan? Any particular road bumps you encounter in a lit review? Leave a note in the comments and we can try to make lit reviewing better :)

Continue Reading

MOOC Week 2: In Which I Cry More than Once

This week in our Machine Learning Coursera class, we had an assignment! So far, we’ve just had formative and evaluative assessments, but today we had to actually program something. I am, let’s say, “under-experienced” with programming. Up until yesterday, my programming accomplishments have been: messing with existing HTML/CSS to make my website pretty, a couple codeacademy courses more than a year ago, and a statistics class, in which I wrestled with R every week to find the correct freaking working directory. Once, with lots of help, I made a button in Javascript. It counted how many times it was pressed. It took hours to make and I cried, but eventually, it worked.

the bowl-shaped plot of a cost function
A cost function, J(θ), for a univariate regression model. Here, θ is a matrix of two values, which are represented on the lower axes: a coefficient for one variable x and a y intercept.

Our assignment yesterday involved programming a Cost Function (in ML, a function mapping the sums of squared errors resulting from potential regression coefficients applied to the same data, which are serving here as training data) and the meaty part of a gradient descent algorithm– a program that will grope around on that cost function (hopefully in an orderly way) to find its minimum. The goal of this exercise is to find the point where the error between the model’s predictions and the actual values are the lowest: the best model to predict future data.

Well. As you can imagine, this was somewhat harder than my hard-won Javascript button. It also involved a lot of matrix algebra, which I had happily forgotten existed up until a week ago.

I made my life significantly more difficult by leaving this assignment to the last day– a day on which I had a brunch to go to and a class to teach. I think you can see where this is going?

Fortunately for me, Brandon took the time while I was teaching to do the assignment first. I would have flunked out last night if it weren’t for him. OK, let’s be honest, I would have flunked out in week 1 if it weren’t for him.

What he discovered, through much annoyance on his part and much to my relief, is that the assignment as written looked very long and complicated (15 pages of instructions!) but really consisted of editing 3 files. It took me a while to believe him and stop reading the assignment instructions, but– let it be known across the Internet (and especially among future Coursera students)– he was right! Saved me hours I did not have to spend parsing the assignment doc.

Of course, it was still difficult. Not only was I having to re-google matrix algebra repeatedly, I had never used Matlab before and had forgotten nearly everything I learned about writing code. The assignment took almost all the time I had available (even with a generous amount of help from B). Repeatedly running code and getting “inner dimensions must agree” was abundantly frustrating. I didn’t have time to take a break and recoup or calm down or be grateful for my progress– I had to get the assignment in by midnight. This is all complicated by my false and self-fulfilling belief that I am inherently bad at math and my long-running battle with a paralyzing fear of failure. By the time I submitted the assignment, I didn’t feel much relief or accomplishment– I felt I was about 11 years old, crying at the dinner table with my dad, trying to get through my algebra homework.

Obviously, we can’t have this happening every weekend for the rest of the summer. So here’s the new plan:

When I feel frustrated, I will take a break. I’ll get a glass of water, take a walk, or lay down for a bit and encourage myself. Remind myself of all the benefits of not getting something right the first time.

We aim to get the assignments done by Tuesday. They are due Sunday night, so we will have plenty of time to be kind to ourselves.

We will keep evaluating the plan so we can make it better if need be.

From this week forward, I’ll be trying to see this class as an opportunity to learn to use failure as a tool for learning (in addition to its curricular topics and Matlab benefits : )

 

Continue Reading

Downs and Ups

It’s finals time!

I thought I’d throw up a quick post to talk about what the end of the semester looks and feels like, rather than waiting until after I’ve recovered.

Here’s what I’ve been up to:

  • At the end of our first year, we do a review of our progress with a committee. It went well, but compiling the materials consumed a lot of time and thought. The process and the review was a very productive exercise in thinking about my career, so it was worth it. I may to a future post about first year reviews.
  • I took an extra class this term, which I don’t recommend. It was interesting, and supposed to only be IMG_2194two credits’ worth of work, but it turned out to be a lot throughout the semester, and it added significantly to my end-of-semester workload. Today we gave our final presentations, and although the class and its content were great, it’s a relief that that’s over.
  • I bit off more than I could chew for a big final project and I’m excited about the outcome enough that I’m going to do the work instead of scaling it down, but intrinsic motivation is a double-sided coin. That’s how you end up with to-do list like this one.
  • I stayed up really late last night on accident. I wasn’t even doing anything useful, I was relaxing with Gilmore Girls!

So this morning, I had to be at school for a talk and a class. It turned out to be well-worth the short sleep– the talk was about how the idea and practice of trigger warnings have evolved, how they are used, and how they do (or don’t) function in the college classroom.

For some reason I can’t explain, I got really sad in the time between that talk and my class. I’m not used to acknowledging my feelings, so I may never get to the bottom of this one, other than that I was tired and it didn’t take much :)

I tried to take a walk, but the outside is full of people. I sent a text to my partner, but it was early on the west coast, and he was still asleep. Eventually, found a comfortable chair somewhat out of the way, made myself some tea, and meditated a bit.

It helped a lot to acknowledge how I was feeling, and that it was OK. Someone from my lab even asked if I was OK; I lied to him (Sorry!– bad socialization) but I really appreciated it. For what it’s worth, I think he– and anyone in my department– would have understood. After all, they were grad students once.

So the end of the semester is challenging, as you’d expect. But even though most of the stress is my own fault, I wouldn’t trade it. I’m excited about my occupation for the first time ever, and I am confident I can make it (also pretty rare for me :).

To conclude, I’ll share this Facebook post I made last week about this same paper (that of the big to-do list). I am very happy to stand behind it, even on this less-than-easy day.

Screen Shot 2016-05-10 at 10.46.50 PM

Continue Reading

De-slackifying

So far, this is going quite well. I found a few things I was dreading doing and putting off because they were unclear, or complicated, or something I had minorly screwed up in the past and was afraid of.

But avoiding the task doesn’t solve any of those problems (in fact, it exacerbates the latter!) so, once I got my head clear and recovered some of my confidence yesterday, I clarified them, broke them down into next actions, and tackled them this morning.

And you know what?

I didn’t fail or die or anything! My cat even still likes me.

Now, it’s time to go grocery shopping and make a whole bunch of burritos so I don’t have to cook next week 😉

Continue Reading

Slacking

I keep noticing that I’m less motivated, productive, and just feeling less on top of things than I did last term. I worry that I’m slipping into old habits or I’ve lost something key and can’t get it back. What if the new environment and the excitement of a new direction in my life is what allowed me to do well, and that productivity was not a skill I developed or evidence of personal growth, but a fluke of circumstance?

Well, last night (when I was aimlessly wandering the Internet at 3 am, because that’s healthy) I stumbled across an interesting analogy.

One of the ways I’ve been slacking is that I’ve gained back about 10 pounds of the weight I lost a year ago. (This has happened slowly over the time since I moved and started grad school, but I only bought a scale recently ; )

So I am back browsing some of the supportive weightloss forums I used to be on, and last night I stumbled on this post from a user who had also gained back some weight she had lost. Reading the comments helped me realize a few things:

  1. What happened yesterday/last month/over the last year has already happened, and feeling bad about it or denying it won’t change anything.
  2. Even if my slacking off had been an awful evil moral failing, the best thing to do would not be to feel bad about it. It would be to do something different today.
  3. The narrative of “I’ve been doing poorly,” or “I am slacking off,” or (possibly worse) “I am a procrastinator” is not helping. We have a bias toward wanting to be right about the world– if we believe we are slacking, it makes sense that we’d continue to do so. Why aim that tendency against your goals?
  4. Being vulnerable with other people about your experience, even if it’s your perceived failure at something they are all working toward and succeeding at, opens you up for their support and encouragement, and may even make other people who aren’t doing as well as they’d like feel less alone.

So I have to stop telling myself stories about being a slacker or about losing my momentum, and start thinking about what I can change. I can open up to people I can trust (or… the whole Internet?) about feeling like I’m not as productive as I’d like.

And, like my supportive partner reminded me, I can build a rat park for myself so I don’t use endless Internet binges to feel better. I’ll write a future post about the idea of rat-parking and my plans, but we talked through some basics on our weekly chat date this week. One of my favorite ideas was to start writing journal entries in draft blog entries. I love journaling, but it has seemed like it might detract from “real work.” I want to write a blog, but it’s intimidating to sit down and write a full, Internet-ready post in one sitting. Well, this way, I can journal, and if I like it, I can post it. If I don’t, I can save it to read or even polish for posting later. And either way, I get the writing practice and the cathartic experience of processing my thoughts.

So, it’s in the spirit of all of these things that I offer this journal-entry-which-is-now-a-blog-post, and this revamped description of reality:

Graduate school is stressful, and I don’t always eat, work, clean, or even relax the way I want to, but if pay attention to my behavior and feelings, be compassionate with myself, and make adjustments based on that understanding and compassion (instead of shame and guilt) I can look forward to the feeling of accomplishment, confidence in myself, and guilt-free relaxing that comes when I get shit done.

Continue Reading

Winter Break 1.0

Long time, no see! I just got back from winter break. You’d think that I’d have had time to write something, wouldn’t you? Well, that’s sort of what this post is about.

Before winter break started, I was really excited about all the work I’d be able to get done. It was a simple formula:

No class = more time

Right? Not exactly

During the last lab meeting before winter break, we went around the table and talked about what we would do before we met next. I talked with my cohort about what winter break would be like. I planned my trip. I knew it was a norm to joke about how difficult it will be to get work done, but I knew I had bitten off less than I could chew. I look back on my confidence with amusement.

Now I see that having no structure in my time is, for me, a disaster. I did get the main priorities for the break completed, and I’m happy with what I’ve done, but let’s just say that if I didn’t return 5 days ahead of time just before a huge snowstorm that gave me an excuse to not leave the apartment for days on end, I would be significantly less happy.

And those few days were like pulling teeth to get anything done, even though I clearly had the time. Like the total cliche I am, I found other things to do. I cleaned, I cooked– I even baked a loaf of bread! (That part was worth it. delicious.)

It’s important to me to learn from this experience for a few reasons:

First, I will have other breaks, including summer, when I will have to get work done.

Second, and hugely, assuming all goes well, I will have a lot of unstructured time during which to write my dissertation. During that time, I need to write my dissertation, not worry about writing my dissertation and then write it all at once at the last possible moment.

Thirdly, I need to have practice imposing and conforming to structure on my time to be a researcher who succeeds in accomplishing her own priorities, rather than what happens to be urgent at that moment.

After a year or two of arranging my life pretty haphazardly and years of historic failure to handle long term projects for school, I’ve been pleasantly surprised with my productive response to the structure of last semester. What that tells me is that, just like studies suggest, I am capable of improving and building new skills. Now it’s time to extend that to productive use of unstructured time. And I’ll have several, but not unlimited opportunities to practice and refine those habits before dissertation time.

I’ll report on my plans and progress in this area, and I hope you’ll leave your suggestions for being productive with unstructured time in the comments!

Continue Reading

Finals week

Finals week itself is actually way less stressful this term than any term in undergrad or my master’s program, given that my semester-end projects were mostly papers, which were challenging, but due last week. So really, I’m just using it as an excuse for light blogging this week.

I do have a take home final for statistics, from which I thought I would procrastinate briefly to bring you a draft of my personal demand function for Corona lager.

Demand(Corona) = b((T – 75) i)l

T = environmental temperature (Fahrenheit)

i = thirst (arbitrary units on subjective scale from 1 to 10)

l = Do I have a lime? (Yes = 1, No = 0)

b = Have I been to the beach today? (Yes = 1, No = 0)

Alas, it is winter on the east coast, where I live now, so my demand for Corona will be stuck at zero until I leave for California in 6 days (!)

Continue Reading

Why is it so hard to read a journal article??

I can’t solve paywalls, journals, and academese, but I can help you get journal articles properly formatted on your Kindle!

Anyone who has tried to read a journal article on a smartphone or eReader knows:

  1. you can technically read a PDF as is
  2. it is a giant pain in the ass

Here. I’ll show you. image2

Most journal articles not only have margins, but columns, so the writing is itsy bitsy on your screen. Kindle has tried to fix this with double-tap zoom, but if you’ve tried it, you know it’s not as easy as all that. Once you get to the bottom of your zoomed screen, you have to figure out how to get to the next one. Even if the convenient side-tap feature works (when you are in the middle of a column), each column is not easily divisible by kindle-screens, so you often have to scan over text you’ve already read to find your place again. It works in a pinch, but I am getting annoyed just thinking about it.

So here’s the solution: k2pdfopt. Doesn’t that just roll off the tongue? Just check out the website.

So it’s about as modern as the website suggests, but let me reiterate: it works! I can comfortably read a two-column journal article on my Kindle! See? image1

Once you have a habit down for using it, it’s pretty easy. I don’t even use any of the third-party UIs; I run it straight out of Terminal so I can feel like a super villain. You run one PDF at a time (I ignore all the options, just drag and drop the PDF, hit enter twice, and wait) then close the program and open it again for the next PDF(! i know.) You’ll find the PDF names appended with k2pdfopt, then I email them to my Kindle, and there they appear!

Do you have a life hack or a piece of software that makes your academic life easier? Let us know in the comments!

Continue Reading