Writing an Effective Literature Review Quickly(ish)

Chances are that, at some point during your research career, you will be asked to write a literature review.

It’s a great way to get to know the existing research on a topic, so I suggest you agree. And then, perhaps you will sit down at your computer with a stack (or perhaps a Zotero folder) of papers and stare at your screen for a long time before you break down and Google “how to do a literature review.”

Every time I sit down to write a literature review (even if I am in the middle of the same project as last time) it seems like I forget how to do it! So when my advisor asked me a few weeks ago to write one, I googled (again). In the interest of not forgetting next time, I took notes on my process. In the interest of sharing the wealth, I am going to write it out here for you :)

What is a literature review? This seems like a silly question for someone to ask if they’ve already written one, but I think it’s worth addressing. I’ve heard of a few things referred to as literature reviews: annotated bibliographies, “related work” sections of papers, stand-alone documents, and published synthesis papers, for example.

Before you start your lit review project, it’s worth trying to assess what kind of document you need. Often, your finished project will need to be a cited piece of prose, which will stand on its own or lay out your “research space” in a larger paper. Make sure that, as you read, you are aware of the context into which this paper will fit– look for rhetorical links to the ideas you’ll present in other parts of your papers.

If you are trying to build familiarity with the research or gather relevant papers for multiple, related projects, your best bet might be an annotated bibliography– something flexible or modular that you can borrow pieces from without collapsing a rhetorical structure. In my case, that’s exactly what I did. I even used workflowy, so I could easily move things around and use an infinite number of levels in my outline (I’ve been known to get carried away).

Process Overview

I search for keywords in Google Scholar. I supplement GS with my University’s worldcat subscription for books or hard to find titles. I read abstracts to find relevant articles, and save those to Zotero using the browser plugins. During this stage, I keep an eye out for new keywords in those articles that I can use to search.

Once I have a handful of seed articles (really, just a few), I start reading the introductions. These aren’t the most important parts to get a sense of the article’s place in the literature, but the are great for finding more papers, related topics, and keywords.

Before I go back and grab all these new papers, I set up a draft outline based on what I’ve read so far and the format I need. Then, I go back and read the methods (if necessary), results, and conclusions of the seed papers and sort them into my outline with a brief description of what you think you will need out of this paper. I put each paper in multiple categories sometimes– it may have more than one contribution and interesting methods, for example. This outline is going to evolve once we get more papers, but it’s worth having some kind of basic model in your head as you keep searching.

Now, I go back and find those cited papers, search for my new keywords, and look at related topics. This is an iterative process, so don’t get caught up on things being in order. For each batch of new papers, read what you need, write yourself some notes, and stick them in your outline.

For any literature review, you’ll need to be looking for larger models that tie papers together, gaps in the research, and disagreements among authors that can become or support your argument: keep an eye out and keep notes.

Then, if you need to, write it up!

Here are some tips and specifics

First, give yourself more time than you think it’ll take. When I was planning the lit review beforehand, I titled the workflowy list I made to keep notes “Lit review in 1 week.” LOL no.

Searching for literature

  • Track your keywords. You may need them later! Also, you can avoid redundant searches.
  • Use the cited by feature. Google Scholar tells you how many people cited a paper, and if you click on that citation count, it will bring you to a list of papers that have cited it. These papers may be similar and relevant, or not! You can search within citing papers by clicking the checkbox underneath the search bar.

Reading

  • Read what you need. I said this before, but it’s worth repeating. Only read in depth what you need for your purposes. If you’ve already read 15 similar intros, start skimming and skipping them. Keep an eye out in the abstract for contributions and novelty: did they use a method you haven’t seen yet? Better read that section.
  • Take notes. These are just for you, so don’t spend too much stress on them. Especially when you are just getting into an area of literature, it’s not always obvious when someone messes up, when there’s a big gap, or when it’s you who is misunderstanding. Feel free to ask these questions and make these critiques in your personal notes; they will be really useful later when half of them turn out to be useless, but the other half can show you the gap your research can address, or just make you look like a great critical thinker 😉
  • Note themes especially early on. These will help you make your structure!

Structure

  • Map themes. When you’ve read a handful of papers, start mapping the themes if an outline structure isn’t already obvious (or dictated by your format). I used the pencil on dead tree format for that this time, and it worked well. I did find myself wishing I had a white board for this step, though. You can draw lines to represent connections, try to group the ideas into themes, or anything else that seems useful to you to start framing your paper. What you want out of this is an idea of what the argument of your paper will be and the skeleton of your outline.
  • Find a story. I hear this all the time in academic writing advice, and I honestly find it a little frustrating. What does that mean?? I could (and perhaps I will!) devote a whole blog post to this subject, but the TL;DR is that your paper should make an argument. Organize your ideas around your argument.
  • Sort papers into your argument or outline. Look for holes to search for more papers.

Writing

  • Create section headings in your document based on the structure you created
  • Re-read your notes for the papers in each section. If you need to fill in some details, go back to the abstract or appropriate section of the paper.
  • Write up the parts of the research that are important to your argument in your own words, and cite carefully.
  • Add your own synthesis or ideas that support your argument
  • Conclude with a summary, what it means, and why it is important.

Productivity tips: I used the Pomodoro technique this time around, and set goals for myself every day. For example, once I got into the reading groove it’d be something like “read, note, and outline 7 papers.” I’d work in intervals of 20 minutes with a 5 minute break in between for up to 5 hours. I had a hard time keeping track in Zotero of which papers I’d done and not, so I would copy and paste the notes I took for the paper into a Note in Zotero and sort the whole folder by how many notes each paper has. Not perhaps how it’s designed, but it worked :)

I hope this all helps! Do you have ideas about how I can improve my plan? Any particular road bumps you encounter in a lit review? Leave a note in the comments and we can try to make lit reviewing better :)

Continue Reading

Summer updates

I am hanging on in the Machine Learning MOOC! Week 9; barely! I am trying to keep my eye on my progress and focus on how much I have actually learned instead of how frustrating it is to repeatedly get something wrong for several hours in a row.

I was talking to some friends about defining characteristics of success in their occupation, and persistence came up as a candidate for computer science. Persistence is definitely a weak spot for me, and I’m learning both machine learning and programming simultaneously, so this class has been a real trial. Hopefully I will be able to look back on it in the future and say, “if I can finish that machine learning class, I can do this!”

I have mostly stopped weekly planning. Travel, having guests in town, committing to a new project, and trying out a new objective setting plan have colluded to derail that project. I learned a lot, though, and I wholly recommend trying it, even if the lessons you learn are from what stops you 😉

Speaking of which, I’ve started daily objective planning. As part of testing an app my partner is building, I’m setting objectives to further my life’s current projects. I can’t wait to share more info about that app with you, but for now, the practice of daily objective setting generally has been really effective for me. Putting each objective under a large-scale project that I believe is important has been just as motivating as the crossing off of them each day. I have a bigger-picture view of my life and a better ability to balance the urgent with the important.

I fly back startlingly soon and have a lot to do! I’m working on a literature review, so I will have lot of actual science content to share with you shortly :)

Continue Reading

How I learned to stop worrying and love the bubble bath

Remember when I said I was going to reflect weekly? Well, the weekly part hasn’t worked exactly, BUT the spirit of the project (continuous improvement to my schedule for work and play) has been consistent and beneficial.

Here’s the biggest thing I’ve learned:

I must make room for my humanity.

This may sound weird and woo to you, which is fair–  you can reword it however you like. The point is, about two weeks ago, I was getting into daily self-soothing spirals of unproductive and un-fun messing around. First, it was reddit and facebook. Literally, I would read one until I ran out of content, then switch to the other. Then, I downloaded “Two Dots.” Ya’ll, this is important: do not download Two Dots. After I deleted Two Dots, I went back to reddit and facebook.

This happens for me when I am stressed out or overwhelmed by something that I should be doing, or if I’m not taking care of myself. Not only was my work suffering, but my relationships were, too. I was grouchy because I felt guilty, and I felt more guilty for being grouchy.

So, I downloaded an extension for Chrome called “Block Site” and instructed it to redirect requests for “facebook.com” or “reddit.com” to list I created in Workflowy of things to do that I would enjoy and would make me feel refreshed. These things were a plausible and attractive alternative to social media loops, and have an ending point– something to accomplish.

There are big things on this list– like “hike Mount Woodson (for real this time),” but they are mostly small things. I could walk to a nearby grocery store for a fruit popsicle and go read with in in a nearby park. I could paint my nails, or watch a TED talk, draw or color. I could read in the bath or on the pier. Or I could write in this blog.

I feel so much better when I’m done with one of these things, and by the time I’m back, I’m usually ready to get back to work. If I’m feeling up for it, my “work” reading is on the same Kindle as the Neal Stephenson novel I’m into right now, so I can adjust my level of work based on my level of energy.

I was trained (as I’m sure many of you were) by school to “NO EXCUSES WORK RIGHT NOW ALL THE TIME.” That doesn’t mean I didn’t procrastinate, it just meant I felt a constant hum of guilt and worry when I did anything other than work. This ethic does not allow for being kind to yourself, discourages you from trying something you might fail at, and it doesn’t prioritize reading your mind and body’s signals that are telling me how much work I can make myself do. When I was ignoring these signals and buying into the guilt-driven work ethic, I would feel bad for so much as getting out of my chair. Therefore, when I felt tired or unsure, I would stay put. And I would open a new tab. Of course, all I had to do was type “f” and my browser knew what to do (-acebook.com) from there!

So far, this has made a big difference in my work and state of mind. Notably, I haven’t been on reddit at all in 2 weeks. I am nicer to be around, and more likely to be available when my friends invite me somewhere. Also, there’s a fancy grown-up coloring book on its way from amazon today : D

Continue Reading

MOOC Week 2: In Which I Cry More than Once

This week in our Machine Learning Coursera class, we had an assignment! So far, we’ve just had formative and evaluative assessments, but today we had to actually program something. I am, let’s say, “under-experienced” with programming. Up until yesterday, my programming accomplishments have been: messing with existing HTML/CSS to make my website pretty, a couple codeacademy courses more than a year ago, and a statistics class, in which I wrestled with R every week to find the correct freaking working directory. Once, with lots of help, I made a button in Javascript. It counted how many times it was pressed. It took hours to make and I cried, but eventually, it worked.

the bowl-shaped plot of a cost function
A cost function, J(θ), for a univariate regression model. Here, θ is a matrix of two values, which are represented on the lower axes: a coefficient for one variable x and a y intercept.

Our assignment yesterday involved programming a Cost Function (in ML, a function mapping the sums of squared errors resulting from potential regression coefficients applied to the same data, which are serving here as training data) and the meaty part of a gradient descent algorithm– a program that will grope around on that cost function (hopefully in an orderly way) to find its minimum. The goal of this exercise is to find the point where the error between the model’s predictions and the actual values are the lowest: the best model to predict future data.

Well. As you can imagine, this was somewhat harder than my hard-won Javascript button. It also involved a lot of matrix algebra, which I had happily forgotten existed up until a week ago.

I made my life significantly more difficult by leaving this assignment to the last day– a day on which I had a brunch to go to and a class to teach. I think you can see where this is going?

Fortunately for me, Brandon took the time while I was teaching to do the assignment first. I would have flunked out last night if it weren’t for him. OK, let’s be honest, I would have flunked out in week 1 if it weren’t for him.

What he discovered, through much annoyance on his part and much to my relief, is that the assignment as written looked very long and complicated (15 pages of instructions!) but really consisted of editing 3 files. It took me a while to believe him and stop reading the assignment instructions, but– let it be known across the Internet (and especially among future Coursera students)– he was right! Saved me hours I did not have to spend parsing the assignment doc.

Of course, it was still difficult. Not only was I having to re-google matrix algebra repeatedly, I had never used Matlab before and had forgotten nearly everything I learned about writing code. The assignment took almost all the time I had available (even with a generous amount of help from B). Repeatedly running code and getting “inner dimensions must agree” was abundantly frustrating. I didn’t have time to take a break and recoup or calm down or be grateful for my progress– I had to get the assignment in by midnight. This is all complicated by my false and self-fulfilling belief that I am inherently bad at math and my long-running battle with a paralyzing fear of failure. By the time I submitted the assignment, I didn’t feel much relief or accomplishment– I felt I was about 11 years old, crying at the dinner table with my dad, trying to get through my algebra homework.

Obviously, we can’t have this happening every weekend for the rest of the summer. So here’s the new plan:

When I feel frustrated, I will take a break. I’ll get a glass of water, take a walk, or lay down for a bit and encourage myself. Remind myself of all the benefits of not getting something right the first time.

We aim to get the assignments done by Tuesday. They are due Sunday night, so we will have plenty of time to be kind to ourselves.

We will keep evaluating the plan so we can make it better if need be.

From this week forward, I’ll be trying to see this class as an opportunity to learn to use failure as a tool for learning (in addition to its curricular topics and Matlab benefits : )

 

Continue Reading

First summer weekly reflection

Today I did my first reflection on my weekly schedule. As I explained last week, I am trying to continuously improve my work habits so that I am as effective as possible by the time I have a huge, independent dissertation project on my plate in a year or so.

Here’s what I observed this week:

  • I did not stick to my schedule on weekends. Although my Saturday is indistinguishable from my Monday, the same cannot be said for my friends! I want to maintain and improve my social connections, which I rarely have the opportunity to do since my move, so it’s important to me to carve out time for friends and family.
  • My starting time is manageable. My first draft of the weekly schedule included 5 hours of work per day, 7 days a week. I worked from 11:30 am to 4:30 pm. I started that late so I could enjoy staying out or up late on an occasional “work night” since every night is a work night :) I found it easy to be working by 11:30, and could consider moving it up a bit if necessary.
  • I don’t need a lunch break. I had scheduled one in last week’s draft, but didn’t use it. I will get rid of it, and I will think this week about breaks. Studies show that they are important for productivity and health, so I will need to include them.
  • Ambiguous work tasks tank my productivity. I spent nearly entire work day completely wasting my time until I realized that the task on my plate was intimidating and confusing because I didn’t know what specific things to do to accomplish it best. “Work on the paper” is too vague for me.

Here’s how I addressed each observation:

  • Weekends. This one is an open question still.Each draft of my schedule includes a section for “open questions” to acknowledge and focus my attention on unsolved problems. I may not have an answer for every problem right away, and I refuse to let the perfect be the enemy of the good in this project. I had to adjust my weekend schedule anyway because I took on a class on Saturdays, so I’ll experiment with this new schedule and see whether it’s easier or harder. I have another idea to solve this waiting in the wings, but I want to change my schedule incrementally so I can get the best picture of what actually works.
  • Starting time. I decided to stick with the 11:30 starting time for another week to solidify the habit of daily work at a comfortable time, but I will consider moving it earlier for the following week. My goal for evaluating start times will finding one that supports the most productivity during the day, not starting as early as possible for earliness’ sake.
  • Breaks. I eliminated the lunch break and brought in another project that had originally been relegated to free time. I’ve added breaks to my open questions and plan to observe my needs and existing habits for breaks closely over the next week.
  • Ambiguous work. This last week, when I wasted a day I redeemed it near the end by using GTD’s project planning technique to clarify my goals, define my desired outcome, and create concrete, goal-oriented “next actions.” This cleared up my thoughts and the next day, I was back on track.

Draft 2 is on paper!

Continue Reading

Summer plans!

I’m trying a new thing this summer. I want to meet some specific goals in terms of each of three priority projects this summer, but I struggle to remain productive with unstructured time.

I’ve devised a plan for my work days that will support those goals and will help me build on what I learn so that when I have to, say, write a dissertation, I won’t have to figure it out from scratch.

Here’s the plan.
1. Get up to date in GTD. GTD is a productivity system laid out in the book “Getting Things Done” by David Allen. I have tweaked it slightly for my own needs and implemented in in workflowy. The system and software work really well with each other and with my mind, when I am actively using them. The end of this last semester threw me off a bit, so my first goal will be to get my list back up to date. This included narrowing down all the projects I am interested in to the ones I am planning to actively work on this summer. It also includes setting up a time for a “weekly review” so that I keep my list up to date and in line with my priorities.

2. Plan out my week. Although it’s not specifically sanctioned by GTD, in order to make sure I’m spending my time in appropriate proportions on my projects (not just on what seems most urgent or interesting that day) I’ve scheduled out work time for each of my projects during the week. I plan to work about 5 hours per day, 7 days a week.

My first draft of the plan started with a list of the projects I’d identified as important. I allocated a number of hours per week to work on each, and then drew a map of each of the seven days. Along with blocks for each project, I included regular calls, lunch, and maintenance tasks like cleaning. I also have a space for “Open Questions” on the planning page that will serve as prompts for the next step.

3. Reevaluate. On the same day as my weekly review, I have time planned to write about what I’ve learned about working when I have autonomy over my time. This will give me a time to reflect and adjust my plans if I need to.  A week will give me a chance to test out every aspect of the plan before altering it, but each new plan will be lower-pressure: it’s not My Life Plan, it’s just what I’ve committed to for the week.

The written reflections will help me learn week-to-week and help me understand the effects of each change. Hopefully, by the time this summer is over, I will have not only a solid routine established, I will also have a better understanding of how I work and what my options for a routine are so that I can apply all of that knowledge to new circumstances in the winter, next summer, and during my dissertation work.

Commit, test, reflect, adjust. That’s the plan.

I’ve got time planned in my week for blogging, so I will keep this up to date :)

Continue Reading

Downs and Ups

It’s finals time!

I thought I’d throw up a quick post to talk about what the end of the semester looks and feels like, rather than waiting until after I’ve recovered.

Here’s what I’ve been up to:

  • At the end of our first year, we do a review of our progress with a committee. It went well, but compiling the materials consumed a lot of time and thought. The process and the review was a very productive exercise in thinking about my career, so it was worth it. I may to a future post about first year reviews.
  • I took an extra class this term, which I don’t recommend. It was interesting, and supposed to only be IMG_2194two credits’ worth of work, but it turned out to be a lot throughout the semester, and it added significantly to my end-of-semester workload. Today we gave our final presentations, and although the class and its content were great, it’s a relief that that’s over.
  • I bit off more than I could chew for a big final project and I’m excited about the outcome enough that I’m going to do the work instead of scaling it down, but intrinsic motivation is a double-sided coin. That’s how you end up with to-do list like this one.
  • I stayed up really late last night on accident. I wasn’t even doing anything useful, I was relaxing with Gilmore Girls!

So this morning, I had to be at school for a talk and a class. It turned out to be well-worth the short sleep– the talk was about how the idea and practice of trigger warnings have evolved, how they are used, and how they do (or don’t) function in the college classroom.

For some reason I can’t explain, I got really sad in the time between that talk and my class. I’m not used to acknowledging my feelings, so I may never get to the bottom of this one, other than that I was tired and it didn’t take much :)

I tried to take a walk, but the outside is full of people. I sent a text to my partner, but it was early on the west coast, and he was still asleep. Eventually, found a comfortable chair somewhat out of the way, made myself some tea, and meditated a bit.

It helped a lot to acknowledge how I was feeling, and that it was OK. Someone from my lab even asked if I was OK; I lied to him (Sorry!– bad socialization) but I really appreciated it. For what it’s worth, I think he– and anyone in my department– would have understood. After all, they were grad students once.

So the end of the semester is challenging, as you’d expect. But even though most of the stress is my own fault, I wouldn’t trade it. I’m excited about my occupation for the first time ever, and I am confident I can make it (also pretty rare for me :).

To conclude, I’ll share this Facebook post I made last week about this same paper (that of the big to-do list). I am very happy to stand behind it, even on this less-than-easy day.

Screen Shot 2016-05-10 at 10.46.50 PM

Continue Reading

De-slackifying

So far, this is going quite well. I found a few things I was dreading doing and putting off because they were unclear, or complicated, or something I had minorly screwed up in the past and was afraid of.

But avoiding the task doesn’t solve any of those problems (in fact, it exacerbates the latter!) so, once I got my head clear and recovered some of my confidence yesterday, I clarified them, broke them down into next actions, and tackled them this morning.

And you know what?

I didn’t fail or die or anything! My cat even still likes me.

Now, it’s time to go grocery shopping and make a whole bunch of burritos so I don’t have to cook next week 😉

Continue Reading

Slacking

I keep noticing that I’m less motivated, productive, and just feeling less on top of things than I did last term. I worry that I’m slipping into old habits or I’ve lost something key and can’t get it back. What if the new environment and the excitement of a new direction in my life is what allowed me to do well, and that productivity was not a skill I developed or evidence of personal growth, but a fluke of circumstance?

Well, last night (when I was aimlessly wandering the Internet at 3 am, because that’s healthy) I stumbled across an interesting analogy.

One of the ways I’ve been slacking is that I’ve gained back about 10 pounds of the weight I lost a year ago. (This has happened slowly over the time since I moved and started grad school, but I only bought a scale recently ; )

So I am back browsing some of the supportive weightloss forums I used to be on, and last night I stumbled on this post from a user who had also gained back some weight she had lost. Reading the comments helped me realize a few things:

  1. What happened yesterday/last month/over the last year has already happened, and feeling bad about it or denying it won’t change anything.
  2. Even if my slacking off had been an awful evil moral failing, the best thing to do would not be to feel bad about it. It would be to do something different today.
  3. The narrative of “I’ve been doing poorly,” or “I am slacking off,” or (possibly worse) “I am a procrastinator” is not helping. We have a bias toward wanting to be right about the world– if we believe we are slacking, it makes sense that we’d continue to do so. Why aim that tendency against your goals?
  4. Being vulnerable with other people about your experience, even if it’s your perceived failure at something they are all working toward and succeeding at, opens you up for their support and encouragement, and may even make other people who aren’t doing as well as they’d like feel less alone.

So I have to stop telling myself stories about being a slacker or about losing my momentum, and start thinking about what I can change. I can open up to people I can trust (or… the whole Internet?) about feeling like I’m not as productive as I’d like.

And, like my supportive partner reminded me, I can build a rat park for myself so I don’t use endless Internet binges to feel better. I’ll write a future post about the idea of rat-parking and my plans, but we talked through some basics on our weekly chat date this week. One of my favorite ideas was to start writing journal entries in draft blog entries. I love journaling, but it has seemed like it might detract from “real work.” I want to write a blog, but it’s intimidating to sit down and write a full, Internet-ready post in one sitting. Well, this way, I can journal, and if I like it, I can post it. If I don’t, I can save it to read or even polish for posting later. And either way, I get the writing practice and the cathartic experience of processing my thoughts.

So, it’s in the spirit of all of these things that I offer this journal-entry-which-is-now-a-blog-post, and this revamped description of reality:

Graduate school is stressful, and I don’t always eat, work, clean, or even relax the way I want to, but if pay attention to my behavior and feelings, be compassionate with myself, and make adjustments based on that understanding and compassion (instead of shame and guilt) I can look forward to the feeling of accomplishment, confidence in myself, and guilt-free relaxing that comes when I get shit done.

Continue Reading

Winter Break 1.0

Long time, no see! I just got back from winter break. You’d think that I’d have had time to write something, wouldn’t you? Well, that’s sort of what this post is about.

Before winter break started, I was really excited about all the work I’d be able to get done. It was a simple formula:

No class = more time

Right? Not exactly

During the last lab meeting before winter break, we went around the table and talked about what we would do before we met next. I talked with my cohort about what winter break would be like. I planned my trip. I knew it was a norm to joke about how difficult it will be to get work done, but I knew I had bitten off less than I could chew. I look back on my confidence with amusement.

Now I see that having no structure in my time is, for me, a disaster. I did get the main priorities for the break completed, and I’m happy with what I’ve done, but let’s just say that if I didn’t return 5 days ahead of time just before a huge snowstorm that gave me an excuse to not leave the apartment for days on end, I would be significantly less happy.

And those few days were like pulling teeth to get anything done, even though I clearly had the time. Like the total cliche I am, I found other things to do. I cleaned, I cooked– I even baked a loaf of bread! (That part was worth it. delicious.)

It’s important to me to learn from this experience for a few reasons:

First, I will have other breaks, including summer, when I will have to get work done.

Second, and hugely, assuming all goes well, I will have a lot of unstructured time during which to write my dissertation. During that time, I need to write my dissertation, not worry about writing my dissertation and then write it all at once at the last possible moment.

Thirdly, I need to have practice imposing and conforming to structure on my time to be a researcher who succeeds in accomplishing her own priorities, rather than what happens to be urgent at that moment.

After a year or two of arranging my life pretty haphazardly and years of historic failure to handle long term projects for school, I’ve been pleasantly surprised with my productive response to the structure of last semester. What that tells me is that, just like studies suggest, I am capable of improving and building new skills. Now it’s time to extend that to productive use of unstructured time. And I’ll have several, but not unlimited opportunities to practice and refine those habits before dissertation time.

I’ll report on my plans and progress in this area, and I hope you’ll leave your suggestions for being productive with unstructured time in the comments!

Continue Reading