This post belongs to a 3-part series on AWK: Part 1 - Part 2 - Part 3

The goal of this tutorial is to show a few of the cool things that the Unix tool AWK can do with tabular data. We will focus on extracting useful information from a transcriptome—because bioinformatics is cool—, but AWK can do its wonders with any kind of text file.

The tutorial has been carefully designed to make it easy for you to play along. So feel free to copy and paste the commands in your Unix terminal.

Dear me from the past,

You’ll be glad to know that in a few years, high school will be over. You will go out drinking with your friends every weekend, you will travel to twenty different countries, and the number of women that are willing to kiss you will increase by an order of magnitude.

I signed up for a Twitter sometime in 2007, but it wasn’t until a few years ago that I understood its utility. Now, I use it to discover interesting strangers and share ideas that I am passionate about.

I just released an update that makes it easy to hide all the color groups in a scatterplot except one. Simply click on “Show one”. This is very useful to carry out exploratory analyses when there are many color groups. You can zoom around the plot by scrolling up and down with the mouse wheel.

Lately I have been summarizing every paper I read on a single piece of paper. Before I started doing that I would invest a lot of time reading a paper, and two months later it would completely disappear from my brain. The only thing I would remember is if I had found it interesting or not (not the most important information when trying to compile a bibliography). Limiting myself to a single piece of paper ensures that I only record what is important, and the fact that the notes are handwritten lets me triple underscore and use all the curvy arrows I need to make important ideas stand out. My two-month recall is now pretty decent.

I’m going to a seminar tomorrow presented by Tim Lu, one of the authors of a Nature Biotechnology paper titled Synthetic circuits integrating logic and memory in living cells. I decided to read the paper and write up a quick summary.

I remember being six years old and crying in my bed one night. I was outraged about something, and wanted my parents to come check on me. Eventually my dad came and did something completely unexpected. He asked me what I wanted for my birthday.

I immediately stopped crying. My birthday wasn’t due in months. Was this some kind of ruse? Confused, I asked him if he planned to buy whatever I asked for. “We will see”, he said. So I told him, “I want a skateboard”, and went to sleep.

My parents never bought me that skateboard, but I can still remember how fulfilling it felt to simply ask for it.

Twenty four years later, crying is no longer an issue, but whining about not wanting to write a research article sometimes gets the best of me and leaves me procrastinating for hours. I usually love my work, and I love writing, but writing research papers is hard. So I recently decided to apply the birthday wish technique that I learned back in my youth.

The Birthday Wish Technique

1. Grab a blank piece of paper and title it: When I’m done working, I’m going to:

2. Keep it beside you while you work. Proximity is crucial.

3. Whenever you think of something you would rather be doing instead of working, write it down.

Why It Works

Writing about something you are planning to do feels, in a way, like you have already done it. Ask any travel guide fanatic. Sometimes thinking about doing something feels even better than actually doing it. Ask anyone that just ate at a McDonald’s.

The majority of the things that you write in your list will instantly lose their appeal the moment you finish working—they were part of the reality distortion field built by your brain to sabotage your attempts at accomplishing difficult tasks.

Spending a few seconds writing down what you crave, and getting back to work, can mean the difference between spending hours browsing the newsfeeds and finishing that damn report in time for salsa class.

Crafting a sentence is just like solving a puzzle. Except that puzzles are solved by arranging every piece in the box until they form the picture that appears on the cover, and sentences are solved by fumbling with the limited set of words in your repertoire until the shifting idea in your head stops moving.

When I read great sentences, I am always surprised at how graceful they seem and I imagine the hours the author spent crafting them to sound just right. So I decided to start with a sentence I liked and go backwards. Consider this one, written by Simon Winchester in his book Atlantic. It talks about the maritime paintings of a Dutch master:

Even now, nearly five centuries on, these are paintings that grab the attention: invariably there is the hungry sea, its waves translucent green and white-capped, the troughs between them deep and dangerous and all providing a savage contrast to the distant comforts of cow-grazed meadows and church steeples.

Now read a variation I won’t publicly confess to having written:

Even though these paintings are almost five centuries old, they are still striking. The sea is usually stormy and full of tall green waves. It seems very different from the calm meadow and the village that are usually in the background.

Designing presentations on paper lets you apply a magnifying glass of common sense to the logical flow of your ideas, making your inconsistencies go up in flames, like tiny ants.

If you are excited about what you are going to present that you don’t know where to start, picking up a paper will help you sort through the jumble of thoughts running around your head and hone in those that can tell your story most effectively.

Avoid the impulse to start with Powerpoint and give your ideas a chance to fight it out and compete for your attention on paper. Doodling prevents early weak ideas from taking the place of more clever ideas that needed a bit more time to pop the surface of your subconscious.

Next time you have to give a presentation, draw it out on paper first. Use as many emphatic curvy arrows and triple underscores as you want.

My appreciation for anyone who has ever written a research paper has skyrocketed during the past few weeks. I had to begin writing my own article to realize how mind-numbingly hard it is. Sometimes I feel like words refuse to leave my brain unless I force them with a corkscrew. Other times I stumble over booby-trapped paragraphs that kill whatever train of thought I had going.

Whenever I manage to write something coherent is because I take it one paragraph at a time, and when that doesn’t work, one sentence at a time. In his article on scientific writing, George Gopen and Judith Swan share some of the most eye-opening insights I have ever read for crafting understandable sentences. I have spent a couple of hours poring over it and distilled it into two main guidelines:

I randomly found this blog post about T cells. It’s written by Jamie Heather, an immunology PhD student in London, who posts his science thoughts online. He posts a few times a month, but he has kept it up for a few years. I only had to read a couple of his older posts to realize that this is a guy I could grab a beer with.

Then I realized that the only content in my blog was four badly formatted posts with broken links, and that I hadn’t posted anything new in two months. If browsing a few posts from a total stranger living in London made me think he was a cool guy, why am I not making it easy for other people to reach the same conclusion about me?

So I’m starting a 30-day challenge:

Every day, for the next 30 days, I will write something on my blog. It doesn’t have to be earth-shatteringly interesting, funny or clever, just whatever I’m thinking about that day. One paragraph is the minimum, two if I’m feeling inspired.

To find time to write I’ve also decided to quit my morning news-reading routine, which usually lasts an hour. I’ve deleted all the newsy links on my toolbar to make it harder to fall into temptation. I’ve also set up filters on Gmail to automatically remove the newsletters I’m subscribed to from my inbox, and to tag them with a special label so I can check them after I’m done working for the day.

With a bit of luck, 30 days will be enough to develop the writing habit, and I’ll be ready to blog about my stay in French Guiana doing viral metagenomics. I should write a post about that soon.

Until then, wish me luck.