My central argument in this post is that research is like a choose-your-own-adventure book: there are many routes to becoming a successful researcher. This is also the most difficult part: many studies show that more options actually make us more unhappy (watch some of Dan Ariely’s TED talks). So the good and bad news: there isn’t a recipe for becoming a successful researcher.

Below I sketch some common routes to becoming a good researcher, list some pitfalls, and give some tips along the way.

Three types of researchers

There are three types of researchers.

The manager has a big strategic vision and constructs a large research team to execute on this vision. The manager is involved in big research projects—probably more than one at the same time. They secure big grants, collaborate across institutions, have several postdocs, and shape the direction of big projects through strategic guidance. Think Yoda or Albus Dumbledore.

The lone ranger is the researcher implementing things on their own: the one writing both the code and the paper. They have very few or no students. They might collaborate with other researchers, but they are still usually the one implementing the solution. The eternal postdoc. They pursue their personal interests with minimal delegation. Think Anakin Skywalker or Severus Snape.

Finally there is a researcher falling in between these two extremes: the player-coach. They try very hard to be both hands-on but also to give guidance to others. They sometimes meddle too much. Think Obi-Wan Kenobi or Hermione Granger. (Hermione is an undercover player-coach: she plays her role in the action, but then she also tries to coach Harry and Ron to become better.)

You can be any one of these. All researchers start out as lone rangers while they are doing their PhD. Most of the superstar academics that you are familiar with are probably managers. But there would also be many mediocre managers who should probably stop trying to be managers and move to one of the other types. Becoming a manager is not a sure-fire way to become a superstar researcher.

I see myself as a player-coach. I am not that keen to implement everything on my own, but I like to try and understand the details. During my sabbatical, I tried to be a lone ranger for a bit. While a good experience, I didn’t enjoy it nearly as much as I thought I would. I forgot what it was like to be stuck for several months on a problem—and that is something the loan ranger should be able to do.

I am also not a manager. The most students that I’ve had was eleven, and I tanked. The manager would set things up to deal with ten, twenty or even thirty students. But for me the optimal number is somewhere between five and seven (spanning both research master’s and PhD students). I don’t get value from postdocs: I end up managing the more junior students regardless of whether I have a postdoc or not. So I fall on the player-coach side of the spectrum. This is not a role I consciously chose: I explored having more students and this didn’t work (so I’m not a manager), but I also do not want only one or two students (lone ranger).

Mistakes in research

There is a lot of freedom in research. But this means it also isn’t clear what exactly you should be doing. You can choose your own adventure, but now you actually have to choose it. How do I know what to do? Often it is easier to figure out what definitely not to do. (And it is funny how often we still end up doing the thing we know we should not.) Below are some common mistakes early-career researchers often make.

Not doing research. This sounds dumb. But so many early career researchers say they are hoping for the future when they will have time to do research. That day will never come. Or stated differently: if tomorrow isn’t that day, then that day does not exist. You need to prioritise, despite all the other things going on in your life (like teaching).

Telling students to write papers is not research. Most academics at least have some postgraduate students. What I have seen some academics do as a “strategy” to produce research is to require their students to write papers. For example, they require students to have a written paper at the beginning of their second year. The academic then takes a shotgun and just submits all the papers somewhere. This is not research. This is adding noise. (It is an effective way to get more publications, so if you just care about promotion—which is largely based on quantity of publications—do this.) No. You need to sit with each paper, with each student, and craft the story. Define the research problem, build the argument, construct the experiments, and interpret whether or not the results confirm the hypothesis. Regardless of what type of researcher you are, this needs to happen.

Thinking that you should first read the textbook. Some early career academics think you should first master all the fundamentals and only then can you do proper research, since you will know where the gaps are. Now I am not saying textbooks aren’t useful. But they are most useful when directed at a problem—when you are reading with a purpose. The best way to learn how to do research: do research. With others. Working with a more experienced researcher on a paper will trump reading a textbook for its own sake. You do not learn how to play piano by reading a lot of music theory (although, of course, the theory is useful, so maybe not the best analogy, but you get the point).

Thinking the work will speak for itself. People do not see what you want them to see if you do not tell them what you want them to see. (And even if you do tell them, and they get it, they will probably not think it is as amazing as you think it is. But if you do not tell them, they will definitely not see the point.) You have to craft a story around your research.

Why do research?

Research is a slog. You struggle all the time. You are never sure that what you are working on will actually pan out (and you shouldn’t be, otherwise you aren’t doing real research). You do not get regular feedback. Feedback is important for job satisfaction (Colin Raffel writes quite a bit about this). The feedback you do get is always mediocre (in terms of both the substance of the reviews and the assessment the reviewers make of your work). If I have to go by the reviews my work receives, I should be labelled as a borderline weak reject/accept researcher. Between all this, you always feel like you are missing out on some research direction or project. There is never enough time.

So why do research? The unfortunate thing is that there isn’t a single right answer. I will sketch out some answers and then give my own. Then you can come up with your own.

The Sunday-school answer: To produce new valuable knowledge. To extend human knowledge. To push the boundaries of what we know. This is fake news: no one actually does research for this reason. There might be some researchers who claim they do. But what they then actually mean is, “I want to produce new valuable knowledge with my name on it”. Personally, I also have a need to be recognised. The need to be recognised isn’t bad in itself. But it becomes dangerous if this is your sole reason for doing research. If you really really believe that the point of research is to produce new valuable knowledge, then go and update Wikipedia articles using an anonymous username. That is the most noble thing anyone can do. I have too much pride.

So now some real reasons why people do research.

“[T]he … thing academia and writing is all about: gaining insight and making it public” (Ahrens 2017). This process can be wonderful.

An interesting problem that you just can’t let go. Many writers talk about how writing is really painful, but they just have to get it out of their system. Some researchers are the same with a particular problem.

Because it is fun.

Because it makes you a better lecturer (see below).

A good friend, Simon Muller, once said the following about the difference between industry and research in academia. Industry takes a solution to a problem and optimises it as far as possible, getting every last percentage out of the system. They optimise locally. Academia should be looking for optima that are way out: solutions that industry isn’t even looking at. In a talk I attended, Percy Liang showed a picture of a street lamp throwing a bit of light on a sidewalk on a dark night. He then made the point that academia should be looking in the dark. I would add to Simon and Percy’s message that sometimes academia should also figure out what industry did when they accidentally stumbled onto something that worked. I resonate with this view of the point of research.

Why do I do research? I like being part of the conversation in my field—or at least listening in. I enjoy visiting other campuses. I like going to conferences. I love telling stories with a technical component to it. I like coding (a bit), writing (a bit), and teaching (a lot). I like making small YouTube videos. I enjoy seeing and being part of academia. I do not care about “producing new useful knowledge”. I like making notes, chatting with people, trying to figure out what has not been looked at before, and whether there is a weird way to look at an old problem. I enjoy getting some new technical insight from a student. These small things are the ends in themself. Research—for me—is just means to that end, not the other way around.

You have to come up with your own answer to “why research?” Sorry. And more options do not make us happy. Sorry again.

Research and teaching

I love teaching. And research. But I am eternally unsatisfied: When I am in a heavy teaching semester, I miss the freedom of exploring through research. When I am in a teaching-free semester, I miss the well-defined structure of teaching.

Some researchers at university do not enjoy teaching (especially undergraduate teaching). They do it because they have to tick a box. That’s fine. Students need to learn on their own. So, in an undergraduate course, as long as the students know what textbook to read and what they will be assessed on, I think bad lecturers are fine. But, at the same time, university academics need to realise that they are not only researchers. Otherwise, just go to a pure research laboratory in industry (like Google DeepMind, which still publishes). You do not need to enjoy undergraduate teaching, but then you need to see the point of mentoring at least at some level in your work (maybe at a postgraduate level).

I love teaching. How cool is it to be paid to figure stuff out. Compared to research, teaching is a joy. It is well-defined and there is regular feedback. In the best case, you will put in effort to, for instance, make a practical more fun, and students will appreciate it. Even in the worst case, you still get feedback: a student telling you directly after a class that your explanation was rubbish. You can plan your day and say, “Today I will finish a video summary on linear regression.” And then, by the end of the day, if you plan well, you will have something to show for your work. You have to give your lectures, so you do not need extra willpower to decide to carve out time to prepare your lectures.

Research is not like that. There are no guarantees. But you still need to put in all the hours, otherwise nothing will ever happen. But you need to find those hours yourself, somewhere, and have the willpower to stick to it. I need to realise that this is a big part of why my brain often says, “You prefer teaching over research.”

But research makes you a better lecturer. It forces you to think outside of the confines of your course, to see new developments, see how what you are teaching fits into the bigger story, and to make sure that what you are teaching is still relevant. To be a good lecturer, you need to be passionate about your area. And if you are truly passionate, you will not be able not to do research.

Successful researchers

A successful researcher has a research story—a story they tell themselves as a strategic vision. It is okay if this story changes (its probably a little bit good if the story doesn’t stay the same over a whole career). They have focus. The successful researcher could work on many different things over their lifetime, but they do not work on many different things at the same time: one or two big themes. It is also easy to see why: two problems that you are very excited about is fun, twenty problems that you are very excited about is not fun. Even the manager-researcher would align several distinct activities under a single theme. This is why research talks are part of the job interview process—to see if the candidate has carved out a story. I’ve also heard this being called a “research identity”.

Successful researchers can switch between the high-level and the detail-level, and know when to do so. They understand the story—or the development of the story—but can then, when required, also rapidly plough into the technical details, especially when they see that something is not correct.

Successful researchers follow Joshua Tenenbaum’s rules of thumb for doing research, or at least hits two of these:

  1. Don’t waste time doing research you don’t love.
  2. Don’t waste time doing research that other people can do better than you can. This sounds pretty extreme and can be a bit crushing. I think it can be softened a bit: you should pick research where you have a unique advantage, skill or opportunity.
  3. Don’t waste time doing research that other people in your field won’t care about. (It’s okay, and probably a good sign, if some people won’t appreciate it, as long as enough people will.)

At this point it is worth just asking, what does successful mean? I aspire to be a good researcher. But a while ago I made a conscious decision that I will not become an A-rated researcher (a South African rating for someone at the absolute highest possible level of research, think Richard Feynman). I want to be a middle-of-the-pack researcher. And that is okay. Other people are at the university mainly to teach. I would encourage them to do a bit of research (because it improves teaching), but it is absolutely fine to not have that be their main reason for being here (just like for me, as explained above, research is really just a means to an end).

More loose thoughts on research

It is often better to pick who you are going to work with rather than picking what you work on.

Early on, it is good to explore to figure out which route to take. But you need to keep pruning.

Don’t submit crap. Don’t wait until something is perfect before submitting.

If you are worried about being scooped, you are working on the wrong thing (Dan Jurafsky said this in a talk at TTIC in 2016). If you can explain what you are doing to someone in one sentence and are then worried that they will quickly be able to implement it and beat you to the publication, you are working on the wrong thing.

If you are worried about falling behind (in terms of e.g. technology), you are working on the wrong thing. This is different from the point above. What I am saying is that, if your overarching research vision requires you to stay absolutely up to date with every single thing, then you are working on the wrong thing. This happens, for instance, if your research aim is to be at the top of some leaderboard. If your aim is to beat scores, your life will not be fun.

You will never be able to explore every interesting question, so it is okay to focus on what has been placed in front of you at the moment.