Like most people who spend part of their working life writing code, I’m quite obsessive about the tools I use. Programming requires a complex mix of activities, some of which require detailed thought and creativity, and some of which are dull and repetitive. Every efficiency saving in the repetitive work leaves more time and (more importantly) attention with which to make creative decisions.
Small efficiency savings in the repetitive work add up to something that really does make a surprising difference to your quality of work and stress levels. To a non-programmer, a text editor which automatically inserts a ) every time you write a ( might not seem important, but never having to type that extra character or worry about whether your brackets match, has a cumulative effect.
At the end of every year, when I look back, I notice that a lot has changed in how I worked in the previous twelve months, and usually for the better. Back when I started working full time there was no such thing as cloud storage. If you wanted to store files to access both at work and at home, you had to run a server yourself and either copy files over manually or use something like rsync. Saving list of links to pages you are part way through reading was a particular pain. If you didn’t want to use something like rsync to store a text file with a list of URLs, you could always email them to yourself, but there wasn’t an easy way to share a list of browser bookmarks. Then in 2002 Delicious was released. It was a revelation! Suddenly you could store bookmarks on a public website, where you could see them from any machine, and send them to other people. Then in 2005 Flock came out. Flock was a new style of web browser, for the social web, and it had tools to manage RSS feeds, write blog posts, and so on. Flock was built on the Firefox code base, and used Delicious to manage favourites. Every time you starred a web page to bookmark it, Flock automatically saved your bookmark on Delicious. This way, all your bookmarks were available to you, whether you were using Flock at home or work. In 2008 Google brought out the Chrome browser, and it soon became normal for web browsers to sync bookmarks automatically.
Each of these small changes made life online that little bit simpler and easier, and as the years went on I began to wish that I had written an annual retrospective on how my working life had changed thanks to technology. Now I look back, I’m aware of just how much has changed that I’ve absorbed and forgotten. In that spirit, this post is a summary of the technology I used regularly in the job I have just left, and how I used it.
To make sense of all this, you need some context. My last job was busy, but not just because I had a lot of work, I also had a wide variety of work. This included around eight taught modules a year (two as module leader, a few as second lecturer, the rest teaching in labs), course leaderships for around two hundred and fifty students, between twenty and fifty personal tutees a year, between ten and twenty BSc and MSc project students, union rep duties, membership of various committees, open days, personal research, and so on. In this context, almost any efficiency saving is useful, but particularly in managing email. Below the fold, this post covers:
- Research notes and long term thinking
- Managing references
- Medium term and permanent information
- To-do lists and Kanban
- Managing email
- The same desktop everywhere
- Coding tools
By the end of my time at Wolverhampton I had started writing notes in LaTeX, in the hope that it would be easier to copy notes over to draft scientific papers for editing. I have already written a blog post about this, so I won’t go into more detail here. Suffice to say, I’m still using this system.
In the past, I have only every seriously tried two other methods of note taking: org-mode (when I still used emacs) and hard-backed notebooks. I still sometimes write notes in Markdown for short-term use, for example if I’m making notes in a presentation or meeting. Writing long-hand notes has an added advantage — the notes are easier to remember. However, I found that I could only ever be quite lazy when writing with a pen; I abbreviated so much that typing up my notes later was very time-consuming.
Every academic needs to maintain collections of references to academic literature. This is always fiddly. Each reference you store needs to have enough accurate information with it that another scientist can find the document you are referring to. Every conference and journal has its own style for formatting references. You probably have notes you want to store with each reference, and you probably want to link that reference to a PDF or similar file on a hard drive or similar.
There are a number of tools and file formats for doing this. Because I don’t generally use Microsoft Word (or LibreOffice) to write papers, some common tools are out of reach for me. However, there are a large number of tools which are either available for Linux platforms, or work in a browser. By the time I left Wolverhampton I had started using Paperpile, just as an experiment, and I had ported over several hundred references to my account there (I now have 831 references there). I’m very happy with Paperpile, but if I’m honest I find it difficult to explain exactly why I find it easier than all the other tools I tried, which were:
- BibTeX files maintained by hand with a text editor, such as emacs
- Gnome bibliographer (now extinct?)
Of these, Zenodo and Medeley are the most similar to Paperpile — they all have a browser button that allows you to click and import a paper from a website to the reference manager. Paperpile, like several other tools, attempts to discover bibliographic information from a document or the web, so most of the time you do not have to tell the tool the title, author, etc. of a paper. Some of these tools allow you to store a PDF along with the reference, but Paperpile stores that PDF in your Google Drive, so it is available everywhere. Somehow, these small changes just fit with my workflow, and Paperpile has been the first tool I’ve ever managed to stick with.
Of the rest, Qiqqa is the one tool that stands out as having something unique to offer. Qiqqa is able to show you semantic connections between papers. This allows you to explore topics in the literature, or track the development of a particular author (see pages fourteen onwards in the manual). Many of the other tools commit the cardinal sin of bibliography tools — they asked the user to supply basic information (author, title, date, etc.) instead of trying to discover it from the document or the web. Worst of all, some expect the user to tab through multiple fields in a complicated form, perhaps on the assumption that all GUIs are easier to use than just writing text. For this reason, I bounced around different bibliography tools for years, and always fell back to maintaining BibTeX files, which ended up being quicker to use than a GUI.
When dealing with a large and varied workload, to-do lists are probably the tool that most people think of as being essential. I used to think that too, but I eventually realised that I needed to capture information that was not immediately useful, but not really long term thinking either. Otherwise I spent too much time trying to retrieve this information when I needed it. Having used Tomboy notes for a while (quite happily) I eventually settled on using Google Keep. By the end of my time at Wolverhampton, I had the following notes in Keep:
- My appraisal targets for the academic year, ticked when completed
- A full list of current BSc and MSc project students, ticked when the student has passed
- A list of personal tutors for courses I led
- My full work address (useful for copy-pasting into emails requesting a reference)
- A list of the assessments that I was moderating and the colleagues who were moderating my assessments
- Notes from each of my modules on what I wanted to improve the next time the module runs
- A list of the software packages I would need installed for my students in the following year
In addition, I had some project-specific notes, but these were largely subsumed into my research notes.
For some years I kept all to-do lists on paper. I had two systems that I used for keeping to-do lists. Firstly, if I needed to write a long list of items (for example, everything I have to do this week I wrote the list on a single page, with tick boxes. Secondly, if I need to write a single item within a larger set of notes (for example, as part of some minutes of a meeting) I wrote ACTION across the page next to the to-do. This worked pretty well, but over the years the number of items I needed to capture increased, and reviewing a paper notebook every day became too time consuming.
The first tool I stuck with was Remember the Milk. RtM had a lot of features. You could email the service, it had great keyboard short-cuts, labels, priorities, per-project lists: everything you can think of. Eventually, after many happy years as an RtM customer, I migrated away. Why? Partly because having a hundred or so to-do items is miserable. I was beginning to find that although I was capturing notes well, I wasn’t executing them efficiently. Secondly, I reached a point where relatively few of my tasks were coming to me via meetings and paper, and most were related to emails. At that point, I dropped to-do lists altogether, and kept almost everything in my email system. The few tasks that didn’t fit, I tracked in Google Keep.
I’m sure a lot of people would use Trello for to-do lists, and the Kanban system. I’ve got a lot of sympathy with this way of doing things, and I’ve tried Trello for a number of projects. I found it worked best when I was collaborating with students on highly structured projects. When managing to-do lists for myself, the Kanban method seemed like overkill. Moreover, when working on projects that involved some programming, Trello boards don’t necessarily match with issue lists or other bug reporting tools, and linking to the relevant issue seems like an unnecessary extra step. There are Kanban tools which sit on top of issue trackers, such as HuBoard for GitHub, but I’ve never been in a project with enough contributors to make this work for me.
These days, managing email seems to be a lifetimes work. It seems to swallow everything it touches, and grows exponentially. Years ago, I used to keep all emails in my inbox, but eventually I switched to something like an Inbox Zero policy, because the volume of email I had to manage meant that organising it was a necessity. So, every email coming in to me gets either deleted, sent to SPAM or labelled and archived.
I label and filter a lot of email automatically, but I have several labels that can only be applied by hand. For these, I take inspiration from Active Inbox. Even if you don’t actually use Active Inbox, the discipline the tool imposes is a really interesting guide. The labels I used (and still use) work like this:
- Each project I am involved with (e.g. a module I am teaching on, an event I help to organise, a committee I sit on, etc.) gets a project label. These start with P-.
- Any email labelled !ACTION requires some action from me. When I have done whatever is required, I remove the !ACTION label.
- Any email labelled !WAITINGON needs some action from me that I cannot progress until someone else has replied.
- The !MAYBE label means that an email might be worth acting on, but this isn’t something I have a commitment to do.
Using this system, I know exactly how many tasks I have to complete, just by counting the emails with the relevant label (although I don’t necessarily know when the tasks need to be finished). By the end of my last job, I had over one hundred !ACTION emails at any given time during teaching semesters. At the end of each semester, once marking and boards had been completed, I could usually get that down to between thirty and fifty. Using the system above, I almost never lost track of a task, even if I couldn’t complete tasks on time.
One minor annoyance when moving between different machines, is having different settings and preferences on each one. Whilst working at Wolverhampton I spent about two hours a day commuting on public transport. I would regularly use a desktop at work, a laptop on the train, and a desktop at home in the same day. Apart from moving files around, it was helpful to me to ensure that every tool I used was set up in the same way. To do this, I stored all my configuration files in a folder on Dropbox. To set up a new machine, I just deleted the default configuration files for each tools I used then created (symbolic) links from the Dropbox folder to
All this talk of productivity, and nothing about programming. Seems strange, doesn’t it? Well, in my last job I rarely had a meaningful choice about which languages and tools I could use. A large proportion of the code I wrote was for teaching purposes, and I had a strict policy that I would only ever use the same tools that my students used. That meant Eclipse for Java, Gedit for Scala and C (Linux), Visual Studio for C (Windows), Notepad++ for PHP (Windows), and Jupyter notebooks for Python. proce55ing and App Inventor came with their own environments. We used SVN for version control, and the plan was to move to GitLab as soon as possible. Whenever possible we used the standard unit testing and build tools for whichever language we were using. The exception to this was C, which doesn’t quite seem to have a standard unit testing tool, but I settled arbitrarily on CUnit.
Other tools and languages came and went. For personal work, I used GitHub (after trying Google Code and BitBucket) and emacs or Sublime, and whichever language was appropriate for the job. I wish I could get more excited about these tools. They all have pros and cons, but the truth is that modern programming tools are converging on standard patterns.