Workshop 3. Advanced CLI and Tools

This workshop covers some more advanced features of using the linux command line that you will find very helpful when using a CLI for bioinformatics tasks.

Command line text editors

Although there are many tools available for editing text that run on your local machine, sometimes it is convenient or necessary to edit the text in a file from the command line itself.

It is therefore important to have familiarity with at least one of the most common CLI text editors: nano, vim, or emacs.

These programs accomplish essentially the same thing (editing text files) but go about it in very different ways.

Linked below are short-ish introductions to how each editor works, but as a quick guide:

  • nano - the easiest to use, but has the fewest features, will likely be installed on every linux system you use
  • vim - the steepest learning curve, but extremely powerful when you learn it well, will likely be installed on every linux system you use
  • emacs - somewhere in between nano and vim with difficulty of use, powerful and highly customizeable, won’t always be installed on every system you use

nano

Super fun! For a couple minutes.

nano is the most basic CLI text editor (though some might disagree) and will be available on nearly every linux-based system you ever encounter.

Basic usage:

  • Easy: type nano or nano <filename> on the CLI and start typing
  • Ctl-O to save (i.e. write Out) a file
  • Ctl-X to exit

Here is a quick tutorial video of how to use the nano editor.

Runtime: ~7 min

vim

NB: you don't have to smoke Marlboros to use vim.

vim is an extremely powerful and mature text editor (it was first released in 1991, but is based on the vi editor releaed in 1976).

It is a modal editor, meaning there are multiple modes of operation, the most important being:

  • Normal mode: keys control navigation around a text document and entering other modes
  • Insert mode: allows inserting and editing text as in other editors
  • Command mode: specify commands for performing text editing functions and routines, e.g. search and replace

When you run vim, you start in Normal mode.

Basic Normal mode usage:

  • h j k l move the cursor ← ↓ ↑ →, respectively
  • 0 $ move the cursor to the beginning or end of current line, respectively
  • gg G move the cursor to the top or bottom of the document, respectively
  • /<pattern> searches the document for the text <pattern>
  • i enter insert mode and begin inserting text before the cursor

Basic Insert mode usage:

  • most keys behave as in any other text editor, adding or deleting characters to or from the document
  • Esc or Ctl-[ exits Insert mode and returns to Normal mode

Command mode is entered by pressing : while in Normal mode.

Basic Command mode usage:

  • :w write (save) the current file to disk
  • :q quit vim
  • :%s/patt/repl/[gi] replace all occurences of patt in the document with repl

These videos are a good quick introduction to vim:

Runtime: ~9 min

Runtime: ~6 min

Runtime: ~6 min

emacs

Everything you could ever possibly need, and way, way more

emacs is an extensible and mature text editor (it was first released in 1976, same year as vi, and they have been locked in mortal combat ever since).

Unlike vim, emacs is not a modal editor, and by default typing characters makes them appear on the screen.

Sophisticated editing commands are mapped to sequences of command characters started with the Ctl-key or meta key (usually either Alt-key or ESC <space> key.

Basic emacs usage:

  • C-x C-c to exit emacs
  • C-x C-f open (or create) a file for editing
  • C-s C-s save current buffer (file)

There are many, many, many commands like those above in emacs, far too many to cover here, but this is the first video in a series that covers them well:

Runtime: ~24 min

Advanced command line techniques

Despite being text-based, the bash shell is a very sophisticated and powerful user interface.

Note

There are many shell programs, of which bash is the most common. Different shells have different capabilities and syntax, and the following material is specific to the bash shell. If you use another shell (e.g. tcsh or zsh) some of these techniques will likely not work.

These short tutorials cover some of the more useful capabilities of the bash shell, but are hardly exhaustive.

There are many guides and tutorials online for more comprehensive study, just a few:

Part 1

  • pipelining
  • silencing stdout and stderr with /dev/null

Runtime: ~10 min

Part 2

  • aliases - user-defined shortcuts for longer commands
  • environment variables - text values stored in variables
  • shell expansion - construct commands by string substitution and special syntax
  • ~/.bash_profile and ~/.bashrc

Runtime: ~10 min

Part 3

  • the history command
  • history scrollback
  • history search (Ctl-r)

Runtime: ~4 min

Useful tools

There are many, many programs installed by default on linux systems, but only a handful are needed in day-to-day usage.

These are some useful commands that you can explore for yourself:

  • file <path> - guess what type of file (i.e. gzipped file, PDF file, etc) is at the location <path>
  • du - show the disk usage (i.e. file size) of files in a directory, or summarized for all files beneath a directory
  • df - show the disk volumes mounted on the current filesystem and their capacity and usage
  • ln -s <src> [<dest>] - create a symbolic link, which is a file that points to file <src> and is named <dest> (same as <src> filename by default)

The following commands are very useful but warrant some explanation and so are described in a final video:

  • pushd and popd - modify your directory stack, so you can switch between different directories quickly
  • find - search for files with particular characteristics (e.g. filename pattern) recursively from a given directory
  • xargs / fim - execute the same command on a list of input

Workshop task

content/workshops/03_advanced_cli_workshop.rst