Workshop 3. Advanced CLI and Tools¶
This workshop covers some more advanced features of using the linux command line that you will find very helpful when using a CLI for bioinformatics tasks.
Command line text editors¶
Although there are many tools available for editing text that run on your local machine, sometimes it is convenient or necessary to edit the text in a file from the command line itself.
It is therefore important to have familiarity with at least one of the most
common CLI text editors: nano
, vim
, or emacs
.
These programs accomplish essentially the same thing (editing text files) but go about it in very different ways.
Linked below are short-ish introductions to how each editor works, but as a quick guide:
nano
- the easiest to use, but has the fewest features, will likely be installed on every linux system you usevim
- the steepest learning curve, but extremely powerful when you learn it well, will likely be installed on every linux system you useemacs
- somewhere in between nano and vim with difficulty of use, powerful and highly customizeable, won’t always be installed on every system you use
nano¶
nano is the most basic CLI text editor (though some might disagree) and will be available on nearly every linux-based system you ever encounter.
Basic usage:
- Easy: type
nano
ornano <filename>
on the CLI and start typing Ctl-O
to save (i.e. write Out) a fileCtl-X
to exit
Here is a quick tutorial video of how to use the nano
editor.
Runtime: ~7 min
vim¶
vim is an extremely powerful and mature text editor (it was first released in 1991, but is based on the vi editor releaed in 1976).
It is a modal editor, meaning there are multiple modes of operation, the most important being:
- Normal mode: keys control navigation around a text document and entering other modes
- Insert mode: allows inserting and editing text as in other editors
- Command mode: specify commands for performing text editing functions and routines, e.g. search and replace
When you run vim
, you start in Normal mode.
Basic Normal mode usage:
h
j
k
l
move the cursor ← ↓ ↑ →, respectively0
$
move the cursor to the beginning or end of current line, respectivelygg
G
move the cursor to the top or bottom of the document, respectively/<pattern>
searches the document for the text<pattern>
i
enter insert mode and begin inserting text before the cursor
Basic Insert mode usage:
- most keys behave as in any other text editor, adding or deleting characters to or from the document
Esc
orCtl-[
exits Insert mode and returns to Normal mode
Command mode is entered by pressing :
while in Normal mode.
Basic Command mode usage:
:w
write (save) the current file to disk:q
quit vim:%s/patt/repl/[gi]
replace all occurences ofpatt
in the document withrepl
These videos are a good quick introduction to vim
:
Runtime: ~9 min
Runtime: ~6 min
Runtime: ~6 min
emacs¶
emacs is an extensible and mature text editor (it was first released in 1976, same year as vi, and they have been locked in mortal combat ever since).
Unlike vim, emacs is not a modal editor, and by default typing characters makes them appear on the screen.
Sophisticated editing commands are mapped to sequences of command characters
started with the Ctl-key
or meta key (usually either Alt-key
or
ESC <space> key
.
Basic emacs usage:
C-x C-c
to exit emacsC-x C-f
open (or create) a file for editingC-s C-s
save current buffer (file)
There are many, many, many commands like those above in emacs, far too many to cover here, but this is the first video in a series that covers them well:
Runtime: ~24 min
Advanced command line techniques¶
Despite being text-based, the bash shell is a very sophisticated and powerful user interface.
Note
There are many shell programs, of which bash is the most common. Different shells have different capabilities and syntax, and the following material is specific to the bash shell. If you use another shell (e.g. tcsh or zsh) some of these techniques will likely not work.
These short tutorials cover some of the more useful capabilities of the bash shell, but are hardly exhaustive.
There are many guides and tutorials online for more comprehensive study, just a few:
- Bash Guide for Beginners
- The Bash Academy (cool, but quite incomplete as of 8/3/2017)
- LMGTFY
Part 2¶
- aliases - user-defined shortcuts for longer commands
- environment variables - text values stored in variables
- shell expansion - construct commands by string substitution and special syntax
~/.bash_profile
and~/.bashrc
Runtime: ~10 min
Useful tools¶
There are many, many programs installed by default on linux systems, but only a handful are needed in day-to-day usage.
These are some useful commands that you can explore for yourself:
file <path>
- guess what type of file (i.e. gzipped file, PDF file, etc) is at the location<path>
du
- show the disk usage (i.e. file size) of files in a directory, or summarized for all files beneath a directorydf
- show the disk volumes mounted on the current filesystem and their capacity and usageln -s <src> [<dest>]
- create a symbolic link, which is a file that points to file<src>
and is named<dest>
(same as<src>
filename by default)
The following commands are very useful but warrant some explanation and so are described in a final video:
pushd
andpopd
- modify your directory stack, so you can switch between different directories quicklyfind
- search for files with particular characteristics (e.g. filename pattern) recursively from a given directoryxargs
/fim
- execute the same command on a list of input
Workshop task¶
content/workshops/03_advanced_cli_workshop.rst