A list of mandatory to know Unix commands that I use often with the list of most common parameters.
[(!) Article updated on 21/11/2011.]
The prompt usually ends with the dollar sign ($), in example
If you want to use multiple parameters, add them one by one, in example instead of running
ls -l -a -h, run
Single letter parameters are passed after single dash, word parameters are passed after two dashed, in example ls
-r or `ls --reverse'
2. Operations on files and directories
2.1 Using wildcards (shell globbing)
Globing is the ability of the shell to match patterns to file and directory names. When glob is found as parameter to any command, the shell "expands" it and returns the filenames that match the pattern. If no files match glob, the shell returns the pattern itself to the command where the glob is used. Globbing is done before executing the command. The two most popular glob characters are
* that matches any number or characters and
? that matches one arbitrary character. To not use globbing, the pattern needs to be wrapped in single quotes. The best way to practice globs is by using the echo command, that is used to print its arguments as standard output:
$ echo * afpovertcp.cfg aliases aliases.db apache2 asl asl.conf $ echo *ap openldap $ echo ap* apache2 $ echo *ap* apache2 com.apple.screensharing.agent.launchd openldap $ echo *ap?che* apache2 $ echo '*ap?che*' *ap?che*
Lists directory contests, by default in the current directory.
Running without params:
$ ls api argumentsProcessor.js config.js entities maps
Usually I want to know the detailed description so I use the following additional parameters:
- -l for long description
- -a for displaying also hidden files (beginning with dot)
- -h for displaying file sized in human readable format (shortened)
$ ls -lah total 128 drwxr-xr-x 17 wojsierak staff 578B 9 Jul 12:05 . drwxr-xr-x 13 wojsierak staff 442B 29 Jun 11:44 .. -rw-r--r--@ 1 wojsierak staff 6.0K 27 May 20:32 .DS_Store drwxr-xr-x 14 wojsierak staff 476B 29 Jun 11:44 api -rw-r--r-- 1 wojsierak staff 3.1K 6 Feb 16:03 argumentsMapper.js -rw-r--r-- 1 wojsierak staff 4.1K 21 Nov 2011 argumentsProcessor.js
Running ls in another directory:
$ ls -lah /etc
Other useful parameters:
- -d: list only directories
- -I, --ignore: lists files not matching pattern, in example `ls --ignore='a*' would exclude files starting from letter 'a'.
- -R, --recursive: prints also content of subdirectories.
- -r, --reverse: sorts files in the reverse order.
- -S, --sort=size
- -t, --sort=time: sorts by last modification time
- -u, --time=access: sorts by last access time
- -X, --sort=extension: sorts by extension names
1: 2: 3: 4: 5: 6: 7: drwxr-xr-x 14 wsierak staff 476B 29 Jun 11:44 api ?UUUGGGOOO 00 UUUUUU GGGGGG #### ^-- date stamp and file name ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | \--- File Size | | | | | | \-------- Group Name (for example, Users, Administrators, etc) | | | | | \--------------- Owner | | | | \---------------------- Links count | \--\--\----------------------------- Read, Write and Special access modes for [U]ser, [G]roup, and [O]thers (everyone else) \------------------------------------- File type flag
Type and access:
There are four parts that make this column.
| File | Permissions | | Type | User | Group | Other | ------------------------------- d rwx r-x r-x
- - (dash): regular file
- d: directories
- l: symlink (symbolic link) Less common:
- p: pipe
- s: socket
The permission sets are as follows:
- user: the user who owns the file
- group: the group a file belongs to, any user in that group takes advantage of these permissions
- other: everyone else in the system
- r: read contents of file or read the contents of directory (view files and sub-directories in that directory)
- w: write to a file or writ to the directory (create files and sub-directories in that directory)
- x: execute programs or scripts for the files or enter into the directory
- -: no permission
-r-x------means that the file can be only read and executed but not written to only by an owner
drwxr-xr-xmeans the file is directory, can be read, written to and content can be accessed by the owner and can be read and accessed by users in a group and other users in the system
2.3 Creating files
There are many ways to create a new file directly from the command line without using any text editor.
The simplest way is by using the
touch command that is used to update the modification time stamp, however when the file doesn't exist, it will create an empty one.
$ touch read.me
2.3.2 Standard output from another command
Standard input and output are also called streams. Processes read and write commands from them. To send output of any command to a file instead of terminal use the redirection in one of the two following forms:
Below will create a new file or overwrite an existing file
$ echo 'hello world' > read.me
Below will create a new file or append to an existing file
$ echo 'hello world' >> read.me
2.4 Reading files
There are many ways of reading the contents of a file without editor.
Cat simply prints contents of one or more file.
$ cat read.me lisez.moi
Less command is used to page through large files. It is also used by the
info command that is used for reading manuals. By pressing the
space bar to move forward and the
b key to move backward the user can conveniently control pagination instead of reading just a portion of a text while the rest would be already gone. To quit press the 'q' button. To search for a word type
/searchword for searching forward and
?searchword for searching backward.
$ less read.me
Less can also read input from piped stream:
$ ls -lah | less
2.4.3 Head and Tail
Both commands show n number of lines from a file or input stream, as the name suggests, the head command shows the first n lines and the tail n last lines. Default num of lines is 10, change num by using the -n parameter.
head -n3 /etc/passwd ## # User Database # tail /etc/passwd # ... will print the last 10 lines from the file.
2.5 Copying, Moving, Renaming, Removing files
With all of the commands listed here, the file names or folder paths can be used as well as globs explained earlier.
2.5.1 Copy files or folders
Copying file into another (cloning) with 'cp' command:
$ cp read.me readme.txt
Copying to another folder:
$ cp file1...filen foldernameorpath
2.5.2 Move files or directories
Renaming files with 'mv' command:
$ mv read.me readme.txt
Moving files or directories to another places:
$ cp file1...filen foldernameorpath
2.5.3 Remove files or directories
Removing files with the 'rm' command:
$ rm read.me
Removing directories with its contents that could be other directories and files is done with two parameters:
r for recursive and
f for forcing:
$ rm -rf myfolder
2.6 Navigating directories
2.6.1 Print current working directory
Use 'print working directory' command:
$ pwd /Users/wojciechsierakowski/Documents
-p parameter to show the true full path with symbolic links. /*
2.6.2 Change current working directory
Unix directory hierarchy starts at the root directory marked as
/. There are two type of paths describing location of files or folders:
- absolute - starting with slash: /etc/tmp
- relative from the current position in the hierarchy - without the slash: applications/program
The single dot
. refers to the current directory, if you are in the home/applications folders, then single dot refers to that folder. Most commands refer to the current folder anyway so this is not used very often.
The double dot
.. refers to the parent folder.
Changing current directory to the root folder:
$ cd /
Changing current directory to the applications directory placed in the users' home directory (
~ is a shortcut for home dir):
$ cd ~/home
2.6.3 Create directory
$ mkdir mydirectory
2.6.4 Remove directory
In most cases
rmdir command is useless as if the directory contains files or subdirectories it will print the following error:
$ rmdir mydirectory rmdir: mydirectory: Directory not empty
Instead use the
rm command with
-r recursive parameters as described earlier.
2.7 More advanced file commands
2.7.1 Search for lines in files containing search text
grep command prints the lines from file or files or input stream matching an expression.
Printing lines containing word 'sigma' in the dictionary file /usr/share/dict/words:
$ grep sigma /usr/share/dict/words sigma sigmaspire sigmate sigmatic
Printing lines containing word 'http' in the /etc/apache2 .conf files:
$ grep server /etc/*.conf /etc/dnsextd.conf:// you could choose to allow anyone with a DNS key on your server to /etc/dnsextd.conf:// nameserver address 127.0.0.1 port 5030; /etc/resolv.conf:nameserver 188.8.131.52
The results contain the files containing the searched keywords as well as the lines themselves.
Most commonly used parameters are:
- /- v that inverts the search to display lines not matching
- /- i for case-insensitive matches
Grep used with the
-E options enables the powerful regular expressions search
2.7.2 Getting file format details
$ file IMAG2674.jpeg IMAG2674.jpeg: JPEG image data, EXIF standard $ file NBp3Stories.xlsx NBp3Stories.xlsx: Zip archive data, at least v2.0 to extract
2.7.3 File difference
find commands shows differences between two files.
For the following two files:
$ cat readme1.txt this is a sample txt file for showing differences $ cat readme2.txt this is the sample text file for showing differences in files
It will print the following output:
$ diff readme1.txt readme2.txt 1,4c1,3 < this < is < a sample < txt --- > this is > the sample > text 5a5 > in files
Notice the differences are only included.
-u parameter prints the output in another format:
$ diff -u readme1.txt readme2.txt --- readme1.txt 2015-07-27 18:10:18.000000000 +0100 +++ readme2.txt 2015-07-27 18:10:48.000000000 +0100 @@ -1,5 +1,5 @@ -this -is -a sample -txt +this is +the sample +text file for showing differences +in files
2.7.4 Finding files (*)
When using find's own pattern matching feature, wrap the search term in single quotes to prevent the shell from using its own globbing.
$ find dir -name file -print
An alternative to
locate command, that uses the index that the operating system maintains and updates periodically. It is much faster than
find but the results may not contain the files added recently.
2.7.5 Sorting lines
Sorting in numerical order with
2.7.6 Changing file permissions
There are two main ways to change permission of a file with the
The first method is to add or remove specific permissions to specific group:
- u for owner
- g for group
- o for others
# Adding read permissions to the group $ chmod g+r file # Removing write permissions from the world/others $ chmod o-w file
Probably the more popular way to change permission is the absolute change by using numbers representing bits. First number represents user, second group, third others. Decimal representation represents 3 bits for read, write, execute: 4-2-1.
If the owner should have execute access and no read or write only the last bit is turned on so the decimal value for 001 is 1. If write access should be enabled, only the first bit should be turned on 010 what gives 2 in decimal. For read, write and execute all bits should be enabled 111 what gives 7.
# Read, write, execute access for owner, read and write for group and read for others: $ chmod 764 filename # Read, write for owner, no rights for others $ chmod 400 filename
The shell default permissions are specified with the
umask command, 022 means that all users will be able to see files and directories created by the user.
$ umask 0022
2.7.7 Creating symbolic links
To create a file that points to another file like the shortcuts from M$ Windows use
ln command with the
$ ln -s ~/projects/programs/ myprograms $ ls -lah lrwxr-xr-x 1 wsierakowski staff 61B 28 Jul 11:21 myprograms -> /Users/wsierakowski/projects/programs $ cat myprograms cat: myprograms: Is a directory
Notice that when trying to see the content of the symbolic link, the user is actually moved to the destination, in this case a directory. To view where the link points to use
ls -l. If the symlink file is copied to another location, the path will become inaccurate and the link will not be pointing correctly to the target anymore.
ln without the
-s parameter to create hard links. (TBC)
3. Other basics
3.1 Getting help
The most popular way for getting help is to use the manual pages
# Finding documentation for the less command $ man less # Finding all documentation pages containing the keyword $ man -k less PPI::Token::End(3pm) - Completely useless content after the _|_END_|_ tag less(1) - opposite of more less(3pm) - perl pragma to request less of something
Right after the manual page name, there is a section number listed in the brackets. Also many books or internet sources refer to a section of man when talking about particular manual pages. The less command we use in terminal appear to be in the section one (1) in the results above. Here is the list of all manual sections:
|3||Library functions, covering in particular the C standard library|
|4||Special files (usually devices, those found in /dev) and drivers|
|5||File formats and conventions|
|6||Games and screensavers|
|8||System administration commands and daemons|
Searching for help in a particular section:
$ man 1 less $ man 5 passwd
Alternative to the
man command is the
info command from the GNU project.
3.2 Shell and Environment variables
Variables are used to store string data to use them in scripts or to control the way how shell or some commands behave.
Assigning to shell variable is done with the equal sign (no spaces around the sign), the variable can be accessed by using the dollar sign:
$ MYVAR=hello # define shell variable $MYVAR $ echo $MYVAR hello
In addition to shell variables, there are also environment variables. The main difference between them two is that an environment variable is a globally available, in a program and its child programs. A shell variable is only available in the current shell. To make a shell variable available as an environment variable, use
export VARNAME (without dollar $).
To list all environment variables, use
env and to list all shell variables, use
$ MYVAR=hello # define shell variable $MYVAR $ echo $MYVAR hello $ env | grep MYVAR # note: no output $ export MYVAR # turn $MYVAR into an environment variable $ env | grep MYVAR MYVAR=hello
Another way to define an environment variable:
$ export MYVAR=hello $ echo $MYVAR hello $ env | grep MYVAR MYVAR=hello
Shell variable PS1 is used to define how the shell prompt looks like:
$ set | grep PS1 PS1='\h:\W \u\$ '
Environment variable LESS is used by the
less command to set the default options. Man pages for commands contain section ENVIRONMENT VARIABLES describing what are the env variables that can be used for that purpose.
Some of the
shell variables evaluate to data that may come useful:
- $$ gives current shell process PID
3.3 Command path
There is a special environment variable dedicated to keep a list of system directories that the shell use when trying for locate commands. When the user runs any program from the terminal, the shell will check the
PATH variable first and only if the command is not found in the locations listed there, it will search for the command in the current working directory.
$ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin $ env | grep PATH PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin
If the shell locates the command in one of the locations, it will not search others.
To add a new directory to the existing
PATH variable, append it to the beginning if you want that location to be searched first or to the end if you want that location to be searched last by using the colon sign. The changes aren't permanent since opening new shell will use the default
# add mypath to the beginning $ PATH=mypath:$PATH # add mypath to the end $ PATH=$PATH:mypath
3.4 Redirecting and piping input and output
To send the output of any command to a file instead of terminal use the single or double redirections
>>. Single overwrites a file if it already exists and double appends to it.
$ ls > lsoutput.txt $ ls >> lsoutput.txt
To send output from one command to another use the pipe character
$ grep https /etc/* | wc
To redirect also standard error in addition to standard output use the
2> redirection where 2 is the streamID of the standard error, and 1 is streamID of the standard out.
$ ls nonexistantpath > stdout.txt ls: nonexistantpath: No such file or directory $ ls nonexistantpath > stdout.txt 2> stderr.txt
In the first case the error was printed as stderr was printed to the terminal, in the second case the same error message got printed to the stderr.txt file.
Saving both outputs to the same file:
$ ls nonexistantpath > out.txt 2>&1
To redirect the input use the
< operator. It is not common to use input redirection as most of the commands accept path to a file as one of the arguments.
# both do the same $ cat < /etc/passwd $ cat /etc/passwd
3.5 Controlling processes
3.5.1 Listing processes
To get a list of running processes with their details such as:
- process IDs (PID)
- terminal device they are running in (TTY)
- process status (STAT)
- amount of CPU time a process used (TIME)
PID TTY TIME CMD 507 ttys003 0:00.12 -bash 28893 ttys005 0:00.04 -bash 38675 ttys005 0:00.40 mongo
Often used parameters:
xto show all running processes for the user calling the command
axto show all running process for all users
uto include more detailed information
wto show full names for commands
Most often the
ps command is used with
aux parameter and it's output is redirected to
grep to find a particular process by name:
$ ps aux | grep apache
3.5.2 Terminating and killing processes
kill command, the user can send a signal to the process with certain
PID. There are many type of signals:
- TERM used by default by the
killcommand to terminate a process
- STOP to freeze
- CONT to resume STOPped process
- KILL (9) to kill a process
$ kill pid # same as above: $ kill -TERM pid # immediately kill: $ kill -9 pid # or: $ kill -KILL pid
Both numbers and names of signals are accepted.
The KILL (9) signal should be used with caution as it is the only signal that immediately stops the process and removes it from the memory without giving time to remove temporary files or do other maintenance job. Processes usually wait for signals and in case of receiving TERM in example, they perform some ending task before closing.
Using CTRL+C while a process is running in the terminal is equivalent to sending INT signal.
3.5.3 Running processes in background
To detach process from the shell to get back control over it we can put a process in the background by using ampersand. This is usually done for the processes that will take significant amount of time to complete their work like compressing a file. The process will also continue to run after logging out or closing the terminal window. To avoid random stdout text from the processes run in background, redirect their output to a file.
$ tar cvf arch.tar myfiles &  50845
The shell returns PID of the process and returns the prompt.
fg to bring the process back.