Unix Primer - Basic Commands In the Unix Shell
The shell is a command programming language that provides an interface to the
UNIX operating system. You can access your shell account via the SSH network
protocol from anywhere on the internet. Two free programs installed on
most IUPUI computers are PuTTY and WinSCP. Both are available through
iuware.iu.edu. To connect, use the hostname:
pegasus.cs.iupui.edu.
1. Directories
The shell should start you in your home directory. This is your individual space
on the UNIX system for your files. You can find out the name of your current
working directory by typing:
pegasus{jbeams}1: pwd
/users/username
(The stuff before the colon ':' designates your command line prompt, and you
should type the letters 'p', 'w', 'd', and then "enter" - always conclude each
command by pressing the "enter" key. The response that follows on the next line
will be the name of your home directory, where the name following the last slash
should be your username.) The directory structure can be conceptualized as an
inverted tree.
No matter where in the directory structure you are, you can always get back to
your home directory by typing:
pegasus{jbeams}3: cd
(without specifying a directory name).
From your home directory, create a new subdirectory named "primer" for working
through this tutorial:
pegasus{jbeams}4: mkdir primer
You can remove an empty subdirectory with the following command (but don't do
this right now):
pegasus{jbeams}5: rmdir primer
(Note: if you do remove "primer", please create it again.)
Now change to the "primer" subdirectory, making it your current working
directory:
pegasus{jbeams}6: cd primer
2. Files
Files live within directories. You can see a list of the files in your "primer"
directory (which should be your current working directory) by typing:
pegasus{jbeams}7: ls
Since you just created the directory, nothing will be listed because the
directory is empty. Create your first file using the pico text editor:
pegasus{jbeams}8: pico first
The pico editor fills the entire console window. You can type text and move the
cursor around with the arrow keys; the bottom of the screen presents the
commands available. Type the sentence: "My first file." Then press "^O" (hold
the left "control" key while pressing 'O') and "enter" to save the file, then
"^X" (hold the left "control" key while pressing 'X') to exit pico. Now when you
list your files, you will see file "first" listed:
pegasus{jbeams}9: ls
first
You can view a text file with the following command:
pegasus{jbeams}10: cat first
My first file.
("cat" is short for concatenate - you can use this to display multiple files
together on the screen.) If you have a file that is longer than your 24-line
console window, use instead "more" to list one page at a time or "less" to
scroll the file down and up with the arrow keys. Don't use these programs to try
to display binary (non-text) files on your console - the attempt to print the
non-printable control characters might alter your console settings and render
the console unusable.
Copy file "first" using the following command:
pegasus{jbeams}11: cp first 2nd
By doing this you have created a new file named "2nd" which is a duplicate of
file "first". The file listing reveals:
pegasus{jbeams}12: ls
2nd first
Now rename the file "2nd" to "second":
pegasus{jbeams}13: mv 2nd second
Listing the files still shows two files because you haven't created a new file,
just changed an existing file's name:
pegasus{jbeams}14: ls
first second
If you "cat" the second file, you'll see the same sentence as in your first
file:
pegasus{jbeams}15: cat second
My first file.
The command more is essential for looking at the contents of long files.
Pressing the space bar advances the display by one screen. To quit the display
before the end press the q key. If you are using more to look at a file with the
command
pegasus{jbeams}16: more filename
then you can go back a screen by pressing the b key.
The command cat (concatenate and display) has a broader usefulness than its name
suggests. For example, the command:
pegasus{jbeams}17: cat filename
copies file filename to the standard output, the screen. If more than one file
is given then they are copied to the screen successively. The standard output
can be re-directed to a file so that the contents of two (or more) files can be
concatenated into a third:
pegasus{jbeams}18: cat infile1 infile2 > outfile
The standard output can be redirected to append to an existing file, thus:
pegasus{jbeams}19: cat infile3 >> outfile
adds the contents of infile3 to the end of file outfile. If cat is invoked
without an argument then it copies keyboard input to the screen. To escape from
this enter <Ctrl D> to signal 'end of file'.
The command more is essential for looking at the contents of long files.
Pressing the space bar advances the display by one screen. To quit the display
before the end press the q key. If you are using more to look at a file with the
command
pegasus{jbeams}20: more filename
then you can go back a screen by pressing the b key.
It is possible to 'pipe' the output of one process into the standard input of a
second. So to examine the output of your own program a screen at a time, as it
is produced, the command is:
pegasus{jbeams}21: a.out | more
3. Input and Output Redirection
A process usually has a standard input and a standard output associated with it.
The default for standard input is the keyboard, while standard output goes to
the screen. If there is an error then an error message may go to the standard
error, which also defaults to the screen.
It is possible to change the standard input and output when a command is issued.
For example, for a compiled program to take input from file indata the command
would be:
pegasus{jbeams}22: a.out < indata
If we also want to redirect the output to file outdata the command becomes:
pegasus{jbeams}23: a.out < indata > outdata
"mv" will allow you to move files, not just rename them. Perform the following
commands:
pegasus{jbeams}24: mkdir sub
pegasus{jbeams}25: mv second sub
pegasus{jbeams}26: ls sub
second
pegasus{jbeams}27: ls
first sub
This creates a new subdirectory named "sub", moves "second" into "sub", then
lists the contents of both directories. You can list even more information about
files by using the "-l" option with "ls":
pegasus{jbeams}28: ls -l
-rw-r--r-- 1 username group 15 May 22 16:26 first
drwxr-xr-x 2 username group 512 May 22 17:11 sub
(where "username" will be your username and "group" will be your group name).
Among other things, this lists the creation date and time, file access
permissions, and file size in bytes. The letter 'd' (the first character on the
line) indicates the directory names.
Next perform the following commands:
pegasus{jbeams}29: cd sub
pegasus{jbeams}30: pwd
/Users/username/primer/sub
pegasus{jbeams}31: ls -l
-rw-r--r-- 1 username group 15 May 22 16:55 second
pegasus{jbeams}32: cd ..
pegasus{jbeams}33: pwd
/Users/username/primer
This changes your current working directory to the "sub" subdirectory under
"primer", lists the files there, then changes you back up a level. The ".."
always refers to the parent directory of the given subdirectory.
Finally, clean up the duplicate files by removing the "second" file and the
"sub" subdirectory:
pegasus{jbeams}34: rm sub/second
pegasus{jbeams}35: rmdir sub
pegasus{jbeams}36: ls -l
-rw-r--r-- 1 username group 15 May 22 16:26 first
This shows that you can refer to a file in a different directory using the
relative path name to the file (you can also use the absolute path name to the
file - something like "/Users/username/primer/sub/second", depending on your
home directory). You can also include the ".." within the path name (for
instance, you could have referred to the file as "../primer/sub/second").
4. Programs
Programs are just files that contain lists of instructions in the form of binary
numbers that can be run or 'executed' by the computer when they are loaded into
memory, as opposed to files that contain lines of text. Typically executable
files actually start as text files containing programs written in a high-level
language like C. Compilers read these program files, and build an appropriate
binary executable file. All the commands you have been typing: ls, cd, mkdir,
etc. are just binary files derived from such programs, also often referred to as
binaries or execuables. Many of the programs from which these executables are
compiled are no longer than the programs you will write in this course. In this
class you will learn how to create your own programs, which in UNIX can then be
accessed and used like any other commands in the system. This simplicity and
extensibility is what makes UNIX so powerful.
5. Getting Further Information
On-line manuals are a standard part of Unix. They may be consulted by use of the
man command. For example, to see the on-line information on the chmod command
described in section 4.6, enter the Unix command:
pegasus{jbeams}37: man chmod
The man command uses the more utility to show one screen of the manual at a
time.
6. Concepts of file permissions.
Because there is typically more than one user on a Linux system, Linux provides
a mechanism known as file permissions, which protect user files from tampering
by other users. This mechanism lets files and directories be ``owned'' by a
particular user. For example, because Larry created the files in his home
directory, Larry owns those files and has access to them.
Linux also lets files be shared between users and groups of users. If Larry
desired, he could cut off access to his files so that no other user could access
them. However, on most systems the default is to allow other users to read your
files but not modify or delete them in any way.
Every file is owned by a particular user. However, files are also owned by a
particular group, which is a defined group of users of the system. Every user is
placed into at least one group when that user's account is created. However, the
system administrator may grant the user access to more than one group.
Groups are usually defined by the type of users who access the machine. For
example, on a university Linux system users may be placed into the groups
student, staff, faculty or guest. There are also a few system-defined groups
(like bin and admin) which are used by the system itself to control access to
resources--very rarely do actual users belong to these system groups.
Permissions fall into three main divisions: read, write, and execute. These
permissions may be granted to three classes of users: the owner of the file, the
group to which the file belongs, and to all users, regardless of group.
Read permission lets a user read the contents of the file, or in the case of
directories, list the contents of the directory (using ls). Write permission
lets the user write to and modify the file. For directories, write permission
lets the user create new files or delete files within that directory. Finally,
execute permission lets the user run the file as a program or shell script (if
the file is a program or shell script). For directories, having execute
permission lets the user cd into the directory in question.
7. Interpreting file permissions.
Let's look at an example that demonstrates file permissions. Using the ls
command with the -l option displays a ``long'' listing of the file, including
file permissions.
pegasus{jbeams}38: ls -l stuff
-rw-r--r-- 1 larry users 612661 May 16 15:31 stuff
The first field in the listing represents the file permissions. The third field
is the owner of the file (larry) and the fourth field is the group to which the
file belongs (users). Obviously, the last field is the name of the file (stuff).
We'll cover the other fields later.
This file is owned by larry, and belongs to the group users. The string -rw-r-r-
lists, in order, the permissions granted to the file's owner, the file's group,
and everybody else.
The first character of the permissions string (``-'') represents the type of
file. A ``-'' means that this is a regular file (as opposed to a directory or
device driver). The next three characters (``rw-'') represent the permissions
granted to the file's owner, larry. The ``r'' stands for ``read'' and the ``w''
stands for ``write''. Thus, larry has read and write permission to the file
stuff.
As mentioned, besides read and write permission, there is also ``execute''
permission--represented by an ``x''. However, a ``-'' is listed here in place of
an ``x'', so Larry doesn't have execute permission on this file. This is fine,
as the file stuff isn't a program of any kind. Of course, because Larry owns the
file, he may grant himself execute permission for the file if he so desires.
(This will be covered shortly.)
The next three characters, (``r-''), represent the group's permissions on the
file. The group that owns this file is users. Because only an ``r'' appears
here, any user who belongs to the group users may read this file.
The last three characters, also (``r-''), represent the permissions granted to
every other user on the system (other than the owner of the file and those in
the group users). Again, because only an ``r'' is present, other users may read
the file, but not write to it or execute it.
Here are some other examples of permissions:
-rwxr-xr-x The owner of the file may read, write, and execute the file. Users in
the file's group and all other users may execute the file
-rw------- The owner of the file may read and write the file. No other user can
access the file.
-rwxrwxrwx All users may read, write and execute the file.
8. Permissions Dependencies.
The permissions granted to a file also depend on the permissions of the
directory in which the file is located. For example, even if a file is set to -rwxrwxrwx,
other users cannot access the file unless they have read and execute access to
the directory in which the file is located. For example, if Larry wanted to
restrict access to all of his files, he could set the permissions to his home
directory /home/larry to -rwx---. In this way, no other user has access to his
directory, and all files and directories within it. Larry doesn't need to worry
about the individual permissions on each of his files.
In other words, to access a file at all, you must have execute access to all
directories along the file's pathname, and read (or execute) access to the file
itself.
Typically, users on a Linux system are very open with their files. The usual set
of permissions given to files is -rw-r-r-, which lets other users read the file
but not change it in any way. The usual set of permissions given to directories
is -rwxr-xr-x, which lets other users look through your directories, but not
create or delete files within them.
However, many users wish to keep other users out of their files. Setting the
permissions of a file to -rw---- will prevent any other user from accessing the
file. Likewise, setting the permissions of a directory to -rwx--- keeps other
users out of the directory in question.
9. Changing permissions.
The command chmod is used to set the permissions on a file. Only the owner of a
file may change the permissions on that file. The syntax of chmod is
chmod {a,u,g,o}{+,-}{r,w,x} filenames
Briefly, you supply one or more of all, user, group, or other. Then you specify
whether you are adding rights (+) or taking them away (-). Finally, you specify
one or more of read, write, and execute. Some examples of legal commands are:
pegasus{jbeams}39: chmod a+r filename
gives all users read access to the file
pegasus{jbeams}40: chmod +r filename
same as above. if aug or o isn't specified, a is assumed
pegasus{jbeams}41: chmod og-x filename
remove execute permissions from people other than the owner
pegasus{jbeams}42: chmod u+rwx filename
let the owner read, write and execute the file
10. Jobs and processes.
Job control is a feature provided by many shells that let you control multiple
running commands, or jobs, at once. Every time you run a program, you start what
is called a process. The command ps displays a list of currently running
processes.
The PID listed in the first column is the process ID, a unique number given to
every running process. The last column, COMMAND, is the name of the running
command. Here, we're looking only at the processes which Larry himself is
currently running. (There are many other processes running on the system as
well--``ps -aux'' lists them all.) These are csh (your shell program), and the
ps command itself. As you can see, csh is running concurrently with the ps
command. csh executed ps when you typed the command. After ps has finished
running (after the table of processes is displayed), control is returned to the
csh process, which displays the prompt, ready for another command.
A running process is also called a job. The terms process and job are
interchangeable. However, a process is usually referred to as a ``job'' when
used in conjunction with job control--a feature of the shell that lets you
switch between several independent jobs.
In most cases users run only a single job at a time--whatever command they last
typed to the shell. However, using job control, you can run several jobs at
once, and switch between them as needed.
How might this be useful? Let's say you are editing a text file and want to
interrupt your editing and do something else. With job control, you can
temporarily suspend the editor, go back to the shell prompt and start to work on
something else. When you're done, you can switch back to the editor and be back
where you started, as if you didn't leave the editor. There are many other
practical uses of job control.
11. Foreground and background.
Jobs can either be in the foreground or in the background. There can only be one
job in the foreground at a time. The foreground job is the job with which you
interact--it receives input from the keyboard and sends output to your screen,
unless, of course, you have redirected input or output, as described above. On
the other hand, jobs in the background do not receive input from the
terminal--in general, they run along quietly without the need for interaction.
Some jobs take a long time to finish and don't do anything interesting while
they are running. Compiling programs is one such job, as is compressing a large
file. There's no reason why you should sit around being bored while these jobs
complete their tasks; just run them in the background. While jobs run in the
background, you are free to run other programs.
Jobs may also be suspended. A suspended job is a job that is temporarily
stopped. After you suspend a job, you can tell the job to continue in the
foreground or the background as needed. Resuming a suspended job does not change
the state of the job in any way--the job continues to run where it left off.
Suspending a job is not equal to interrupting a job. When you interrupt a
running process (by pressing the interrupt key, which is usually Ctrl-C), the
process is killed, for good. Once the job is killed, there's no hope of resuming
it. You'll must run the command again. Also, some programs trap the interrupt,
so that pressing Ctrl-C won't immediately kill the job. This is to let the
program perform any necessary cleanup operations before exiting. In fact, some
programs don't let you kill them with an interrupt at all.
12. Backgrounding and killing jobs.
Let's begin with a simple example. The command yes is a seemingly useless
command that sends an endless stream of y's to standard output. (This is
actually useful. If you piped the output of yes to another command which asked a
series of yes and no questions, the stream of y's would confirm all of the
questions.)
The y's will continue ad infinitum. You can kill the process by pressing the
interrupt key, which is usually Ctrl-C. So that we don't have to put up with the
annoying stream of y's, let's redirect the standard output of yes to /dev/null.
/dev/null is a special file in UNIX systems that acts as a ``black hole'' for
data. Any data sent to it disappears. This is a very effective method of
quieting an otherwise verbose program.
pegasus{jbeams}43: yes > /dev/null
Ah, much better. Nothing is printed, but the shell prompt doesn't come back.
This is because yes is still running, and is sending those inane y's to
/dev/null. Again, to kill the job, press the interrupt key.
Let's suppose that you want the yes command to continue to run but wanted to get
the shell prompt back so that you can work on other things. You can put yes into
the background, allowing it to run, without need for interaction.
One way to put a process in the background is to append an ``&'' character to
the end of the command.
pegasus{jbeams}44: yes > /dev/null &
[1] 164
pegasus{jbeams}45:
As you can see, the shell prompt has returned. But what is this ``[1] 164''? And
is the yes command really running?
The ``[1]'' represents the job number for the yes process. The shell assigns a
job number to every running job. Because yes is the one and only job we're
running, it is assigned job number 1. The ``164'' is the process ID, or PID,
number given by the system to the job. You can use either number to refer to the
job, as you'll see later.
You now have the yes process running in the background, continuously sending a
stream of y's to /dev/null. To check on the status of this process, use the
internal shell command jobs.
Sure enough, there it is. You could also use the ps command as demonstrated
above to check on the status of the job.
To terminate the job, use the kill command. This command takes either a job
number or a process ID number as an argument. This was job number 1, so using
the command
pegasus{jbeams}46: kill %1
kills the job. When identifying the job with the job number, you must prefix the
number with a percent (``%'') character.
Now that you've killed the job, use jobs again to check on it:
The job is in fact dead, and if you use the jobs command again nothing should be
printed.
You can also kill the job using the process ID (PID) number, displayed along
with the job ID when you start the job. In our example, the process ID is 164,
so the command
pegasus{jbeams}47: kill 164
is equivalent to
pegasus{jbeams}48: kill %1
You don't need to use the ``%'' when referring to a job by its process ID.
13. Stopping and restarting jobs.
There is another way to put a job into the background. You can start the job
normally (in the foreground), stop the job, and then restart it in the
background.
First, start the yes process in the foreground, as you did before:
pegasus{jbeams}49: yes > /dev/null
Again, because yes is running in the foreground, you shouldn't get the shell
prompt back.
Now, rather than interrupt the job with Ctrl-C, suspend the job. Suspending a
job doesn't kill it: it only temporarily stops the job until you restart it. To
do this, press the suspend key, which is usually Ctrl-Z.
While the job is suspended, it's simply not running. No CPU time is used for the
job. However, you can restart the job, which causes the job to run again as if
nothing ever happened. It will continue to run where it left off.
To restart the job in the foreground, use the fg command (for ``foreground'').
The shell displays the name of the command again so you're aware of which job
you just put into the foreground. Stop the job again with Ctrl-Z. This time, use
the bg command to put the job into the background. This causes the command to
run just as if you started the command with ``&'' as in the last section.
And you have your prompt back. Jobs should report that yes is indeed running,
and you can kill the job with kill as we did before.
How can you stop the job again? Using Ctrl-Z won't work, because the job is in
the background. The answer is to put the job in the foreground with fg, and then
stop it. As it turns out, you can use fg on either stopped jobs or jobs in the
background.
There is a big difference between a job in the background and a job that is
stopped. A stopped job is not running--it's not using any CPU time, and it's not
doing any work (the job still occupies system memory, although it may have been
swapped out to disk). A job in the background is running and using memory, as
well as completing some task while you do other work.
However, a job in the background may try to display text on your terminal, which
can be annoying if you're trying to work on something else. For example, if you
used the command
pegasus{jbeams}50: yes &
without redirecting stdout to /dev/null, a stream of y's would be displayed on
your screen, without any way for you to interrupt it. (You can't use Ctrl-C to
interrupt jobs in the background.) In order to stop the endless y's, use the fg
command to bring the job to the foreground, and then use Ctrl-C to kill it.
Another note. The fg and bg commands normally affect the job that was last
stopped (indicated by a ``+'' next to the job number when you use the jobs
command). If you are running multiple jobs at once, you can put jobs in the
foreground or background by giving the job ID as an argument to fg or bg, as in
pegasus{jbeams}51: fg %2
(to put job number 2 into the foreground), or
pegasus{jbeams}52: bg %3
(to put job number 3 into the background). You can't use process ID numbers with
fg or bg.
Furthermore, using the job number alone, as in
pegasus{jbeams}53: %2
is equivalent to
pegasus{jbeams}53: fg %2
14. Scripts
Let's say that you use a series of commands often and would like to save time by
grouping all of them together into a single command. For example, the three
commands
pegasus{jbeams}54: cat chapter1 chapter2 chapter3 > book
pegasus{jbeams}55: wc -l book
pegasus{jbeams}56: cat book
concatenates the files chapter1, chapter2, and chapter3 and places the result in
the file book. The second command displays a count of the number of lines in
book, and the third command cat book prints book to the screen.
Rather than type all these commands, you can group them into a shell script. The
shell script used to run all these commands might look like this:
#!/bin/csh
# A shell script to create and show the book
cat chapter1 chapter2 chapter3 > book
wc -l book
cat book
Shell scripts are just plain text files; you can create them with an editor such
as emacs, vi, or the pico editor discussed earlier.
Let's look at this shell script. The first line:
#!/bin/csh
identifies the file as a shell script and tells the shell how to execute the
script. It instructs the shell to pass the script to /bin/csh for execution,
where /bin/csh is the shell program itself.
The second line is a comment. Comments begin with the character # and continue
to the end of the line. Comments are ignored by the shell--they are commonly
used to identify the shell script to the programmer and make the script easier
to understand.
The rest of the lines in the script are just commands, as you would type them to
the shell directly. In effect, the shell reads each line of the script and runs
that line as if you had typed it at the shell prompt.
Permissions are important for shell scripts. If you create a shell script, make
sure that you have execute permission on the script in order to run it. When you
create text files, the default permissions usually don't include execute
permission, and you must set them explicitly. See the discussion of file
permissions for details. Briefly, if this script were saved in the file called
makebook, you could use the command
pegasus{jbeams}1: chmod u+x makebook
to give yourself execute permission for the shell script makebook.
You can use the command
pegasus{jbeams}57: makebook
to run all the commands in the script.
15. Getting Out of Trouble and Finishing off
In most cases if you get stuck, for example, in a looping program or in a
utility that does not seem to be doing anything then pressing <Ctrl C> will get
you back to the shell command prompt. If a process is taking terminal input then
<Ctrl D> signals an 'end of file'. At the command prompt <Ctrl D> will exit from
the shell, and on some systems if this is the login shell then you will be
logged out.
If you have become nested in shells invoked within shells, then you will have to
exit them one by one before you can logout with exit or the logout command. If
Unix declines to log you out (because there are 'stopped jobs' for example), try
issuing the logout command a second time.
If you are working remotely from a PC then breaking your connection or
re-booting the PC will usually log you out. Do not re-boot or switch off a Unix
workstation, except in extremis, but never walk away and leave yourself
logged-in on a public workstation either!
16. Checking you disk space.
du -s * .[a-zA-Z]* | sort -n