Introduction to Unix

Research Computing, Dartmouth College

This presentation can be found at:

http://caligari.dartmouth.edu/~rc/classes/unix1/

Course Handout: (last update Thursday, 07-Jul-2022 23:30:10 EDT)

Introduction to UNIX

The goals of this course are to enable you to:

Example commands are shown like this. Most commands are shown with links to their full man pages (ls)
Output from commands is shown like this
Optional items are shown in brackets, [ like this ]

Some descriptions in these notes have more detail available, and are denoted like this:

More details of this item would appear here. The printed notes include all of the additional information




UNIX is simple and coherent, but it takes a genius (or at any rate a programmer) to understand and appreciate the simplicity.

- Dennis Ritchie

If you have any trouble sounding condescending, find a Unix user to show you how it's done.

Dilbert newsletter 3.0, 1994
- Scott Adams

Best viewed with ANY browser Powered by Vi

Unix use at Dartmouth

Research Computing Group, Computing Services

The Research Computing group provides:

Hardware

Consulting

History of Unix

UNIX history map

UNIX history map pdf

The name "UNIX" is a trademark, originally of AT&T, but since sold several times - most variants have names other than "UNIX", but all derive from AT&T code or API specifications originally.

Features of Unix

How to get a Research Computing account

General Research Account

Discovery Cluster accounts

Gateway (remote access) accounts

Logging in from a Macintosh or PC

There are many ways to talk to a UNIX machine. Terminal programs typically implement the secure shell (SSH), telnet and remote login (rlogin) protocols. In addition to those, X server software provides the necessary control over the display that remote graphical software needs. Information on recommended terminal and Xserver software for MacOS and MS Windows is at www.dartmouth.edu/~rc/help/faq/connect.html.

Logging in to UNIX

Where programs run; the login model.

Getting Help

research . computing @ dartmouth

System error messages

foo: Command not found.
Probably the commonest error message you will see. Every command given to a Unix system, whether it is typed at a command line, read from a script, or generated by clicking an icon, actually tells Unix to find a program and run it. If the command is mistyped, or incorrectly installed, or your account tells the system to look in the wrong places, you will see this error message.

If you are getting a repeatable error message from any Unix command, and you mail us for help, try to include the exact error message (cut and paste it into a mail program if possible).

Panic Buttons.

logoff
To stop a program
To stop output
When in doubt
exit, ctrl-d, logout
ctrl-c
'q', or quit (works on man pages, more, etc)
'?' or help or 'h' will sometimes bring up help.

Learning to fish

Man Pages are your Friend

command name (man page section)

Name
Brief description of the command.

Syntax
This tells you how to invoke the command and includes command line options.

Description
Tells you more about the command.

Options
Explains the options that a command can accept, command line options modify the way in which a command works.

Examples
Shows you how to use the command, this can be one of the most useful sections if you find an example that does what you want.

Files
Tells you which files this command uses or depends on. Especially useful for system administrators types.

Bugs/Warnings/Restrictions/Diagnostics ...
Commands are qualified near the end of the man page, this area can usually be ignored for simple invocations of commands but is the first place to look if you encounter problems.

See also
This is usually a list of related commands, or a list of a family of commands. Think of it as a primitive hyperlink.

Man page example

nimbus:~>man who

who(1)

Name
who - print who and where users are logged in
Syntax
who [who-file] [am i]
Description
The who command, without an argument, lists the login name, terminal name, and login time for each current UNIX user. Without an argument, who examines the /etc/utmp file to obtain its information. If a file is given, that file is examined. Typically the given file will be /usr/adm/wtmp, which contains a record of all the logins since it was created. Then who lists logins, logouts, and crashes since the creation of the wtmp file. Each login is listed with user name, terminal name (with /dev/ suppressed), and date and time. When an argument is given, logouts produce a similar line without a user name. Reboots produce a line with `~' in the place of the device name, and a fossil time indicative of when the system went down. With two arguments, as in `who am I' (and also `who are you'), who tells who you are logged in as.
Files
/etc/utmp
See Also
getuid(2), utmp(5)

Searching for man pages by keyword

To find the man pages on a subject use:
man -k [subject]
man ps
man -k system
man -k talk

Man page sections:

1 .......... User commands.
2 .......... Programming interface (system calls).
3 .......... Programming interface (external functions and subroutines)
4 .......... File Format
5 .......... System (administrative) commands. Sometimes also (8)
+ .......... Other sections.

Storing Information -- The File System

ls
The ls command lists files in a directory, and various information about the files. You will use this a lot.
ls -l
The -l flag (long) lists most of the information stored about a file.

The UNIX file system (cont.)

The file system can be thought of as an upside down tree with "root" as the starting point.

The file system contains directories (folders) and files.

Directories can contain other directories as well as files. A directory is just a special type of file, containing file names and pointers to the internal data structures which hold the permissions, ownership etc. for the file. Directories are manipulated using special utilities.

Unix filesystem tree image

The UNIX file system (cont.)

File systems can be remote, actually attached to some other computer on the network. They can normally be treated just the same as local file systems, except they may be a bit slower. The Northstar system keeps all the user files on central servers. Some Central Research machines keep local files in addition to using the remote file servers.

Most Unix systems can also mount disks with "foreign" file systems (e.g. MAC HFS, DOS FAT, NT, OS/2). Operations on such file systems are restricted to conform to the procedures of the foreign system (permissions, owner, filename limits)

Home Directories

Working with Directories

Creating, moving and removing directories.
mkdir frog
Makes a directory named "frog".
mv frog toad
Moves a directory, removes the old directory (effectively a rename). This only works if they are on the same filesystem.
cp -r toad lizard
Copies one directory to another. The "-r" flag says to recursively copy files and subdirectories. This is only one (not the best) of several ways to copy a directory hierarchy.
rmdir toad
Removes a directory, if the directory is empty.
rm -r toad
Will remove a directory even if it is not empty. Again, "-r" indicates recursive action, deleting all the contents first. Use with caution.
ln -s lizard snake
Links (similar to a Macintosh alias) one file to another. Think of it as creating a "pointer" to a file or directory; good for creating easy access to a shared resource. Take a look at this with a ls -l.. "lizard" is the real file, while "snake" is the alias for it. 

lrwxrwxrwx 1 jwallace web   6 Oct 14 20:02 snake -> lizard

Navigating Directories

Commands to try

pwd
Print Working Directory. Shows you where you are in the file system - usually this information is configured to be part of the shell prompt too.
cd [directory]
Change Directory. With no argument, this changes the current directory back to your home directory. With a directory name, it tries to change directory to the named location, as long as it exists and you have permission to go there. The directory name can be an absolute or relative path (more on this later).
cd ~username
Shortcut to go to the home directory of username.
cd ..
Moves up one directory
ls
Lists the files in the current (working) directory.
ls path_name
E.g. ls /usr/local/bin Lists the files in the named directory.
ls -al
Shows a "long" listing of the files. Include "hidden" files because -a was given.
ls -FR
Recursively lists all sub-directories and displays executable files with an appended "*" and directories with an appended "/". Don't do this starting at the root directory!

Command history in the tcsh: try pressing the arrow keys to recall previous commands and edit them before re-executing.

Pathname Examples

Files: Creating, Listing and Examining

Sidebar: standard I/O and redirection

Most commands send their output to "standard output". The shell provides a mechanism for redirecting this to a file, or to another program. The default is to the terminal. Many programs also read some input (default is the keyboard of the terminal). Again, there is a mechanism to tell any program to read its input from a file or another program instead.
> filename
send output to the named file, creating it if needed. This is most common way new files are created
>! filename
send output to the named file, creating it if needed, and forcing overwrite
>> filename
append output to the named file, creating it if needed
< filename
read input from the named file
prog1 | prog2
run "prog1" and send its output to the input of "prog2". Both programs execute in "parallel". Actually they are usually timeshared, but appear to both run simutaneously

These facilities are available to all programs, but it is up to the individual programs whether they respect the conventions.

Creating a file

echo " This is a test file" > test_file
This creates a file using "echo" and "redirecting" the output to a file.
ls -a > ls_file
This creates a file by taking the output(standard output) of the "ls" command and " redirecting" the output to a file.
ls -FR /people2 | wc > how_many
This creates a file by taking the standard output of the "ls" command and "piping" it in to the "wc" (word count) command and then redirecting" the output to a file.
touch empty_file
Creates a empty file! Useful way to signal an event.

Looking at files

more [file_name]
Will display the contents of a file one "page" at a time. Can also display the contents of a data "stream" one page at a time. Ex. ls -FR | more
more /usr/man/cat1/cc.1
The space bar will move you to the next page and the return key will move you down a line. A "q" will end the more (more can also search through a file see the man page).
cat /usr/man/cat1/cc.1
Streams the file to "standard output". What else can cat do??
head [file_name]
Display the first few lines of a file
tail [file_name].
Display the last few lines of a file
Most of these utilities are intended for examining text files, with lines separated by "Newline" characters. Most files in Unix tend to be text files. The internal structure of binary files is application specific, and with the exception of cat, these utilities are not very useful with them. Two utilities, strings and od are very useful for examining binary files outside of their specific applications.

Files: Copying, Moving and Deleting

cp how_many not_enough
Copies the first file to the second file, does not remove the first file.
cp -r unix2 unix2copy
Copies an entire directory. Creates a new directory.
mv not_enough booty
Moves file "not_enough" to file "booty". If the destination name is on the same device as the source, this is just a rename (and so it is very fast). For diffferent devices, a mv is equivalent to a cp followed by rm
rm not_enough booty
Removes files "not_enough" and "booty". BEWARE of filenames with embedded spaces (see below).
Type in "which rm" (which tells you what command is being run, many times it will tell you the path) . Try rm -f on a file.

Unix Editors

UNIX editors are powerful tools but (sometimes) not very user friendly. edit is often aliased to "your favourite editor", but can be linked to ed on some systems.

Note that these are editors, not word processors or text formatters.

The vi editor

Vi has modes. Insert mode and command mode. When you first enter vi you are in command mode and anything you type will be interpreted as a command. This of this as selecting actions from the menus in MS Word.
Vi needs to be able to address the terminal window you are working in, so the setting of the $TERM variable is important. Some systems get confused by resizing the terminal window while vi is running.

When you are in command mode many commands which affect more than one line of the file are prefaced with a ":". An example is :w which writes out or saves the current file.

When you are in insert mode you can type in text, when you are in command mode you can issue commands to vi (search for a string, save the file ...).

Basic vi commands:

vi file_name or vi
Enters vi using the named file as the file to edit.
view file_name
Opens a file in read only mode. Means that you cannot easily change the file. If you make a change to the file you can force it to write with :w!.
<esc>
The escape key, tells vi to leave insert mode and go into command mode. You cannot exit vi unless you are in command mode.
ZZ
Exit vi.
:set showmode
Shows insert mode in the lower right hand corner of the screen. Very helpful when first learning vi.
:q!
Force a quite without saving (if you mess up)

Vi navigation

Vi commands continued.

i, I
Enter insert mode before current character. I inserts at start of line.
a, A
Enter append mode (like insert, but after current character). A appends to end of line
o, O
Enter insert mode on a new line below current line (open). O goes to insert mode on a new line above the current line.
ESC
The Escape key stops insert mode and returns to command mode. Pressing Escape in command mode just beeps.

Vi navigation

On almost all terminals and terminal emulation software, the arrow keys will work as expected. Alternately, in command mode, the h,j,k,l keys behave as arrow keys.

Vi commands

Positioning in a file

[ctrl] - F
Forward one screen.
[ctrl] - B
Backward one screen.
[ctrl] - D
Scroll down half screen.
[ctrl] - U
Scroll up half screen.
:1
Go to first line in file.
:$
Go to last line in file.

Interrupting, Canceling

[esc]
End an insert or incomplete command.
[ctrl] - C
Interrupt.
[ctrl] - L
Redraw screen.

Deleting and removing.

dd
Delete the current line.
x
Delete current character.
dw
Deletes a word.
de
Deletes a word but leaves punctuation.
3 dd
Deletes 3 lines.

Other commands

:e
Re-reads the last saved version of the file.
:wq
Writes the file and quits.
:q!
Quits and does not write out changes.

The login shell

More on shells

The Environment

The environment variables are a way to pass information to programs - any program can examine these strings and modify its behaviour (e.g. PRINTER is looked at by the lpr program. Some of these environment variables can be changed, others are preset by the system.

env
HOME=/classes/rcomp1
SHELL=/usr/local/bin/tcsh
TERM=vt100
USER=rcomp1
PATH=/bin:/usr/bin:/usr/local/bin:/usr/bin/X11:/usr/ucb:.
HOSTTYPE=sun4
VENDOR=sun
OSTYPE=solaris
MACHTYPE=sparc
SHLVL=1
PWD=/classes/rcomp1
LOGNAME=rcomp1
GROUP=users
HOST=sunray.dartmouth.edu
MANPATH=/usr/local/man:/usr/man:/usr/share/man
PRINTER=berry-public

Command Line Arguments

The other way information is passed to applications is through the command line arguments. Everything following the program name on the command line, except things interpreted directly by the shell, is made available to the program. Typically these are option flags (to modify the behaviour of the program) and filenames (for the program to act on). The program may then also have a direct interaction with the use via a text or graphical interface. Most program use all of the methods.

Many featuresof Unix are designed to help in automating procedures (Unix users are lazy). Passing information via the environment and the command line is scriptable, while direct interaction via the keyboard and mouse is not.

Job control

Shells also allow you to place jobs in the background and run multiple processes from one shell session, although multiple windows are easier for this.
ls -FCR > save_it &
The "&" says make this a background job.
^Z
control-Z sends a "suspend" signal to a currently running process
bg
continue execution of a suspended process in the background (must not try to read from the keyboard, but can write to the screen)
fg
resume execution of a suspended program in the foreground
jobs
list background jobs running in current shell session.

Shell commands

Some quick shell commands to try:

These commands are all csh/tcsh internal commands, designed to make life easier at the command line
source .cshrc [.login]
Re-reads your log in files as if(almost) you had logged in again. Good for making changes to login files and testing them. Don't make changes to log in files and then log out and back in to test them. You might not be able to log back in.
history
Shows you the history of commands that you have issued. You can raise or lower this number.
[Up Arrow]/[Down Arrow]
The Up/Down arrow keys will move through the history list of previous commands, so that you can re-execute them with minmal retyping.
!!
Execute the last command again.
!23
Execute command number 23 again.
alias
Shows aliases. An alias is a command shortcut. Can make things much simpler and faster.
alias his history
Creates an alias called "his" for the command "history".

Printing

In Unix, the functions of preparing a file for printing (translating formats etc.) and sending the file to the printer, are usually separate. At Dartmouth essentially all public printers expect to be sent postscript, and the print spoolers detect when a file is already in postscript form and pass it unchanged. Anything else is assumed to be pure text, and is automatically translated to postscript. Programs which prepare their own formatted print files need to be set for postscript output.

All print jobs are queued. Access to the printer is controlled by a spool queue. Printers may be local or remote - there is no difference in the way they are used. All public printers at Dartmouth are remote. The "built-in" print command provided by many applications actually prepares a temporary file with the material to be printed, and calls one of the following utilities.

lpr [-Pprinter] file [file ...]
lpr sends the named file(s) to the printer (actually, append this job to the queue for the printer). If no printer is named, the variable $PRINTER is used. If it is not set, the system default printer is used. If no files are named, standard input is read, so that lpr can be the end of a pipe.
lpq [-Pprinter]
Check the queue for the named printer.
lprm [-Pprinter] job-number
Remove a print job from the queue, if it hasn't started printing yet.

lp [-dprinter] file [file ...]
lp is functionally similar to lpr. On most modern systems, both commands are available. "lpr" comes from the BSD heritage, while "lp" is from the SysV heritage. The "lpstat" and "cancel" commands are also part of the SysV printing tools.
lpstat
Check printer status, similar to "lpq"
cancel
Cancel a print job from the queue, similar to "lprm"

Utilities to provide more control over the formatting of plain text files vary from system to system. Check out pr and lwf on Nimbus and Northstar, or postprint on Sunray.

Other UNIX commands

An assortment of other useful UNIX commands --- remember that many commands can be "joined" together with the pipe "|" and that output from a command can be sent to a file with the redirect symbol.

Line oriented commands

grep
Searches a file for the string and reports on all matching lines.
find
Searches though a file system. E.g. find /usr/share -name \*zip\* will search the filesystem below "/usr/share" for filenames with "zip" in them.
sort
Sorts (alphabetic or numeric) lines in a file.
cut
Cuts fields out of a file.
who
Show who is logged to the system and what they are doing. The command w is almost identical to who
mail
A very basic mail user interface. Not recommended for reading mail, but very handy for sending mail. Typically used to mail yourself a notification that a long-running computation has finally finished.
at
Execute a command at a future time - useful for background jobs
script
Capture a terminal session to a file. Saves what you and the computer type to the screen (stdout). Saves the output to file "typescript".
sed
A stream editor, really nice in pipes.
awk
Pattern matching language. Very powerful.

Screen oriented commands

talk
Talk to another person on the internet.
irc
Internet relay chat. Talk with lots of people on the internet.
pine
A mail reader. There are other mail readers available too. You can even use a mail program at the end of a pipe. E.g. w | grep sam | mail $USER. This runs the "w" command and looks for user (string) sam; the output is then mailed to $USER (always set to your username).
textblitz
A purely text-based interface to the Blitzmail system
top
Show the most active processes on the system. An animated, sorted, ps listing.

References, Resources, Man pages etc.

The following is a sampling of the many available books on the subject.
The links are to publishers web sites, or Amazon.com.
These notes may be found at http://caligari.dartmouth.edu/~rc/classes/unix1 (multi-frame version), or http://caligari.dartmouth.edu/~rc/classes/unix1/s5.shtml (S5 slide show).
The online version has many links to additional information and may be more up to date than the printed notes.
(last update   Thursday, 07-Jul-2022 23:30:10 EDT)   ©Dartmouth College