From Mageia wiki
Jump to: navigation, search


Synopsis:
These pages introduce and explain basic shell commands and mechanisms.

Contents

Using The Shell

The Ultimate Power Tool

Every once in while you get to hear the litany that 'the command line' is 'out of date', 'arcane' and so on. And that people and operating systems should get rid off it.

The truth is: you can use Linux without knowing anything about the shell. You can boot your system directly into X and shut it down from there. You can use the Mageia Control Center and Webmin to configure each and every aspect of your system.

It is my conviction, however, that using Linux without the shell is like driving your car in first gear only. Sure, it's easier at first glance and you get where you want eventually, but it's slow and takes the fun out of driving.

And that's it: the command line is fun. It's like the biggest collection of building blocks you've ever seen. And these blocks can do the most amazing things and what's more, you can make them cooperate, thus achieving highly complex tasks within a few lines. This is because in Unix the shell isn't just a simple command interpreter like the DOS prompt in Windows, but a fully fledged programming environment.

That isn't to say it is easy learning your way around the shell. Well, it's learning, so what do you expect? ;-) But believe me that it can be highly rewarding. And you will be regarded as a computer wizard by most outsiders in a rather short period of time *grin*.

In order to get some idea what a shell is, a little background knowledge might be useful.

Terminals, Xterms And Shells

Back in the days when Unix was born, the now dominant breed of 'personal computers' was completely unknown. The machines then referred to as 'computers' were bulky, room-sized brutes using tapes and magnetic memory (hence the term 'core' for system memory). With the PDP-11 by DEC (now Compaq) introduced in 1970, a 'small' (thus called 'mini') and rather cheap computer appeared on the market and turned almost instantly into a big hit with universities, most of which were for the first time able to buy a computer (the PDP-11 was a real bargain at $10,000).

The operating systems for these machines were written in assembler, machine code, thus highly efficient but also totally and utterly unportable. Each computer vendor sold his operating system along with his machines.

The awkwardness of this situation was soon realized and attempts were made to create an operating system which could run on more than one brand of machine. In 1969 an employee of AT&T Bell Labs named Ken Thompson wrote the first code of what was to become known as Unix (a pun on MULTICS, an earlier project Thompson had been working on). However, things really got started when Dennis Ritchie came up with a new programming language for this new operating system, called 'C'.

Although Unix was less efficient than the vendor supplied operating systems, it had three decisive advantages: it could be ported to any machine you happened to have around, the included 'C' language made programming several orders of magnitude easier and it was free. Soon, universities all over America began equipping their machines with Unix.

Terminals

So Unix was (and is) an operating system running on a wide variety of machines. What about the people who used these machines? These people connected to these machines via 'dumb' terminals, i.e. machines consisting of a keyboard, a monitor and enough electronics to hold a connection to the central computer. What users at these terminals did was basically TeleTYping, thus the string 'tty' for terminal device files or the name of the 'getty' command.

You may ask yourself what all this has got to do with the topic at hand. Well, the vendors of these terminals couldn't agree on a standard. That is, each brand of terminal had its own keyboard layout, its own method of displaying characters on the screen, its own idea as to what signals sent and received represented which characters or control codes, and so on.
In order to cope with this chaos, a central file containing all the different and differing terminal capabilities was created -- the termcap. Open /etc/termcap in a text viewer and stand amazed (or frightened ;-)).

Linux terminals mostly use either 'vt100' or 'linux' as their terminal type.

On a Mandrivalinux system you can access windows which emulate such terminals via the key sequence CTRL+ALT+F1 to CTRL+ALT+F6. ALT-F7 brings you back to the current X session (see below). They are 'special files' in /dev: /dev/tty1 to /dev/tty6.


Xterms

In the early 1980's, the X Window System, a graphical subsystem for Unix, was developed. XFree86 was forked from this system in the early 90s in order to write a version more stable on the then fledgling Intel-based Unix clones like FreeBSD, NetBSD or Linux. Later, x.org forked from XFree86. x.org is by now the default graphical system for Mandriva.

The first and foremost use of X was running multiple 'virtual' terminals on it. X even came with such an application, xterm. Therefore you'll find that 'xterm' and 'virtual terminal' are often used synonymously. When you read 'open an xterm', you don't actually have to install and use the xterm program, any other terminal emulator, like rxvt, konsole, aterm, eterm, wterm, etc, will do, too.

A terminal emulator (another name for 'virtual terminal') connects to the system via a pseudo tty device file and uses its own display standard. This might lead to a slightly different behavior of some keys or applications on different terminal emulators, depending on how good these emulators emulate the xterm display standard.

On Mandriva, you can get access to these emulators through the KDE Main Menu -> System -> Terminals. They are special files in /dev/pts ('pts' = 'Pseudo-TerminalS'): /dev/pts/1, /dev/pts2 etc.

Shells

In order to run programs on a terminal, a shell is needed. The shell is that part of an operating system which communicates with the user and allows communication between commands.

The first 'real' Unix shell, sh, was written around 1975 by Steve Bourne, and is thus also called the 'Bourne shell'. Soon others developed, some based on the original Bourne shell like (pd) ksh or zsh, which are often used as the standard shell on proprietary Unixes, or the shells which implemented features from the 'C' programming language, like csh or tcsh.

In Linux, the standard shell is 'bash', the GNU Bourne-Again Shell (well, Unix humor ...). It is very powerful (some would say, bloated), the compressed (!!) man page alone weighs some 50 KB.

First Steps On The Shell

Note:
First of all, you should not use the shell as 'root' for everyday tasks, especially if you are just beginning to use it. You can't damage the system accidentally (or even intentionally) when using a shell with regular user permissions; with root permissions, however, a simple typo can have unforeseen and severe consequences.


The first thing you see after logging in or opening an xterm window is the prompt. The standard prompt on Mandriva linux consists of your user name, the name of the machine you are logged into (if none is set, 'localhost' is used), the directory where you are (the 'present working directory') and the prompt sign:

[tom@belbo tom]$

I'm logged in as user 'tom' on my machine which I've called 'belbo' and I'm in my home directory, '/home/tom'. The prompt for 'root' looks like this:

[root@belbo root]#

You see that apart from the different name, the prompt sign has changed from a '$' to a '#'. It is traditional on Bourne shells to end a user's prompt with '$' and root's prompt with '#'.

Every aspect of the prompt is customizable. You will read about that later on.

To run a command, you type the name of the command after the prompt and then hit the "ENTER" key. The shell searches for the command in its search path (more on that later, too), runs the command, prints the output of the command (if any) on the terminal and presents you with a new prompt when the command is finished:

[tom@belbo tom]$ whoami
tom
[tom@belbo tom]$

By the way, it does not matter where the cursor is when you hit the ENTER key, the shell will always read the complete line.

Basic commands are ls (list directory), cp (copy), mv (move / rename), cd (change directory), each coming with a wealth of options listed in their respective man page (man ls, man mv etc).

When you press "TAB", then a list of all available commands is displayed, when you enter some characters and press "TAB" afterwards, then a list of all commands starting with these characters is displayed.

Before you start off into shell land, a quick introduction into terminology. Commands take options and/or arguments:

[user@computer ~]$ mv -i file dir

The '-i' is an option to the mv command, whereas 'file' and 'dir' are arguments. Available options are explained in detail in the command's man page (man mv in this example), whereas arguments are supplied by you. Options modify how a command works, whereas arguments determine on which data the command should be performed.

So far, this looks very much like the DOS shell almost everyone hates and despises (and rightfully so). Time to introduce some mechanisms which make your shell life a much more pleasurable experience.

Using the Command-line Editor (shell) II: The Very First Shell Commands to Experiment With

If you're a beginner, have fun playing with these first commands in a console.


To open a console:

  • Press simultaneously Alt+Ctrl+Fn keys (Fn being a function key between 1 and 6): you can come back to your classical graphical environment with Alt-F7
  • Or click on K Menu -> Tools -> Konsole.

Be careful not to be logged on as 'root' but as a regular user.


Knowing 'Where You Are'

  • The pwd command displays the name of the working directory (sometimes called current directory).

To remember more easily: pwd is the acronym for "print working directory".


Creating an Empty File

  • Use the command touch file_name.

If file_name is an existing file, only its date and hour will be changed. Its content will not be modified.


Adding a Line at the End of a File Without Removing Its Contents

  • A combination of the echo command and the [redirection operator|BasicsBshell4] '>>' (be careful not to confuse with '>' described below):

echo "blah blah blah" >> file_name


Reading the Contents of a File

  • The command cat file_name displays the contents of a text file.


Replacing the Contents of an Existing File With Another Contents or to Create a File With Specific Contents

  • Similar to the last-but-one command, but using the [redirection operator|BasicsBshell4] '>' this time:

echo "blah blah blah" > file_name

(if the file doesn't exist, it will be created, without using the 'touch' command)


Reading the Contents of a File Spreading Across Several Pages

  • The command less file_name displays the contents of a file and allows scrolling through it.


Changing the Name of a File or a Directory

  • The command mv name1 name2 renames the file or directory name1 to name2.


Creating a New Copy of a File

  • The command cp file1 file2 creates file2 by copying the contents of file1 into it.


Seeing What's Inside a Directory

  • The command ls -l lists the contents of the working directory.
  • The command ls -la lists the contents of the working directory and includes hidden files.
  • The command ls -l dir lists the contents of the 'dir' directory.

Moving Around

  • The command cd directory_name changes the working directory to directory_name.


Cleaning Up

  • The command rm file removes a file.


Creating or Removing Directories

  • The command mkdir directory creates a directory.
  • The command rmdir directory removes an empty directory.


Put a File Somewhere Else

  • The command mv file directory moves the file 'file' to the directory 'directory'.


Searching for Some Text in a File

  • The command grep text_to_look_for file_name looks for a text in a file.
  • The command grep -i text_to_look_for file_name does the same thing but do not distinguish between uppercase and lowercase letters.


To Know What is Happening Right Now

  • The command top lists programs using the most your CPU (they will be up in the display). Use 'q' to quit.
  • The command ps ux lists all active programs belonging to the current user.
  • The command ps aux lists all active programs in the system.


Looking at What You've Done

  • You can recall and see the previous commands you've typed using up and down arrow keys on your keyboard.
  • The command history displays a list of the last commands used.


To Become Administrator Temporarily

  • The command su - switches you to the administrator mode, i.e. you become 'root' (be careful: don't stay under root to experiment with all these commands. A tragic mistake can be made very quickly!). Please note dash; Never use just su!
  • The command exit or Ctrl+D keys bring you back to your previous identity.
  • Alternatively you can simply type sudo before the command to execute as root, once you have configured sudo.

Using The Shell III

Unix (and its younger brother Linux) were born on the command line. Because of this, the command line in Unix features a lot of mechanism to spare you menial typing. This page introduces some of them.


Auto-Completion

What is the shortest way to switch your current working directory from your home directory to the directory /usr/src/linux/Documentation/isdn/ using the cd ('change directory') command? It's [user@computer ~]$ cd /u\<TAB\>sr\<TAB\>l\<TAB\>/D\<TAB\>is\<TAB\>

This is called 'automatic command line completion' and it is indispensable. Let's have a closer look at the example:

[user@computer ~]$ cd /u\<TAB\>

expands to "[user@computer ~]$ cd /usr/". Easy. Next

[user@computer ~]$ cd /u\<TAB\>sr\<TAB\>

expands to "cd /usr/src/". If you just enter "cd /u\<TAB\>s\<TAB\>", you will be presented with a choice of three subdirectories of /usr which all match this pattern ('cd /u\*/s\*'): /usr/sbin, /usr/share and /usr/src.

So the "TAB" key is a handy tool for searching directories for files or subdirectories you know the first letters of. For example

[user@computer ~]$ ls /usr/bin/zip\<TAB\>

Gives you a list of all files and subdirectories in /usr/bin that start with the letters 'zip'. Of course there are much more powerful commands for this tasks, but it does the trick when you're in a hurry.

Furthermore it comes in handy with really long filenames. Say you want to install an RPM called boomshakalakwhizbang-4.6.4.5-mdk586.rpm. You type:

[user@computer ~]$ rpm -i boom\<TAB\>

And if there are no other files in this directory that match this pattern, the shell will fill in the rest by itself.

[user@computer ~]$ cd /u\<TAB\>sr\<TAB\>l\<TAB\>

Expands to "cd /usr/src/linux" and waits for a decision. There are two directories in /usr/src that match: /usr/src/linux-... and /usr/src/linux. How do you tell the shell you want the latter? Append a slash ('/'), thus indicating the end of this last name.

Presume you are not sure if it was /usr/src/linux/Documentation or /usr/src/linux/documentation. As you may know, Linux discriminates between upper and lower case. If you have read carefully until here, you know the answer already:

[user@computer ~]$ cd /u\<TAB\>sr\<TAB\>l\<TAB\>/d\<TAB\>

expands to /usr/src/linux/drivers/. Looks like it was 'Documentation' (with a capital 'D') then.

This kind of completion works for commands, too:

[user@computer ~]$ gre<TAB> grecord grefer grep

[user@computer ~]$ gre

Here the shell presents me with a list of all the commands it knows about which start with the string 'gre'.

Commandline History

Using the up-arrow key you can scroll through all the shell commands you have issued on that console recently. Using the down-arrow key you can scroll back again. Together with the SHIFT key, you can scroll through previous output on the console. You can also edit 'old' command lines and issue them again.

Pressing CTRL + r puts the shell into "reverse-i(ncremental)-search" mode. Now type the first letter of the command you are looking for:

CTRL + r and then typing 'i' may change this line to:

(reverse-i-search)`i': isdnctrl hangup ippp0

If you now press the ENTER key, this command will be executed again. If you press the left or right cursor key or ESC instead, you will have this command on a normal command line where you can edit it.

Commandline Editing

You can navigate and edit the command line with the cursor and the function keys ('Home', 'End' etc), if you like, but there are also keyboard shortcuts for most standard editing tasks:

  • CTRL + k: delete ('kill') from cursor position to the end of the line
  • CTRL + u: delete from cursor position to the beginning of the line
  • ALT + d: delete from cursor position to the end of the current 'word'
  • CTRL + w: delete from cursor position to the beginning of the current 'word'
  • CTRL + a: move cursor to the first character of the line
  • CTRL + e: move cursor beyond the last character of the line
  • ALT + b: move cursor to the first character of the current 'word'
  • ALT + f: move cursor to the last character of the current 'word'
  • CTRL + y: insert latest deleted 'word'
  •  !$\: repeats the last argument of the previous command.

Example: You created a directory with mkdir peter/pan/documents/tinkerbell. Now you want to change into that directory with cd. Instead of typing the path again, you type

[user@computer ~]$ cd !$

And the shell will append the path from the previous mkdir command to the current cd command.

As you venture deeper into Linux land, you'll find that these keyboard shortcuts are also used in other applications when it comes to entering text, for example in browser input fields.

Available Shell Shortcuts

Mandriva Linux comes with a raft of shortcuts, some are native features of bash, some are pre-configured (you'll learn later on how to configure your own shortcuts).

Since the home directory is the focal point of activity for every user, many Unix systems provide special shortcuts for it.

For example, "~" (tilde) is a short form for the name of your home directory. Let's say you are in some far away directory and want to copy a file called 'sometext' to the directory 'docs' in your home directory. Instead of typing "cp sometext /home/myusername/docs", you type "cp sometext ~/docs" which is much shorter, but has exactly the same effect.

In theory, this also applies to the cd command. In reality, cd would take you to your home directory, no matter where you are. But even that was considered too much typing. Just type cd and you're back home.

Mandriva Linux provides you with a set of pre-configured shortcuts (called 'aliases'):

  • cd.. executes 'cd ..' (go to parent directory).
  • d executes 'ls' (list directory).
  • l executes 'ls' (list directory).
  • la executes 'ls -a' (list complete directory, i.e. including files starting with a dot)
  • ll executes 'ls -l -k' (list directory in long format, i.e. with file attributes, print file size in KB and not in bytes)
  • ls executes 'ls -F &#150;&#150;color=auto' (list directories, append file type indicators and use colors)
  • lsd executes 'ls -d \*/' (list subdirectories only, no files)
  • md executes 'mkdir' (create directory)
  • p executes 'cd -' (go back to previous directory)
  • rd executes 'rmdir' (delete (empty) directory)
  • s executes 'cd ..' (go to parent directory)

Now that you are a bit more familiar with the shell and some of the shortcuts it provides, it's time to have a look what you can actually do with it besides running simple commands.

Using The Shell IV

Command Queuing

In time you will become more aware of frequently used combinations of commands. You might want to issue all commands in one line and then direct your attention elsewhere. No problem. The shell lets you put various special queuing characters between commands. This section introduces two of the most important.

Notice that I will use white spaces around these characters for the sake of clarity. You don't need them in real life, there's no difference between ls -a ; du -hs and ls -a;du -hs.


[user@computer ~]$ command1 ; command2

Executes command1 first and then command2, even if command1 returned an error.

Example:


[user@computer ~]$ ls -a ; du -hs

Will first print a complete directory listing to the screen and then the total amount of space the files in this directory and its subdirectories take up.


[user@computer ~]$ command1 && command2

Runs command2 if and only if command1 completed without an error.

Example:


[user@computer ~]$ ls -a bogusdir && du -hs

Will return ls: bogusdir: No such file or directory and du won't be executed at all (presuming you don't have a directory called 'bogusdir', that is ;-)). If you'd have used ';' instead, the 'du' would have been executed.

The classical example to demonstrate the difference between ';' and '&&', and the usefulness of command queuing in general, is the compilation and installation of the Linux kernel.


To compile and install the Linux, you need to execute a set of commands one after the other: make dep, make clean, make bzImage, make modules, make modules_install and make install. It would be annoying to issue one command, wait till it has completed, then run the next, wait, the next, wait and so on. On the other hand every command in this row will only work correctly if all the previous commands completed with no errors. If you use ';' to queue these commands and one of the commands fails, the rest will be executed regardless, and you might end up with an faulty kernel image in the /boot directory. So use '&&':


[user@computer ~]$ make dep && make clean && make bzImage && make modules && make modules_install && make install

This will compile the kernel and its modules, install them, without requiring any form of interaction.

Command Jobbing

When you run a command or start a program from a terminal, that terminal is blocked as long as the command or program is running. In Unix, we say that command or program is running in the 'foreground'. If you need a terminal to run another command, you have to open a new one.

But there's a more elegant way, called 'jobbing' or 'backgrounding'. When you 'job' or 'background' a command, it will release the terminal immediately, so that terminal can be used for new input right away. To do this, all you have to do is to put an "&" (ampersand) behind the command:

[user@computer ~]$ gqview &

Tells the shell to execute the graphical viewer 'GQview' in the 'background' (i.e. to run it as a 'job').

The jobs command tells you which commands and programs are running as jobs on this terminal window:

[user@computer ~]$ jobs
[1]+ Running gqview &


This is important when you consider closing a terminal window, because upon closing the terminal all jobs running on it are terminated. That is, if I would close this terminal, the GQview program started from it would be closed as well.

But what if you already have a foreground program running and want it to run in the background instead? No problem:

gqview
(press "CTRL z")
[1] Terminated gqview

[2]+ Stopped gqview<

Then

bg kview
[2]+ gqview &


The key combination CTRL z suspends a program running on that terminal, so you can now use the bg command to send it to the background.

Notice that there is a situation where starting a graphical application from a terminal in the foreground is useful. This way all error messages this application issues are printed to the terminal, and while they might not be of use for you, they can provide useful hints of what's going wrong to the person you ask for help when in trouble.

Some graphical programs, usually in their testing (or Beta) period, still send messages to their controlling terminal even if they have been backgrounded. If you are annoyed by this behavior, you can turn it off with

[user@computer ~]$ command &>/dev/null &

This sends the program not only to the background, it also sends all terminal output from command to the /dev/null file. Just so you know, /dev/null is the system's data shredder, everything you send there promptly vanishes without a trace.

Command Substitution

Command substitution is a very handy feature. Let's say you want to have a look at the README.mouse file from the XFree86 documentation, but you don't know where it is. Since you are a clever user, you've already heard of the 'locate' command and have the 'slocate' package installed. So you do a:

[user@computer ~]$ locate README.mouse

and you find out it is located in /usr/X11R6/lib/X11/doc. Now you either use the terminal pager less or your file manager to display that file, providing that path. Command substitution makes that much easier for you:

[user@computer ~]$ less $(locate README.mouse

Accomplishes the same in one step. The output of the command locate README.mouse (= /usr/X11R6/lib/X11/doc/README.mouse) is used as the argument for the less pager, which now displays this file.

The syntax for this mechanism is:

[user@computer ~]$ command1 $(command2)

Instead of '$()', you can use single backquotes:

[user@computer ~]$ command1 `command2`

That's less to type, but harder to read in scripts and more easily to confuse with 'normal' single quotes, which the shell won't accept as substitution indicators. I prefer the first method, but in the end it's up to you.

Here's another example. Let's say you want to kill some stubborn program called rob. You can do this by finding out what its Process ID is with the pidof command and then issuing the kill command with this PID as its argument. Instead of typing

[user@computer ~]$ pidof rob
567
[user@computer ~]$ kill 567

You type:

[user@computer ~]$ kill `pidof rob`

It is more efficient, isn't it?

Next, I'll introduce two more useful shell mechanisms, file name globbing and output redirection.

Using the Shell V

Filename Globbing or Expansion

Filename globbing allows you to provide more than one filename to a command without having to write all filenames in full. You use special characters for this, called 'wildcards'.

Say you want to delete all files in a directory that end with the string .bak using the rm command. Instead of typing each filename as an argument to rm, you use the '*' wildcard:

[user@computer ~]$ rm *.bak

'*' matches zero or more characters. In this example, you tell the shell to expand the argument to the rm command to "all file names ending on or consisting of the string .bak", the shell does so and passes that expanded argument to the rm command.

As you will see, it is important to note that the shell reads and interprets the command line before the command does it. This has the advantage that you can use wildcards with (almost) all shell commands which take strings (file names, directory names, search strings etc) for an argument.

Let's play a bit more with the '*' wildcard. You have a directory which contains the files 124.bak, 346.bak and 583.bak. You want to keep 583.bak. What you do is this

[user@computer ~]$ rm *4*.bak

The shell expands '*4*.bak' to "all file names which contain the number '4' and the end on the string .bak".

Notice that rm 4*.bak would not have worked, since this would only have encompassed file names beginning with the number '4'. Since there are no such files in this directory, the shell expands this pattern to an empty string and rm issues an error message:

rm: cannot remove `4*.bak': No such file or directory

Now you want to keep the file 346.bak but delete 124.bak and 583.bak. That's trickier since the files which are to be deleted have nothing in common except the ending. But lucky as you are, you can also define files by what they not have:

[user@computer ~]$ rm *[!6].bak

This reads: "delete all files which end on the string .bak except for those which end on the string 6.bak". You have to put the negation sign '!' and the character to be negated (here '6') into brackets, because otherwise the shell interprets the exclamation mark as the beginning of a history substitution. Negation works with all globbing patterns introduced in this article.

Notice that it is very easy to shoot yourself in the foot with the '*' wildcard and negation. Guess what

[user@computer ~]$ rm *[!6]*.bak

does? It deletes all the files, even the one which does contain a '6' in its filename. If you put '*' wildcards before and after a negation, it renders the negation practically useless, because the shell expands this to "all file names which do not have that character at any given position". In our example, the only file name on which that pattern would not have matched, would have been 666.bak.

The second wildcard is the question mark, '?'. In a globbing pattern, a question mark represents exactly one character. To demonstrate its use, let's add two new files to the three example files, 311.bak~ and 'some.text'. Now list all files, which have exactly four characters after the dot:

[user@computer ~]$ ls *.????

does this. The question mark wildcard is also a useful means to avoid the 'negation trap' mentioned above:

[user@computer ~]$ rm *[!4]?.*

This expands to "all files except for those with a '4' in the second to last position before the dot" and deletes all files except for 346.bak.

Is there more? You bet. So far, you've only seen globbing patterns which match one character at a certain position. But nothing keeps you from matching more than one:

[user@computer ~]$ ls [13]*

lists all files which either begin with the character '1' or the character '3'; in our test case the file 124.bak, 311.bak~ and 346.bak match. Notice that you have to enclose the pattern in brackets, otherwise the pattern would match only files which begin with the string '13'.

Now all that's left for ultimate happiness to ensue is the possibility to define ranges of matches:

[user@computer ~]$ ls *[3-8]?.*

lists all files whose second to last character before the dot is a number between '3' and '8'. In our example, this matches the files 346.bak and 583.bak.

Quoting Special Shell Characters

These powerful mechanisms have one drawback, though: the shell will always try to expand them, and it will do so before the command. There are several cases in which this can get in your way:

  • File names with special characters. Assume you have another file in that directory with the name !56.bak. Try to match it with a globbing pattern:

[user@computer ~]$ rm !*
rm
rm: too few arguments

The shell interprets '!*' as a history substitution ('insert all arguments from previous command'), not as a globbing pattern.

  • Commands which take special characters as arguments themselves. A row of Linux command line tools like (e)grep, sed, awk, find and locate for example use their own set of what is then called 'regular expressions'. These expressions may look strikingly similar to globbing patterns but are in some cases interpreted differently.
    But in order to enable the command to interpret them in the first place, the shell must be prevented from interpreting them as globbing patterns first:

[user@computer ~]$ find . -name [1-9]* -print
find: paths must precede expression

Properly:

[user@computer ~]$ find . -name '[1-9]*' -print
./346.bak
./124.bak
./583.bak
./311.bak~

You can quote such special characters like ! , $, ? or the empty space either with a back slash:

[user@computer ~]$ ls \!*
!56.bak

or with (single) quotes

[user@computer ~]$ ls '!'*
!56.bak

Notice that using quotes may need some deliberation on where to put them.

[user@computer ~]$ ls '! *'

would look for file called !\* since the second wildcard is now quoted, too, and thus interpreted literally.

Output Redirection

The Unix philosophy is to have many small programs, each excelling at a certain task. Complex tasks are not fulfilled by complex programs but by tying together several programs with a handful of shell mechanisms. One of them is redirecting output.

1.1 Redirecting between two or more commands

This is done via 'pipes', denoted by the pipe symbol |. The syntax is

[user@computer ~]$ command1 | command2 | command3

Etc.

You've certainly seen them already. They are often used to direct the output of a program to a pager like 'more' or 'less'.

[user@computer ~]$ |ls -l | less

The first command provides the directory listing and the second displays it in a scrollable manner. A more complex example:

[user@computer ~]$ rpm -qa | grep ^x | less

The first command puts together a list of all installed RPMs, the second filters (grep) those that start ('^') with an 'x' and the third displays the results in a paged and scrollable list.

Redirecting From or Into Files

Sometimes you want to save the output of a command in a file or feed it from a file. This is done via the operators '>' and '<'.

[user@computer ~]$ command > file

saves the output of command in file overwriting all previous content of file:

[user@computer ~]$ ls > dirlist

saves the listing of the current directory to a file called dirlist.

[user@computer ~]$ command < file

uses file as the input for command:

[user@computer ~]$ sort < dirlist > sdirlist

feeds the content of dirlist to the sort command, which sorts it and puts the sorted output into the sdirlist file. Of course, if you're clever, you'd do that in one step:

[user@computer ~]$ ls | sort > sdirlist

A special case is command 2> file. This puts just the error messages of command into file. You may need that from time to time...

Another operator is '>>'. This one appends the output to a existent file:

[user@computer ~]$ echo "string" >> file

This would append string to the content of the file file. A quick way to edit a file without opening an editor first1.1.

There is an important restriction to the '<' and '>' operators, though: Something like

[user@computer ~]$ command < file1 > file1

will erase the content of file1. However,

[user@computer ~]$ command < file1 >> file1

will work fine and append the processed content of file1 to the same file.

That's a lot isn't it? ;-) No need to panic, you can learn everything step by step at your own pace. Practice makes perfect ...

Once you are familiar with the most common shell mechanisms, you might feel the urge to customize your environment. You will find some ideas on the next two pages. In the last part of this article, you'll also find a short FAQ dealing with the most common shell error messages and some minor configuration settings.

Using The Shell VI

bash Configuration Files

When you do a

[user@computer ~]$ ls .bash*

in your home directory, you will see a list of files:

  • .bash_history contains a list of the commands you have entered,
  • .bash_logout contains a list of commands to be executed when you leave the shell,
  • .bash_profile contains a list of commands to be executed when you log in and
  • .bashrc contains a list of commands to be executed every time you open a new shell.

Notice that there's a difference between the last two: .bash_profile is read once at the beginning of a session, whereas .bashrc is read every time you open a new terminal (e.g. a new xterm window). In a traditional setup you would define variables like PATH in .bash_profile, and things like aliases and functions in .bashrc. But since '.bash_profile' usually is pre-configured to read the content of .bashrc anyway, you might as well save some effort and put all your configuration into .bashrc.

These files define per user settings. System wide settings are stored in /etc/profile, /etc/bashrc and the files in /etc/profile.d. You should prefer using the per user configuration files, though editing them doesn't require you to be 'root', and they also allow you to set up things differently for 'root' and each user account, which can be a good thing. In case of conflicts between user settings and system settings, user settings prevail.

The Prompt

The prompt is the first thing you see every time you open a console or an xterm. It looks like this:

[user@computer ~]$

In its default setting it shows your user name, the hostname of your machine (or 'localhost' is you haven't assigned one) and the current working directory ('~' is the Unix shortcut for your home directory).

Traditionally, the last character is set up to indicate whether you are a user ($) or 'root', in which case it changes to a hash (#).

You can set or change your prompt via changing the content of the $PS1 variable. The command

[user@computer ~]$ echo $PS1

displays the current setting. Available special characters and their meaning are listed in man bash, section PROMPTING.

Need some ideas on what might be a better prompt? Well, for starters the default setting isn't very friendly to forgetful people, since it only shows the last part of your current path. If you see a prompt like

[tom@localhost bin] $

your current working directory could be /bin, /usr/bin, /usr/local/bin or /usr/X11R6/bin. Sure, you can type

[user@computer ~]$ pwd

to find out where you are, but shouldn't there be a way to tell the shell to do that on its own accord?

There is. The appearance of the prompt - and for most of the other settings discussed here - is set in /etc/bashrc. You may also change it on a per user basis by editing .bash_profile and .bashrc in your home directory.

Parameters are described in man bash, chapter 'PROMPTING'. You can add nifty things like the current time in different formats or the history number of the command, even different colors are possible.

My currently favored setting in my user ~/.bashrc is:

PS1="PS1="\[\033[1m\][\w]\[\033[0m\] "

'root's ~/.bashrc has:

PS1="\[\033[0;31m\][\w]\[\033[0m\] "

And the prompt I get, is this:

[/usr/bin]

And this when I'm 'root':

[/usr/bin]

I've chopped the host- and user name part, since I have no need for it. But I want to see at first glance if I am logged in as a user or as 'root' on this console. Note that the user prompt will be white on dark backgrounds and black on light ones.

A more moderate setting might be

PS1="\u: \w\\$ "

which will result in prompts like these:

user_name: /usr/bin$

but who cares about being moderate? :-)

You can test various settings on the fly by using the export command e.g.: export PS1="\u: \w\\$ ". If you've found a prompt command which suits you, put it into your .bashrc. That way it will be applied automatically to every console or terminal window you open.

You may even 'theme' your prompt, i.e. use different colors or make it look like an good ol' C64 prompt. If you are interested in this, have a look at Bashish.

Changing $PATH

'$PATH', like '$PS1', belongs to the group of environment variables. Type

[user@computer ~]$ set

to get a full list of all currently defined environment variables.

The environment variables you see here are defined in the shell's configuration files, either by the user in his/hers set of shell configuration files or system-wide by 'root' via the shell configuration files in /etc. If you are on X, some more variables are set by the start-up files of X and your window manager or desktop environment.

You shouldn't temper with the settings of most these variables unless you know what you are doing and why. Knowing how to change the $PATH variable, however, can be useful, since it determines the names of directories where the shell looks for commands and programs, i.e. for executable files. If the command you want to run is in a directory which is listed in $PATH, you do not need to supply the full path to that command, just the command name. Some third-party software does not install its executable files into the standard Linux directories for commands and programs. Adding their non-standard installation locations to the $PATH is a possible workaround. Furthermore it will also teach you how to deal with environment variables in general.

First off you have noticed that the names of environment variables are written in all capital letters. This is just a convention, but since Linux discriminates between capital and small letters, it is important to keep that in mind. You can define a variable like '$path' or '$pAtH', but the shell won't use it.

The second thing is that variable names are sometimes preceded by an '$', sometimes not. To set a variable, you use its name without a preceding '$':

PATH=/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin

to access the variable and its value, you need to put the '$' in front of it:

[user@computer ~]$ echo $PATH
/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin

because otherwise, the variable name is treated as a simple text string:

[user@computer ~]$ echo PATH
PATH

The third item is especially important for dealing with the $PATH variable. You can not only replace variables, but also just add new strings to their values. Most of the time, you don't want to do something like 'PATH=/some/directory' because that would erase all the other directories from your $PATH and force you to supply the full path name to every command you want to run on that terminal. Instead, just add it:

PATH=$PATH:/some/directory

Here PATH is set to its current value (represented by $PATH), plus one more directory.

So far, you've only set a $PATH variable for the terminal you have typed that command in. If you open a new terminal and issue a echo $PATH, you will see the old $PATH without the directory you just added. You have defined a local environment variable (restricted to the terminal you defined it in).

In order to define a global variable which will be recognized by all terminals you are going to open in this session, you need to export the local variable with the 'export' command:

export PATH=$PATH:/some/directory

If you now open a new terminal and type echo $PATH, you will see the new value of $PATH in this terminal, too. Notice, however, that the 'export' command only sets or changes variables for the terminal it is run in and terminals which are started after it has been run. Terminals which were already open will still have the old $PATH.

In order to add a directory permanently to your $PATH, just add the above 'export' command line to your .bash_profile.

Do not use .bashrc for PATH settings which involve appending directories to the existing PATH. .bashrc is sourced every time you open a new shell, so if you append a directory to PATH in it, it will be appended again and every time you open a terminal. This leads to a PATH variable with an ever growing number of duplicate directory entries.

The last part of this article will introduce two more advanced configuration methods and a FAQ dealing with minor configuration and error message issues.

Using The Shell VII

Remembering all sorts of commands and their options and typing them each time is tedious work. Fortunately you don't have to. You can define shortcuts for frequently used commands. These shortcuts can either be defined in the relatively simple form of command aliases or in the somewhat more complex syntax of shell functions.


Command Aliases And Shell Functions

Command Aliases

For example, I use this command to upload my stuff to MUO:

[user@computer ~]$ rsync -e ssh -z -t -r -vv &#150;&#150;progress /home/tom/web/muo/rsmuo/docs muo:/www/mandrakeuser/docs

Obviously I'd go nuts if I had to type this command line each time. So I have defined what is called an 'alias' in ~/.b/fre/ashrc:

alias upmuo='rsync -e ssh -z -t -r -vv --progress /home/tom/web/muo/rsmuo/docs muo:/www/mandrakeuser/docs'

Now all I have to do to upload my stuff is to type upmuo.

The syntax for defining an alias is:

alias shortcut='command'

The quotes are necessary when the command contains empty spaces (e.g. between command and option). Notice that you either quote using single quotes or using double quotes. If you've got a hitch that there's a difference between those two, then you are right ;-).

Single quotes strip the special meaning from all characters included, double quotes from all characters except for '$' (parameter substitution) and '`' (command substitution). Which means in order to use variables or command substitution in aliases, you have to use double quotes. If you look at the example above, I could define a variable called MUOHOME in .bashrc:

export MUOHOME=$HOME/web/muo/rsmuo/docs

To use that variable in the alias above, I would have to use double quotes:

alias upmuo="rsync -e ssh -z -t -r -vv &#150;&#150;progress $MUOHOME muo:/www/mandrakeuser/docs"

because otherwise the alias would search for an directory or file called $MUOHOME.

You can create aliases 'on the fly' with the alias command on the command line, or list them in ~/.bashrc (per user), or in /etc/profile.d/alias.sh (for every user and 'root'), in pre-8 Mandrake Linux releases, /etc/bashrc fulfills that function. To delete an alias, simply type: unalias alias. Just running alias will list all the defined aliases on your system.

If you have a look at ~/.bashrc and /etc/profile.d/alias.sh, you'll see that there are already some aliases defined. You can define more than one alias for the same command. Of course, you must make sure that your alias isn't the name of some other program, something like alias rm='ls -l' won't work. You can try this by typing the shortcut you want to use on the command line. If the shell can't find a command with this name, you can use it as an alias.


Some aliases that might be useful (don't forget the quotes1.1 ):

  • alias rpmq='rpm -qa | grep': Now rpmq string will list all installed RPMs which contain string in their name.
  • alias ls='ls -ho &#150;&#150;color | more': ls will now print a colored and paged listing with file sizes in KB.
  • alias use='du &#150;&#150;max-depth=1 | sort -n | more': use gives you a paged list of subdirectory sizes ordered by size.
  • alias dkd='cd /usr/src/linux/Documentation': Frequently used directories can be aliased as well. Other prospective candidates are for example the subdirectories of '/mnt'.

Aliased directories may also be on removable media: alias dlm='/mnt/cdrom/Mandrake/RPMS/'.

One mnemonic hint: try to start aliases with a similar function always with the same letter. For instance, begin all your directory aliases with a 'd'.

I am sure you will find lots of possibilities for using this feature.

Shell Functions

Writing shell functions already boarders on the topic of shell scripting, which is beyond the scope of this article (and mine ;-)). In fact, shell functions are shell scripts, but they have the advantage of being preloaded and being executed in the same shell (whereas a shell script opens at least one sub-shell).

With shell functions, you can do a lot of things you can't do with aliases. One example:


function apros() { apropos $1 | egrep -v '(3|\(n\)'; }


This defines a new command, called 'apros'. apros name will execute 'apropos name' (i.e. a search command for man pages) and pipe (|) the output of that command through 'egrep' which filters out all man pages from sections '3' and 'n', which usually are not of interest, but tend to mess up the output of the apropos command.

Functions allow you to use arguments given to the function name at any place of the function command. With aliases, only one argument is allowed and that argument has to be at the end of the command line (like in the 'rpmq' alias above).

'$1' is a so-called 'positional parameter', it's a placeholder for the first argument given to the function. Of course there are more.

function apros() { apropos $1 | egrep -v "\($2"; }

If you now run the apros command like this:

apros name man_section_number

it searches for name, but excludes all man pages from man_section_number:

apros menu 3

returns all man page titles which contain 'menu', except those from section 3 (programming). Notice that you have to quote twice and that you have to use double quotes:

  • You must quote the 'egrep' search pattern in order to protect it from the shell.
  • You must use double quotes to get the second positional parameter interpreted correctly.
  • You must quote the round bracket again in order to tell 'egrep' to take it literally and not as a special control character .

Tricky, ain't it? ;-).

Shell functions are handled just like aliases: put them into your .bashrc to have them around permanently.

Where To Go From Here

This article is but the beginning. Shell scripting can help you to automate a lot of tasks, to fix errors in scripts by others yourself and to accustom your Mandrake Linux system to your liking in (almost) every aspect. If you plan to learn one of the more complex programming languages out there, shell scripting is a good place to start because the basic concepts are similar.

The BASH Programming - Introduction HOW-TO will explain the topics covered in this article in more depth and introduce you to the world of shell programming. You can then continue with the very recommendable (and free) Advanced Bash-Scripting Guide by Mendel Cooper.

If you prefer books, I can recommend S. Veeraraghavan's Teach Yourself Shell Programming, Sams Publishing. In contrast, I found Learning the Bash Shell by Newham/Rosenblatt, O'Reilly, rather unhelpful and confusing, but maybe that's just me ;-). [Note: Learning the Bash Shell is an excellent reference. Mark_1830]

Apart from that: practice, practice, practice. Read shell scripts by others and try to understand what they do, how and why. Don't run your test scripts as 'root'. Have fun.

Shell FAQ

How do I turn that &\*# beep off?

With this command:

[user@computer ~]$ setterm -blength 0

Why do I get "bash: command: command not found"?

If it isn't a typo, most likely the command you are trying to execute is located in a directory which is not part of your $PATH. Supply the full path to the command. If you are in the same directory as the command, do ./command. 1.1 Why do I get "bash: command: Permission denied"?

In order for a file to be executable, the execute [permission | BasicsBpermis] must be set for the user who wants to execute the file. Do a:

[user@computer ~]$ chmod 755 file

If that doesn't work, read the article on permissions.

The execution bit is set, I have permissions to execute but permission is denied anyway. Why?

Check the '/etc/fstab' entry of the partition where that file is. Make sure it doesn't contain the option 'noexec'. If it contains the option 'user', the option 'exec' must be set, too.


How do I change the file listing colors?

Copy '/etc/DIR_COLORS' to your home directory and rename it to '.dir_colors'. Everything you need you'll find in that file.


I've put a script 'foo' into my ~/bin directory, but every time I try to run it, a different command also called 'foo' is started. Why?

Have a look at your $PATH and you will see that your personal ~/bin directory is the last or at least very close to the end of the $PATH. The shell will search the directories listed in $PATH one after the other. As soon as it finds the first matching command, it executes this command. If there is a command on your system with the same name as your script, it is likely that this command will be executed, not your script. So rename your script.

What does "bash: command: bad interpreter" mean?

This usually happens with third party installation binaries or Java applications. These applications need a shell of their own, so instead of

command

you have to run

sh command

My terminal freezes everytime I press "<CTRL s>"

Don't do that then ;-). CTRL s sends a scroll lock command to the terminal. To release this lock, just press CTRL q.

Related Resources

Jerry Peek: 'Why Use a Command Line Instead of Windows?'
Eric S. Raymond: 'A Brief History of Hackerdom'
Text-Terminal-HOWTO
~MdkRef. 3,3
~MdkRef. 3,4
Filename Expansion
~MdkRef, 3.5


Revision / Modified: Mark_1830 08:00, 13 July 2009 (UTC)
Revision / Modified: Feb. 28, 2002
Author: Tom Berger


Other resources

To be added


Author : ptyxs (march 2006)