Debugging with the BASH debugger

This file describes the BASH debugger, the BASH symbolic debugger.

This is the 3.1-0.09 Edition, 27 June 2007, for BASH.

Copyright (C) 2002, 2003, 2004, 2006, 2007 Rocky Bernstein

Appendices

Indexes (nodes containing large menus)

1. Summary of the BASH Debugger

The purpose of a debugger such as the BASH debugger is to allow you to see what is going on “inside” a bash script while it executes.

the BASH debugger can do four main kinds of things (plus other things in support of these) to help you catch bugs in the act:

• Start your script, specifying anything that might affect its behavior.
• Make your script stop on specified conditions.
• Examine what has happened, when your script has stopped.
• Change things in your script, so you can experiment with correcting the effects of one bug and go on to learn about another.

Although you can use the BASH debugger to debug scripts written in BASH, it can also be used just as a front-end for learning more about programming in BASH. As an additional aid, the debugger can be used within the context of an existing script with its functions and variables that have already been initialized; fragments of the existing can be experimented with by entering them inside the debugger.

1.1 A Sample BASH Debugger Session

You can use this manual at your leisure to read all about the BASH debugger. However, a handful of commands are enough to get started using the debugger. This chapter illustrates those commands.

Below we will debug a script that contains a function to compute the factorial of a number: fact(0) is 1 and fact(n) is n*fact(n-1).

$ bashdb -L .  /tmp/fact.sh
Bourne-Again Shell Debugger, release bash-3.1-0.09
Copyright 2002, 2003, 2004, 2006, 2007 Rocky Bernstein
This is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.

(/tmp/fact.sh:9):
  9:	echo fact 0 is: `fact 0`
bashdb<0> -
  1:    #!/usr/local/bin/bash
  2:    fact() {
  3:    ((n==0)) && echo 1 && return
  4:    ((nm1=n-1))
  5:    ((result=n*`fact $nm1`))
  6:    echo $result
  7:    }
  8:    
  9:==> echo fact 0 is: `fact 0`
bashdb<1> list
 10:   echo fact 3 is: $(fact 3)

The command invocation uses the option “-L .” Here we assume that the bashdb script and the debugger files are in the same location. If you are running from the source code, this will be the case. However if bashdb has been installed this probably won't be true and here you probably don't need to use “-L .” Instead you would type simply bashdb /tmp/fact.sh.

Position information consists of a filename and line number, e.g. (/tmp/fact.sh:9) and is given parenthesis. This position format is similar to that used by the Perl debugger and is also in the same format used by my GNU make debugger ( http://bashdb.sourceforge.net/remake and the Extended Python Debugger http://remake.sourceforge.net/pydb. GNU Emacs and DDD can parse this and in fact the same regular expression is used on the 3 debuggers.

The first debugger command we gave -, we listed a window of lines before where we were executing. Because the window, 10 lines, is larger than the number of lines to the top of the file we printed only 9 lines here. The next command, list, starts from the current line and again wants to print 10 lines but because there are only one remaining line, that is what is printed.

bashdb<2> step
(/tmp/fact.sh:9):
fact 0
9:	echo fact 0 is: `fact 0`
bashdb<(3)> <RET>
2:	fact() {
bashdb<(4)> <RET>
3:	((n==0)) && echo 1 && return
bashdb<(5)> print $n

bashdb<(6)>

Ooops... The variable n isn't initialized.(1)

The first step command steps the script one instruction. It may seem odd that the line printed is exactly the same one as before. What has happened though is that we've “stepped” into the subshell needed to run `fact 0`; we haven't however started running anything inside that subshell yet though.

To indicate that which piece of the multi-part line echo fact 0 is: `fact 0` we show that part all by itself fact 0. If nothing is shown then it means we are running the beginning statement or in this case the outermost statement.

To indicate that we are now nested in a subshell, notice that the command number, starting with 3, or the third command entered, now appears in parenthesis. Each subshell nesting adds a set of parenthesis.

The first step command steps the script one instruction; it didn't advance the line number, 9, at all. That is because we were stopping before the command substitution or backtick is to take place. The second command we entered was just hitting the return key; bashdb remembers that you entered step previously, so it runs the step rather than next, the other alternative when you hit <RET>. Step one more instruction and we are just before running the first statement of the function.

Next, we print the value of the variable n. Notice we need to add a preceding dollar simple to get the substitution or value of n. As we will see later, if the pe command were used this would not be necessary.

We now modify the file to add an assignment to local variable n and restart.

bashdb<6> restart
Restarting with: /usr/local/bin/bashdb -L . fact.sh
(/tmp/fact.sh:10):
10:	echo fact 0 is: `fact 0`
bashdb<0> list 1
  1:    #!/usr/local/bin/bash
  2:    fact() {
  3:    local -i n=${1:0}
  4:    ((n==0)) && echo 1 && return
  5:    ((nm1=n-1))
  6:    ((result=n*`fact $nm1`))
  7:    echo $result
  8:    }
  9:    
 10:==> echo fact 0 is: `fact 0`
bashdb<1> s 3
(/tmp/fact.sh:3):
3:	local -i n=${1:0}
bashdb<(2)> step
(/tmp/fact.sh:4):
4:	((n==0)) && echo 1 && return
bashdb<(3)> print $n
print $n
0

This time we use the list debugger command to list the lines in the file. From before we know it takes three step commands before we get into the fact() function, so we add a count onto the step command. Notice we abbreviate step with s; we could have done likewise and abbreviated list with l.

bashdb<(4)> <RET>
(/tmp/fact.sh:4):
4:	((n==0)) && echo 1 && return
echo 1
bashdb<(5)> <RET>
(/tmp/fact.sh:4): 
4:	((n==0)) && echo 1 && return
return

Again we just use <RET> to repeat the last step commands. And again the fact that we are staying on the same line 4 means that the next condition in the line is about to be executed. Notice that we see the command (echo 1 or return) listed when we stay on the same line which has multiple stopping points in it. Given the information above, we know that the value echo'ed on return will be 1.

bashdb<(6)> <RET>
fact 0 is: 1
(/tmp/fact.sh:12): 
12:	echo fact 3 is: $(fact 3)
bashdb<(7)> break 5
Breakpoint 1 set in file fact.sh, line 5.
bashdb<(8)> continue

We saw that we could step with a count into the function fact(). However above took another approach: we set a stopping point or “breakpoint” at line 5 to get us a little ways into the fact() subroutine. Just before line 5 is to executed, we will get back into the debugger. The continue command just resumes execution until the next stopping point which has been set up in some way.

(/tmp/fact.sh:5):
5:      ((nm1=n-1))
Breakpoint 1 hit(1 times).
bashdb<(8)> x n-1
2
bashdb<(9)> s
(/tmp/fact.sh:5):
6:     ((result=n*`fact $nm1`))
bashdb<(10)> c
fact.sh: line 6: ((: result=n*: syntax error: operand expected (error token is "*")
bashdb<(7)> R
Restarting with: bash --debugger fact.sh 
11:	echo fact 0 is: `fact 0`
bashdb<0> l fact
 2:    fact () 
 3:    { 
 4:       local -i n=${1:0};
 5:       (( "n==0" )) && echo 1 && return;
 6:       (( nm1=n-1 ));
 7:       ((fact_nm1=`fact $nm1`))
 8:       (( "result=n*fact_nm1" ));
 9:       echo $result
10:    }

In addition to listing by line numbers, we can also list giving a function name. Below, instead of setting a breakpoint at line 5 and running “continue” as we did above, we try something slightly shorter and slightly different. We give the line number on the “continue” statement. This is a little different in that a one-time break is made on line 5. Once that statement is reached the breakpoint is removed.

bashdb<1> continue 5
One-time breakpoint 1 set in file fact.sh, line 5.
fact 0 is: 1
(/tmp/fact.sh:5):
5:	((nm1=n-1))
bashdb<(2)> s
6:	((fact_nm1=`fact $nm1`))
bashdb<(3)> s
2:	fact() {
bashdb<(4)> T
->0 in file `fact.sh' at line 2
##1 fact("3") called from file `fact.sh' at line 12
##2 source("fact.sh") called from file `/usr/local/bin/bashdb' at line 154
##3 main("fact.sh") called from file `/usr/local/bin/bashdb' at line 0
bashdb<(5)> c
fact 3 is: 6
Debugged program terminated normally. Use q to quit or R to restart.

When we stop at line 5 above, we have already run fact(0) and output the correct results. The output from the program “fact 0 is: 1” is intermixed with the debugger output. The T command above requests call stack output and this confirms that we are not in the fact(0) call but in the fact(3) call. There are 4 lines listed in the stack trace even though there is just one call from the main program. The top line of the trace doesn't really represent a call, it's just where we currently are in the program. That last line is an artifact of invoking bash from the bashdb script rather than running bash --debugger.

The last message in the output above ‘Debugged program exited normally.’ is from the BASH debugger; it indicates script has finished executing. We can end our bashdb session with the bashdb quit command.

Above we did our debugging session on the command line. If you are a GNU Emacs user, you can do your debugging inside that. Also there is a(nother) GUI interface called DDD that supports the BASH debugger.

1.2 Interactive Line Tracing Session

One of the things I had found annoying about “set -x” tracing is that no position information given in the trace output, in particular the line number and the file name. However I learn in http://raz.cx/blog/2005/08/handy-bash-debugging-trick.html that one can set $PS4 to rectify this.

Here's what I use:

  export PS4='(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]}\n'

This can be set in one's .bashrc, or added as a line in your script before “set -x”.

There is also facility inside the bash debugger showing position information when tracing a script. Here's a simple session.

/usr/local/bin/bashdb /tmp/fact.sh
Bourne-Again Shell Debugger, release bash-3.1-0.09
Copyright 2002, 2003, 2004, 2006, 2007 Rocky Bernstein
This is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.

(/tmp/fact.sh:11):
11:	echo fact 0 is: `fact 0`
bashdb<0> set linetrace on
bashdb<1> cont
(/tmp/fact.sh:11):
level 1, subshell 1, depth 0:	echo fact 0 is: `fact 0`
fact 0
(/tmp/fact.sh:2):
level 1, subshell 1, depth 1:	fact() {
(/tmp/fact.sh:3):
level 1, subshell 1, depth 1:	    local -i n=${1:0}
(/tmp/fact.sh:4):
level 1, subshell 1, depth 1:	    ((n==0)) && echo 1 && return
(/tmp/fact.sh:4):
level 1, subshell 1, depth 1:	    ((n==0)) && echo 1 && return
echo 1
(/tmp/fact.sh:4):
level 1, subshell 1, depth 1:	    ((n==0)) && echo 1 && return
return
fact 0 is: 1
(/tmp/fact.sh:13):
level 1, subshell 0, depth 0:	echo fact 3 is: $(fact 3)
(/tmp/fact.sh:13):
level 1, subshell 1, depth 0:	echo fact 3 is: $(fact 3)
fact 3
(/tmp/fact.sh:2):
level 1, subshell 1, depth 1:	fact() {
(/tmp/fact.sh:3):
level 1, subshell 1, depth 1:	    local -i n=${1:0}
(/tmp/fact.sh:4):
level 1, subshell 1, depth 1:	    ((n==0)) && echo 1 && return
(/tmp/fact.sh:5):
level 1, subshell 1, depth 1:	    ((nm1=n-1))
(/tmp/fact.sh:6):
level 1, subshell 1, depth 1:	    ((fact_nm1=`fact $nm1`))
(/tmp/fact.sh:6):
level 1, subshell 2, depth 1:	    ((fact_nm1=`fact $nm1`))
fact $nm1
(/tmp/fact.sh:2):
level 1, subshell 2, depth 2:	fact() {
...
level 1, subshell 4, depth 4:	fact() {
(/tmp/fact.sh:3):
level 1, subshell 4, depth 4:	    local -i n=${1:}
(/tmp/fact.sh:4):
level 1, subshell 4, depth 4:	    ((n==0)) && echo 1 && return
(/tmp/fact.sh:4):
level 1, subshell 4, depth 4:	    ((n==0)) && echo 1 && return
echo 1
(/tmp/fact.sh:4):
level 1, subshell 4, depth 4:	    ((n==0)) && echo 1 && return
return
(/tmp/fact.sh:7):
level 1, subshell 3, depth 3:	    ((result=n*fact_nm1))
(/tmp/fact.sh:8):
level 1, subshell 3, depth 3:	    echo $result
(/tmp/fact.sh:7):
level 1, subshell 2, depth 2:	    ((result=n*fact_nm1))
(/tmp/fact.sh:8):
level 1, subshell 2, depth 2:	    echo $result
(/tmp/fact.sh:7):
level 1, subshell 1, depth 1:	    ((result=n*fact_nm1))
(/tmp/fact.sh:8):
level 1, subshell 1, depth 1:	    echo $result
fact 3 is: 6
(/usr/local/bin/bashdb:260):
level 1, subshell 0, depth -1:	
Debugged program terminated normally. Use q to quit or R to restart.
bashdb<2> 

An explanation of the output. The level is how many invocations of BASH are in effect before the statement shown is executed. The subshell is how many subshells you are nested in. Subshells are used by command substitution—`..' and $(...)—as well as arithmetic expressions ((...)). The depth is the function depth or how many calls you are nested in. A “source” command also increases this depth.

Notice also that in contrast to set -x tracing, the line shown is exactly as you entered it in the source. So if you indented statements in a meaningful way, it will help you understand the statement nesting level. But as before, if a line contains multiple statements, you are not executing the first statement in the line and set showcommand is not turned off (by default it is on), that statement is shown in addition below the multi-statement line. Such an example can be seen right at the beginning where fact 0 is shown.

If what you want to do is trace the entire script as was done above (and not stop in the debugger when the script is over), you can get the same effect by using the -X or --trace option on the bashdb command:

/usr/local/bin/bashdb -X /tmp/fact.sh
Bourne-Again Shell Debugger, release bash-3.1-0.09
Copyright 2002, 2003, 2004, 2006, 2007 Rocky Bernstein
This is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.

(/usr/local/bin/bashdb:272):
level 1, subshell 0, depth -1:	  . $_source_file
(/tmp/fact.sh:11):
level 1, subshell 0, depth 0:	echo fact 0 is: `fact 0`
(/tmp/fact.sh:11):
level 1, subshell 1, depth 0:	echo fact 0 is: `fact 0`
fact 0
(/tmp/fact.sh:2):
level 1, subshell 1, depth 1:	fact() {
(/tmp/fact.sh:3):
level 1, subshell 1, depth 1:	    local -i n=${1:0}
...
level 1, subshell 2, depth 2:	    echo $result
(/tmp/fact.sh:7):
level 1, subshell 1, depth 1:	    ((result=n*fact_nm1))
(/tmp/fact.sh:8):
level 1, subshell 1, depth 1:	    echo $result
fact 3 is: 6
(/usr/local/bin/bashdb:285):
level 1, subshell 0, depth -1:	

If you issue a break (e.g. send a SIGINT signal) while the program is running you will go into the debugger (assuming your program doesn't trap SIGINT).

Free software

the BASH debugger is free software, protected by the GNU General Public License (GPL). The GPL gives you the freedom to copy or adapt a licensed program—but every person getting a copy also gets with it the freedom to modify that copy (which means that they must get access to the source code), and the freedom to distribute further copies. Typical software companies use copyrights to limit your freedoms; the Free Software Foundation uses the GPL to preserve these freedoms.

Fundamentally, the General Public License is a license which says that you have these freedoms and that you cannot take these freedoms away from anyone else.

Free Software Needs Free Documentation

The biggest deficiency in the free software community today is not in the software—it is the lack of good free documentation that we can include with the free software. Many of our most important programs do not come with free reference manuals and free introductory texts. Documentation is an essential part of any software package; when an important free software package does not come with a free manual and a free tutorial, that is a major gap. We have many such gaps today.

Consider Perl, for instance. The tutorial manuals that people normally use are non-free. How did this come about? Because the authors of those manuals published them with restrictive terms—no copying, no modification, source files not available—which exclude them from the free software world.

That wasn't the first time this sort of thing happened, and it was far from the last. Many times we have heard a GNU user eagerly describe a manual that he is writing, his intended contribution to the community, only to learn that he had ruined everything by signing a publication contract to make it non-free.

Free documentation, like free software, is a matter of freedom, not price. The problem with the non-free manual is not that publishers charge a price for printed copies—that in itself is fine. (The Free Software Foundation sells printed copies of manuals, too.) The problem is the restrictions on the use of the manual. Free manuals are available in source code form, and give you permission to copy and modify. Non-free manuals do not allow this.

The criteria of freedom for a free manual are roughly the same as for free software. Redistribution (including the normal kinds of commercial redistribution) must be permitted, so that the manual can accompany every copy of the program, both on-line and on paper.

Permission for modification of the technical content is crucial too. When people modify the software, adding or changing features, if they are conscientious they will change the manual too—so they can provide accurate and clear documentation for the modified program. A manual that leaves you no choice but to write a new manual to document a changed version of the program is not really available to our community.

Some kinds of limits on the way modification is handled are acceptable. For example, requirements to preserve the original author's copyright notice, the distribution terms, or the list of authors, are ok. It is also no problem to require modified versions to include notice that they were modified. Even entire sections that may not be deleted or changed are acceptable, as long as they deal with nontechnical topics (like this one). These kinds of restrictions are acceptable because they don't obstruct the community's normal use of the manual.

However, it must be possible to modify all the technical content of the manual, and then distribute the result in all the usual media, through all the usual channels. Otherwise, the restrictions obstruct the use of the manual, it is not free, and we need another manual to replace it.

Please spread the word about this issue. Our community continues to lose manuals to proprietary publishing. If we spread the word that free software needs free reference manuals and free tutorials, perhaps the next person who wants to contribute by writing documentation will realize, before it is too late, that only free manuals contribute to the free software community.

If you are writing documentation, please insist on publishing it under the GNU Free Documentation License or another free documentation license. Remember that this decision requires your approval—you don't have to let the publisher decide. Some commercial publishers will use a free license if you insist, but they will not propose the option; it is up to you to raise the issue and say firmly that this is what you want. If the publisher you are dealing with refuses, please try other publishers. If you're not sure whether a proposed license is free, write to licensing@gnu.org.

You can encourage commercial publishers to sell more free, copylefted manuals and tutorials by buying them, and particularly by buying copies from the publishers that paid for their writing or for major improvements. Meanwhile, try to avoid buying non-free documentation at all. Check the distribution terms of a manual before you buy it, and insist that whoever seeks your business must respect your freedom. Check the history of the book, and try to reward the publishers that have paid or pay the authors to work on it.

The Free Software Foundation maintains a list of free documentation published by other publishers, at http://www.fsf.org/doc/other-free-books.html.

2. Getting in and out

This chapter discusses how to start the BASH debugger, and how to get out of it. The essentials are:

• type ‘bash --debugger script-name’ or ‘bashdb script-name’ to start the BASH debugger. Or...
• type ‘bashdb -c command string’ to give a string to run under the debugger. Or ..
• modify your program to enter the debugger at a particular point: source ../bashdb-trace and _Dbg_debugger.
• type quit or C-d inside the debugger to exit.

There are also two front-ends available as well. One can also enter the debugger inside emacs via the command M-x bashdb after loading Emacs' Grand Unified Debugger, gud. See Using the BASH debugger from GNU Emacs. And there is support in a DDD for bash.

2.1 Starting the BASH debugger

Note: it is important to use a debugger-enabled bash. You will get an error message if the debugger is run under a version of BASH that does not have debugging support.

As mentioned above, one can enter the BASH debugger via Emacs or DDD. However you don't have to use either of these. And these still need a way on their own to get things started.

There are in fact two other ways to start the BASH debugger. The first way is to pass the ‘--debugger’ option to bash with the name of your script the scripts arguments following that, or with a command string (-c).

bash --debugger script script-arguments...
bash --debugger -c command-string...

This calls a debugger initialization script. It works much like a BASH login profile which may set variables and define functions. But this shell profile is customized for debugging and as such arranges for itself to get called before each statement is executed. Although there are some problems at present in I/O redirection that the method described next doesn't have, it is expected that over time more features will be enabled in bash when the ‘--debugger’ option is in effect. By default, both debugging in Emacs via GUD (Using the BASH debugger under Emacs) and debugging via DDD work via this method.

The form ‘bash --debugger -c ...’ can be used to get into the debugger without having to give a script name to debug. Sometimes you may want to do this just to see how the debugger works: try some debugger commands or maybe get online help. If you run ddd --bash without giving a script name, it in fact uses this form.

In order for the ‘--debugger’ option to work however, you must have the debugger scripts installed in a place where the BASH debugger can find them. For this reason, in developing the BASH debugger, I use a second method more often; it doesn't require the bash debugger to be installed. This method uses another script called bashdb which allows for giving its own options, the final option is signaled by adding --). After this, the name of the script to debugged and any the arguments to pass to that script are given. Using this method, one would start the debugger like this:

bash path-to-bashdb/bashdb bashdb-options -- script script-arguments...

If you don't need to pass dash options to your program which might get confused with the debugger options, then you don't need to add the --.(2)

As with the first method, bash should be a debugger-enabled bash. If bashdb has the path to bash in it at the top (e.g. via #!), and bashdb can be found in your program-search path, then this might be equivalent to the above:

bashdb bashdb-options -- script script-arguments...

There are two or three disadvantages however of running a debugger this way. First $0 will have the value bashdb rather than the script you are trying to run. For some scripts this may change the behavior of the debugged script. Second a traceback will contain additional lines showing the “source”-ing of the debugged script from bashdb. And third, although this way works better than the first method, over time this way may come into disuse.

An option that you'll probably need to use if bashdb isn't installed but run out of the source code directory is ‘-L’ which specifies the directory that contains the debugger script files.

You can further control how bashdb starts up by using command-line options. bashdb itself can remind you of the options available.

Type

bashdb -h

to display all available options and briefly describe their use.

When the bash debugger is invoked either by the bashdb front-end script or bash --debugging, the first argument that does not have an associated option flag for bashdb or bash (as the case may be) is used as the name a the script file to be debugged, and any following options get passed the debugged script.

Options for the bashdb front-end are shown in the following list.

2.1.1 Command-line options for bashdb script

You can run the BASH debugger in various alternative modes—for example, in batch mode or quiet mode. If getopts support is available the script also allows long-form command options.

-h | --help

This option causes the BASH debugger to print some basic help and exit.

-V

This option causes the BASH debugger to print its version number, no-warranty blurb, and exit.

-c | --command cmd

Run the string instead of running a script

-B | --basename

This option causes the BASH debugger to print its version number and no-warranty blurb, and exit.

-n | --nx | --no-init

Do not execute commands found in any initialization files. Normally, BASH executes the commands in these files after all the command options and arguments have been processed. See section Command files.

-q | --quiet

“Quiet”. Do not print the introductory and copyright messages. These messages are also suppressed in batch mode.

-t | --terminal | --tty tty

“Terminal output”. Set the file or terminal that you want debugger command output to go to. Note that the debugger output is independent of the debugged script output.

o

-L | --library directory

Set directory where debugger files reside to directory. The default location is ../lib/bashdb relative to the place that the bashdb script is located. For example if bashdb is located in /usr/local/bin/bashdb, the default library location will be /usr/local/lib/bashdb which may or may not exist. If it doesn't you'll get an error when you run bashdb. Only if the default location is incorrect, should you need to use the -L option.

-T | --tempdir directory

Set directory to use for writing temporary files.

2.2 Quitting the BASH debugger

An interrupt (often C-c) does not exit from the BASH debugger, but rather terminates the action of any the BASH debugger command that is in progress and returns to the BASH debugger command level. Inside a debugger command interpreter, use quit command (see section Quitting the BASH debugger).

There way to terminate the debugger is to use the kill command. This does more forceful kill -9. It can be used in cases where quit doesn't work.

2.3 Calling the BASH debugger from inside your program

Running a program from the debugger adds a bit of overhead and slows down your program quite a bit, and addressing this better would mean some serious changes to BASH internals. If you have a configure script generated by autoconf, and you want to stop in the middle of the script, it can take quite a while.

Furthermore, by necessity, debuggers change the operation of the program they are debugging. And this can lead to unexpected and unwanted differences. It has happened so often that the term “Heisenbugs” (see http://en.wikipedia.org/wiki/Heisenbug) was coined to describe the situation where the addition of the use of a debugger (among other possibilities) changes behavior of the program so that the bug doesn't manifest itself anymore.

Another way to get into the debugger which adds no overhead or slowdown until you reach the point at which you want to start debugging. However here you must change the script and make an explicit call to the debugger. Because the debugger isn't involved before the first call, there is no overhead and the script will run at the same speed as if there were no debugger.

There are two parts to calling the debugger from inside the script, “sourcing” in the debugger code and then making the call calling the debugger to issue a “step” command. The name of file in source command is bashdb-trace and it is located in the directory where the other bash debugger files live. For example on GNU/Linux if it is in directory /usr/local/share/bashdb, you would first add to a BASH script the line:

    source /usr/local/share/bashdb/bashdb-trace

Although I said that running under the debugger adds overhead which slows down you program, the above command in of itself will not cause any slowdown. If possible, it's best to put this somewhere in the main-line code rather than in a function or in a subshell. If it is put in a function of subshell and you step outside of that, some of the global variables set up in bashdb-trace may be lost. One the other hand if you know your debugging will be confined to just the scope of the source command there is no problem.

The second thing that needs to be done is put add statements to start stepping code, and this line looks like this:

     _Dbg_debugger ; : ; :

(For reasons that are a mystery to me, the stepping seems to kick in only two statements after the call.)

Putting these two components together, here's an example:

  # Lots of stuff here. ...
  x=2
  source <path-to-program>/_Dbg_debugger
  echo There can be lots of other statements between the above
  echo and below line.
  _Dbg_debugger ; : ; :
  y=3

You will be stopped before y=3 in the debugger.

You can also supply options to _Dbg_debugger just as you would to the debugger itself. All of the options listed in Command-line options for bashdb script can be used with the exception of -c (run a command) and of course you don't supply the name of a BASH script.

For example to suppress the banner you could use _Dbg_debugger -q in the above example.

2.3.1 Program-Controlled Line Tracing

You can also turn on and off line tracing. Here's an example

  source <path-to-program>/bashdb-trace
   ...
  _Dbg_linetrace_on
  for i in `seq 10` ; do
     echo $i
  done
  _Dbg_linetrace_off
   BASHDB_QUIT_ON_QUIT=1  

The BASHDB_QUIT_ON_QUIT variable make sure the program doesn't stay inside the debugger after it quits. It can also be set earlier in the program.

Again <path-to-program> is whatever path needed to located <bashdb-trace>. For example it might be </usr/local/share> on some GNU/Linux installations.

2.3.2 Having the debugger intercept signals

You can also use the debugger's signal handler to print signals or go into the debugger when geting a specific signal.

Here are some examples:

    _Dbg_handler INT print showstack nostop  # this is the default
    _Dbg_handler INT                         # same thing
    _Dbg_hander                              # same thing
    _Dbg_handler HUP print stop              # stop in debugger when getting

3. Script Setup inside the BASH Debugger

3.1 Starting your script

After invoking the debugger you should be on the first stoppable line of your program to be debugged. At this point you can issue debugger commands to set breakpoints (see section Setting breakpoints), or watchpoints (see section Setting watchpoints), or start continue the execution of the program (see section Resuming Execution).

restart [args]

run [args]

R [args]

Use the restart command to restart your script under the BASH debugger. Without any arguments, the script name and parameters from the last invocation are used. The BASH debugger tries to maintain the settings, watchpoints, breakpoints, actions and so on. Internally it uses line numbers and filenames to record he position of interesting places in your porgram; so if your program changes some or all of these numbers may be off. Environment variable BASHDB_RESTART_FILE is and a temporary file are used to signal a restart, so you shouldn't uset BASHDB_RESTART_FILE (or any environment variable starting with BASHDB_.

3.2 Command files

A command file for the BASH debugger is a file of lines that are the BASH debugger commands. Comments (lines starting with #) may also be included. An empty line in a command file does nothing; it does not mean to repeat the last command, as it would from the terminal.

When you start the BASH debugger, it automatically executes commands from its init files, normally called ‘.bashdbinit’(3). During startup, the BASH debugger does the following:

1. Reads the init file (if any) in your home directory(4).
2. Processes command line options and operands.
3. Reads the init file (if any) in the current working directory.
4. Reads command files specified by the ‘-x’ option.

The init file in your home directory can set options (such as ‘set complaints’) that affect subsequent processing of command line options and operands. Init files are not executed if you use the ‘-x’ option (see section bashdb script options).

On some configurations of the BASH debugger, the init file is known by a different name (these are typically environments where a specialized form of the BASH debugger may need to coexist with other forms, hence a different name for the specialized version's init file). These are the environments with special init file names:

You can also request the execution of a command file with the source command:

source filename

Execute the command file filename.

The lines in a command file are executed sequentially. They are not printed as they are executed. If there is an error, execution proceeds to the next command in the file.

3.3 Your script's arguments

The arguments to your script can be specified by the arguments of the restart command. They are passed to a shell, which expands wildcard characters and performs redirection of I/O, and thence to your script.

restart with no arguments uses the same arguments used by the previous restart, or those set by the set args command..

set args

Specify the arguments to be used if your program is rerun. If set args has no arguments, restart executes your program with no arguments. Once you have run your program with arguments, using set args before the next restart is the only way to run it again without arguments.

show args

Show the arguments to give your program when it is started.

3.4 Your script's input and output

By default, the script you run under the BASH debugger does input and output to the same terminal that BASH uses. Before running the script to be debugged, the debugger records the tty that was in effect. All of its output is then written to that. However you can change this when using the ‘bashdb’ script using the ‘-t’ option.

info terminal

Displays information recorded by the BASH debugger about the terminal modes your program is using.

Another way to specify where your script should do input and output is with the tty command. This command accepts a file name as argument, and causes this file to be the default for future restart commands. It also resets the controlling terminal for the child process, for future restart commands. For example,

tty /dev/ttyb

directs that processes started with subsequent restart commands default to do input and output on the terminal ‘/dev/ttyb’ and have that as their controlling terminal.

An explicit redirection in restart overrides the tty command's effect on the input/output device, but not its effect on the controlling terminal.

When you use the tty command or redirect input in the restart command, only the input for your script is affected. The input for the BASH debugger still comes from your terminal.

3.5 Script/Debugger Interaction

The BASH debugger and your program live in the same variable space so to speak. BASH does not have a notion of module scoping or lexical hiding (yet) as is found in modern programming langauges and in modern versions of the Korn shell. This then imposes some additional care and awareness.

Most of the variables and functions used inside the BASH debugger start _Dbg_, so please don't use variables or functions with these names in your program.

Note: there are some other variables that begin with just an underscore (_); over time these will be phased out. But until then, avoid those or consult what is used by the debugger. Run ‘bashdb --debugger -c "declare -p"’ to list all the variables in use including those used by the debugger.

A number of environment variables are also reserved for use; these start with BASHDB_. For example: BASHDB_INPUT, BASHDB_LEVEL and, BASHDB_QUIT_ON_QUIT (see section Debug), BASHDB_RESTART_FILE (see section Starting), to name a few. Finally, there are some BASH environment dynamic variables and these start with BASH_. For example BASH_SUBSHELL (see section Debug), BASH_COMMAND (see section Command Display), BASH_LINENO, and BASH_SOURCE to name a few.

Inside the debugger some variables may be redefined. In particular IFS and PS4, and various dollar variables $?, $1, $2, etc. The values before entering the debugger are saved and those variables have their old values restroed when leaving the debugger. However you may notice these difference in various debugger commands. For example examine PS4 might not return the same value as eval declare -p PS4. The former is picking the debugger value while the eval is careful to restore the value to what it was before entering the debugger.

In order to do its work The BASH debugger sets up a DEBUG trap. Consequently a script shouldn't reset this or the debugger will lose control. The BASH debugger also sets up an EXIT handler so that it can gain control after the script finishes. Another signal intercepted is the an interrupt or INT signal. For more information about signal handling, see section Signals

4. BASH Debugger Command Reference

You can abbreviate the long name of the BASH debugger command to the first few letters of the command name, if that abbreviation is unambiguous; and you can repeat the next o rstep commands by typing just <RET>. Some commands which require a parameter, such as print remember the argument that was given to them.

4.1 Command syntax

A BASH debugger command is a single line of input. There is no limit on how long it can be. It starts with a command name, which is followed by arguments whose meaning depends on the command name. For example, the command step accepts an argument which is the number of times to step, as in ‘step 5’. You can also use the step command with no arguments. Some commands do not allow any arguments.

A blank line as input to the BASH debugger (typing just <RET>) means to repeat the previous next or step command.

Any text from a # to the end of the line is a comment; it does nothing. This is useful mainly in command files (see section Command files).

4.2 Getting help (‘help’)

Once inside the BASH debugger, you can always ask it for information on its commands, using the command help.

help

h

You can use help (abbreviated h) with no arguments to display a short list of named classes of commands:

bashdb<0> help
bashdb commands:
List/search source lines:                 Control script execution:
-------------------------                 -------------------------
 l [start|.] [cnt] List cnt lines         T [n]        Stack trace
                   from line start        s [n]        Single step [n times]
 l sub       List source code fn          n [n]        Next, steps over subs
 - or .      List previous/current line   <CR>/<Enter> Repeat last n or s 
 w [line]    List around line             c [linespec] Continue [to linespec]
 f filename  View source in file          L            List all breakpoints
 /pat/       Search forward for pat       b linespec   Set breakpoint
 ?pat?       Search backward for pat      del [n].. or D Delete a/all breaks
                                                         by entry number
Debugger controls:                        skip         skip execution of cmd
-------------------------                 cl linespec  Delete breakpoints by
 H [num]         Show last num commands                line spec
 q [exp] or ^D   Quit returning exp       R [args]     Attempt a restart
 info [cmd]      Get info on cmd.         u [n]        Go up stack by n or 1.
 !n or hi n      Run debugger history n   do [n]       Go down stack by n or 1.
 h or ? [cmd]    Get help on command      W [var]      Add watchpoint. If no
 info [cmd]      Get info on cmd                       no expr, delete all
 show [cmd]      Show settings            We [expr]    Add Watchpoint arith 
                                                       expr
 so file         read in dbg commands     t            Toggle trace
                                          en/di n      enable/disable brkpt,
 set x y         set a debugger variable               watchpoint, or display
 e bash-cmd      evaluate a bash command  tb linespec  Add one-time break
 disp expr       add a display expr       a linespec cmd eval "cmd" at linespec
 M               Show module versions     A            delete all actions
 x expr          evaluate expression      ret          jump out of fn or source
                 (via declare, let, eval) finish       execute until return
 deb             debug into another       cond n exp   set breakpoint condition
                 shell script
 !! cmd [args]   execute shell command "cmd" with "args"
 file filename   Set script filename to read for current source.

Data Examination: also see e, t, x
-------------------------                 
 p variable      Print variable 
 V [[!]pat]      List variable(s) matching or not (!) matching pattern pat
 S [[!]pat]      List subroutine names [not] matching pattern pat

Readline command line editing (emacs/vi mode) is available.
For more help, type h <cmd> or consult online-documentation.

help command

With a command name as help argument, the BASH debugger displays short information on how to use that command.

bashdb<0> help list l linespec List window lines starting at linespec. l min incr List incr lines starting at 'min' linespec. l List next window of lines. l . Same as above. Long command name: list.

In addition to help, you can use the debugger command info to inquire about the state of your script, or the state of the BASH debugger itself. The listings under info in the Index point to all the sub-commands. See section Command Index.

info

This command (abbreviated i) is for describing the state of your program. For example, you can list the arguments given to your script with info args, or list the breakpoints you have set with info breakpoints. You can get a complete list of the info sub-commands with help info.

bashdb<0> help info List of info subcommands: info args -- Argument variables (e.g. $1, $2, ...) of the current stack frame. info breakpoints -- Status of user-settable breakpoints info display -- Show all display expressions info files -- Source files in the program info functions -- All function names info line -- list current line number and and file name info program -- Execution status of the program. info signals -- What debugger does when program gets various signals info source -- Information about the current source file info stack -- Backtrace of the stack info terminal -- Print terminal device info variables -- All global and static variable names info warranty -- Various kinds of warranty you do not have bashdb<1> info source Current script file is parm.sh Contains 34 lines.

4.3 Quitting the BASH debugger (‘quit’, ‘kill’)

quit [expression]

quit [expression [subshell-levels]]

q

To exit the BASH debugger, use the quit command (abbreviated q), or type an end-of-file character (usually C-d). If you do not supply expression, the BASH debugger will try to terminate normally or with exit code 0. Otherwise it will terminate using the result of expression as the exit code.

A simple quit tries to terminate all nested subshells that may be in effect. If you are nested a subshell, this is normally indicated in a debugger prompt by the number of parentheses that the history number is inside — no parenthesis means there is no subshell in effect. The dynamic variable BASH_SUBSHELL also contains the number of subshells in effect.

If you want only to terminate some number of subshells but not all of them, you can give a count of the number of subshells to leave after the return-code expression. To leave just one level of subshell return does almost the same thing. (See see section Returning) There is a subtle difference between the two though: return will leave you at the beginning of the next statement while quit may leave you at the place the subshell was invoked which may be in the middle of another command such as an assingment statement or condition test.

If the environment variable BASHDB_QUIT_ON_QUIT is set, when the program terminates, the debugger will also terminate too. This may be useful if you are debugging a script which calls another script and you want this inner script just to return to the outer script.

kill

k

In situations where quit doesn't work we provide an alternative and more forceful quit command: kill. This sends to the OS non-maskable KILL signal with the debugger process number. No cleanup of temporary files is done by the program.

4.4 Stopping and Resuming Execution

One important use of a debugger is to stop your program before it terminates so that if your script might run into trouble, you can investigate and find out why. However should your script accidently continue to termination, the BASH debugger has arranged for it not to leave the debugger without your explicit instruction. That way, you can restart the program using the same command arguments.

Inside the BASH debugger, your script may stop for any of several reasons, such as a signal, a breakpoint, or reaching a new line after a debugger command such as step. You may then examine and change variables, set new breakpoints or remove old ones, and then continue execution.

4.4.1 Breakpoints, watchpoints (‘break’, ‘tbreak’, ‘watch’, ‘watche’...)

A breakpoint makes your script stop whenever a certain point in the program is reached. For each breakpoint, you can add conditions to control in finer detail whether your script stops.

You can set breakpoints specify the place where your script should stop the break command and its variants (see section Setting breakpoints). These commands allow own to specify the location by line number and file name or function name.

A watchpoint is a special breakpoint that stops your script when the value of an expression changes. There is a different command to set watchpoints (see section Setting watchpoints).

But aside from that, you can manage a watchpoint like any other breakpoint: you delete enable, and disable both breakpoints and watchpoints using the same commands.

You can arrange to have values from your program displayed automatically whenever BASH stops at a breakpoint. See section Automatic display.

The BASH debugger assigns a number to each breakpoint when you create it; these numbers are successive integers starting with one. In many of the commands for controlling various features of breakpoints you use the breakpoint number to say which breakpoint you want to change. Each breakpoint may be enabled or disabled; if disabled, it has no effect on your script until you enable it again.

Watchpoint numbers however are distinguished from breakpoint numbers by virtue of their being suffixed with the either an upper- or lower-case `W'. For example, to enable breakpoint entry 0 along with watchpoint entry 1 you would write ‘enable 1 2w’, the “2w” refers to the watchpoint; “2W” would work just as well.

4.4.1.1 Setting breakpoints (‘break’ ‘tbreak’)

Breakpoints are set with the break command (abbreviated b).

break function

Set a breakpoint at entry to function function.

break linenum

Set a breakpoint at line linenum in the current source file. The current source file is the last file whose source text was printed. The breakpoint will stop your script just before it executes any of the code on that line.

break filename:linenum

Set a breakpoint at line linenum in source file filename; filename has to be one of the files previously read in and has to be specified exactly as the name used when read in. For a list of read-in files, use the ‘info files’ command.

break … if cond

Set a breakpoint with condition cond; evaluate the expression cond each time the breakpoint is reached, and stop only if the value is nonzero—that is, if cond evaluates as true. The expression is evaluated via the let builtin funtion. ‘…’ stands for one of the possible arguments described above (or no argument) specifying where to break. The word “if” is often optional and is necessary only ‘…’ is omitted. See section Break conditions, for more information on breakpoint conditions.

Examples:

bashdb<0> break fn1 Breakpoint 1 set in file parm.sh, line 3. bashdb<1> break 28 Breakpoint 2 set in file parm.sh, line 28. bashdb<2> break parm.sh:29 Breakpoint 3 set in file parm.sh, line 29. bashdb<3> break 28 if x==5 Breakpoint 4 set in file parm.sh, line 28.

tbreak args

Set a breakpoint enabled only for one stop. args are the same as for the break command, and the breakpoint is set in the same way, but the breakpoint is automatically deleted after the first time your program stops there. See section Disabling breakpoints.

info breakpoints [n]

info break [n]

info watchpoints [n]

Print a table of all breakpoints, watchpoints set and not deleted, with the following columns for each breakpoint:

Breakpoint Numbers (‘Num’)

Enabled or Disabled (‘Enb’)

Enabled breakpoints are marked with ‘1’. ‘0’ marks breakpoints that are disabled (not enabled).

Count

The number of times that breakpoint or watchpoint has been hit.

Condition

The arithmetic expression

File and Line (‘file:line’)

The filename and line number inside that file where of breakpoint in the script. The file and line are separated with a colon.

If a breakpoint is conditional, info break shows the condition on the line following the affected breakpoint; breakpoint commands, if any, are listed after that.

info break displays a count of the number of times the breakpoint has been hit.

info break with a breakpoint number n as argument lists only that breakpoint.

Examples:

bashdb<4> info break Breakpoints at following places: Num Type Disp Enb What 1 breakpoint keep y parm.sh:3 2 breakpoint keep y parm.sh:28 3 breakpoint keep y parm.sh:29 4 breakpoint keep y parm.sh:28 No watch expressions have been set. bashdb<5> info break 4 Num Type Disp Enb What 4 breakpoint keep y parm.sh:28 No watch expressions have been set.

the BASH debugger allows you to set any number of breakpoints at the same place in your script. There is nothing silly or meaningless about this. When the breakpoints are conditional, this is even useful (see section Break conditions).

4.4.1.2 Setting watchpoints (‘watch’, ‘watche’)

You can use a watchpoint to stop execution whenever the value of an expression changes, without having to predict a particular place where this may happen. As with the print (see section Examining Data), the idiosyncracies of a BASH or any POSIX shell derivative suggest using two commands. The watch command is just for a single variables; the watche command uses the builtin “let” command to evaluate an expression. If the variable you are tracking can take a string value, issuing something like ‘watch foo’ will not have the desired effect—any string assignment to foo will have a value 0 when it is assigned via “let.”

watch var

Set a watchpoint for a variable. the BASH debugger will break when the value of var changes. In this command do not add a leading dollar symbol to var.

watche expr

Set a watchpoint for an expression via the builtin “let” command. the BASH debugger will break when expr is written into by the program and its value changes. Not that this may not work for tracking arbitrary string value changes. For that use watch described earlier.

4.4.1.3 Breakpoint command lists (‘commands’)

commands [bnum]

… command-list …

end

Specify a list of commands for breakpoint number bnum. The commands themselves appear on the following lines. Type a line containing just end to terminate the commands.

To remove all commands from a breakpoint, type commands and follow it immediately with end; that is, give no commands.

With no bnum argument, commands refers to the last breakpoint, watchpoint, or catchpoint set (not to the breakpoint most recently encountered).

Pressing <RET> as a means of repeating the last debugger command is disabled within a command-list.

You can use breakpoint commands to start your program up again. Simply use the continue command, or step, or any other command that resumes execution.

Any other commands in the command list, after a command that resumes execution, are ignored. This is because any time you resume execution (even with a simple next or step), you may encounter another breakpoint—which could have its own command list, leading to ambiguities about which list to execute.

If the first command you specify in a command list is silent, the usual message about stopping at a breakpoint is not printed. This may be desirable for breakpoints that are to print a specific message and then continue. If none of the remaining commands print anything, you see no sign that the breakpoint was reached. silent is meaningful only at the beginning of a breakpoint command list.

The commands echo, output, and printf allow you to print precisely controlled output, and are often useful in silent breakpoints.

For example, here is how you could use breakpoint commands to print the value of x at entry to foo whenever x is positive.

break foo if x>0
commands
silent
printf "x is %d\n",x
cont
end

One application for breakpoint commands is to compensate for one bug so you can test for another. Put a breakpoint just after the erroneous line of code, give it a condition to detect the case in which something erroneous has been done, and give it commands to assign correct values to any variables that need them. End with the continue command so that your program does not stop, and start with the silent command so that no output is produced. Here is an example:

break 403
commands
silent
set x = y + 4
cont
end

4.4.1.4 Deleting breakpoints (‘clear’, ‘delete’)

It may desirable to eliminate a breakpoint or watchpoint once it has done its job and you no longer want your script to stop there. This is called deleting the breakpoint. A breakpoint that has been deleted no longer exists; it is forgotten.

With the clear command you can delete breakpoints according to where they are in your script. With the delete command you can delete individual breakpoints, or watchpoints by specifying their breakpoint numbers. Note: as described below under the “clear” command, “d” is an alias for “clear”, not “delete”.

It is not necessary to delete a breakpoint to proceed past it. the BASH debugger automatically ignores breakpoints on the first instruction to be executed when you continue execution.

clear

Delete any breakpoints at the next instruction to be executed in the selected stack frame (see section Selecting a frame). When the innermost frame is selected, this is a good way to delete a breakpoint where your script just stopped.

It may seem odd that we have an alias “d” for “clear.” It so happens that Perl's debugger use “d” for its delete command and the delete concept in Perl's debugger corresponds to “clear” in GDB. (Perl doesn't have a notion of breakpoint entry numbers). So in order to be compatible with both debugger interfaces, “d” is used as an alias for “clear.” Clear?

clear function

clear filename:function

Delete any breakpoints set at entry to the function function.

clear linenum

d linenum

Delete any breakpoints set at or within the code of the specified line.

delete [breakpoints]

Delete the breakpoints, watchpoints specified as arguments.

If no argument is specified, delete all breakpoints (the BASH debugger asks confirmation, unless you have set confirm off). You can abbreviate this command as de.

Note that for compatibility with Perl's debugger, d means something else: clear.

4.4.1.5 Disabling breakpoints (‘disable’, ‘enable’)

Rather than deleting a breakpoint or watchpoint, you might prefer to disable it. This makes the breakpoint inoperative as if it had been deleted, but remembers the information on the breakpoint so that you can enable it again later.

You disable and enable breakpoints, watchpoints, and catchpoints with the enable and disable commands, optionally specifying one or more breakpoint numbers as arguments. Use info break or info watch to print a list of breakpoints, watchpoints, and catchpoints if you do not know which numbers to use.

A breakpoint, watchpoint, or catchpoint can have any of four different states of enablement:

• Enabled. The breakpoint stops your program. A breakpoint set with the break command starts out in this state.
• Disabled. The breakpoint has no effect on your program.
• Enabled once. The breakpoint stops your program, but then becomes disabled.
• Enabled for deletion. The breakpoint stops your program, but immediately after it does so it is deleted permanently. A breakpoint set with the tbreak command starts out in this state.

You can use the following commands to enable or disable breakpoints, watchpoints, and catchpoints:

disable [breakpoints]

Disable the specified breakpoints—or all breakpoints, if none are listed. A disabled breakpoint has no effect but is not forgotten. All options such as ignore-counts, conditions and commands are remembered in case the breakpoint is enabled again later. You may abbreviate disable as dis.

enable [breakpoints]

Enable the specified breakpoints (or all defined breakpoints). They become effective once again in stopping your program.

Except for a breakpoint set with tbreak (see section Setting breakpoints), breakpoints that you set are initially enabled; subsequently, they become disabled or enabled only when you use one of the commands above. (The command until can set and delete a breakpoint of its own, but it does not change the state of your other breakpoints; see Resuming Execution.)

4.4.1.6 Break conditions (‘condition’)

The simplest sort of breakpoint breaks every time your script reaches a specified place. You can also specify a condition for a breakpoint. A condition is just a BASH expression.

Break conditions can be specified when a breakpoint is set, by using ‘if’ in the arguments to the break command. See section Setting breakpoints. A breakpoint with a condition evaluates the expression each time your script reaches it, and your script stops only if the condition is true.

There is also a notion of a “one-time” breakpoint which gets deleted as soon as it is hit, so that that breakpoint is executed once only.

Conditions are also accepted for watchpoints; you may not need them, since a watchpoint is inspecting the value of an expression anyhow—but it might be simpler, say, to just set a watchpoint on a variable name, and specify a condition that tests whether the new value is an interesting one.

Break conditions can be specified when a breakpoint is set, by using ‘if’ in the arguments to the break command. See section Setting breakpoints. They can also be changed at any time with the condition command.

condition bnum expression

Specify expression as the break condition for breakpoint bnum. After you set a condition, breakpoint bnum stops your program only if the value of expression is true (nonzero).

condition bnum

Remove the condition from breakpoint number bnum. It becomes an ordinary