DESIGN.txt - Gdb Implementation Notes
This document describes succinctly the design of Vim's gdb interface.

1. File list
2. gdb.c
3. Vim source
4. Known bugs
5. Changes log

The gdb module provides an objet oriented type interface to Vim.

GDB annotations are used: gdb can be extended (for example adding a
'registers' window) by implementing an oobfunc_T function and adding it to the
oobfunc array. This function is invoked at each GDB 'prompt' annotation. The
gdb window status line, assembly buffers and breakpoints are implemented
this way.

Memory leaks can be traced with GNU mtrace. Except for the gdb_T structure, no
memory allocation is done by gdb before the first GDB process is started.
Virtual memory used by the gdb buffer is limited to 64 K.


==============================================================================
1. File list

List of all files changed or added to vim62 source distribution.

	gdb.c				(new)
	macros/gdb_mappings.vim		(new, not required by gdb)

Main changes to Vim source code, they are described in the paragraph 'Vim
source':

	buffer.c
	os_unix.c
	gui.c		(GUI support: follows os_unix.c model)
	gui_gtk_x11.c	(GUI support: follows os_unix.c model)
	gui_x11.c	(GUI support: follows os_unix.c model)
	normal.c

The following Vim files have been changed or added for housekeeping purpose:

    o Gdb is a feature:

	feature.h	(include '+signs' when FEAT_GDB is defined)
	configure.in
	config.mk.in
	Makefile
	config.h.in
	version.c
	eval.c		(support for the new builtin function gdb())

    o Gdb adds new code to Vim source code:

	globals.h
	structs.h
	proto.h
	proto/gdb.pro	(new)
	main.c		(gdb_new() and gdb_delete())
	gui.c		(gdb_delete() in gui_start())
	buffer.c	(clean up in free_buffer())
	window.c	(gdb window cannot be last one in only_one_window())
	screen.c	(status line gdb label in win_redr_status())
	option.c	(options 'gdbprg' and 'asm')
	option.h	(options 'gdbprg' and 'asm')

    o gdb syntax:

	syntax/gdb.vim	    (exhaustive implementation of gdb syntax)
	syntax/gdbvim.vim   (gdb window syntax, gdb.vim is sourced)

    o enable the window cmdline for the input-line and customize
      this window for gdb: <Tab> and CTRL-Z mappings, gdbvim file type

	ex_getln.c


==============================================================================
2. gdb.c

gdb.c defines a gdb_T structure that can be viewed as a class. External code
(i.e. Vim code) passes an opaque handle to this structure. There is only one
instance of this structure for the whole Vim session.

		     gdb.c	     user interface

  GDB -->----  gdb_process_output--
		       |	   |
		       |	'prompt'
	     ----------|---<-- annotation
	    |	       |	   |
	    |	       |	   |
	  O |	  update_output    |
	  u |	       |	   |
	  t |     (pending cmd)    |`Window input-line'
	    |	       |	   |
	  o |	        ----       |   f_gdb("")
	  f |    normal_cmd |      |	  |
	    v	     |      v      v	  v
	  b |	     |      |      |	  |
	  a |   gdb_winput.......gdb_setwinput
	  n |	     |      |
	  d |	     |      |
	    |   ---y or n   |
	  d |  |     |      |	       `GDB command'
	  a |  |     |      |
	  t |  |    gdb_docmd------<---f_gdb("cmd")
	  a |  |     |
	    |  |     |
  GDB --<- send_cmd -


gdb_process_output: parse GDB output for annotations and keep track of the
current command state

gdb_docmd: parse gdb user command before sending them to GDB

update_output: display a new stack frame source line or highlight a
breakpoint; this function may be called at about the lowest level in Vim:

    main_loop	 } main.c
	normal_cmd  } normal.c
	    safe_vgetc		} getchar.c
		vgetc		}
		    vgetorpeek	}
			inchar  }
			    ui_inchar   } ui.c
				mch_inchar  } os_unix.c
				    gdb_process_output	} gdb.c
					update_output	}

The gdb.c code is plugged in the Vim source code at two locations, mch_inchar()
and normal_cmd() as described in the next paragraph. Vim internal data is not
written to directly by gdb.c except:

	got_int			(described in the next paragraph)
	clear_cmdline		(left TRUE by win_do_lines())
	RedrawingDisabled	(to force redrawing)
	emsg_skip		(to avoid displaying unwanted error msgs)
	cmd_silent
	msg_didout

Gdb memory leaks are traced when Vim is compiled with '-DGDB_MTRACE
-DHAVE_MTRACE'. Set MALLOC_TRACE environment variable to an mtrace log file
and use mtrace on this file to search for memory leaks. There must be none.

Highlighting breakpoints in assembly code:
-----------------------------------------
This section describes in pseudo code how oob functions are involved
in highlighting an asm breakpoint when its asm buffer is not yet
disassembled. The matter is complicated by the fact that GDB may
send ANO_BP_INVALID when stepping or continuing, even though the
table did not change.

States:
    BS_BP_HIT	    got ANO_BREAKPOINT and ('enable once' or ! in bpinfo)
    BS_INVALID	    got ANO_BP_INVALID
		 or got ANO_BREAKPOINT and ('enable once' or ! in bpinfo)
    BS_FR_INVALID   got ANO_FRAME_INVALID
    BS_BP_SET	    a break GDB command is being processed

get_lastbp()
    if asm != 0			// assembly is on
	    && BS_INVALID	// GDB reports bp table changed
	    && BS_BP_SET	// a break GDB command is being processed
    {
	set this->asm_add to address of last bp in GDB bp table
	so that function is disassembled in get_asm()
    }

get_asm()
    disassemble this->asm_add
    reuse an old buffer if needed, in this case, in the reused buffer:
	remove Vim buf signs and corresponding bps in bpinfo list
	remove frame sign
    edit buf containing this->asm_add

get_bp()
    When asm is on, the GDB bp table must be fetched whenever stepping
    as we may be editing a newly disassembled function and therefore
    must set again all bps belonging to this buffer:

    if BS_INVALID			// GDB reports table changed
	    || (asm && BS_FR_INVALID)	// in case stepping in a new asm buf
	process_record() for all bps in GDB bp table

process_record()
    if record already in bpinfo list
	update
	change sign if needed
    else
    {
	if an asm bp (<function+in>)
	    search all buffers in the asm buffer pool whose name starts
	    with function and find the one containing instruction address
	else
	    get source file and line number

	if BS_INVALID
	    || (asm && BS_FR_INVALID && buf == fr_buf)
	{
	    edit the file or buffer
	    add bp sign
	    restore the frame sign if needed

	    if editing the frame buffer (fr_buf)
		    && (BS_BP_HIT || (asm && BS_FR_INVALID))
	    {
		set frame sign on top if needed
		set the cursor at frame sign
	    }
	}
    }


==============================================================================
3. Vim source

The '+signs' feature behaves differently when gdb is running. These changes
are implemented in buffer.c:

sign_mark_adjust()
    sign->lnum stay unchanged, whatever the changes made to the buffer, to stay
    consistent with GDB's breakpoint line numbers

Regarding exchanges with GDB, two issues need to be explained here:
    how and when the gdb.c code processes GDB output so that it is ultimately
    displayed in the gdb output window

    how the gdb.c code manages to pop up the gdb window input-line while Vim is
    doing its intricate 'vi' job

Both points are implemented by changes in os_unix.c and normal.c.

The first point is achieved by:
    at the top level, in normal.c, safe_vgetc() is redirected so that GDB output
    is processed only when the low level Vim functions mch_inchar() and
    RealWaitForChar() are invoked from normal mode commands functions

    at the low level, mch_inchar() and RealWaitForChar() cooperate so that
    gdb_process_output() is invoked whenever enabled by the top level, and
    there is something to read from GDB's pseudo terminal and the timeout is
    not zero

The second point is better explained by a diagram which describes what happens
when gdb_process_output() must pop up the window input_line:

    main_loop

	normal_cmd

	    ...

		vgetorpeek

		    ...

			mch_inchar

			    gdb_process_output
				gdb_setwinput
				    set the string to be inserted later
				    in the window input-line
				    got_int = TRUE
				    return TRUE

			mch_inchar
			    return 0

		    ...

		vgetorpeek
		    got_int is TRUE so:
			flush all input
			flush stuff and typeahead buffer
			interrupting all pending mappings
			and commands

	    ...

	normal_cmd
	    gdb_winput
		launch the window input-line

    main_loop


==============================================================================
4. Known bugs

1 - Setting a bp at an invalid instruction address in asm
    Vim option asm=10
    pgm stopped at asm buffer (for example usleep) and frame sign hilighted
    in buffer
    set a bp in usleep at an invalid instruction address
    continue
    error:  two asm buffers usleep-asm and usleep-asm1, one containing the
            frame sign, one containing the bp sign
	    the reason is get_lastbp() sets this->asm_add to invalid address,
	    get_asm() does not find address in existing usleep-asm and
	    therefore disassembles usleep in another identical usleep-asm1
	    buffer
    status: a bug report has been sent to bug-gdb on setting bp at invalid
	    instruction address that causes inferior process to get SIGSEGV,
	    this is not a GDB bug
	    should get_asm() erase the buffer when this->asm_add is not found ?


==============================================================================
5. Changes log

VIM62 patch 4 (cvs vim62-gdb1-4):
    gdb.c:get_lastbp()
	see above section named "Highlighting breakpoints in assembly code"
    gdb.c:get_asm()
	correct handling of new asm buffer names
    gdb.c:parse_note()
	restrict annotation parsing to exact token match, not just the
	beginning (ANO_BP_END was never parsed)

VIM62 patch 3 (cvs vim62-gdb1-3):

    gdb.c:get_bp(), process_record() and process_annotation()
	get_bp() is a new out of band oobfunc_T function that fetches
	the breakpoint info table.
	Support of breakpoints annotations, providing the following
	features:
	    1) breakpoint sign text is the last two digits of the
	       breakpoint number
	    2) disabled breakpoints; GDB commands 'tbreak', 'thbreak',
	       'disable', 'enable once' and 'enable delete' are supported.
	       Disabled breakpoints are highlighted with a different color.
	       An 'enable once' bp that is hit becomes disabled but does not
	       generate an ANO_BP_INVALID annotation, therefore we must track
	       the 'disposition' and 'enable' state of a bp to know
	       when to trigger get_bp() in this case.
	    3) breakpoints in assembly buffers. If the assembly buffer does
	       not exist yet at the time the breakpoint is set, the 'sign'
	       will be 'place'd the next time it is disassembled.
	       In order to do that:
		    a) trigger get_bp() whenever program stops at a bp
		       unknown of bpinfo list and asm option is set
		    b) get_asm() is run before get_bp(), thus we must handle
		       the case where a bp sign ends up on top of the frame
		       sign, which may occur in the above case or if the
		       buffer had been previously wiped-out; this is
		       done in process_record()
	    4) when receiving a frame annotation (ANO_FRAME_BEGIN or
	       ANO_SOURCE), do not edit this buffer if the related
	       breakpoint contains a 'commands' with the 'continue'
	       statement
	Since we do not need anymore to parse gdb buffer for breakpoint
	data, a lot of regexp have been removed as well as the functions:
	update_output, bp_delete, bp_parse_set, bp_parse_delete

    gdb.c:process_annotation()
	Do not parse frame annotations when running a backtrace GDB
	command and receiving an ANO_FRAME_BEGIN annotation.

VIM62 patch 2 (cvs vim62-gdb1-2):

    gdb.c:fr_lite()
	Remove previous frame sign:
	GDB sends ANO_FRAME_INVALID annotations whenever stepping, running,
	etc... and these annotations invoke fr_unlite() that turns off the
	previous frame sign.
	However, when moving along the stack frame with GDB 'up', 'down',
	'frame' commands, we don't get annotations from GDB and must turn off
	the previous frame sign before setting a new frame sign.

    GUI support:gui.c, gui_gtk_x11.c and gui_x11.c
	These changes made to support the gdb interface with a GUI, follow
	the same model as the one used in os_unix.c for a plain terminal.
	Instead of registering as a listener on select() calls, we register a
	call back function. This has been tested with GTK. The changes made
	for GTK have been made for Motif and Athena in gui_x11.c but NOT
	tested, NOT even compiled.

VIM62 initial patch 1 (cvs vim62-gdb1-1):

    Makefile
	Run 'make autoconf' before 'make' to rebuild 'auto/configure' and
	include feature '+gdb'. This was not required with Vim61.

    buffer.c:buf_getsigntype()
	No changes required anymore. Changes were made in Vim61-gdb to reverse
	signs display order.

    buffer.c:buf_addsign()
    buffer.c:buf_delsign()
    buffer.c:insert_sign()
    structs.h - struct signlist
	Use the changes introduced in Vim62 for FEAT_NETBEANS_INTG to reorder
	properly signs list (last sign on top).

    gdb.c:edit_file()
	Do no set curbuf to 'nobuflisted' when failing to edit an asm buffer
	after user canceled the command. This bug still exists in Vim61-gdb.

    gdb.c
	screen.c:update_debug_sign() has been improved in Vim62: the phantom
	sign (sign id #2) must be added *before* the frame sign (sign id #1) is
	deleted, and deleted *after* the frame sign is added.
	This avoids having the displayed buffer with an empty sign list which
	would cause a repaint without the sign columns
	[redraw_buf_later(buf, NOT_VALID); in buffer.c:buf_delsign()]
	and thus screen flickers.

    ex_getln.c:ex_window()
	When setting folds to a plain window, the window input-line gets also
	empty fold columns.
	Fix: set option foldcolumn=0
	This bug still exists in Vim61-gdb.
	This problem also shows up in the standard window command-line.

VIM61 (cvs vim61-gdb1-1):

    Initial version.

    buffer.c:buf_getsigntype()
	return the last added sign->typenr, not the first, so that the screen
	displays a frame highlighting on top of a breakpoint highlighting and is
	not hidden