|   |
A Very Nasty Bug!
Introduction
The cNewMenu ActiveX DLL allows you
to create as many new popup menus as you want. and allows your popup menus to have
an unrestricted number of sub-popup menus. This feature is not supported anywhere in Visual
Basic so I had to create the sub-popup menus using the Windows API. And this allowed me
to discover an obsure problem affecting API menu projects. Well, I didn't discover it;
David Isham sent me a screen shot showing a seriously nasty looking failure of the component -
a whole pile of new and rather random sub-menus had been created. And the worst problem was
that the fault
wasn't occuring in any repeatable way, just that it happened from time to time, with different
combinations of new menus and mixed up items!
The Problem
Identifying non-repeatable bugs is never any fun. But I got a hint from the fact that new
sub-menus seemed to be appearing. This made me think a little about the way Windows identifies
menus, and turned out to be the cause of the problem. Now a little discussion.
How Windows Manages Menus
Windows allows you to create pop-up menus using the API call CreatePopupMenu. In the
same way as all windows have handles to identify them on the system (the hWnd property), likewise
all menus have a hMenu handle to identify them. (An irritating ommision from the
VB Menu control is that there is no built-in way to identify the hMenu handle for a given menu
item. If you could do this, all kinds of menu manipulation would be a lot easier in VB).
Within a menu itself, each item you add has to have a unique ID value (a long integer).
When a menu
item is highlighted or clicked, windows sends the ID number to your code which allows you to
identify which item has been selected. It is up to the programmer creating the menu items to
specify this ID number. In the cNewMenu I do this by initialising an ID counter to &H800
and incrementing it for every menu item added. So far so good.
However, Windows has a bizarre way of re-using the menu item ID parameter to identify a sub-menu.
If you attach a sub-menu to a menu item, Windows erases the ID you gave the menu item and
replaces it with the handle to the popup menu you have attached.
This is where it all goes horribly wrong. I (somewhat naively) imagined that Windows would
understand that a submenu was a completely different entity to a menu item ID. It doesn't.
Instead it can get itself horribly confused.
Nothing Comes Easy
The problem shows itself when Windows creates a menu handle which is exactly the same as
the ID for an existing menu item. It turns out that this happens quite frequently. The
CreatePopupMenu call has no idea about where the menu you create using it will be
attached, and therefore can't ensure that the handle returned doesn't clash with any existing
menu IDs. It turns out that Windows can't tell the difference between a menu item ID and a
menu handle. As a consequence if you try to show a menu which has the same ID for an item
as a menu handle, Windows shows items on a random basis - it could be the menu item, it could
be the pop-up menu, or even more bizarrely, it could be a mixture of the two.
If you have tried to use this component, and ended up with incorrect menus, or menus which drew
part of the screen behind the menu instead of the menu, then this was the cause of the problem.
Why Are You Being So Reasonable Now?
The fix for this problem is really quite simple, and is implemented in two parts:
- Whenever you create a new menu item ID, you have to check not just that the ID is
unique compared to existing IDs you have created but also against all sub-menu handles.
- Whenever you call CreatePopupMenu, you have to check that the menu handle it returns
doesn't match any existing menu item IDs. If it does, you have to delete that menu and retry
the call again until you get a unique ID.
To prevent my component killing your
application by hitting a continuous loop here, the code I implemented has a retry
counter to ensure it doesn't try more than a maximum of 100 times to get a unique ID.
If after 100 tries you still
don't get one, the code fails a little more gracefully by refusing to add the pop-up menu.
You are very unlikely to get into this position. Even with more than 255 sub-menus
in a popup menu, the code has never had to retry more than about 4 times in my testing.
Anyone Can Make a Mistake
The thing that alarms me about this problem is there seems to be no reference to any issues
with clashing menu item IDs and sub-menus anywhere in the Win32 SDK documentation about menus.
There's no guidance about how you should go about choosing menu item IDs either! How is a
programmer supposed to work this sort of thing out without finding out the hard way?
Correct
me if I'm wrong here.
I suppose you could argue that I shouldn't have been trying to create multiple sub-menus at
run-time anyway - you're not allowed to do it in Visual Basic, after all. Don't even go
there! VB's menu system has not changed for as long as VB has existed, and has yet to offer
an easy to use designer or a way to create new menu sublevels. This deficiency is highlighted
by the fact that
many other applications clearly give you access to unlimited menus created at run time.
Lets take some examples: Explorer, Internet Explorer, the Start Menu. Since these are part
of the core of the operating system (although with IE I say this under advisement pending
the outcome of MS'
anti-trust trial :) there surely can't be any excuse for these facilities to be impossible in VB...
 Back to top
 Back to Popup Menu ActiveX DLL
 Back to Source Code
|
  |
