A more reliable test bed

I've been working on putting together a test bed for the product that my company currently offers. My first cut at the test bed used AutoIt to drive the GUI and NUnit for the testing framework. This worked, but I felt it was too fragile (and perhaps too complex) for our QA folks. I therefore wrote an embedded test framework using my simple command parser that worked better.

First, it didn't require multiple executables (e.g., a separate application to drive the GUI using AutoIt). This reduced the complexity of the system, which was immediately reflected in the amount of documentation and support I had to produce.

Second, one can go off and do other tasks (e.g., read email) while the tests are running. This happened because the test bed no longer sent mouse and keyboard events to the application, so there was no contention for those resources. Responsiveness isn't what I would like, but that's the price I pay (until I get a machine with a multi-core CPU). Note that the window of the application running the tests may come to the top, so you may lose keystrokes if you are typing into another application. This shouldn't, however, interfere with the tests. E.g., the application with the embedded test-bed should ignore those keystrokes.

Third, I realized that this does away with synchronization issues. I am intimately familiar with the problems of dealing with variable wait times. For example, when using a GUI driver such as AutoIt or Silk, the testing application that drives the GUI may need to wait for a network to respond or for a long-running calculation to finish. It is usually unclear how long to allow the GUI driver to wait before assuming that something went wrong. Since this framework is "synchronized" with the application, this issue just disappeared. Admittedly, I should probably add some "time-out" feature (in the future) to handle any hangs, but I'll wait for the QA folks to ask for that before I put any effort into it.

Fourth, since there is no need for the test framework to wait, the test sequence will run faster.

The downside is that adding more tests requires some coding in the application. This, however, is usually straightforward. I find where the event from the GUI is handled and add a little hook function to call that same code, with parameters as necessary.

It's a pleasant suprise to have some positive unintended consquences, instead of (more common) negative side effects.

Ageism?

Don't bother applying for a job at Facebook if you got your college degree before 1980.

The dropdown for "Graduation Date" on their job application doesn't go back that far.

Addendum:

In an industry survey, a majority of technology companies candidly said they would not hire anyone over 40.

New York Times, article in "This Week" section on June 22, 2008, titled "For a Good Retirement, Find Work. Good Luck."

Today's lesson...

I went searching for a line continuation character in DOS. I needed to make my CMD scripts more readable (if such scripts can ever be made readable - I'll use Ant next time). Searching around I found that the caret ("^") was supposed to be the continuation character. I tried it and it didn't work at first, until I realized that I was using it in the middle of a quoted string.

Re-jiggered the long command to put the caret(s) outside of quoted strings and it works like a champ. Not that I want to claim to be an expert in CMD scripts...

A Stitch in Time

I have written a bunch of multi-threaded code in the past few years. (I realize that a "bunch" isn't very quantifiable, but bear with me.) It seemed to work fairly well; the server product I worked on was able to make use of multiple CPUs whenever possible. I tested it on multiple CPU machines and saw all CPUs being fully used under load. Also, others used that product on 8-way boxes with good reliability. I have found and fixed threading bugs in my own code and in other's code. I still, however, find myself looking for patterns to help me deal with threading, to increase my confidence in my (and other's) multi-threaded code.

I don't know if it is the indeterminacy of it, or if it's that I don't feel I can hold a lot of multiple states in my head at once. I also know that the few times I've tried to have my threading code reviewed by others they gave up long before I got any feedback of any kind. I don't know if that was because the code was opaque (though there wasn't very much of it...) or because they weren't able to deal with multi-threaded code any better than I was.

I had a number of concerns about the use of mutexes in that product. My primary concern was the sheer number, both of types and of instances. I felt that I had to make sure that the whole system couldn't deadlock, but given those numbers there was no way that I (or anyone I know) could do that.

The principles I will follow in the future are as follows:

• Don't be afraid of multiple mutexes.
• Do as little as possible while holding locks.
• Never hold a lock infinitely.

I don't know if this will make it easier (and more comfortable and reliable) for me to work on multi-threaded code, but I know I can't go about it as I have in the past.

[title reference]

Visual Basic command parser example

OK, I left the Visual Basic version of this code as an "exercise for the reader," but it turned out that I needed it myself. I translated (by hand) the C# example but couldn't get the Visual Basic version to find the method in the InvokeMember call (even after the FindMembers call had found it). I poked around some more, thinking that once I had the MethodInfo I should be able to use it directly without having to use InvokeMember.

Sure enough, after poking around a bit I found Invoke (on the MethodInfo itself), and it works like a charm.

Maybe some day when I'm feeling more ambitious I'll go back and rework the C# example to match the Visual Basic example, but probably not until I really need to. I actually hope I don't, because that means that I will have a need to use C#, and I'd rather stay cross-platform with something like Java, C++ or Python.

The Third Commandment of Blogger

Don't use Blockquote!

I'm still struggling to figure out what Blogger is doing to my formatting. I'm
not sure if it's this particular template, or something deeper down in Windows, or perhaps some backwards compatibility issue.
I say "deeper down in Windows" because I see the same effect in both Internet
Explorer and Firefox on my Windows XP machine. I don't (currently) have any Linux desktop machines to test this.

First, every time I try to use blockquote both browsers show succeeding paragraphs with a tighter line-spacing than the previous paragraph. This happens with the default font; I haven't tried other fonts because no one it paying me to test this stuff. This paragraph is an example of this effect. Secondly (because of its definition), blockquote takes line endings literally, making for some very strange line breaks as shown above. I like indention to set off text, but not when it comes with this side-effect.


For commandments One and Two, see my previous post. That post shows the same incorrect line-spacing in the numbered list. Also note the extra line-space above this paragraph, which seems be be caused by saving and then re-editing the post. Well, at least I'm not paying anything for these bugs.

We Don't Need No Stinking Comments

I am trying to put together some coding standards for the company where I currently work. Yeah, another exercise, but not (I hope) a futile one. As always happens, someone wanted to have a standard for comments.

My only answer to that was this: "If a program acts differently from the way the end-documentation says it should, which do you trust?" I feel the same way about comments. It's not that I don't appreciate them, but I just don't trust them as much as I trust the code. Admittedly it's usually a harder slog through the code, but once you've figured out what the code does, the comments are usually not useful.

As Kernighan and Plauger say on page 142 (1974 edition) of "The Elements of Programming Style:"

The problem with comments that do not accurately reflect the code is that they may well be believed subconsciously, so the code itself is not examined critically.

I wish I could come up with some good way to measure the effectiveness (or in-effectiveness) of comments, but I'm just going to have to go with my gut on this one. I don't think we can have any standard for comments that anyone will agree upon. We may be able to come up with some recommendations, but as for enforceable standards? In "The Practice of Programming" (Kernighan and Pike, 1999), page 24 generously says:

Comments can be useful, of course.

And the rest of the page goes on to show some good usages. This page, however, is surrounded by a list of things NOT to do with comments. Seems like it's easier to tell people what not to do, rather than what to do.

[If you don't know the American pop culture reference from the title, check here or here.]