Software Craftsmanship

Adding RSS feeds in Google Chrome

2009-11-28T10:00:00.001-08:00

Whilst in Google Chrome, I tried clicking on an RSS feed icon in Craigslist and was rewarded with a bunch of XML. To avoid figuring out which piece I needed to add a feed to Google Reader, I did the following:

Went into Firefox
Clicked on the RSS icon
For "Subscribe to this feed using:", I clicked on Google
Then clicked on "Add to Google Reader"

I haven't verified that this actually worked, so I may have to go back and wrestle with the XML, but I'm hoping I don't.

Google Android ArrayAdapter vs SimpleCursorAdapter

2009-10-25T12:03:00.000-07:00

I've been trying to create a "mover" interface in Android. I use the term "mover" to describe a interface displaying two lists where you can move items from one list to the other. This can be done with arrow buttons if you have enough screen real estate, but I don't think that will work on the small screen of a cell phone.

In any case, my first attempt contained two ListViews, one with an ArrayAdapter of Strings, the other with a SimpleCursorAdapter. For reasons which I haven't yet figured out, the items in one list were a different font size than in the other list. The SimpleCursorAdapter font size was smaller than the ArrayAdapter. I tried a number of changes in the layout file (e.g., changing weights and such), but none of them worked.

I ended up reworking my data storage so that I could use SimpleCursorAdapters for both ListViews, and the font sizes matched. I'm guessing that the Adapters (which I consider the "Controller" in the model-view-controller paradigm) are making some decisions "behind the scenes".

Onward to the next Android mystery...

The More Things Change...

2009-10-21T07:18:00.000-07:00

I went to the StackOverflow DevDay in San Francisco on Monday. Lots of information, most of it useful. Mark Harrison's presentation where he "deconstructed" a Peter Norvig Python script gave me a few new pointers about Python. It also got me to thinking about list comprehensions, specifically the "why"? Performance? Possibilities for parallelization?

This morning's Google search was, therefore:

Python "List Comprehension" advantages

which turned up this:

http://techorama.comyr.com/tag/list-comprehension/

He mentions performance (as well as parallelization):

Simple for loops implemented in python are handled by the interpreter, and are slow. List comprehensions and map() have a in-built, fast, internal C implementation which does away with the overhead of the for loop.

Good to know about, but mildly ironic in that it leads back to C code for performance. The 70's meet the 00's.

I miss my Java 1.5

2009-09-07T07:10:00.000-07:00

I've been experimenting with the Blackberry development environment for the last couple of weeks.

On the plus side, it's great that:

It's a free download.
It integrates with Eclipse.
It provides emulators for various Blackberry models.

On the flip side:

I hit a problem using Subversion that I was never able to figure out (and no one posted any answers to my question on the Blackberry forum). I ended up doing a "dual-directory structure" kludge that allows me to continue development, but it's a klumsy kludge.
I miss some of the features from "contemporary" Java. The Blackberry system offers Java J2ME 1.2, which doesn't include features like String.split and reflection. I'm working around these issues, but I'm not sure what else I'll hit as I continue.
There are some anomalies with getting the sizes of controls such as edit fields and buttons. I'll figure it out eventually, but it's getting in the way of making my app portable to non-Storm Blackberry models.

"SQL for Smarties" Errata (part 1)

2009-08-28T07:48:00.000-07:00

I need to learn more about SQL, and someone I respect recommended Joe Celko's book. I've been delving into it over the past few weeks. I'm learning a lot, but have hit a few typographical errors. Give the size of the book, it's not surprising. I would still (even given these errors) recommend this book for someone who has a good knowledge of SQL and wants to learn more. I would, however, caution them to run the SQL examples themselves to verify that the syntax is correct.

Here is the first set of errata. If any of these seem incorrect, don't hesitate to post a correction to my correction.

Code typos:

Page 73:

One of the "course_name" columns should be "course_code"

Page 461:

Condition "AND P2.salary_amt < P1.salary_amt" gives bottom n salaries, not top n.

Page 741:

I can't parse the example "color || = 'Purply Green')". I think it should be "color || '' = 'Purply Green')"

Page 746:

The two SELECT examples are inconsistent. One has a CAST operator and the other doesn't.

Non-code typos:

Page 33, bottom sentence:

"correction" -> "correct"

Technology for Education and Entertainment

2009-07-27T07:05:00.000-07:00

I hadn't had an eye exam in a couple of years, and my doctor has recommended once a year. I made an appointment at the UC Berkeley Optometry Center and showed up at the appointed hour.

To show the eye charts, they now use a flat screen connected to a Windows computer instead of a projector. In the middle of the exam the screen saver kicked in and the screen went black! I just started laughing. The intern was relieved that I found it funny instead of vexing. I actually looked at her computer to see if I could fix it. The screen saver interval was locked down, however, so I left it up to her and their IT department.

They decided to take pictures of the insides of my eyes. It added a little to the fee, but (after I asked) they said they could email the pictures to me. To take the pictures, they had a device made by Canon with a digital camera back (that was then connected to a computer). They tried a couple of times, but couldn't get the picture to show up on the computer. They then called the tech guy. He just unplugged the camera from the computer, plugged it back in, turned off the camera and the device and turned them both back on. And it all worked.

I could have done that!

Is someone gaming Google "Web Clips"?

2009-06-29T18:54:00.000-07:00

In GMail today, I noticed that the "web clip" at the top of the list of emails just said:

Dictionary.com Word of the Day - Presented By: - 18 hours ago

Unclear as to what that meant, I clicked on it, and found myself on what looked like a cleverly-disguised ad for one of these "work-at-home" marketing scams, complete with misspellings:

"We knew we had to do something, so we put our heads together and started trying online job opporuntities."

Can we trust Web Clips any more?

AOL to Google "conversion" project

2009-06-09T12:26:00.000-07:00

I know a couple of folks who are still using AOL as their email (and contact and favorites) service. Some of them are receptive to moving to Google, so I've started working on helping them. A few weeks ago I actually put together a "business plan" for a service that would provide this functionality:

http://docs.google.com/Presentation?id=dgjndsm4_0c68v7vcn

In the meantime, however, Google came out with their conversion service:

http://gmailblog.blogspot.com/2009/05/import-your-mail-and-contacts-from.html

It's not complete, but it's free, and thus that business plan hits the dust. I am, however, going to continue my efforts to help folks that do want to move their data from AOL to Google:

http://www.chriskeith.me/ck/Home/overview

I probably won't be updating this blog with every change that goes here, but will occasionally mention significant milestones. Feel free to pass this on to anyone that uses AOL (if you actually know anyone who does...) but wants to move to Google.

Erratum (part 3) for D. S. Malik's "Data Structures Using C++"

2009-05-25T19:30:00.000-07:00

On page 738 is an explication of Dijkstra's shortest path algorithm. Everything makes sense to me except step 2:

Set smallestWeight[vertex] = 0

This is because step 1 is:

Initialize array with smallestWeight[u] = weights[vertex, u]

Unless I misread something (or the weight from a node to itself isn't zero), I assume that step 1 subsumes step 2. Or am I missing something?

See also other errata.

Time to stop trusting Google Spreadsheets?

2009-05-25T12:06:00.000-07:00

Somehow Google Spreadsheets lost the most recent version of one of my documents. I had edited it (and actually printed it out). When I went back, however, my changes were gone.

I cleared my Firefox cache and restarted my machine, but still didn't get the version that matched my printed version.

I had shared it with someone, so maybe there's a bug somewhere in their sharing code.

Autosave is a nice feature, EXCEPT when it doesn't work. At least I lost less than an hour of work.

I don't think it's a virus or other malware, because I run reasonably heavy security on both my machine and my router.

How about giving us a "Save" button, Google folks?

... a few minutes later..

It looks like a display problem. I can see some of the old documents in the revision history, but they (Firefox? Google?) won't display my changes.

Google doesn't like my domain name?

2009-05-21T09:16:00.000-07:00

Does Google automatically reject any domain names that contain the string "google"?

Visual Studio C++ Release vs. Debug

2009-05-19T17:26:00.000-07:00

Timings for anagram program that was the final assignment in my data structures class:

Release:

Found 199741 words.
Built multimap in 532 milliseconds.
Created anagram file in 125 milliseconds.

Debug:

Found 199741 words.
Built multimap in 2796 milliseconds.
Created anagram file in 3891 milliseconds.

Erratum (part 2) for D. S. Malik's "Data Structures Using C++"

2009-05-14T08:30:00.000-07:00

I know this isn't real code (only pseudocode), but I still think it would be clearer to say this on page 540 (4th line from the bottom)

if (HT[hIndex].key == insertKey)

Unless, of course, I'm completely missing the intent of the example.

See also the first erratum.

Fricasse of Crow

2009-05-13T10:34:00.000-07:00

In my last post I showed a 2-buffer merge sort that I thought was fast. In support of that claim I now offer this picture:

http://picasaweb.google.com/chris.keith/AcademicStuff#5335363116849662498

However, I did some more work with other sorts (and my professor changed his random number generator) and now I get this (be careful, the horizontal scale is different):

http://picasaweb.google.com/chris.keith/AcademicStuff#5335363118138126626

Note that both quicksorts beat my mergesort (thus I anticipate crow for dinner tonight).

Also note that the recursive quicksort is faster than the iterative one, which doesn't seem right. I just pulled that code off the web (and recoded it from C# to C++). Perhaps there's a better iterative one, but that's an investigation for the future.

MergeSort with minimum allocations

2009-04-30T09:09:00.001-07:00

The most recent assignment in my data structures class was writing a sort comparison, including a MergeSort. I realize that MergeSort can't be in-place, but I wondered if there was a way to keep the memory allocations to a minimum. I came up with this, which uses one extra buffer (the same size as the source array). After each pass, the source buffer becomes the target and vice versa. Timings indicate that (for arrays containing random numbers), this beats a quicksort. (Apologies for reformatting introduced by HTML...)

int *IterativeMergeSorter::Sort() {
int* src = mArray;
int* tgt = new int[mLength];
for (int swapSize = 1; swapSize < mLength; swapSize *= 2) {
for (int j = 0; j < mLength; j += (2 * swapSize)) {
int start = j;
int sizeOfSrc1 = min(swapSize, mLength - start);
int src2Index = start + swapSize;
int remainder = max(0, mLength - src2Index);
int sizeofSrc2 = min(swapSize, remainder);
Merge(&tgt[start], &src[start], sizeOfSrc1,
&src[src2Index], sizeofSrc2);
}
int* temp = src;
src = tgt;
tgt = temp;
}
if (src != mArray) {
memcpy(mArray, src, mLength * sizeof(int));
tgt = src;
}
delete[] tgt;
return mArray;
}

Erratum (part 1) for D. S. Malik's "Data Structures Using C++"

2009-04-27T08:00:00.000-07:00

I am currently taking a Data Structures course at Chabot College. Let's just say I'm aspiring to be a Junior Programmer (only half-kidding...). The course uses the textbook (copyright 2003) mentioned in the title of this post. I realize that no textbook is perfect, but there is at least one error that will have repercussions on the grades of the students that use this text.

In table 5-7 on page 331, the text describing the pop_front method on the STL list container says

Removes the last element from the list.

Unless I'm missing something, this should read "Removes the first element from the list." See also this.

This will have repercussions on student's grades because there is a question in the on-line materials that was directly lifted from this sentence. When I took that quiz I kept getting this question wrong, and couldn't understand why. I then went back and re-read the chapter and saw this.

Caveat Emptor.

Going Paperless: Day 5

2009-04-06T07:00:00.000-07:00

We get three newspapers at my house: the New York Times, the San Francisco Chronicle (as long as it remains in business) and the weekly freebie San Leandro Times. I go for the NYT first and my wife goes for the Chron. I will scan the headlines of the main and "Local" sections of the Chron, and (more often than not) read the comics in the lulls during preparing dinner.

The laptop just won't work for this.

First, it sleeps too frequently (probably because it's on battery). This can be changed.

Secondly, I don't like the idea of getting food on it (as opposed to the physical paper).

Thirdly, I read articles in the paper by first scanning them, taking in the first sentence of each paragraph. If I then want more information, Ithen go back and read the article in more detail. The laptop screen isn't big enough for this.

So I'm 1 out of 3:

Not really worth reading the news on the laptop
Not worth replacing pencil and paper for jotting notes
Definitely worth watching Hulu (and whatever comes after)

Going Paperless: Day 4

2009-04-06T05:49:00.000-07:00

I have a habit of jotting notes to myself. These notes are usually list of items (sometimes related items, sometimes not) that are just reminders. Each item is usually only a few words, but just enough to put me on the path to the full idea.

The second part of this "paperless" test was to determine if I could use the laptop for that purpose. Most of the notes that I jot to myself end up in electronic form anyway (usually expanded with more context). Wouldn't it be good to avoid that intermediate step?

I don't think this is going to work. It's like the time you realize (when trying to write a networked application) that bandwidth is not enough, you also need low latency. Laptops (and TVs and cellphones) are getting slower to "boot up," not faster.

I don't consider this test a failure. Learning something always counts, so I don't consider the experiment a waste of time. Perhaps if I ever find a cellphone I like and can afford, that will replace pen-and-paper.

Going Paperless: Day 3

2009-04-03T05:41:00.000-07:00

This morning I read the whole front page of the New York Times paper edition while my laptop was installing the latest Windows updates.

I also realized that the primary reason I don't like the NYT web site is that it is too hard to quickly scan the headlines, unlike the paper edition. Hello, Google News?

Thirdly, while making cheese & tortillas for breakfast (I'm a FIFO kinda guy when it comes to the refrigerator), I realized that getting cheese grease on the keyboard was probably not a good idea. (Also, I can't afford to spend thousands of dollars on a Toughbook.)

It looks like the laptop is losing as far as reading the paper is concerned, at least for the way that I like to absorb the news.

Going Paperless: Day 2

2009-04-02T09:54:00.000-07:00

Today I read some of the paper edition of the local freebie weekly while waiting for my laptop to reboot after a failed shutdown. After the laptop came back up, I went from the main NYT page to "Today's Paper" and read some articles from there. I also tried to get to the New York Times "Reader" (which looks like it shows images of the paper edition) but couldn't find a version that I didn't have to pay for. I already pay for the paper edition. I'll give the Reader another try tomorrow.

Last night I plugged the headphones in and watched some TV on Hulu. Enjoyable, but I wonder how Hulu is going to stay in business with only public service announcements for "advertising?"

Going Paperless: Day 1

2009-03-31T17:00:00.000-07:00

As an experiment I've decided to try to reduce the amount of paper that I consume. As with all experiments, it may or may not work. But what does "work" mean here? I may, for example, be able to read the New York Times on-line. Will that experience allow me to read more, or to understand more of what I read?

It's not like I use a lot of paper, and my family is conscientious about recycling. I'm not, therefore, expecting to save a lot of paper.

Today I read a few articles using NYT on-line. I feel like I read less than I do from the paper, but I liked the fact that (with my ageing eyes) I didn't need an extra light source to read the articles. It also seems like the font size in the browser is slightly larger than the font size in the paper edition.

My issue today is that I like the NYT editors making choices for me. I find the web page has a lot on it, and I have to "think more" about what I really care about. With the paper edition, I sort it as follows:

main section
arts
business
sports (which I never read anyway)

Now I have to look at each individual article title to figure out if I want to read it or not. I feel that slows me down, and I read less of each actual article.

We'll see what happens tomorrow.

Beward of tilde characters in file and directory names

2009-03-12T10:00:00.000-07:00

I work on an application that occasionally goes across the network to look for files. I had written some code that used the GetFiles and GetDirectories calls from My.Computer.FileSystem. One of our test-beds was a remote computer (going across the public internet, rather than within a company intranet). The calls to GetFiles and GetDirectories using the SearchAllSubDirectories option were agonizingly slow on some networks, but not all.

I tried many different workarounds (including writing my own recursive GetDirectories on the theory that something was wrong in the Microsoft implementation). None of them worked until I took a deep breath and made a long list of things to try (many of them actually just data-gathering under different configurations).

Luckily, the first item on my list fixed the problem. I had been using a tilde character ('~') to help directories and files sort in a certain order. I remembered that the "shortened file name" workaround in Windows used a tilde to generate "intermediate" filenames for older applications that used fixed size buffers for 8.3 filenames.

I changed the tilde to an equal sign and the problem went away.

VBScript vs the Rest of the World

2009-02-20T16:00:00.000-08:00

I tried using VBScript since I was tired of the lack of control structures, functions, etc, in the CMD language. I spent a few hours rewriting (or trying to rewrite) a couple of CMD scripts.

I ended up going back to CMD. My reasons (in no particular order) are:

VBScript is different enough from Visual Basic that it really qualifies as a different language. Very few of my VB habits worked for VBScript.
I hit a security issue when I was trying to copy files. I had written a CMD script, a Cygwin shell script and a Python script that all worked, but VBScript refused to do the copy. It wasn't worth the time to figure out which security setting to tweak.

Also, I don't really want to learn yet another language that locks me into Windows. So I get to live with CMD (whose behavior I do understand) until I can convince myself that my team can deal with Python.

Simple Software Scheduling

2008-12-15T16:14:00.000-08:00

How about scheduling this way? When you're in the SWAG (Scientific Wild-Ass Guess) stage (or even later), just pick one of the following for each task:

1 day
1 week
1 month
1 quarter
forever

Note that there is no "year" category. If you think something is going to take a year you're not scheduling at the right granularity (and maybe even a quarter is too much).

C++ References "syntatic sugar" or not?

2008-10-12T08:00:00.000-07:00

When I first learned about references in C++, I dismissed them as "syntactic sugar." That is, they are equivalent to

* const

See the Dan Saks article at http://www.embedded.com/story/OEG20010311S0024 for a good explanation. Lately, though, I've be re-thinking that stance.

I can now see how collapsing the * const into a & might make the code easier to understand. I say "might" because I can also see how it can cause confusion for beginners, since the "&" symbol is also used as the "address-of" operator. However, it does reduce the general confusion caused by pointers in more complex type declarations. such as

const SomeType* const varName

See Marshall Cline's article at http://www.parashift.com/c++-faq-lite/const-correctness.html#faq-18.5 for an explanation of how to actually read and understand these declarations.

The thing I still don't like about references is that they give no clue whether an object is a stack-based local object or a parameter passed into the method. I can always hope that whoever wrote the method kept it short enough so I don't have to keep scrolling up and down to look at locals and the parameter list. That doesn't, however, always happen. At least some Interactive Development Environments like Visual Studio allow you to hover over a variable to see its type. I haven't used Eclipse in quite a while, so I can't recall whether the various language plug-ins support it. I assume (or at least I hope) that it works for Java.

Who's Fooling Who?

2008-09-18T07:58:00.000-07:00

This morning our San Francisco Chronicle didn't arrive by its appointed time, so I called the Chron's 800 number. I got the usual voicemail with voice recognition and started answering the prompts as instructed. After a few prompts I realized there was a subtle difference about the system. Instead of "dead air" while the system was processing my replies, there was the sound of someone typing on a computer keyboard.

I was vastly amused. Then (per my usual "look on the dark side" view of the world) I started wondering how many people actually believe they are talking to a human being. Or did the human interface guru or gurus determine that (even if customers knew that it was a machine) the sound made the interaction more pleasant? As I said, it made ME grin...

It is P. T. Barnum's "there's a sucker born every minute" or actual useful social engineering?

Yesterday's Minor Victory

2008-09-14T08:07:00.000-07:00

A few years ago (31.Aug.2004, if GMail remembers rightly), I switched from AOL Mail to GMail. I still have my AOL account, as that is still the primary email for my wife. I also still monitor my AOL account to see where that email address has gotten to, either through my own actions or someone else’s. After the first few months of using GMail I had cleaned out almost all references to that address, but a couple eluded me. I kept getting email from Dartmouth College, where I got my undergraduate degree. I followed their unsubscribe instructions, only to get an automated response saying that the AOL email address was not in their list.

Yesterday I remembered an incident from a few years ago. I manage the email list (using Google Groups) for the United Parents of San Leandro, the PTA equivalent at our local high school. As an aside, I still do this, even though my kids are years gone from the institution. I do try to be a good citizen. At one point I got a list of email addresses from a former member of United Parents, supposedly of individuals who would like to get our emails. I added all those email addresses to the list. Within a few days I started getting emails from one individual threatening us with a lawsuit for spamming her. I went back and reviewed the list of addresses in the Google Group, but couldn’t find the address that this person was sending from. The threatening emails kept coming.

I then realized that this person could have set up a forwarding mechanism from some other email address. I went back to the list and searched for email addresses that were (in any way) similar to the return address on the threatening emails. I found one that looked somewhat close, and took it off the list.

The threatening emails stopped, and no one else complained about not getting their emails.

Coming back to the more recent past, I began wondering if I had put that AOL address as a secondary (e.g., forwarding) address for any other email address. I realized that Dartmouth did provide an email address for alumni, and went to check that. Voila. There was my old AOL address. I don’t know how long ago I put there, but I’ve used AOL since before 2001. I lost my emails previous to that year, so I don't know how far back they really go. I do, however, remember trying to get onto AOL with a 2400K Baud modem, and that must be further back than 2001. In any case, I put that AOL address into the Dartmouth system many years ago.

I changed the reference from that old AOL address to my (no longer new) GMail address. We’ll see what happens, but I’m assuming (and hoping!) that I will not get any more emails from Dartmouth to that AOL address.

Now the only reference to my AOL address is from Pacific Bell (oops, sorry, now AT&T), and I can't get to their page where I (supposedly) can remove my old address from their list. One of these days I'll debug that problem.

Six vs. Twenty-Four Jams: Too Many Choices

2008-09-13T08:09:00.000-07:00

As much as I have fond remembrances of Perl, I'm glad languages like Python are around. As I mentioned in a previous post, one of the reasons I left Perl behind was TMTOWTDI. Note the reference in this Wikipedia article to the Python principle that espouses the opposite philosophy. I've now found more support for the idea that more is not more, and is sometimes less.

I've read a few articles that cited a study where consumers were presented with different numbers of choices for buying jam. Searching for those terms turned up this article. I found it interesting that more choices made more people stop and look, but more choices reduced the number of people that actually bought any jam. I, like many others, like to feel that I know enough to make an informed choice. I also realize, however, that I am just as happy to have someone (who I hope is more knowledgeable than me) winnow down those choices to a set to which I can actually give decent consideration. This also reduces the time required to make a choice.

I know that for Perl, once I figured out how to do something, I actively rejected trying to find other ways to do that same thing. I never felt the need. I also never had a case where performance was an issue (which would have forced me to look at other ways). I also, thankfully, never had a Perl prophet try to convert me to the "Many True Ways" of Perl.

This also makes me think of all the sales vice presidents that have touted their "pipeline," but weren't able to translate that into actual sales. It's the end result that matters (both the quality of that result and how long it took to get there), whether you're on the making, selling or buying side of things.

XMLSerializer, part 2

2008-08-21T08:45:00.000-07:00

After spending more time fixing bugs and digging into XMLSerializer (see my previous post), I've come the the following conclusions:

I'm still satisfied with my design where I split the "data" into a separate class. It makes it much easier to quickly see what is being serialized and what is not. If I hadn't done this, I would have reorganized the fields in my objects to put all the serializable fields together, so I could put all the unserializable fields (with their corresponding noisy attribute tags) in a separate section of the class definition.
I figured out my problem (mentioned in bullet item 3 in the previous post). I had not provided a default value for an enum in the class definition. Once I did that, the uninformative error message went away and my object was sitting happily on disk.

On to the next learning experience...

Dealing with .NET XMLSerializer

2008-08-01T07:29:00.000-07:00

I've found that the clearest (e.g., fewest lines of code) and quickest (e.g., fewest surprises) way to deal with XMLSerializer is to create "data" classes. These instances of these classes are then owned and used by instances of "code" classes. This gets around the following issues:

XMLSerializer will only serialize Public fields. It isn't good programming practice to have to declare fields as public. To reduce this ugliness, I moved the serializable fields down into the data class and made the reference to the data class Private. I then put Public Property(s) on the code class that reference the field in the data class.
XMLSerializer doesn't support hashtables, which I find much more useful (and natural in many cases). I used hashtables on the code class. I then added code to a GetSerializable method that sits on the code class that populate the corresponding array on the data class on demand.
XMLSerializer doesn't provide very good error messages. My first attempt at changing a class to user XMLSerializer hit many problems with many different fields in the existing class. Using a data class, I don't have to figure out all the problems all at once. I can move fields from the code class to the data class incrementally, and figure out the particular problem with that field before moving on to the next. One specific example of this are issues with using Public Property. I haven't figured out what they are yet, but I'm suspicious of either:
Public Properties that aren't simple "return" and "assignment" Properties.
Public Properties that wrap a field that is also Public.

The downside is that I have a few extra public fields on various objects that are only used for serialization. The upside is fewer lines of code and a (somewhat) straightforward paradigm for dealing with XMLSerializer.

Thank you, EasyBCD!

2008-07-30T07:40:00.000-07:00

And I don't mean Binary Coded Decimal.

I recently set up a test machine with a dual-boot XP/Vista configuration. Being the cautious type, I went to make a disk image of the Vista side. That is the side I would be using for testing, and would need to be able to quickly and easily restore it to a clean state (e.g., not have to tediously reinstall Vista each time).

I first tried an old version of DiskImage. Nope, won't run off an NTFS drive and I have no floppy in that machine. Toss the CD and recycle the manual.

Next tried a 2003 version of Norton Ghost. Things seemed to go well for a while, but went south really suddenly. I believe that it choked when it tried to deal with the Vista BCD (Boot Configuation Data) stores. Toss that CD and recycle that manual.

Trying to then reinstall Vista failed repeatedly because the reboots kept trying to reboot a corrupted operating system. I couldn't get to the BCD data from the XP side until some Googling turned up EasyBCD, which saved my sorry butt. Now I'm re-installing Vista.

I think I'll next try Ubuntu with PartImage to back up the disk image. I've paid Symantec enough money over the past few years.

Cassandra, Take a Bow

2008-07-04T07:17:00.000-07:00

I've always considered myself a realist. Some may call me a pessimist, but many of my predictions have turned out correctly. I don't like being seen as negative, so I do try to lighten up a bit. This led to an amusing incident at my current job.

After telling my boss a number of times that I thought an expectation was too optimistic, I decided to tell him the legend of Cassandra (because I was starting to feel like the prophet-not-listened-to). After I told him the story, I told him (jokingly) that I considered myself the Cassandra of my group.

A couple of days later, my business phone rang. I get very few calls on that line, since my group tends to use GTalk if possible. I therefore get a higher percentage than usual of wrong numbers. I was in the middle of debugging something, so I was focused on that task rather than the phone call. I picked up the phone and someone said, "Is Cassandra there?"

I almost hung up, but decided to be polite and ask the person what number they dialed (expecting that it was a wrong number).

It was my boss. Given my past history with him, I should be careful what I ask for, because he does have a keen sense of humor.

A more reliable test bed

2008-06-30T09:29:00.000-07:00

I've been working on putting together a test bed for the product that my company currently offers. My first cut at the test bed used AutoIt to drive the GUI and NUnit for the testing framework. This worked, but I felt it was too fragile (and perhaps too complex) for our QA folks. I therefore wrote an embedded test framework using my simple command parser that worked better.

First, it didn't require multiple executables (e.g., a separate application to drive the GUI using AutoIt). This reduced the complexity of the system, which was immediately reflected in the amount of documentation and support I had to produce.

Second, one can go off and do other tasks (e.g., read email) while the tests are running. This happened because the test bed no longer sent mouse and keyboard events to the application, so there was no contention for those resources. Responsiveness isn't what I would like, but that's the price I pay (until I get a machine with a multi-core CPU). Note that the window of the application running the tests may come to the top, so you may lose keystrokes if you are typing into another application. This shouldn't, however, interfere with the tests. E.g., the application with the embedded test-bed should ignore those keystrokes.

Third, I realized that this does away with synchronization issues. I am intimately familiar with the problems of dealing with variable wait times. For example, when using a GUI driver such as AutoIt or Silk, the testing application that drives the GUI may need to wait for a network to respond or for a long-running calculation to finish. It is usually unclear how long to allow the GUI driver to wait before assuming that something went wrong. Since this framework is "synchronized" with the application, this issue just disappeared. Admittedly, I should probably add some "time-out" feature (in the future) to handle any hangs, but I'll wait for the QA folks to ask for that before I put any effort into it.

Fourth, since there is no need for the test framework to wait, the test sequence will run faster.

The downside is that adding more tests requires some coding in the application. This, however, is usually straightforward. I find where the event from the GUI is handled and add a little hook function to call that same code, with parameters as necessary.

It's a pleasant suprise to have some positive unintended consquences, instead of (more common) negative side effects.

Ageism?

2008-06-10T11:56:00.000-07:00

Don't bother applying for a job at Facebook if you got your college degree before 1980.

The dropdown for "Graduation Date" on their job application doesn't go back that far.

Addendum:

In an industry survey, a majority of technology companies candidly said they would not hire anyone over 40.

New York Times, article in "This Week" section on June 22, 2008, titled "For a Good Retirement, Find Work. Good Luck."

Today's lesson...

2008-06-04T14:47:00.000-07:00

I went searching for a line continuation character in DOS. I needed to make my CMD scripts more readable (if such scripts can ever be made readable - I'll use Ant next time). Searching around I found that the caret ("^") was supposed to be the continuation character. I tried it and it didn't work at first, until I realized that I was using it in the middle of a quoted string.

Re-jiggered the long command to put the caret(s) outside of quoted strings and it works like a champ. Not that I want to claim to be an expert in CMD scripts...

A Stitch in Time

2008-05-13T08:00:00.000-07:00

I have written a bunch of multi-threaded code in the past few years. (I realize that a "bunch" isn't very quantifiable, but bear with me.) It seemed to work fairly well; the server product I worked on was able to make use of multiple CPUs whenever possible. I tested it on multiple CPU machines and saw all CPUs being fully used under load. Also, others used that product on 8-way boxes with good reliability. I have found and fixed threading bugs in my own code and in other's code. I still, however, find myself looking for patterns to help me deal with threading, to increase my confidence in my (and other's) multi-threaded code.

I don't know if it is the indeterminacy of it, or if it's that I don't feel I can hold a lot of multiple states in my head at once. I also know that the few times I've tried to have my threading code reviewed by others they gave up long before I got any feedback of any kind. I don't know if that was because the code was opaque (though there wasn't very much of it...) or because they weren't able to deal with multi-threaded code any better than I was.

I had a number of concerns about the use of mutexes in that product. My primary concern was the sheer number, both of types and of instances. I felt that I had to make sure that the whole system couldn't deadlock, but given those numbers there was no way that I (or anyone I know) could do that.

The principles I will follow in the future are as follows:

Don't be afraid of multiple mutexes.
Do as little as possible while holding locks.
Never hold a lock infinitely.

I don't know if this will make it easier (and more comfortable and reliable) for me to work on multi-threaded code, but I know I can't go about it as I have in the past.

[title reference]

Visual Basic command parser example

2008-05-07T01:00:00.000-07:00

OK, I left the Visual Basic version of this code as an "exercise for the reader," but it turned out that I needed it myself. I translated (by hand) the C# example but couldn't get the Visual Basic version to find the method in the InvokeMember call (even after the FindMembers call had found it). I poked around some more, thinking that once I had the MethodInfo I should be able to use it directly without having to use InvokeMember.

Sure enough, after poking around a bit I found Invoke (on the MethodInfo itself), and it works like a charm.

Maybe some day when I'm feeling more ambitious I'll go back and rework the C# example to match the Visual Basic example, but probably not until I really need to. I actually hope I don't, because that means that I will have a need to use C#, and I'd rather stay cross-platform with something like Java, C++ or Python.

The Third Commandment of Blogger

2008-05-02T08:11:00.000-07:00

Don't use Blockquote!

I'm still struggling to figure out what Blogger is doing to my formatting. I'm
not sure if it's this particular template, or something deeper down in Windows, or perhaps some backwards compatibility issue.
I say "deeper down in Windows" because I see the same effect in both Internet
Explorer and Firefox on my Windows XP machine. I don't (currently) have any Linux desktop machines to test this.

First, every time I try to use blockquote both browsers show succeeding paragraphs with a tighter line-spacing than the previous paragraph. This happens with the default font; I haven't tried other fonts because no one it paying me to test this stuff. This paragraph is an example of this effect. Secondly (because of its definition), blockquote takes line endings literally, making for some very strange line breaks as shown above. I like indention to set off text, but not when it comes with this side-effect.

For commandments One and Two, see my previous post. That post shows the same incorrect line-spacing in the numbered list. Also note the extra line-space above this paragraph, which seems be be caused by saving and then re-editing the post. Well, at least I'm not paying anything for these bugs.

We Don't Need No Stinking Comments

2008-05-01T07:00:00.000-07:00

I am trying to put together some coding standards for the company where I currently work. Yeah, another exercise, but not (I hope) a futile one. As always happens, someone wanted to have a standard for comments.

My only answer to that was this: "If a program acts differently from the way the end-documentation says it should, which do you trust?" I feel the same way about comments. It's not that I don't appreciate them, but I just don't trust them as much as I trust the code. Admittedly it's usually a harder slog through the code, but once you've figured out what the code does, the comments are usually not useful.

As Kernighan and Plauger say on page 142 (1974 edition) of "The Elements of Programming Style:"

The problem with comments that do not accurately reflect the code is that they may well be believed subconsciously, so the code itself is not examined critically.

I wish I could come up with some good way to measure the effectiveness (or in-effectiveness) of comments, but I'm just going to have to go with my gut on this one. I don't think we can have any standard for comments that anyone will agree upon. We may be able to come up with some recommendations, but as for enforceable standards? In "The Practice of Programming" (Kernighan and Pike, 1999), page 24 generously says:

Comments can be useful, of course.

And the rest of the page goes on to show some good usages. This page, however, is surrounded by a list of things NOT to do with comments. Seems like it's easier to tell people what not to do, rather than what to do.

[If you don't know the American pop culture reference from the title, check here or here.]

Technology for Good and Bad

2008-04-21T16:30:00.000-07:00

I closed an account at Bank of America a few months ago. After I showed my driver's license to the teller, he swiped it through the magnetic card reader at this station. Perhaps I haven't been dealing with banks and bank tellers much lately, but I was intrigued and amused.

Then, of course, he proceeded to try to sell me a Bank of America credit card. I told him I already have enough credit cards and he proceeded to enter that little bit of information into his terminal.

Perhaps the programming elves at Bank of America implemented the "read the DMV magnetic strip" as an ease-of-use measure, or perhaps they are trying to reduce the number of data entry errors by their front-line people. In any case, it certainly made it easier for me, since I wasn't carrying the bank card that came with the account.

This use of technology I appreciate.

I didn't, on the other hand, appreciate Bank of America making the teller try to sell me something, and I certainly didn't appreciate that they then make him record why. I've already had to deal with a number of telesales calls from them (until I told them to put me on their Do Not Call list). I wonder if my interaction with the teller just took me back off their Do Not Call list. If so, I'll just go through getting off it again.

Appropriate Technology

2008-04-14T07:00:00.000-07:00

One of the side effects of having a Prius with the radio-key is that I sometimes leave the key in my pocket. When I leave the car I don't need to pull the key out of the slot, so I had a personal pattern set where I just get out, close the door and press the Lock button on the outside of the door.

Once (and only once) that got mixed up with my personal pattern for our van. I got out of the van, pressed the lock button on the inside of the door and merrily shut the door. It was only later that I realized that I had locked my keys into the van. What about the warning buzzer, you ask? Well, it has been malfunctioning for a number of years now, and always buzzes when the door is open (regardless of the state of the keys or the headlights). The van is old and I want to spend as little as possible keeping it running.

I called AAA and they sent one of their guys. He got out of his truck with a little pillow with an inflater bulb on it. I watched in fascination as he slid the pillow between the door and the pillar and inflated it to force the door open just enough so he could get a long plastic wand through the opening. He then used this wand to push the Unlock button on the door. Nothing electronic at all.

Much to my chagrin, I had to call AAA again later that day when I went to leave work and found that I had also left the lights on. I hadn't noticed it at lunch.

Maybe I should get that buzzer fixed.

What I Learned Today

2008-04-08T15:30:00.000-07:00

I started using the Lightning calendar add-on with the Thunderbird email client a few weeks ago, and have been wondering what the "CW" stood for in its date display.

Country Western?
Conventional Wisdom?

Used Google search and after going through a few search terms that either gave me too much or too little, I used "calendar CW lightning," which turned up this:

https://addons.mozilla.org/en-US/thunderbird/discussions/comments.php?DiscussionID=7662&page=1#Item_0

Ah, "Current Week!"

... and today's rant...

If I had time, I'd love to work on the Thunderbird text editor. I've gotten to the point where (if I'm going to use any formatting) I just compose my emails in Word, paste them into Thunderbird and then clean up as necessary. I do this because I have not yet been able to figure out how Thunderbird stores its formatting, and I can't find any way to make the "markup" visible. Is this because Thunderbird uses some low-grade Microsoft editing control, or some other reason? (Not that the editor in OutLook Express, or whatever its current name is, is much better...)

Maximum Method Magnitude

2008-04-04T09:25:00.000-07:00

While I was roughing out a series of coding standards for the company I currently work at, I decided to quantify what I mean by "a method that is too long." I prefer to have methods (and functions, for the non-object-oriented among you, or you functional programmers) that I can see all at once. I don't like to scroll up and down (or left and right) to see the whole method. I therefore decided to quantify how much I could see on my screen at once so I could put some specific numbers in the standard.

Note that this restriction is relaxed for methods that are case statements with many similar, small cases, or other simple and repetitive structures. Since these are more "data"-like than "code"-like, having to scroll up and down doesn't interrupt my flow of thought as much as with regular code.

I currently have a Dell flat-screen that I measure at 22" (559 mm) visible diagonal size (regardless of what the Dell sales literature says...). I run it at its default resolution (1680 horizonally by 1050 vertically). I have a few icons on my desktop on the left (heavy use) and right (lighter use), so that reduces my available screen size a bit. I also like to leave room at the bottom of the screen for the taskbar.

I bounce back and forth between three editors, my old standby Epsilon, the Microsoft Visual Studio Editor and Eclipse.

I run Epsilon at 8-point Courier New, which gives me approximately 60 lines vertically and 200 characters horizontally. I spend more time in this editor than the others, so I've spent more time getting it to settings that work for me.

I never changed the default settings for Visual Studio. When I configure it as I usually work (one column of panes on the right), I get about 45 lines vertically and 150 characters horizontally. Note that this is reduced during debugging, because more panes appear. Tools, Options, Environment, Fonts and Colors tells me that I'm using 10-point Courier New.

I also never changed with the font settings in Eclipse. Preferences, General, Appearance, Colors and Fonts tells me that I'm using 10-point Courier New. This gives me 40 lines by 140 characters with the default set of panes on the right, left and bottom.

For Visual Studio and Eclipse, I could probably go back and change the font size to be closer to my Epsilon setting and get similar results (if I'm ever motivated enough).

One added restriction. I really don't like lo-o-o-ong lines. There is a reading phenomenon called "doubling" where, when lines are too long, you lose track of which line is the next one you need to go back to when you read to the end of the current line. This is less of a problem in code than in text, given the "poetic" (as opposed to "text") layout of code. I still, however, find it reduces readability for me, especially when I have to scroll a couple of screenfuls to the right horizonally and then jump all the way back to the left and locate the next line of the code.

Given all of the above, I'm going to put 60 lines vertically by 100 characters horizonally into the standards document. We'll see what my team members think of my numbers.

Finally, apologies for the forced alliteration of the title. I just finished reading a new translation of Sir Gawain and The Green Knight (by Simon Armitage), where there is heavy use of alliteration. If you like that sort of book, this one's a good read.

CSharp command parser example

2008-03-29T09:17:00.000-07:00

I've added a CSharp example to the command lookup example code at http://code.google.com/p/dead-simple-command-lookups/. This, along with the C++ example, are as far as I plan to go. I (currently) don't use other langauges, and thus have no need to keep other versions readily available.

Ah, the truth comes out. I didn't really put this up for others to use (although they are certainly welcome to do so, given that the code is under the MIT license). I just wanted to have this code readily available when I needed to use it.

No Whining! (...at least not today...)

2008-02-13T17:18:00.001-08:00

Rather than complain about something, today I want to contribute. I have been working on some very simple command lookup programs in various languages. I've finished most of them (one more to come - the C++ version). They are in Google Code at http://code.google.com/p/dead-simple-command-lookups/. Note that I've put the files into the download section as well as under Subversion for those that don't want to bother with getting a Subversion client running.

Cell Phones Are Still a Compromise

2008-01-30T19:00:00.000-08:00

I have a cellphone but it's off most of the time. I've given out the number to very few people, and they know that I keep the phone off most of the time. Since I don't have call blocking enabled (that I know of!) anyone I call can grab my number. Since I don't, however, call that many people, I don't worry about who they are.

A number of months ago I interviewed at Google. I didn't get the job (sigh) but that's bits into the bucket. One of the pieces of information that came across in one of the emails (from their recruiter) was that they preferred that their interviewees NOT use cell phones during the phone screens.

This intrigued me. I'm not sure whether this is because cell phone networks in some countries aren't up to snuff or for some other reason(s). I do know from my own personal experience that about half the cell phone calls I make have dropouts or clarity problems, even in "civilized" places like the San Francisco Bay Area. Which actually is the primary reason I don't turn my cell phone on unless absolutely necessary.

JavaScript Strikes Again

2008-01-24T20:00:00.000-08:00

In a recent job interview, I was asked how does one detect (in a JavaScript function) whether a parameter actually had a value passed to it. It's been a long time since I've done any JavaScript, so I answered that I assumed there was a way to test for "null". The interviewer said that I had the appropriate concept, but the actual language keyword to test against was "undefined". We then had a brief discussion about null versus undefined, but I wanted to find out more.

Google turned up this page which has some very concise and useful examples. I was not amused, however, to read that testing for null will return true regardless of whether the value is actually null or undefined. I would hope that testing for undefined doesn't return true for null, and I hope to have time to test this sometime in the future. This brought me back to the first time I realized that JavaScript didn't (in previous incarnations, anyway) syntax check code that was never executed. I'm used to that happening for shell script, but it brought me up short for a language that actually seemed to be used in many shipping products.

I guess one works with what one is given, but it doesn't make me any more confident in JavaScript as a robust language.

Today's Formatting Hobgoblin

2008-01-17T08:13:00.000-08:00

One of the little things that bothers me about some of the code I work with is the variation in handling of the "if" statement. In this code an "if" with a single statement in the "if" block (and no "else" block) will end up on one line:

if (condition) do_something_else();

I find this interrupts my reading of the code. I realize that putting brackets around the statement in the "if" block and putting the statement on its own line will add two lines to the code. I find, however, the consistency of formatting makes the code easier to read (for me, anyway).

This may be a foolish consistency or not. I looked back to my favorite coding standard for support. The Linux kernel coding style says "Don't put multiple statements on a single line unless you have something to hide." This doesn't mention brackets, but that follows from other standards.

At least I'm not the only voice crying in the wilderness...

Copy/Paste Considered Harmful

2007-12-23T08:11:00.000-08:00

When I first started coding years ago, it was in a very early version of Basic that ran on the timesharing computer at Dartmouth. I had no formal training and committed many of the sins of self-taught programmers, including programming by copy/paste. After a few times of having to fix bugs in multiple places I started learning about functions in Basic so I could stop that nonsense. I also became aware of how messy it was to add new features in copied code.

At this point in my career I am more aware of this problem, and tend (when I see it in existing code) to refactor before going on with my planned task(s). If there are automated QA tests with the appropriate coverage I assume they will catch problems. If not, I construct a few automated tests of my own to increase my confidence that the refactoring didn't change any existing behavior.

I was talking with a co-worker a few weeks ago about coding practices, and the topic of copy/paste came up. I went into my usual rant/sermon about it, only to find out that he actually had a professor who promoted copy/paste as a method of "code reuse." I don't know where my co-worker went to school, or what this professor had a degree in, but it took me aback to hear this.

Now there are a few times where copy/paste is appropriate. For example, if we are late in the QA cycle and about to release a product and find a real show-stopper of a bug. After analyzing the bug and determing the cause and the fix, we realize that it is deep in some code that is called from many other places. Given the time frame, we're not able to test all those places to the level of quality that we would like. In this case, I have occasionally done some copy/paste to make sure that the side effects of my change are as minimal as possible. I always, however, put a TODO (or TBD, or other cleanup marker, depending on the coding standards of the company I'm at) into the copied code with a reference back to the source of the copied code. This then goes as a task at the top of my cleanup list for the beginning of the next development cycle.

I don't see how to produce long-lasting quality code without doing this.

[Minor pop quiz: where does the title of this post come from?]

Setting Expectations

2007-11-25T19:51:00.000-08:00

I was talking with my young niece a few days ago about getting shots. She told me she really hated them. I then told her a story about my brother who had always had a deathly fear of shots. At one point he and I were talking about this (as children), and it came up that he had always thought that when the doctor pushed the plunger in, it was making the needle go further into his arm. He said he was really frightened that it would go into his bone.

At that point my niece's eyes got big and she said, "You mean, it doesn't push the needle further in?"

I was intrigued that she had the same mis-information as my brother. It made me think of how hard I work to give correct information in order to set expectations. I feel that one characteristic of good engineering is to set expectations correctly. This may sometimes go against the wishes and hopes of the marketing and sales folks, but it has to be done. I do, however, feel that (in the long run) setting expectations correctly works for both me as a producer of software and my customers as consumers of software. By "customer" here I mean both internal (e.g., within the company) and external (e.g., paying) customers.

If I set expectations too high and don't meet them, my credibility is damaged, and people have a long memory for failure.

Lowest Common Denominator Wins This One

2007-11-19T17:33:00.000-08:00

Today I came across this article from Component Developer Magazine about ODBC. I was especially amused (and bemused) by this quote:

At the time of its release, the SQL Server team at Microsoft believed OLE DB would supersede ODBC. This is not longer the case and ODBC's future is completely secure. ODBC is more widely used that OLE DB and it is better suited to some key scenarios that I will discuss later in this article.

I was originally planning to title this post "Old APIs Never Die, And They Never Fade Away Either." The message here, though, seems to indicate (if by omission only) that OLE DB's future is not completely secure. Or maybe I'm just trying to dig too far into the sub-text.

In any case, I believe this is a case where ODBC (as clumsy as some think it is) has won precisely because it is low level, focused on one task, and does not have a lot of proprietary technology that needs to be duplicated across platforms. While ODBC on non-Windows platforms such as Solaris and Linux still has a lot of rough edges, it's generally usable except for the occasional configuration nightmare.

I wonder, on the other hand, what will happen with efforts such as Mono, which are attempting to duplicate a much higher level of functionality and more compilicated interface. As Wikipedia says (today, anyway):

The Microsoft compatibility stack provides a pathway for porting Windows .NET applications to Linux. This group of components include ADO.NET, ASP.NET, and Windows.Forms, among others. As these components are not covered by ECMA standards, some of them remain subject to patent fears and concerns.

Sometimes one's eyes are bigger than one's stomach.

Share the Wealth

2007-11-17T13:04:00.000-08:00

When I was leaving one of my previous jobs, I was working with a younger developer who was going to take over some of my responsibilities. While I had been working there I had written documents describing procedures and processes that I had put in place for myself. I had been keeping these documents (as well as some code for personal regression tests) in a source code repository using CVS. The repository was on the laptop machine with the working copy on the desktop machine. I used CVS because the company source code system (ClearCase) wasn't set up to allow personal archives. I also prefer not to use ClearCase when possible, but I'll save that rant for another post.

The day I left I cleaned out the CVS directories in my local copy of the directory tree, zipped it up and emailed that zip file to him. I didn't want to give him the CVS repository itself because I didn't want to confuse him any more than necessary.

I also changed the password on my email (Lotus Notes) and made sure that all my emails had been put into my local email database. I then gave him a copy of that database along with the password and made sure that he could open that local database on his machine. He looked at me, slightly puzzled, and asked if there weren't emails that I didn't want him to see. I told him that I knew from the day I started that these emails were really property of the company and they didn't belong to me. I never, therefore, used it for any personal purposes or sent any emails that I wouldn't want "read out loud in the town square" (to quote my brother-in-law).

I don't know if he ever actually looked at any of this, or whether he considered it wealth or dross. I did, however, feel better leaving the information with him, rather than letting it be wiped when my work machines were returned to the company pool.

The "Dead Leaves" Guy

2007-11-11T17:33:00.001-08:00

My family and I have lived in the same house for the past 24 years. Once or twice a year I squeeze myself between our "Florida Room" and the neighbor's garage to clean out the oak leaves that end up there. It's a tight fit, and not a task I enjoy. When it's done, I step back, dust off my hands (figuratively and literally) and take the green waste bin out to the street for pickup.

I've always gotten along well with our next door neighbors, who have been there for more than a decade. Last weekend I noticed the husband up the roof of his garage, merrily sweeping the oak leaves into the space that I have been squeezing into all these years. He looked a little sheepish, but said he had been doing this all along. I told him that if I hadn't been cleaning them out, we would have had oak trees trying to grow between our buildings. I then assured him that it was OK with me, and we went to our various tasks.

I have also been cleaning out a lot of "dead leaves" that others have put into our code at work. I am concerned when I see code that is so obviously cut-and-pasted instead of refactored. I sometimes wonder what people think when they are doing this. Are they concerned because they don't understand the code well enough to do refactoring? Are they too overloaded to think about the fact that what they put in may sprout into oak trees in the worst possible place? Are they too rushed to think a few months down the road, or a year or two?

It concerns and mystifies me.

Glosses on Linus Torvalds Coding Style

2007-10-24T19:55:00.000-07:00

A gloss is a marginal note (see picture in the Wikipedia entry for an example). While one can't very well annotate someone else's web page, I will add my voice to the chorus of those who have commented on Linus Torvalds coding style document.

I strongly agree with almost everything said in this document, and have been glad for the "backup" when I've had to argue a point with some of my fellow workers. There are, however, a couple of items where I diverge.

The first point that I would gently argue is that 8-space tabs are too deep. I haven't "been looking at [my] screen for 20 straight hours" for a couple of years, but at the very outset I worked with 4-space tabs, and I find them very readable. I haven't worked with anyone who has strongly defended 8-space tabs. I will admit that I have had to try to persuade people to switch from 2-space to 4-space tabs, but that was back in the Apple ][ Basic days when people were trying to keep their programs as small as possible (remember cassettes?). I don't know of any readability studies on this topic, and would be interested to see them.

My second argument (or shall I say "disagreement" to be less disagreeable...) is with his distaste for mixed-case names. I find mixed-case easier to read (maybe it's just me), but, more importantly, I find it a lot easier to type than underscores. I know I'm not the best touch-typist in the world, but I always find myself looking down to make sure that my ring-finger is indeed hitting the dash/underscore key and not one of its neighbors. Using camelCase keeps my fingers on the home row much more, makes my typing faster and reduces the number of errors.

These are just friendly disagreements, however, and don't diminish the fact that I'm glad this document exists, and is available to us all.

Pythonics Anonymous

2007-10-20T14:30:00.001-07:00

I think I need a twelve-step program to wean myself from Python.

When I first looked at Python a few years ago I was put off by the fact that white space was significant. I also thought that having to do "self." to indicate object fields (vs. global variables or parameters) was just noise. I got tired, however, of always looking up Perl syntax, and I really didn't care for the Perl TMTOWTDI (I prefer one good way to do things, not many). I then came across Eric Raymond's article and decided to learn and use Python.

Now, after a few years of working with the language I appreciate the fact that white space is significant (no more bracket-placement wars!). I also appreciate the cross-platform-ability and the fact that (even though it is open-source) it isn't GPL, which has allowed us to use it at a company I used to work for. I like having the various supporting packages (although the minidom has a problem when pretty-printing XML that I had to work around).

However, I now find myself thinking that I'd better go back to Java. I was put off Java because the reality took a really long time to catch up with the hype. Remember "Write once, run anywhere?" (which was really "Write once, debug everywhere?") It also seemed too heavyweight to me. I liked being able to run the Python interpreter in command line mode and experiment without having to fire up an editor, create files, save them and run them. It was also satisfying to be able to be able to write a one-line "Hello World".

It seems now, however, that the world (at least the world of enterprise software) has decided that It's A Java World. I guess I better get back on board. I know I'll fall off the wagon occasionally, but it's time to put Python back in the corner and get back to business.

Data Always Goes In A Database, Right?

2007-10-09T11:30:00.000-07:00

Not necessarily.

I have worked on a number of software products where data (usually metadata) is stored in text or binary files. The code opens the file, reads from the file and parses the text (if necessary) into the structures used internally by the code. Over the years, the question has been asked more than once: why don't you put this data into a database?

There are a number of reasons not to use a database. First, all the data needs to be in memory for performance, and there is no advantage to keeping parts of it out on the disk. Second, the data is usually not appropriate for being "table-ized." That is, the data is complex and tree-structured rather than row-structured. I know that one can always flatten data into multiple tables with the appropriate keys and joins, but parsers built with tools such as Yacc and Lex will always be faster than doing multiple database joins and reads.

A third reason is that upgrading the data in the database requires scripts to add or modify tables or columns (deletes not being done to reduce the potential for referential integrity problems). Parsers can recognize a version number and do an upgrade-on-read, which doesn't require another program to run to perform the upgrade.

Fourth, files are easier for end-users to handle. For example, if a user needs to send some of his data to tech support, it's much easier for him to package up one or more files, rather than having to run some database dump utility and send the dump file. There is also no issue with (once having that database dump) the possibility that tech support does not have the particular version of the database software that the user is running.

A fifth reason (for text files at least), is that in a pinch the file can be opened and edited in a text editor if there are problems with it. With a database you have to run the appropriate database client software, then tease out the table relationships to understand where changes need to be made.

When all you know how to use is a hammer, everything looks like a nail.

Fail Early (But Not Often)

2007-09-22T10:50:00.000-07:00

Civic-minded citizens have long been reminded to "Vote Early". A corrupt Chicago politician extended it to "Vote Early And Often." Failures should be handled as early as possible, but shouldn't (under most circumstances) happen very often.

My primary principle is "The earlier the failure, the less the consequences." For example, if I didn't check the return code on a file open, the code would merrily continue on and try to write to the bogus file descriptor returned from the "file open". Most current operating systems give you a bad handle error, but earlier ones would just return junk at best and might do something grotesque to the file system at worst. Also, the code has probably already gone past the point where it can put up a coherent error message that includes the filename and system error code. This the leads to my second principle about failure handling, which is: "The better the error message, the less tech support hates my guts." One annoying error message that I've had to deal with (as an end user) is "The [someLibrary.DLL] is in use by another program." Which program? Do I have to go through my Task Manager, or use "ps" on *nix, trying to kill one program at a time (not a good idea anyway!) to find out which is the offender?

Most contemporary programming languages have some sort of exception mechanism to help me write code to handle failures. In a language like C++ where I can throw just about anything, I create a superclass designed to handle failures (subclassed as appropriate) combined with a macro that catches all possible types that any code in my executable might throw. As long as I put my catch blocks in the right places (e.g., at the top of a thread or in top-level event handlers) I can return errors without bad side effects. Java, on the other hand, will only allow me to throw descendants of Throwable, which removes the need for the "glue" class and macro that I described for C++, but adds the complexity of having to deal with multiple subclasses of Throwable. There's no such thing as a free lunch.

What if I am working in a language that supports exceptions, but I am using some API that doesn't? For example, suppose I needed to use fopen (the low level 'C' routine) while working in C++. In this case I always check the return code from that call (as well as all other file system calls). If an error is returned from any of those calls, I convert it into a coherent exception and throw that exception.

To move farther back in the evolutionary ladder of languages, what happens when I am working in a language (such as shell script) where there is no exception mechanism at all? Here I check for errors after every possible line of shell script that can possibly fail ("assume the worst"). I keep the code readable by putting the status check off to the right of the "meaningful" code. This means that when I read the code I can concentrate on what it should be doing in most cases, rather than the edge cases of failure. For example, this shell script contains a shell function that displays an error message, the name of the script and how to get instructions for running the script.

Note that the error might not be caused by inappropriate usage, but this at least gives the person who ran the script something to see before reading the code in the script.

In the spirit of continuous improvement, I've tried moving the status test down into the shell function, but I discovered (at least on Cygwin) that making the function call changes the global status variable ($?). I even briefly considered doing something like this where the _chk_ function runs lines of script and then checks the error code.

_chk_( 'first line of script' )
_chk_( 'second line of script' )

I rejected this, however, because (a) the noise level is too high, and (b) I had serious concerns about quoting problems within each line of shell script code.

Sometimes you have to know when to stop refactoring.

The Two Commandments of Blogger

2007-09-11T06:28:00.000-07:00

Learned from bitter experience...

Don't copy and paste from anywhere else, unless you have no special formatting such as lists or fonts. If you do, you will end up retyping your post anyway because Blogger incorrectly converts the formatting. I have tried this from a number of different sources including Google Documents and somehow the text after lists always ends up with the wrong linespacing.
Don't trust the preview. In scenario above, the preview looks fine. When I post, however, the linespacing is messed up and I have to "Save as Draft" to pull the post and re-edit it. This happens in both Firefox and Internet Explorer.

Thanks to my son for his help and suggestions.

The Costumer's Handshake

2007-08-26T10:28:00.000-07:00

What is it about code formatting standards? After all, the important thing is that the code compiles, links and passes its tests, right?

No. All psychological and behavioral arguments aside, there is something attractive about well-formatted code. Admittedly, well-formatted code does not guarantee functional correctness, adequate performance and maintainability, but it is clear (at least to me and the folks that I prefer to work with) that it helps readability and therefore maintainability. And it certainly makes a better first impression than a slap-dash bunch of statements thrown together.

What does this have to do with costumers and handshakes?

My wife likes to sew costumes for my daughter, who is deeply involved with the San Francisco Dickens Faire and other such semi-historical pastimes. At one of the Dickens Faires my wife approached a woman and asked her about her costume. This is not an uncommon occurrence at these events, and most people are only too happy to talk about their work. After chatting for a few minutes and establishing a rapport, my wife reached out, looked closer at the garment, and turned the edge of the woman's bodice to examine the stitching and the lining. The woman chuckled and said, "Ah, the costumer's handshake!" and we both immediately knew what she meant.

People who care about the quality of their (and other's) work know to look inside and underneath. Costumers know the handshake. Woodworkers know to open the drawers of a cabinet to check out craftsmanship of the joints inside. They look to see if something is veneer (likely to come off in time), or solid wood, which will last longer. Printers know to hold up a page of a book to the light to check that the imposition of back-to-back pages is in alignment, and to check the quality of the paper.

I don't know if such quick evaluations can help judge the quality of a body of code. There are obvious things to look for such as error handling and formatting consistency, but problems such as duplicated code and bad naming are harder to spot without spending more time getting to know the code.

I haven't worked with many open source projects, but the one I've spent the most time with (Blender - see www.blender.org) certainly doesn't pass a lot of my tests. Does that mean it's not successful? If open-source success is defined as "usage and visibility" then Blender certainly qualifies as a success. A lot of the commercial closed-source code I work with doesn't pass my tests either. Given that success for commercial closed-source is defined as profitability for the company that produces it, I can't comment on that in public.

I do know, however, that (by the craftsmanship standard), much of the code that I work with might not stand up to having its "lining" examined. But that is one of the goals I aim for.

Paranoid Programming

2007-08-22T14:14:00.000-07:00

I read and enjoy Joel on Software (and used his CityDesk product for my website). I have to respond, however, to some of his comments in this article.

I strongly agree with most of it. For example, I'm glad to see conventions such as "Don’t put closing braces more than one screen away from the matching opening brace." (And I wish I could enforce that standard at the place I work at!) Also, he writes: "I have to admit that I’m a little bit scared of language features that hide things". I have seen very confused code caused by "features" in C++ like the parentheses operator on iterators. I always prefer to see a real method name such as "Next()" (thanks to Nick Pouschine for putting into words the solution to my unease with that syntax). And I never have, and never will, overload the plus operator for any reason.

I don't, however, see any reason to write off exceptions. I depend very heavily on exceptions in all the C++ and Python code I write. Joel's example of:

dosomething();
cleanup();

just seems wrong. The cleanup code should be on the destructor (or some equivalent in non-object-oriented languages), which should always get called at some point. Note that I'm assuming you've adopted the convention of "every object is correctly owned" so that memory is always freed at the appropriate point in the code, regardless of whether errors occurred or not.

[Apologies in advance for the CAPITALS.] I don't EVER need to "investigate the entire call tree of dosomething()" to see if there’s anything problematic in there. Rather I ALWAYS assume the worst, which is that any line of can cause an error (divide by zero, anyone?). I'm not paranoid, they really are out to get me!

I therefore write all my code to handle errors anywhere. That way, if an error happens in my code or some third-party code that I'm using, it is handled. Once I "accepted" exceptions, then I wrote code that I found more readable. I also was more confident in its error handling capabilities.

For example, instead of having all that nesting in the code from Raymond Chen's article that Joel references, I could put the variables into fields in an object, all the cleanup code into a destructor and then "flatten out" the error checking (I'll try to put an example in here later...). I don't claim that there are fewer lines of code (there may be more), but the error handling code doesn't get in the way of my reading and understanding of the intent of the code. This is especially true when errors have to be propagated back up a long call stack, where there may be many intermediate functions that would have to return the error. With exceptions, I only care about where they are thrown and where they need to be caught, not all the intermediate points in between.

To me, having the "other channel" of exception handling removes noise from the code and makes it more readable. I agree with Raymond that exceptions are tricky and you have to know how to do the right thing (e.g., don't throw pointers to stack-based objects and make sure to understand threading issues). Once, however, I had those principles understood and enforced, I was able to keep the error handling code "out of the way" of the rest of the code. This allows me to focus on the intent of the code as I read it, and not have to mentally "filter out" all the error handling code.

I certainly don't think that I'm smarter than Joel or Raymond. I do know, however, that I prefer using exceptions rather than having every function return error codes, both for code readability and robustness. I know they make me a more productive programmer.

Buffer Overflows Are So Twentieth Century

2007-08-10T06:41:00.000-07:00

At one of the companies I worked for in the past, the engineering team was able to agree on a set of standards that met the criteria outlined in my previous post. These standards were short, usually 1-2 page documents. As this was at least Year 3 BW (Before Web), the documents were stored in Lotus Notes.

One of the standards that we quickly agreed on was titled "No unsafe string functions." That is, we banned (except for the very occasional exception case) the use of C-language functions such as strcpy, strcat and sprintf that can cause buffer overflow. We wrote a series of small safe functions that would make sure that buffer overflows would not happen. These functions all took a maximum buffer length as well as the target buffer pointer. We didn't want to convert to std::string because there had been a number of cases where some engineers didn't understand their usage and had caused very serious performance problems.

I wrote a Bourne Shell script to search across the appropriate files for all occurrences of the unsafe functions. I ran this once a week or so. Once in a while, I would find an incorrect usage that had crept in, but usually all the engineers understood the need for such a standard and would adhere to it.

A few years later the company was acquired and a new group of engineers was brought on board. We started over again with the standards process, but something strange happened with this particular standard. This new group could not accept it. I never fully understood why. I re-wrote the standard a number of times and had a long email correspondence with the leader of the group (who I have great respect for). However, those of us on the original team were unable to convince the newer members of the need for this standard.

A year or so later, one of the application engineers that used our product came to me and reported a crash. The product that I worked on was a server product, and crashes were taken seriously. I analyzed the crash and found it was caused by the use of one of those unsafe functions. I fixed the problem. Luckily the crash did not happen at a customer site, so we were able to prevent it before the particular application went live.

I then re-wrote the standard (yet again), and sent a pleading email to the leader of the group asking if they could now accept it.

They could, and did.

And now they understand.

What's a Coding Standard?

2007-07-29T15:08:00.000-07:00

There are two kinds of coding standards, although those of you who are less binary than I am may prefer to think of a spectrum of types rather than just two. (You know, of course, that there are two types of people, those that categorize everything into two categories and those that don't.) The first type is a MUST standard. These "MUST" standards have less to do with what the code looks like and more with what the code does. That is, content over form.

An example of this type of standard for C++ would be "All pointer fields in an object must be initialized to NULL or a legal pointer in the object constructor." This is driven by the reality that we should always delete all pointers (that point to memory owned by an object) in that object's destructor. Setting all pointers in the constructor guarantees that, regardless of what happens next and as long as any other code that changes those pointers does so in a proper manner, the destructor can be safely called at any point.

The second type of standard is a SHOULD standard. These standards are more concerned with form, not content. Here things are a more subjective. Here I put standards such as this document containing Linus Torvald's coding standards for Linux (or Google for "linus torvalds coding standards" for some more juicy reading). I agree with all of these except for the 8-space indentions and CamelCase (see, we're already in contention). I'll go over my reasons for disagreement in another post, but this already illustrates my point.

How do we know when something is actually a standard, and not just a document written by someone sometime somewhere? Here I have to defer Nick Pouschine, one of the best people I've ever worked for. His definition of a standard is:

All code in the project actually follows the standard. Note that this means older code also, since most coders spend their time looking at code, not looking at coding standards. If there is code that does not match the standard, then the coders will not take the standard seriously.
The standard must be agreed upon by all the coders working on the project. If it is not agreed upon, different coders will do what they want anyway and the standard is not a standard.
The standard must have at least one reason for being a standard. That reason can be something like the example for constructor usage above, or something like Linus' "A human brain can generally easily keep track of about 7 different things, anything more and it gets confused."

Without these, a standards document is just a document, and not a standard.