Interface Design: APIs as a User Interface

This article closely parallels Interface Design: World of Warcraft vs. Excel. In that article, I claim that UI designers can learn from game designers. Now I'm going to claim that library authors can learn from UI designers.

Library authors write code that is used by other programmers. Their interface is called their API, and, like a GUI, it is all the user has to see in order to use the product. An API doesn't look much like a GUI, of course. It is a highly specialized UI that can only be used by someone trained in their use.

Usability concerns are certainly important for APIs. If a programmer doesn't understand your API, then they will not be willing to use all of the features that you put into the library. They'll have bugs in the corners or the library that they thought they understood, too. Confusing APIs directly contribute to unreliable code and lower programmer productivity.

What can library authors learn from usability experts? This isn't as stupid a question as it sounds. Usability experts have figured out a lot about how human brains learn how to do things. Programmers have brains that work almost exactly like everyone else's brain. (I may get flamed for that sentence, but I'll warn you ahead of time that I can defend it.) There may be something worth learning.

Patterns

Patterns can help a lot. If I'm on a web page, my brain has been trained to try to click on underlined text. This is a valuable pattern that usability folks can use when they want users to click on some text. Likewise, programmers like patterns.

If I'm using a modern C++ library, and I want to get access to a collection of items, I'll try to grab an iterator range using functions called "begin" and "end". Once I get those iterators, I know exactly how to access the contents of the collection. My brain can then abbreviate a lot of different tasks as "this class acts like an STL collection". Suddenly, getting access to a collection in a library becomes easy.

Programmers have been trained to see and use a wide variety of patterns. Generally a framework will define the patterns that users will get trained on. Ruby on Rails, for example, trains users to get access to items referenced through foreign keys using the name of the table that you're trying to get to. You can write a rails library without doing this, but it's something extra that users of your library will have to learn. Instead of thinking "table A has a one to many relationship with table B", you're users will have to remember "table A has a one to many relationship with table B that can be accessed using another function name". It's more information that has to be remembered, and it's likely to be forgotten quickly.

Different users will understand different patterns. A usability engineer would tell you that you should understand your users. This means that you should understand what patterns your users can use easily and what you'll have to teach them. Your goal is to avoid teaching them.

Explorable Interfaces

Usability folks also like to create explorable interfaces. That is, they want to make it safe to try things out. My web browser does this by allowing me to hit the back button after I click on the wrong piece of underlined text. That's great for users since they get to explore the web without losing much time. Library authors can also try to do the same thing.

How can a library be safe to explore? There are a lot of different tricks that authors have used. A combination of multiple techniques is probably the best idea.

A good library for a compiled language will fail to compile when you try to use it incorrectly. It might even give you a clear error message when you do something wrong. An easy way to do this is to use type checking of function parameters. You can go a lot farther, though. For example, if one thing has to be done before another, then you could have the first thing return an object that has to be passed to the second thing. If the compiler helps a programmer figure out the library, they will be more willing to explore different parts of the library. To a usability engineer, the cost of trying something new is reduced.

Making it easy to write tests for your code can also make things easier to explore. Interpreted languages frequently use this as a substitute for the first technique, but it's useful for compiled languages as well.

Documentation is a great way to make a library easy to explore. If you can look up exactly what a function does, then it's a lot easier to use it correctly, and it becomes a lot easier to try new things out with the library.

Usability Testing?

I've never heard of anyone doing usability testing on their library, but it does happen accidentally. I frequently write libraries for use by programmers that I work with every day. If a library fulfills a real need, then those programmers will use it. They'll try to use it in ways that I didn't foresee, and I may have to modify my library to allow their new use. They might make some bugs that I didn't anticipate as well. In some cases, I can go back and change the API to make it more difficult to create those bugs.

This accidental usability testing is about as much as a typical library author might do. Could you do a real usability test? How might it work?

I can imagine telling a coworker to go through a simple use case for a new library. I could watch them try to figure out what's going on. Is it obvious that a mutex has to be held while you make a certain set of calls? Is the name of one of the functions unclear? Is the documentation incomplete somewhere? These are exactly the sorts of questions that usability testing should be able to uncover.

If a library had to be easy to use, this sounds like it would be the way to do it. I've never even heard of anyone trying it in an organized way. Is there a good reason for this, or are library designers just lazy?

This originally appeared on PC-Doctor's blog.

Books that all Programmers Should Read

I read a lot of programming books. However, I haven't read many that all programmers should read. Programmers do a lot of different things, and it's pretty darn hard to write something for all of them.

In fact, I can only think of two books. If you know of another, I'd love to hear about it.

How to Write Code

All programmers write code. Furthermore, writing code is a skill that transcends language. If someone can write really good JavaScript code, then they probably understand most of the principles in writing Scheme code. The skill does not depend on language.

Steve McConnell does a great job of explaining this in Code Complete. This is a classic, and everyone should read it. The first edition had examples from a lot of different languages. Between 1991 and 2004, a lot of languages died out. The second edition doesn't try as hard to be language independent. It's still good, and the points he makes are still language neutral.

It talks about how to write code at the very lowest level: How to name variables. How to format argument lists. When to write comments. Et cetera. McConnell found a large number of obscure studies that prove some of his points, too. It's a great book.

User Interface Design

Sooner or later, every programmer has to write an interface of some sort. Yes, even library authors are creating an API. Even authors of tools that run only on servers have users. If it doesn't have a user, then there's really no point in writing it.

To write a UI design book that can apply to any of that and also apply to a web site requires a fundamentally different approach. Again, I'm going to bring up a classic.

Donald Norman wrote the Design of Everyday Things almost 20 years ago. It's not about which widget to put in the upper left corner of your web page. It's about human psychology and how people learn to use something they've never seen before. Humans are the same no matter what kind of tool they're using. This is the important part of UI design.

American culture does change, and you'll have to keep reading new articles to learn what users are capable of at any given point in time. Norman's book is about the stuff that will remain true forever.

It's also a fun book to read. After you read it, you'll never open a door to an art gallery the same way again.

That's it?

That's all I could come up with. There might be some room for a book on project management since all projects need to be managed. However, project management goes through the same series of fads that programming goes through. To add a book to this list, I'd like it to be timeless. I suspect a book like that exists, but I don't have it.

This originally appeared on PC-Doctor's blog.

What's that Data Structures Class for?

I assume that when computer science students go through college, they all take a required course in data structures. If I were designing a course like this, I'd make them learn how a variety of useful data structures worked. Certainly if you read a book on data structures you'll learn this sort of thing.

How many programmers actually need this information? In today's world, there are a lot of libraries out there that have a reasonable implementation of AVL tree, hash table, and B tree. Certainly some people need to learn to write these things. Why does your typical student programmer care about different ways to handle collisions when inserting into a hash table?

Okay, I'll admit that programming without this knowledge for me would feel a bit like skiing naked. It'd be a bit too weird to choose an algorithm because someone told me that I should always use AVL trees because their worst case performance and therefore their predictability in the face of ignorance is better. For me, I'd rather be able to use a sorted array to improve locality of reference even if I don't know that there's a problem anywhere. I'm sure that at least 95% of the time that I've used an immutable sorted array it hasn't made a difference. I certainly don't check with a profiler unless a real problem exists.

Every so often performance does matter, though. It sometimes matters a lot. In that case, you might say that you need a programmer who knows to check the hash tables to make sure they aren't behaving horribly. However, a decent profiler is likely to tell you a lot of useful information without having any knowledge of data structures. Since these cases are rare for typical programmers, wouldn't they be just fine if they knew a collection of data structures that they could swap in until they got something that worked better?

I can't remember many times when I've had to swap out a data structure because of performance issues. The last time was inserting into the end of a very large STL vector. The copies were too expensive due to C++'s tendency to copy things too often. (Even this case will be fixed with the next C++ standard.) Anyway, STL has some other data structures that can be used. I was able to replace my data structure and things immediately improved. I can't remember enough details to know what knowledge I needed. It's also possible that I guess correctly more often than an ignorant programmer would. Who knows how many times they might need to swap things randomly?

A C# programmer would have it even easier. The System.Collection namespace in .NET doesn't have a lot of different options. It'd be pretty easy to try all the possibilities pretty quickly. If none of the options solves the problem, it's entirely possible that there's something that could be done elsewhere.

Memory and speed performance are pretty much the only times you might care about the differences between a hash table and an AVL tree. A few years from now, application programmers may just add a bit of concurrency if they want more speed. Few web applications run low on memory. Are data structures classes useful anymore for typical programmers?

I've left a lot of unanswered questions in this post. I'm really curious about the answers. I'd love to hear from you.

This originally appeared in PC-Doctor's blog.

Model View Controller and Beautiful Code

I've been reading Beautiful Code. It's a fun book with chapters written by a few well-known programmers about some beautiful code that they've written or seen. They all have wildly different views on what makes code beautiful, so it's pretty entertaining. It's fun seeing what programmers I've heard of think is beautiful. Brian Kernighan, for example, wrote about some string processing code. I found that extremely entertaining and ironic. Yukihiro Matsumoto had an impressively concise description of what it takes to make beautiful code. It was refreshingly in character with the Ruby philosophy.

Most of the authors were completely unknown to me. However, even though the most obscure is more famous than me, I'm still going to tell you about some beautiful code that I run into frequently.

The Model View Controller (MVC) design pattern is, perhaps, one of the oldest out there. It's been used for almost 30 years now, and it is still an extremely clean way to divide up the responsibilities of a user interface. When I see it implemented well, it's extremely satisfying.

For those of you who grew up using Microsoft's GUI frameworks, you may have never seen this done well. .NET Forms and MFC make it difficult to implement it correctly. (In fact, this topic was inspired by me working simultaneously on a Rails and a MFC project.) Ruby on Rails, by the way, revolves around the MVC design pattern.

What's so beautiful about the pattern?

It cleanly separates three different activities. The separation is clean enough that the pattern is relatively easy to use. It's usually easy to figure out whether some functionality should go in the model, the view, or the controller.

Furthermore, even in a highly interactive GUI, the program becomes easily testable. The view is the only part that interacts directly with the user, and, other than interacting with the user, the models and controllers have all of the functionality. (This statement becomes less true if you're writing custom controls such as a text editor.) If you test the models and controllers, then you've tested a large fraction of the code.

I always smile when I see some MVC code that works well.

This originally appeared on PC-Doctor's blog.

Futexes are Fun

Futexes are a synchronization primitive that is new to the 2.6 Linux kernel. If you're like me and had never heard of them before a few days ago, then I recommend reading Ulrich Drepper's article, Futexes are Tricky. To a Windows programmer like myself, futexes are a wonderful idea that would be lots of fun to play with. Essentially, they're the antithesis of Window's synchronization model.

Hmmm. I guess I'm showing how much I dislike Window's synchronization model...

Let's start there. In Windows, if you want to use a synchronization object, you go to the Micrsoft-approved zoo of Win32/.NET synchronization objects, and you find one that's close to what you need. You then modify your problem to fit this synchronization object.

Okay, the zoo of synchronization objects that Microsoft gives you isn't bad. There are some extremely useful ones in there. If you're using .NET, you even get to use a monitor, which was missing from Win32.

In fact, if you're doing relatively normal programming in Windows, you'll never even realize that you're missing futexes. However, if you're trying to build an application that has to have optimal performance, then you may want something unusual.

You may want to do a WaitForMultipleObjects on 65 mutexes. You may want to optimize a mutex implementation as much as possible, and you're willing to give up features like re-entrance. There are a lot of things you might need, and Microsoft deliberately doesn't give you all of them. Thomas Becker has tried to implement something that's not in Microsoft's zoo, and the difficulties that he ran into are instructive.

POSIX doesn't have a universal solution, either. However, it does have a monitor, and, assuming performance isn't an issue, you can solve any problem using this.

Futexes are all that and more. They let a programmer write their own synchronization object in user mode. They do this by avoiding doing anything that's unnecessary. A futex lets you have a thread join a wait list and sleep until it is woken. The user mode code decides when a thread will enter the wait list, and when threads in the wait list will get woken up. They're still a bit new, and they don't let you control which thread gets woken up with much precision. Even if it's missing some features, however, it's still an exciting idea. It allows user mode libraries almost complete control over the performance and features of a synchronization object.

That's pretty darn clever. My next goal is to find an application at PC-Doctor that needs a futex. It sounds like a lot of fun, but it won't work on Windows.

This originally appeared on PC-Doctor's blog.

Bonus Day at PC-Doctor: They Gave Me a Coffin!

This morning was PC-Doctor's 14^th anniversary! To celebrate, the management team at PC-Doctor put in overtime and bought a lot of goodies to give away to employees.

Wow. That's pretty cool.

Maybe I'll win a 50" TV or some Raiders tickets or the massage chair.

The management team picks names, ostensibly at random, and when someone's name comes up, they run over to a pile of stuff they choose and claim it. Sounds great! I don't own a TV, and they've got a bunch of them here. Maybe I can get one?

It's Andy's turn! I work a lot with Andy, but his kids were sick today, so I got to pick for him. He sounded pretty happy about his plasma TV on the phone.

There aren't many choices left, but there are still a few good options remaining.

When's it going to be my turn?

Okay, now it's really hard to find anything left. What's still around?

Uh, oh. It turns out that I'm the last person.

What's still there?

Woot! I get a Lady of Guadalupe Casket! Rob says that it'll get delivered next week.

While I'm not a coffin expert, it's at least got a long list of features. It's pretty long, too, so my legs would only be slightly bent!

I'd told my wife that I might win a TV. Somehow, I'd forgotten to mention that I might win a casket. In retrospect, this seems like poor planning on my part.

One question remains: Is there enough space in my cubical for my new coffin?

This originally appeared on PC-Doctor's blog.

RTL Languages and International Usability

When you localize a product into a right to left language such as Hebrew or Arabic, how much do you have to change the position of UI elements?

It's a pretty simple question, and I think the answer is also pretty simple. If you have to have your eye on the right side of the page to start reading a sentence, then your eye is going to start on the right side of the page.

As far as I can tell, that's the extent of the problem. What are the implications?

You can't assume that users will start in the upper left corner and scan down to the bottom right of a page. Important navigation tools should be on the right or the top. Tab order should go from the right to left.

Is that the whole story? As someone who knows absolutely nothing about the subject, it's tempting to say yes. Unfortunately, Saleh, a Persian blogger, knows better. I recommend reading his blog.

This originally appeared on PC-Doctor's blog.

Fairness in Win32 Lock Objects

Windows Vista has given up fairness in their synchronization objects. In fact, Windows XP isn't guaranteed to be fair, but Vista goes quite a bit further.

The issue and the solution is explained pretty well by Joe Duffy over here. At first glance, it looks as though this could cause some really bad thread starvation in some circumstances.

The change to unfair locks clearly has the risk of leading to starvation. But, statistically speaking, timing in concurrent systems tends to be so volatile that each thread will eventually get its turn to run, probabilistically speaking. Many more programs would suffer from the convoy problems resulting from fair locks than would notice starvation happening in production systems as a result of unfair locks.

The problem they're trying to solve and the problem that I'm worried about are pretty similar, so it's worth explaining it carefully. If you have a mutex that protects a small chunk of code, and you have a large number of threads trying to get that mutex, then you have the potential to have a lock convoy. You can get this easily if you wake up a large number of threads with the same priority and have them all immediately try to get another lock.

Here's what happens. A bunch of threads are waiting for a mutex. A thread releases the mutex. Under Windows XP, the next thread in line to get the mutex now owns the mutex and gets a temporary priority boost. Windows does a context switch to the new thread, and it performs a short operation and releases the mutex. (Note that this will also boost the priority of the next thread in line, but that thread will have the same priority, so no context switch is performed.)

These context switches are expensive. Joe Duffy claims that it's about 1000 cycles. Avoiding these switches would be a good thing, so Vista changes the way mutexes work. When a thread releases a mutex under Vista, the mutex does not become owned by the next thread in the queue. Instead, it becomes unowned. This allows a third thread that's already in the running state to grab the mutex and keep going without any context switches at all. That's much cheaper, much less fair, and has the potential to starve the waiting threads.

One problem with this is that you're giving priority to running threads over waiting threads. As the number of cores in your CPU goes up, then the number of running threads gets higher. Eventually, isn't there going to be some starvation of the waiting threads?

I tried for a while to construct an artificial scenario where a thread could be starved for a long period of time. I've got to admit that I couldn't do it. The reason is that there is always some randomness in the scheduling. In order to make it so that there is a problem, I had to increase the fraction of time that a thread holds the synchronization object. Once you do that, you're going to limit the number of threads that can run without having to run into problems with a lack of fairness. It feels as though that might be true of all problems where this change becomes bad, but I couldn't prove it.

In short, Vista gives up the pretense of fairness in synchronization objects, and it does so without guaranteeing that it doesn't starve a thread. However, it looks to me as though thread starvation will never be a huge issue.

This originally appeared on PC-Doctor's blog.

Interface Design: World of Warcraft vs. Excel

Blizzard's World of Warcraft (WoW) game is extremely successful. For those who haven't been paying attention, it's a game where you coinhabit a fantasy world with thousands of other players. There are over 9 million people who play the game enough to pay Blizzard a subscription to play. The game is successful, and, in part, it's because of the interface. There are some lessons in Blizzard's interface that can be directly applied to conventional desktop or web applications.

WoW is a game that's designed to be appealing to both the casual user who doesn't play much and a smaller but more dedicated group of people who put a lot of effort into getting really good at the game. Microsoft Excel is another great example of a user interface that has to appeal to both casual and expert users, but it's a lot more fun to talk about video games than spreadsheets.

In terms of appealing to both casual and expert users, WoW is far better than any other application that I've ever used. In part this is because Blizzard is trying to convert as many players as possible from casual to expert. For many people, it's not possible to remain interested in a game for a long time without becoming an expert. Because of the subscription business model of WoW, it is critical for Blizzard to keep players interested for a long time.

In this blog, I'm going to look carefully at how Blizzard does this conversion. Microsoft Excel makes a similar attempt, but it is fundamentally different as well. Excel must be usable for both my mother who wants to make a shopping list and my wife who wants to model a wastewater treatment plant design. Microsoft does not need to convert my mother into someone who understands Excel as well as my wife. Microsoft will make the same amount of money from my mom and my wife, so there's no need to convert between the two types of users. WoW makes a lot more money off of someone like me who plays regularly than someone who plays for a bit, gets confused about how to do something, and then gives up out of boredom or frustration.

If you look carefully at WoW, a huge amount of effort has gone into slowly teaching a user about how the world works and how the avatar that the user is playing works. In fact, making the game easy to discover and learn has received an enormous amount of effort from Blizzard.

I'd like to think that authors of non-gaming applications can learn from this. Becoming an expert in an application is extremely satisfying. It's really fun to use Excel if you're comfortable using exactly the right features you need to accomplish a task. It's also really fun to play WoW after you've become enough of an expert to accomplish a difficult task in the game.

There are three things that an expert WoW user knows how to do. First, they know what their avatar can do in the world. Second, they have to understand the interface that Blizzard has provided for the game. Finally, they have to understand the "physics" of the world that their avatar is interacting with; they have to understand how the world reacts to what their avatar does.

Starting out in the world

When you start out playing the game, you don't have much to worry about. There are about 6 spells, abilities, or items that a starting avatar can use. All of the interface options are turned off, so the interface itself is almost empty. Furthermore, there's nothing that might attack you anywhere near you, so much of the complexities of the world are eliminated. Everything is extremely simple.

It's really easy to see what you're supposed to do next, too. There's a person standing nearby with a giant yellow exclamation point over their head. Later, you'll learn what this means. When you're starting out, it's something that draws your attention. You'll try to interact with that person, and this will give you your first quest. You have rapidly learned a bunch of new things about the interface at this point.

The first quest gives you something to do, and something to learn about. Frequently, it just involves walking a ways over to someone else. This is really easy, and it's impossible to be discouraged by this quest. Furthermore, you'll learn how to move around a bit.

Meanwhile, Blizzard is putting a bunch of little "tips" about your avatar and the world on your screen. If you want to, you can read these, but the design holds up without these. Only a few of these are really required to figure out what's going on.

Like Excel, WoW makes it easy to get by without knowing much. In Excel, you can type a list of grocery items without knowing anything about optimizers. In WoW, you can play with the 6 abilities that you're given at the beginning. It won't help to cast Healing Touch on yourself, but you can see what it does almost immediately. The casual user portion of both interfaces is easily discoverable.

Advancing through the world

As you progress through the game, Blizzard will try to teach you about different parts of the world, different abilities of your avatar, and different parts of the interface.

The interface can be played with right away. There are option screens that allow you to turn various features on or off. When starting out, they are all off. This is a lot like Excel hiding the less popular menu items. (But it's not annoying because the interface is changed directly by you. There are no surprises about moving menu items in WoW.) Furthermore, there are some buttons that are tied to actions that aren't needed for a while. Blizzard doesn't do a lot to make sure you know about some of these features, but you'll learn quickly about some of the more useful ones from other people you meet in game. Blizzard also has some information on them on their web site for those who are actively trying to learn about the interface. Some of the features that are only needed by experts are explained only on third party web sites.

This is a great model to use for some non-game applications. Focus on the necessary stuff. Make features that only experts need available, but don't try to get everyone to use them. Put your energy into making the casual features easy and the expert features possible.

The physics of the world becomes apparent only slowly. Blizzard has designed it so that only experts need to understand most of the physics. If you don't want to think and optimize what you're doing, it's perfectly fine to have the wrong impression of what's going on. You'll progress at a slower rate, and you might not be able to do some of the things in the game that are designed for experts, but you will be able to do a lot.

This is also a great model. If things are happening in the background in your application, make sure that the users don't have to know exactly what's going on. When I'm using Excel, I shouldn't have to know what its rules are for when it re-evaluates each cell. It should just work automatically 99% of the time. Experts might have to know, and they can look it up somewhere, but all of the programmer's energy should be devoted to making sure you don't have to know about it rather than explaining how it works.

The abilities of the avatar are slowly learned as you progress. This makes them almost trivial to teach. Every so often you'll kill enough monsters that you'll be able to learn a small number of new spells. Some might simply be more powerful versions of a spell you already knew. Sometimes, you'll learn a completely new spell. At these points, it's easy to convince a user that they should play with the new abilities they've been given.

I don't think Microsoft could hope to do this well with Excel. If Microsoft tried to force users to make a shopping list before they were allowed to use the SUM() function, you'd have a lot of bloggers like myself ridiculing them. The ability to teach a user in this way may be unique to video games.

Experts only

Blizzard has put a lot of time into creating content that experts can explore. I have to admit that I haven't explored any of the content that they've created that requires 25 people working as a team. However, I think I've got a reasonably good understanding of the behavior of monsters and the abilities of my avatar.

There are things in the game that are extremely complex, possibly even dangerous, and Blizzard doesn't do much to explain them. Users are expected to figure it out, look it up online, or not use it at all.

This is a good thing.

An expert user has to be able to think they are an expert. It's no fun being completely confused about a game after playing for days on end. There have to be things in there that are difficult to use or do correctly in order to convince people that they know what they're doing. They shouldn't be impossible to figure out, but a challenge that is overcome is a lot of fun.

Lots of people say that they are experts at Excel on their resumes. A very, very small number of people understand how to use all of the features of Excel. (Many of them are probably QA employees at Microsoft.) Users like to think that they're experts. To the developer, it really doesn't matter if they are experts or not. Both Blizzard and Microsoft make exactly the same amount of money when their so-called expert runs to the store to buy the next expansion pack or upgrade. They don't care if they really are experts.

Let's look at an example of how you can make someone feel like an expert in WoW. Last week, a small group of friends and I were playing WoW in an area that we hadn't been to before. A friend cast a spell that he almost never uses called Soulshatter. Disaster struck immediately, and we all died.

Soulshatter is a spell that makes you feel like an expert after you understand it enough to use it safely. The source of in-game info for the spell says this:

Soulshatter
8% of base Health
Instant
5 min cooldown
Reagents: Soul Shard
Reduces threat by 50% for all enemies within 50 yards.

Most of this is nonsense for someone who isn't already at least familiar with the game, but some of it is actually nonsense to a lot of people who think that they're experts!

First of all, the spell costs some small amount of health to cast, and it also takes a "Soul Shard" to cast. A Soul Shard is an item that requires some time and energy to create, so this cost will discourage people from trying it when they aren't really interested in what the spell does. This is good, because the spell shouldn't be cast without careful thought.

What does it do? It doesn't behave at all like what the tooltip says, even if you understand what threat is, what a cooldown is, and what the "instant" might mean. It causes all monsters that are even remotely near you to be interested in attacking you! If they're already attacking someone, then they'll become less interested in attacking you (that's the point of the spell), but if they're pretty far away and haven't noticed your avatar yet, they're going to come running over and attack you (that's why we died).

I don't know about you, but that's not at all what I would have guessed after reading the description. Clearly, this is a spell that's designed only for experts. I read a description of the spell before we played, so I felt like an expert when I explained it to my friend. Is it really sufficiently complicated that I deserved to feel like an expert? Absolutely not. That's not the point. I felt good making my friend feel bad, but now we both understand it. Next time, he'll feel good about using it safely.

Notice that the link I used is not to a Blizzard owned web site. This is yet another great lesson for application developers. Online communities can and will help you out with expert-only features. If you're not writing for a game with 9 million people playing it, then you may have to create your own, but if you do, then people will have a place to go to find out the obscure parts of your program that you shouldn't be spending much time perfecting.

Experts can also be entertained in ways that don't impact casual players at all. For example, It is possible to get an item that makes you completely immune to any damage for 10 seconds. If used at exactly the right time, this will wow and amaze your friends and enemies. If used very carefully, an expert might even be able to do something useful with it, but it's mostly an item that's good for a laugh. Getting the item is mildly painful, but it's available even to fairly inexperienced players. This is a "feature" of the game that can benefit an expert and keep them entertained for an hour or so, but it also helps out casual players. Again, if someone uses this successfully, they will feel like an expert.

Many other examples exist in the game. I really enjoy discovering how Blizzard has made the game both easy and fun to discover. Not all of the work they've done is applicable to desktop and web applications, but it's frequently fun to try to imagine how it might be applicable.

This originally appeared on PC-Doctor's blog.

C++/CLI vs. C#

We're thinking about writing a new application in .NET. I tend to believe that there's not much difference between any of the languages that Microsoft has created for .NET. VB is essentially C# with different keywords and punctuation, for example. You're certainly not going to change the design of your code because you had to write end instead of }, so, for a long time, it seemed irrelevant to discuss anything but C#.

Then Microsoft fixed their C++ implementation with C++/CLI. This is actually a different language from C#. It tries pretty hard to be C++ with access to the .NET runtime and some extra pointer syntax. In particular, you can still have deterministic destructors, free functions, and generic programming. You'll write a different program with C++/CLI than you would with C#, so it's worth thinking about which language you'd choose.

Deterministic Destructors

Actually, you just have destructors. There is a way to create the whole IDispose mess in a C++/CLI class, but it's largely irrelevant since the destructor actually works. This blog explains how to do it, if you're curious.

With real destructors, you can use the resource acquisition is initialization (RAII) idiom to make your code exception safe and easy to read at the same time. C# supports exceptions, but it relies on some quirky syntax to make them safe to use. If the programmer forgets to use the quirky syntax in a few places, then the code can do some really weird things. (No, there aren't any memory leaks. People like to claim that that's a serious problem with C++. Releasing resources is a more general problem that garbage collection does not solve. RAII solves the whole problem rather than a corner case.)

Anyway, I've gotten lazy as a C++ programmer, and I'm not sure I'm willing to go back to a world where exception safety is something you have to work hard to achieve, and releasing resources correctly is not something that the compiler does automatically for you.

To me, this is enough to make me not want to touch a complex application in C#, but let's keep going.

Free functions

C# does not have free functions. Every function is a member of a class. This sort of restriction really irritates programmers like me. Why does the compiler writer get to decide what the right solution to a problem is? Why can't I use the tool that's right for my problem?

In C#'s defense, I think they're using theory that it's harder to kill yourself with a blunt knife. There are a lot of bad programmers out there in the world. Some of them don't know when to use a free function and when to use a static member function. The designers of C# decided that in a fully OO application, there shouldn't be much need for free functions. In the majority of the cases, free functions are, therefore, a bad idea. If you don't let programmers use them at all, then programmers who don't want to worry about getting things right have one less decision to make. C#'s solution is, therefore, ideal for them.

It's annoying for me.

For some reason, I picked on free functions. I'm a fan of them, but there are other things missing from C#: multiple inheritance, a preprocessor, etc. It's all the same story.

Generic Programming

Some people are violently against this stuff, so I won't try to argue that it's got its place. I'd certainly miss it, but I doubt I'd use much of it here at PC-Doctor.

Refactoring Tools

C++ is a frighteningly complex language. Creating a C++ complier is an enormous amount of work. Generally, by the time companies are done with the C++ compiler, they've run out of time and budget.

Refactoring tools never seems to get put in the budget. Of course, changing the name of a type and having to figure out where all the instances that need to be changed are can be both difficult and occasionally impossible in C++.

C#, on the other hand, has some great refactoring tools. This makes some things a heck of a lot easier.

Conclusion

C++ and, by extension, C++/CLI has a lot of features. There are a lot of things in there that are dangerous to use in the wrong circumstances. However, if you spend all your time doing something, you want to have all the tools available to you. You're willing to learn the extra parts that aren't useful every day. I want to have enough tools to hurt myself.

The refactoring tools will be missed, and I hope that someone will get around to making some good ones for C++ eventually.

This originally appeared on PC-Doctor's blog.

PC-Doctor's Robotics Lab

In August 2007's IEEE Spectrum they had an interesting article about Microsoft's robotics lab.

I do have some highly speculative and mildly interesting thoughts about PC-Doctor's role in such an industry. Before I get there, though, I want to comment on how, when Bill Gates wants to put together an "elite" team of programmers to do something that's new for Microsoft, they take people who have been Microsoft employees from birth. After all, if they've learned anything outside of Redmond, they might not want to do things the Microsoft way!

Okay, that's all the ranting I'm going to do. Other than that, they seem to be doing some cool stuff over there. Essentially, they've created a robotics lab who's goal is to create some software tools that will standardize the way sensors and actuators are hooked together, and they also want to make a centralized database to hold navigation information. The goal is to make home robotics happen on a large scale.

The article says that the current market for robotics is about $11B. That's not much from Microsoft's point of view. However, it's pretty easy to imagine that, if home robotics does get to the point where everyone has a few robots in their house, then it'll be a heck of a lot bigger than that.

Okay. We're finally past the ranting and background material. Why should a PC diagnostics company be interested in this?

A robot or two in everyone's home is a lot of hardware to manage. Camera lenses can get dusty, bearings can start to fail, DC motors can overheat, batteries can lose power, dogs can chew the video cables, and dust bunnies can be unreachable because of obstacles. There's a lot of diagnostic possibilities.

From our point of view, the best part of Microsoft's entry into the market with a bunch of Windows geeks is that they're almost certain to want to make robotics into something like Windows. Windows is incredibly good at getting a bunch of strange hardware to work together despite the different manufacturers not even knowing about each other. Anything that a bunch of Windows programmers will create is almost certain to have similar abilities. This is critical for PC-Doctor. We wouldn't be able to create a diagnostic for each IR camera out there. We'd have to make a single IR camera diagnostic that could work with a large fraction of IR cameras out there.

Would our diagnostics for robotics be substantially different from PC hardware diagnostics? To some extent, they wouldn't have to be. There's certainly a CPU and memory bank on the robot that has to work. For the purposes of a highly speculative blog post like this, let's ignore that.

Certainly, some diagnostics could be very, very different. A fancy robot contains a variety of sensors and actuators. A Roomba, for example, has a steering servo, a drive motor, a pressure plate on the front, and a sensor on the bottom to detect stairs. This is about the minimum number of sensors you can have on a useful robot. (That's why Roomba is successful. It's $120.) Testing these sensors and actuators in meaningful ways might not look a lot like PC hardware.

One difference here is that, if you want to test a sensor on the floor that detects stairs, you'll have to find some stairs. That means that the diagnostic would have to actually use the robot in order to test it. If you want to test the DVD drive on a computer, you don't really have to worry about the graphics card. To test a stair sensor, we'd have to figure out if it's even possible to test it. (It might be in a single floor house. It might be a Roomba and not have any useful navigation capability at all!)

Another difference is that a lot of robots have sensors that measure how well the actuators are working. For example, a motor that controls an arm joint might also have a angle sensor on that same joint. Feedback would be used to ensure that the arm is moving correctly. It's tempting to assume that this would reduce the need for diagnostics. However, the feedback loop actually creates more opportunity for diagnostics. For example, you could keep track of the amount power required to accelerate the arm under various circumstances. If that varies over time, then something bad might be happening. If you wanted to get really complex, you could test an angle sensor by pointing the camera from another robot at the arm and moving it. That would require multiple robots and be a lot of fun to write. (Hey, Ken! Can I write that one?)

Obviously, there's no market for this yet, and we don't have a robotics broom closet in the back of our R&D lab. (I suppose I wouldn't be talking about this on a blog if there was.) It is a fun topic to think about, though.

This originally appeared on PC-Doctor's blog.

Google Web Toolkit: First Impressions

I've starting converting a complex web app from being purely JavaScript and browser plugin based to using the Google Web Toolkit for the JavaScript part of the project. See my previous blog post about this.

Unfortunately for me, this project is not high enough priority to play with it for long periods of time. However, I have managed to start work and get some basic functionality working. It was certainly interesting!

I'm going to comment on a few of my first impressions. I'll state a strong opinion now, but I reserve the right to change it completely as I get more experience with the library and tools.

Speaking of Tools...

Wow. This is really slick. You get to debug the Java code directly without looking at the JavaScript that it compiles to! Eclipse is a great editor for Java code, and GWT integrates nicely with it.

The debugger is nice, the compiled code looks good, and the warnings and errors that the development web server generates are extremely helpful. This is probably the strongest selling point for me with GWT.

Don't touch the DOM!

This was a shock to me. The web browser's DOM is an extremely efficient place to store information about the document you're displaying. A heck of a lot of effort has gone into making this efficient and safe. CSS is an extremely powerful tool that is closely coupled to the document's DOM.

GWT's designers don't want you to touch the DOM, though.

GWT gives programmers only limited abilities to write to the DOM, and they work extremely hard to make it difficult to read the DOM. For example, the DOM.setAttribute call can be used to assign a id attribute to an HTML tag. That sounds like a really useful call, doesn't it? I could use that to make it easy for our graphic artist to design some CSS to describe the appearance of the application. It's deprecated, though. You're not supposed to use it.

GWT is not for documents.

I shouldn't be too harsh to GWT about the DOM, though. This might be obvious, but GWT is for complex, full featured applications that happen to reside in your web browser. It's not for displaying documents. There are some corners of the library that allow you to put HTML directly wherever you want to, and I suspect they work great. They should even work with an external CSS file.

An application is not a document, and I might be having trouble getting over that. I'll get back to you later when I form a stronger opinion.

Writing JavaScript directly?

If you want to do something that simply cannot be done through GWT, you're in luck. GWT has a JavaScript Native Interface that is analogous to Sun's JNI. It works great! I use it extensively to talk to my browser plugin, for example. It's easy to use, and it doesn't screw up the tools any more than they should be.

This, of course, allows you to create whatever back door you want to allow you to manipulate the DOM. However, until I get a better understanding of the GWT techniques, I'm going to try to do things the GWT way. I may have more to say on this subject later.
There are back doors to get there, but it's not a real GWT application if you rewrite their tools to do it your way. Instead, you do things the SWIG way.

GWT == SWIG

Yes, that's right. GWT has more in common with a SWIG application than a JavaScript/DOM/CSS application. Is that bad? It's certainly a great thing if you like SWIG. I don't have much experience with it, so I don't yet have a strong opinion.

However, I do think it's a great idea to emulate a well used framework instead of writing a completely new one. There are a lot of SWIG developers and ex-SWIG developers out there, and I'd guess that they'll feel pretty comfortable in this framework.

Documentation?

The documentation is fine, and I shouldn't complain about it. However, I could have really used this blog post before I'd started. If you start out as a native JavaScript/Prototype/Ruby on Rails developer like I did, then what I've said here is not completely obvious.

The Google group is great, though. I've gotten responses to my queries immediately.

This originally appeared on PC-Doctor's blog.

Why Aren't PCs as Pretty as Macs?

If you walk into an Apple store, and you manage to find the corner that sells Macs, you'll find some fairly attractive looking machines. A lot of them aren't big black boxes, for example. Why can't I buy a PC that looks like this?

You probably want me to say that gaming PCs are designed to be attractive. That's true, as long as you believe that any money spent on appearance looks good. Alienware right now is selling desktops that look like alien insects. That's not what I want, though. I want a machine that tries to blend in and look nice at the same time. The current round of iMacs do this admirably.

Maybe a gaming PC isn't the answer? I can imagine that. Gamers who want to slaughter thousands of cartoons might not want something attractive. How about a high end home theater system? Voodoo PC makes some right now that are designed for exactly this. Are they attractive? Well, they're still big boxes, but, since you're paying $5k for them, you get to choose any color you want, and it's got a cute display on the front.

No, I'm not looking for this. I want a PC that was designed from the ground up to look good. I want a piece of furniture that does what it has to do and doesn't take up any more of my attention than it deserves. It's got an on button and a DVD drive. It doesn't deserve my attention.

Is cost and performance a problem? If a desktop manufacturer has to use notebook technologies to cram everything they want in an unusual form factor, perhaps things are either too slow or too expensive to sell.

I doubt this is realistic. Voodoo PC manages to sell $5000 home theater PCs. I'm going to guess that they could use some notebook technology and not lose much of their profit margin. Performance isn't that important for the PC that I want. All my PC does is surf the web, play DVDs, and occasionally edit photos. I'm even willing to pay extra!

Is there a technical problem with this? Perhaps the ATX motherboard specs force PC manufacturers to have a large rectangle available for the board, and they can't make the crazy shapes that Apple can get away with. While this is a reasonable cop out for a small shop, what about large manufacturers? Dell might be screwed because people don't normally get to see Dells before they buy them. A PC like this would be centered around its appearance. Then what about HP? They could afford to make a motherboard in whatever shape they wanted. Users would get to see the machine in the store before they buy it. It'd be just like a piece of furniture.

Could copyright restrictions be the problem? Apple is extremely possessive when it comes to its designs. Is it possible that I'm too picky, and Apple has prevented anyone from making anything remotely similar to their machines? I sure hope not!

So far, I'm not coming up with any answers. I hope someone out there can shed some light on this.

Computer: Heal Yourself!

The Autonomic Computing Initiative at IBM tries to do some really interesting things. The goal for IBM is to make server hardware run without much human intervention. IBM breaks the problem down into four different parts:

1. Automatically install and configure software
2. Automatically find and correct hardware faults
3. Automatically tweak software and hardware for optimal performance
4. Automatically defend itself from potentially unknown attacks

This is an ambitious goal, of course. They don't intend to complete the project right away. #2 is the interesting one from the point of view of PC-Doctor. However, I'd like to try to look at it from IBM's point of view. They (unlike PC-Doctor) have a lot of influence on hardware standards. The question they should be asking is "What sensors can be added to existing hardware technologies to enable us to predict faults before they happen?". Fault prediction isn't the whole story, but it's an interesting one.

I'd better admit right away that I don't know much about computer hardware. Uh... "That's not my department" sounds like a good excuse. However, I hang out with some experts, so it's possible that a bit has rubbed off. We'll find out from the comments in a week or two!

Hard drives:

This is an easy one. The SMART (http://www.seagate.com/support/kb/disc/smart.html) standard already allows software to look at correctable failures on the hard drives. If you look at these errors over time, you may be able to make a guess about when a hard drive will fail.

This is nice because the hardware already sends the necessary information all the way up to application software running on the computer.

Flash memory:

Flash memory can also fail slowly over time. I don't know of any effort to standardize the reporting of this information, but, at least on the lowest level, some information is available. There are two things that could be looked at.

First, blocks of flash memory will fail periodically. This is similar to a hard drive's sector getting marked as bad. Backup blocks will be available. Some errors during the fabrication of the device will also be marked as bad and replaced before it ends up on a computer. Device manufacturers probably don't want to admit how many blocks were bad from the beginning, but a company like IBM might have a chance to convince them otherwise.

Second, you could count the number of times that you write to the memory. Manufacturers expect a certain number of writes to cause failures in the device, but I don't know how good these measurements would be at predicting failure.

Fans:

A lot of servers these days can tell you when a fan has failed. They might send some email to IT staff about it and turn on a backup fan. It'd be more impressive if you could predict failures.

Bearing failures are one common failure mode for fans. This frequently creates a noise before it fails completely. A vibration sensor mounted on a fan might be able to predict an imminent failure. You could also look at either bearing temperature or the current required to maintain fan speed. Both would provide some indication of increased friction in the bearing.

Network card:

Some Marvell network cards can test the cable that's plugged into it. The idea is to send a pulse down the cable and time reflections that come back. The Marvell cards look at failures in the cable, but you could do a more sensitive test and measure when someone kinks the cable or even when someone rolls an office chair over it. If you constantly took measurements of this, and you kept track of changes in the reflections, you might get some interesting info on the cable between the switch and the computer.

Printed wiring boards:

You could do some similar measurements with the connections on the PWB that the motherboard is printed on. This might help you learn about some problems that develop over time on a PWB, but I have to admit that I have no idea what sorts of problems might be common.

Shock, vibration, and theft

Can you get some useful information from accelerometers scattered throughout a computer? Notebooks already do. An accelerometer placed anywhere in a notebook can detect if it's in free fall and park the hard drive heads before the notebook lands on the floor.

A typical server doesn't enter free fall frequently, though. One thing you could look for is large vibrations. Presumably, large vibrations could, over time, damage a server. Shock would also damage a server, but it's not obvious when that would happen.

Security might be another interesting application of accelerometers. If you can tell that a hard drive has moved, then you could assume that it has been taken out of its server enclosure and disable it. This might be a good defense against someone stealing an unencrypted hard drive to read data off of it. This would require long term battery backup for the accelerometer system. It would also require a pretty good accelerometer.

IBM sounds as though they want to make some progress on this. It would be really nice to be able to measure the health of a server. Most of my suggestions would add some cost to a computer, so it may only be worthwhile for a critical server.

Now, after I've written the whole thing, I'll have to ask around PC-Doctor and see if anyone here knows what IBM is actually doing!

This originally appeared on PC-Doctor's blog.

Multithreaded Programming for the Masses

Writing software on multicore CPUs is a hard problem. The chip designers have told us that they're not going to do all the work for us programmers anymore. Now we have to do something. (Here's a good description of the problem from Herb Sutter.) Writing multithreaded apps is not easy. I've done a lot of it in C++, and the tools, the libraries, and the design patterns just don't make it trivial. I don't mean to say that it's impossible, however. It's certainly possible if you're careful, you plan for it from the beginning, and you know what you're doing. The real problem is that 99% of programmers aren't all that good. A defect in multithreaded code can be really hard to track down, so it's expensive if you get it wrong. A lot of companies and research groups have been spending a lot of time trying to figure out how to make it easier. I haven't been doing any research on the topic, but, like a good blogger, I'm going to comment on it anyway.

First of all, do we have to speed up all applications by the number of cores in your CPU? Your typical Visual Basic programmer isn't any good at threading, but do they have to be? They can create dialog boxes and display data and let you edit that data. Do they have to know anything about threads for this?

Probably not.

It's still possible to speed up that dialog box, too. If there's some text entered by the user, the grammar/spelling checker might be running on a different core from the UI widget. This wasn't done by the Visual Basic programmer. The dialog might have been drawn using a graphics engine that used threads or even offloaded some computations to the video card. Again, this wasn't done by the programmer who's never worried about threads.

So, we don't always have to know about threads. Some substantial fraction of the programmers in the world don't have to speed up their code using multiple cores. That's good news. We really need to admit that not all programmers need to make their programs faster. The libraries they use can be made faster without compromising the reliability of the program itself.

That's not the whole story, though. Let's look at the other end of the spectrum. How about a video game?

Graphical processing is astonishingly easy to parallelize. GPUs will continue to get faster over time because of this. A lot of calculations can be done on individual pixels. It's relatively rare that two pixels that are far apart will affect each other substantially. (There are some techniques that defy this statement, but a lot of them rely heavily on expensive matrix calculations which may be parallelizable. I know nothing about these techniques, but I'm going to guess that they're easy to parallelize, too.)

If video cards are going to continue to get faster, then the CPU had better keep up. The CPU is going to have to generate enough data to avoid starving the GPU. As the GPU gets infinitely fast compared to a single core of the CPU, this will require multiple cores.

Uh, oh. Does this mean that all game designers will need to use multiple threads?

I'm sure that it doesn't. Couldn't the game engine programmers do all of the hard work and let the average game programmer use a safe API that avoids threads? Certainly a level designer who's scripting an in-game event in Lua would not be forced to worry about threads!

What about an AI programmer? I don't know enough about game engine design to say for sure, but I'd be willing to bet that a cleverly designed game engine could avoid exposing a lot of the multithreaded mess to many programmers, but at some point AI programmers will have to do things on multiple cores at the same time.

While the AI programmer might be fairly smart, and they might be trained in multithreaded programming techniques, that does not mean that it's a good idea to subject him or her to mutexes, condition variables, thread creation, deadlock avoidance, etc. That's not a good use of a programmer's time.

What can be done to help a programmer who really does have to run code on multiple cores?

Functional programming languages are a potential solution. The creators of languages like ML, Haskell, and Erlang realized that your compiler can do a lot of fancy things if the programmer isn't ever allowed to modify a value. If things can't be modified, then they can be easily parallelized. Once you create an object, as many cores as you want can read it without even bothering to lock it. Of course, this will frustrate a Visual Basic programmer who is used to changing a variable as the object that the variable represents changes. It requires some significantly different programming techniques.

Once again, this is not for everyone.

Futures are a pretty slick way to abstract away threads and some locks. It doesn't eliminate the headaches of multithreaded programming, but it has the potential to make things simpler.

A future is a chunk of code that a programmer wants to execute. Instead of calling the function that contains the code and waiting for the return value immediately afterwards, the programmer separates the calling and the waiting. First you tell your library that you want your future to run. The library can do whatever it wants at this point. It could execute it immediately. It could put it in a queue to be run by a thread pool. It could ignore it completely. The program can then do whatever it wants. At some point, it may want the result of the future. Then the program will ask the library for the result of the future. The library may have to wait for the result to be ready, but eventually the result will be returned.

The great part about futures is that threads don't have to be created. If you can divide your program's logic into N bits, you can let your library and compiler figure out how to use that to speed up the program using multiple cores.

Unfortunately, futures don't eliminate the need to worry about locks. Since that's probably a more difficult problem than thread management, futures are not a panacea.

There are some other ways to easily parallelize your code. SQL statements are a great example. Databases are really good at optimizing SQL for this sort of thing.

Intel has a strong interest in making it easier for programmers to use their new chips. Their Thread Building Blocks library uses STL-style algorithms and protected iterator ranges to express parallelism. It also has a "task scheduler" that behaves at least a bit more like a futures library. This seems like a really great idea. An STL-style algorithm that describes the parallelism to the library is not always sufficiently flexible to describe a problem. However, if it is sufficient, it's extremely easy to use. The task scheduler is more conventional and would probably be much nicer with a good lambda library.

OpenMP is another library designed to abstract away many of the details of parallizing code. It's not strictly a library, however. It requires some compiler support as well. The programmer uses a function that behaves much like a Unix fork() command. OpenMP then manages several threads to handle the different branches of the fork. While I'm certainly not an expert, this doesn't seem like either a clean solution or a sufficiently flexible solution.

I'm sure there are other research projects out there. If you know of any interesting ones, please post a comment about it below.

This originally appeared on PC-Doctor's blog.

Exploiting an Industry's Culture

Sometimes, an industry becomes uncreative or stops taking risks. This lets an outsider come in and gain market share by exploiting the mistakes made by an entire industry. It's fun to look for these industries and understand what they're doing wrong.

My favorite example is the video game market. (Roger Ehrenberg has a good summary of the Xbox side of this.) Ten years ago, everyone in the video game industry was happy thinking that all gamers were young males. The fact that other people spent enormous amounts of time playing Solitaire and Minesweeper didn't seem to bother anyone.

Then The Sims came out. This should have been a wakeup call. To a young male such as myself, it was a complete waste of time. To Electronic Arts stockholders, it was gold. That happened in early 2000. Believe it or not, nothing much happened for a long time. People at least talked about why nothing happened, though. It was apparently pretty hard to convince a publisher to risk large amounts of money on something that wasn't a clone of a successful game. All successful games involved things that young guys like to do. (Solitaire didn't make any money for Microsoft.)

You can tell where this is going.

The Wii was the next big one. Now you can play bowling and have enough fun doing it that your grandmother will join you. (Trust me, she doesn't like Halo.) Even so, the industry sat around for a few months saying that the Wii was just a fad and it would be back to young males sometime soon. (The definition of "young" seems to change, though. 35-year-olds play Halo 3.)

I think the video game industry has finally woken up. They've given a term to gamers who aren't young males! That could well be what was missing before. "Casual gamers" don't like to play first person shooters, and now even Microsoft wants to lure them to the Xbox.

Okay, that's a story of an industry's culture having major problems and some companies exploiting that. That story is almost complete. There's another one that's happening right now.
Of course, I'm talking about cell phones and the iPhone. Many years ago when I bought my last cell phone, I really wanted a good user interface. Despite spending a lot of time designing user interfaces, I didn't want to figure out someone else's bad user interface. There were a lot of them back when I bought mine. The Sony Ericsson collaboration seemed to be doing okay, so I got one.

Apparently my $200 purchase towards a decent UI didn't motivate the entire industry to work on improving, however. Apple has figured out how to make and sell fashionable and easy-to-use consumer electronics, and they've exploited Sharp's inability to do the same.

Now, it's still not obvious that the iPhone is a runaway success. It should be noted, however, that everyone at least knows what an iPhone is. I'm a bit jealous of my friends who have them, too. That's going to help attract buyers to a cheaper phone if Apple came out with one.

The cell phone story isn't complete, but I'm betting that it'll end up with Apple doing well. They didn't do well at first with the iPod, either.

Alright, now we've talked about the story that's mostly complete and the story that's happening right now. What about an industry that has a cultural problem right now but hasn't yet been exploited? One of the best ways places to look is a small industry that doesn't have a lot of players in it. If one company is dominating a market, then any cultural issues that that company has could be exploitable by a complete newcomer.

A few of you have probably figured out where I'm going with this one, too.

Is the hardware diagnostics field vulnerable? PC-Doctor is the big player here and we might have some problems. Several potential vulnerabilities might be there.

First, are our diagnostics any good? Well, I happen to know something about our diagnostics. I find it really hard to believe that a newcomer (or even a current player) can do as well here. This is what we do, and we do it well.

Are really good diagnostics what people want, though? Well, it is what companies like HP, Lenovo, Dell, or Apple want on the machines that they ship to their customers. They want to be able to trust those diagnostics, and they can when they run PC-Doctor.

Here's a potential exploit, though: What do the customers of those big companies want? Do they want fast and trustworthy diagnostics? I don't think they do. They want something that says that their machine is broken, why it's broken, and how they should fix it. They don't care if it works 95% of the time or 99% of the time. They just need it to work this one time.

Furthermore, they don't care if the problem is hardware or software. That's a critical question to a PC manufacturer who only warranties hardware defects. That's not the right question for a lawyer in Florida who wants to know if he should download a new driver or buy a new hard drive. Right now, PC-Doctor doesn't deal with software issues.

How could this be exploited? Well, suppose a company made some really good software diagnostics. Then they could add some fairly bad hardware diagnostics to it. The big companies might not be impressed by these hardware diagnostics, but the end users might be since the software solves the problem that they want to solve. It would take a new entrant to the diagnostics industry a while to build up a complete set of hardware diagnostics, but they might be able to do it by focusing on what the consumer needs instead of what the big companies need.

I could also be part of the culture that's screwing up. If that's the case, then I wouldn't even know that there was something else wrong. Is our user interface so bad that no consumer would ever use it without a tech support guy telling them what to do? Are we completely unaware of this?

Should I be worried about this? I'd love to hear what you think.

This originally appeared on PC-Doctor's blog.

Explanations in user interfaces are bad!

When I was doing the design for the BTO Support Center website, I had some troubles explaining to some coworkers why helpful text shouldn't be added to explain the interface. At the time, I couldn't explain it well, but now that I've thought about it a while, I think I have a better way to describe it.

My new argument assumes that the interface is explorable. Let's start with that.

If a user is comfortable with a user interface, they will happily play with it until they get it to do what they want. This is called an explorable interface, and it's required in any good interface. For example, I'm typing this on the blogging software's built in editor. I've never used this editor before, but the cost of just pushing buttons randomly on it is low. I can undo them easily, so I'm not worried about pushing the wrong thing. The editor doesn't normally pop up a useless dialog box that I have to get rid of, so pushing the wrong button is unlikely to waste much of my time. It is laid out in a way that explains what buttons are relevant, so I don't even have to scan most of the web page. The cost of not knowing what I'm doing is low, so I've never read most of the text on the page.

The economist Herbert Simon described this behavior as "satisficing". This is a combination of satisfy and suffice, and Herbert Simon used it to describe people's behavior when confronted with a choice that is expensive to resolve. If the cost of reading an entire web page to find the optimal solution is expensive, then people are perfectly happy to use the first thing they find that might work. The interface designer has to do a lot of work to make sure that the first thing they find is the correct one.

In an explorable user interface, users don't have to figure out exactly what the designer was thinking. They don't have to read every bit of text on the dialog. They can just pick a button that might do what they want and push it. This, it turns out, is often substantially cheaper than trying to understand the interface completely. After all, you can just undo it afterwards.

Once you understand this, a bit of text to describe your interface is clearly the wrong answer. A user will not actually read the text on the web page if exploring might work. If exploring doesn't work, the user will become frustrated rather than resorting to your helpful bit of text. In some cases, they will be perfectly happy to assume that your interface doesn't support the option they were hoping to find.

Don't explain your interface. Make it easy to explore.

This originally appeared on PC-Doctor's blog.

Choosing a programming language: Is type safety worth it?

I'm a big fan of the strongly typed language vs weakly typed language debate. It's old, but it's also important.

I'm revisiting this topic because I'm trying to decide exactly that problem on a project that I'm working on. In my case, I'm torn between client side JavaScript on the web browser and Google Web Toolkit (GWT) compiled to JavaScript. The only reason I have to mention this is because I want to eliminate the quality of the libraries from the debate. GWT has fairly rudimentary libraries, but they're growing, and you can add JavaScript libraries if you really have to.

I'm going to try to have a debate with myself about it. I can't claim that I'm unbiased, and I'm going to ignore arguments that I don't consider significant. I'd love to hear further arguments from you in the comments.

Argument for weak typing

Weak typing allows you to write code faster. Well, that's what the proponents of it claim, anyway. I haven't seen any good measurements of this. There's certainly less typing to do, though. You don't have to declare the type of your variables. (In JavaScript, you do have to declare your local variables, however.) (It is worth pointing out that I have never written software and been limited by my typing speed. I'm skeptical of this argument.)

If you're designing a scripting language, then weak typing is easier to explain to inexperienced programmers. There's a whole step that doesn't have to be done. In some circumstances, this is a clear win. (In my case, this is irrelevant.)

Argument for strong typing

Strong typing allows the compiler to do more work for you. Essentially, you're constraining how variables can be used, and the compiler can use those constraints to detect errors or perform optimizations.

The optimization part is important for a few people. The error detection is great for everyone.

Counterargument from weak typing.

Unit testing is the weakly typed solution to error checking. (It's also the strongly typed solution, but for a different class of error.) It's either hard or impossible to write a unit test that checks that function arguments have the correct properties before going into a function. However, you can write a unit test that ensures that functions behave correctly in a variety of different circumstances. That's what programmers really care about, anyway.

Also, suppose we want to have a function that supports multiple types? In a strongly typed language like Java, we can only support different types if they share an interface. In a weakly typed language, we can support multiple types if they support the same operations. If I want a function that adds all the elements of a collection together, I can write one function that supports both strings and complex numbers correctly. (This is possible with generic programming as well in a strongly typed language, but that's not really what we're arguing about here.)

Rebuttal from strong typing

Unit testing is a good guarantee, and it's one that's required by strongly typed languages as well. However, it's a fundamentally different guarantee than you get by having the compiler check the types of variables as they're used by the code. First, a compiler checks every line of code even if it's hard to get to or you forgot to write one of the unit tests. Second, the combination of tests and strong typing is a better guarantee than tests alone.

Here's a Ruby example of some dangerous code:


function foo( a, b )
  if a < 0.0
    print( b )
  end
end
foo( -12, 'This string is printed' )
foo( 5 )


This code has two "overloads". One takes a string and a negative number. The other takes a non-negative number and might take a string. The problem is, there is no good way to tell this based on the function signature. You can only tell by looking at the implementation. This means that you can't safely change the implementation of your function.

Unit testing partially guarantees that functions have working implementations. Strong typing partially guarantees that they are used correctly.

Conclusion

I'm going to go with Google Web Toolkit, which imposes strong typing on top of JavaScript. The mediocre library support may bite me later, but I'll be happier knowing that my compiler knows a bit about what my code will do.

I'm hoping I get flamed by someone, so please post your comments!

This originally appeared on PC-Doctor's blog.

ActiveX is still around!

ActiveX has been around a while. When Microsoft was battling Netscape, they needed a way to put custom, active content on web pages. Java was being used by Netscape, and people thought it was great. Microsoft needed something they could develop quickly that would let programmers put new types of content on the web browser. ActiveX was born.

The basic idea behind ActiveX is really simple. A programmer creates a DLL that can be accessed by anyone. Some introspection is added, and now a web browser can call native code! Once you're in native code, you can do whatever the heck you want, so Microsoft's work was done. Of course, Microsoft added a bunch of ways to make it complicated, but the basic architecture is extremely simple.

That was back when Bill Gates thought that no one would pay money for security. The security model Microsoft used was also extremely simple: All ActiveX DLLs are signed. If someone hijacks thousands of computers using your DLL, then Microsoft will know who's responsible!

Of course, Microsoft signed some DLLs that had some big holes in them. In fact, lots of legitimate companies did. For the next decade and a half, a whole team of Microsoft employees dealt with the consequences of these design decisions.

ActiveX is still here, though.

Even in Vista, ActiveX is still available. It's still possible to run whatever code you want in Internet Explorer under Windows Vista. If you don't believe me, go over to your Vista machine and head over to this URL: Microsoft Windows Update. This page can replace your drivers and reboot your computer!

So, what has Microsoft changed? Is it still business as usual in the land of ActiveX? No, it's not. A lot has changed.

First of all, enough warnings pop up around an ActiveX control that both programmers and users avoid them like the plague. Back in the early days, programmers were supposed to put UI widgets on the browser window because Microsoft said it was easier to do it that way than by using HTML. (This conveniently prevented the page from loading under Netscape, so no one actually took this advice.) Now, almost no one makes ActiveX controls. Once you've got some video players, Flash, and a few others, you're done. No one else has to write them anymore! Certainly no one has to write one that requires administrative access to the computer. Once Windows Update was finished, the designers probably concluded that that was all you needed.

There's very close to no documentation on the subject, but it's still possible to have your ActiveX control run as an administrator. The strange part is that now, instead of being a mainstream programmer, you have to put on your dark sunglasses and visit some very murky areas. Microsoft won't tell you exactly what to do, but they do put clues in a variety of blog postings and tech notes. Bugs will haunt you as you make your way toward what you need, and you'll never really know if you're exploiting the OS or doing it correctly.

It's amazing what's changed. It's even more amazing how little has changed.

This originally appeared on PC-Doctor's blog.

What is usability testing all about?

The phrase usability testing gets thrown around a lot. It sounds great when you're planning a project. If you say you'll do some usability testing, then people get a warm feeling about your project plan.

After discussing it with a few people, I've concluded that there are a lot of myths out there about usability testing. I'll outline all of them that I've either heard from someone or thought to myself.

First let me explain what it is.

No. I lied. That's a huge topic, and I'm going to bypass it here. Instead, I'll refer you to the best summary I've read on the subject: Dumas and Redish, A Practical Guide to Usability Testing. It's a bit old, and there are likely better summaries out there, but linking it nicely lets me avoid explaining what usability testing is.

I will, however, explain what the goals of usability testing are:

The idea is to watch your users interact with your product (or something similar to your product) in a way that allows you to see how well the product works for the users. In addition to finding problems, usability testing also tries to gather data that the testers can use to figure out how the problems should be corrected.

That description is designed to shoot down several of the bigger myths that I've run into.

Myth: Usability testing gathers statistical evidence that you can use to make decisions about solving a usability problem.

Running a good usability test on a single user is expensive and time consuming. There's a lot of data that gets analyzed, and I've found it to be a heck of a lot more useful to get more data about a single user than it is to test multiple users.

You end up running tests on very small numbers of people. I generally spend a fair amount of time before and after a single test preparing the test and analyzing the results. The test will find some problems, but you correct those problems before you analyze how bad the problems were. If a problem is bad, then you're likely to run into it. If a problem is small, you're not likely to be bothered by it again. It's much better to just assume that all problems you run into are large enough that they should be solved.

All forms of usability testing that I've done follow this formula: Watch a single user use your product. For every problem that user runs into, figure out what caused the problem, and decide if and how you need to fix it. All problems a user runs into are considered real problems until proven otherwise.

Myth: Usability testing requires a lot of fancy equipment.

I hear this sometimes after people read about what large software companies use when they do usability testing. I've never seen it, but I've heard rumors about rooms filled with hidden cameras, one-way mirrors, and eye tracking devices. It sounds like fun, but it's really not needed.

When I do some usability testing, I try to understand the user as well as possible. I want to know what they're thinking when they click the wrong menu item. I tend to be in their face a lot more than someone standing behind one-way mirrors would be, but, for a lot of problems, a user being in an artificial environment with a couple of engineers breathing down their neck isn't as bad of a problem as it sounds. Certainly, a lot can be done this way.

Myth: Any old user will work.

The background of your user has a huge impact on how they view your product.

In the early stages of testing, I like to use coworkers as much as possible. They're easy to sucker into these tests; the first few times you do it, they even think it's fun! However, the data you generate from these tests requires so much interpretation that you can quickly get to the point where it's not worth your time. (They're great for the early tests, though. If nothing else, they can help test your testing procedure!)

Myth: Usability testing has to take a long time.
Myth: Usability testing can be done very quickly.

These myths are different, but the answer is the same. Usability testing can take as long as you need it to. It's fairly difficult to anticipate how long you'll need, however. Unexpected usability problems frequently come up. They need to be fixed in code, and if you're not lucky, that might take time.

If the dialog box or web page that you're testing isn't all that important, then it's possible to run tests that will fail to catch smaller problems. This can greatly speed things up, and it is possible to test something quickly and get away with it.

One of my favorite easy test to run is one that tries to decide between a small number of completely different approaches. This can be quick, but it's also possible that none of the approaches tried works all that well. I haven't been great at predicting the time required for usability tests that I've done, and I claim that it's a problem with usability testing rather than my own shortcoming.

One of the more dangerous forms of this myth is the belief that a large, complex product can be tested comprehensively in an amount of time that management will be happy with. Usability testing of a significant amount of UI code is a major project, and it really needs to be done continuously over the entire lifecycle of the product.

Myth: There aren't any other myths.

As I find some more time, I'll come back and address some more issues that I've run into. I think I've gotten the biggest ones that I've seen, though.

This originally appeared on PC-Doctor's blog.

How to become a programmer in ten easy steps.

A cousin of mine recently told me that he wants to switch jobs. He wanted to become a programmer! Specifically, he wanted some advice on becoming a programmer quickly. He's a really smart guy. I have no doubt that he can learn everything he needs to become a programmer.

Can he do it quickly? Well, I didn't learn very quickly. I started somewhere in elementary school, and I did it as a hobby for a long, long time. I still have a ways to go before I can unequivocally say that I'm good at it.

That's not really what he's asking, though. He's asking what it takes to switch jobs to become a programmer. It's much better to get paid while you learn, so he wants to land a job as a programmer and continue learning. What does that take?

I discovered that I have no idea. No one hired me when I was learning in elementary school. I wouldn't have hired me until I'd learned how to deal with complexity in software. There are a lot of really bad programmers who have been hired, but my cousin asked me, so I answered with a description of how to deal with complexity.

I should point out that he's already an engineer, and he expressed some interest in academic research as well. Programmers obviously have more fun than other engineers, so I had to rebut those possibilities as well.

I think the real breakthrough that I made that started me on my way to becoming a good programmer is to realize that it's all about psychology and only minimally about algorithms. Most beginning books that I've flipped through don't talk about complexity at all. Let me explain...

Computer software these days is unbelievably complicated. A common metric for measuring the complexity of software is to measure the number of lines of code in a program. Windows is around 10 million lines, for example. (I'm making the number up.) That's 10 million individual parts that all have to work reasonably well together. This is the sort of complexity that's impossible with mechanical parts; mechanical parts simply aren't reliable enough to make something that complex. The only competition for complexity is in circuit design, and I can argue that this isn't as complex as software, either.

Computer programming ends up being about the management of complexity much more than about algorithms. I get into about 100 times more conversations at work about hiding complexity or managing information than I do about algorithm design.

Once I figured that out, my programming skills got enormously better. Now that I've figured that out, writing code is a lot like writing an essay. I'm writing to an audience, and the audience has to understand my point immediately. It's a bit harder than writing a essay, though, because you also have this secondary audience of the complier/interpreter/computer. The compiler has to have the same understanding as the primary audience.

This probably does a lot to remove the glamor from software development for you. However, you should understand that making a million lines of code (or even 10 kloc) understandable to a human being is a very serious undertaking. It's a really non-trivial problem, and people are still trying to figure out how to do it easily. (This includes university research groups, but the funding is higher within companies.)

Here's an example of a really serious problem that hasn't been solved, yet. I'll assume you know about Moore's Law. Until somewhat recently, Intel and AMD have made CPUs faster and faster by accomplishing more per clock cycle and performing more clock cycles per second. Now the speed of light has gotten in the way, and they have to make collections of CPUs on one chip that work together to solve a problem. This has the interesting side effect of moving the problem of speeding up computers from the chip designer to the software developer!

In order to solve the problem, software developers have to write software that does more than one thing at a time. This has been done a lot by computer programmers before, but only by good programmers! It's not easy to do. Suddenly, Moore's Law is in jeopardy not because of lithography limits but because software is too complex to write! It's a potential crisis that's looming in the industry now, and a lot of people are working hard to solve it.

I hope my cousin does become a programmer. He's a bright guy, and if he finds it interesting, I'm sure he'll do well eventually. I also wonder if I'm hoping that he won't learn quickly. I certainly didn't. Maybe I just wish that the code I wrote way back when I was learning was a lot better than it was!

This originally appeared on PC-Doctor's blog.