Namespace visibility in C#

Java has package scoping. It allows an element to be visible only within the same namespace. It's a wonderful thing.

Here's how it works in Java:


package com.pc-doctor.mynamespace;
package class Foo { ... }


The class Foo is only visibile within mynamespace.

Even though I'm not a Java programmer, this immediately strikes me as extremely useful. Frequently, helper classes are only needed by code that lives close by.

There are two reasons to want namespace visibility to be enforced by your compiler:

1. If you can make those classes invisible outside the namespace, it will make life a lot easier for clients of that namespace. Having only the useful classes appear in Intelisense is a big win.
2. Having helper classes be invisible also helps construction of the component that is in the namespace. If the compiler doesn't let anyone make calls to the helper classes, then we can make much stronger assumptions about how our clients use the code.

C# does not offer any support for namespace visibility. However, there are three ways to accomplish it. None of them are perfect, and one of them is a bit bizarre.

The Microsoft Way

Microsoft expects you to make the classes internal. This prevents anyone outside of the assembly from using the class.

However, you have to make a separate assembly for each namespace that you want to do this with.

Frankly, that's painful enough that few people do it.

The C++ Way

C++ also lacks namespace visibility. The folks over at Boost use a separate namespace underneath the main namespace for helper classes (and functions). They've standardized on the name "detail" for this namespace, and it works fine for problem #1. It doesn't do anything for #2, though.

This is an easy thing to do, but it doesn't do much in C#. C# programmers use a lot more using directives than C++ programmers should. The only place to put a using directive in C# is at the top of the file. This means that, if a single function needs a namespace, then the whole file will get it.

The end result is that a lot of detail namespaces will be visible. This means that you'll have to make a using directive for the detail namespace that you really want, and you'll have to avoid typing "detail" to get to an element.

Even with those problems, it's probably worth doing in C#. Create a detail namespace under each namespace and put things that you'd like to have package scope in there.

Getting the C# Compiler to Enforce Namespace Visibility

C# does have a feature that can be used to emulate namespace visibility. This solution is a bit weird.

I should point out that it uses a feature of C# that wasn't designed for what we're going to use it for. In C++, this kind of behavior is encouraged and well supported. In C# (and Java), you're not supposed to deviate from the party line.

I'm sure Microsoft doesn't have any tests written for this behavior; I've already found one bug in the compiler from this technique.

What is a namespace? It allows many classes to be placed in the same scope even when they're stored in different files.

A partial class does the same thing, and this can be used to emulate a namespace. Partial classes were designed to allow Microsoft's code generators to create part of a class and put those portions of the code in a separate file. C#'s designer tool makes heavy use of this.

It's also very close to a namespace!

If you use a partial class as a namespace, then you can put multiple classes in different files and have some of them be invisible outside of the "namespace".

A private class inside the "namespace" is visible to other classes in the namespace, but it is not visible outside of the namespace. Likewise, a public class is visible outside of the partial class.

All of this is enforced by the compiler, too. We get both of the benefits of namespace visibility with this technique.

Unfortunately, it doesn't work perfectly.

For example, using declarations don't work for a partial class. If you're not in the namespace, you will always have to use the partial class's name to access public members of the namespace. This may be annoying in some cases, but, if you limit this technique's use to cases where there isn't a lot of access to the namespace from outside of it, then it's not serious.

It also looks wrong in the IDE. The IDE has no idea that your class is really a namespace, so it gets colored incorrectly. The severity of this problem is a matter of opinion. It doesn't bother me.

This is all awkward enough to prevent me from completely replacing namespaces. Instead, I use this when I want to expose an extremely simple API and perhaps a type or two. It's really only worth the trouble if you get to hide a lot by using this.

The syntax is also verbose:

namespace pcdoctor {
  partial public class fakeNamespace {
    private class NamespaceVisibilityClass {}
  }
}

There's actually a hidden benefit of this technique. It's possible to make a "free function" in the fake namespace. A static member function behaves a lot like a free function. It's accessible from any of the classes in the namespace. If it's public, then it's accessible from outside the namespace.

I mentioned that I found a bug in the compiler. Unfortunately, I didn't track down exactly what the bug was. Instead, I just found a solution and moved on.

However, if you find that some of your code gets executed more than once when the fakeNamespace type is instantiated, then you might want to find another solution.

Good luck! Just remember that the central authority that controls C# programmers doesn't want you to do this. You're on your own here.

The Visitor Pattern as an Alternative to OOP

The visitor pattern from the GoF is frequently overlooked by programmers who are used to object oriented programming. However, in some cases, it is significantly cleaner and easier to use than an overridden function. Unfortunately, it's easier to misuse as well, and, when it is used poorly, it can be a real mess.

I was going to tell you about my static analysis project and how I'm using the visitor pattern there. Then I took a glance at the wikipedia article on the visitor pattern. It's clearly written by a OOP fanatic who's never seen the alternatives, so I'm going to contrast my implementation of visitor with the one there.

The contrast is useful because wikipedia's implementation is written using object oriented principles. Part of my goal with this post is to explain about OO alternatives. My implementation is written using compile time polymorphism rather than runtime polymorphism. As we'll see, this is significantly prettier and more flexible than runtime polymorphism.

Java-Style Visitors

I recommend clicking on the wikipedia link and reading a bit. Like all good OOP C++ programmers, he's got loads of virtual functions in a variety of interfaces.

Some of those are used to implement a completely unnecessary runtime double dispatch mechanism. The double dispatch functions can be eliminated by avoiding the use of the interfaces in the first place. This immediately cleans up his classes a bit.

If the compiler always knows the type of a class, then the accept() and visit() messiness disappear. The accept function essentially acts as a safe cast from the base interface to the derived type. If we never let the compiler forget what the derived type is, then we don't need a cast.

Yes, I'll admit that I laughed when the author confidently says that those functions are required. They're only required because the author wanted to do dispatch at runtime instead of compile time. The visitor pattern does not require this.

There's another interesting point to notice about the interface. He's got one virtual function for every single derived class in his visitor. There are no functions that handle a collection of derived classes. This is because he's forced to use the class hierarchy to tell the compiler which function to call. He cannot use any other properties of the derived classes for this. Without that flexibility, he'll often be forced to write a separate function for each derived class.

Traditional OOP techniques

I'd like to briefly touch on what would happen if a visitor pattern wasn't used at all.

In this case, the visitor class in the wikipedia code gets replaced by a virtual function that gets implemented for every single derived class in the hierarchy. If you want to perform that function, you simply call it. This is often the correct thing to do, too.

However, it's worth looking at some of the disadvantages that are solved by using a visitor. First, the code in each of those functions gets spread out over the entire class hierarchy. If those functions are more tightly coupled to each other than to the object that they're attached to, then this will be awkward to maintain.

Second, each one of those derived classes has to be known and available for modification. You can't make it so that third parties can add additional derived classes, for example. Every derived class has to be found as well. In some cases, that can be a serious problem.

Finally, the functionality provided by your new function has to be advertised to every single author and user of the hierarchy. If the functionality is obscure and only useful in limited cases, then a visitor (or a free function) should be considered instead. The only one who has to know about the functionality provided by the visitor is the author of the visitor. Nothing else gets touched.

Compile Time Polymorphism

The visitors that I'm writing for my static analysis project are dramatically different.

They act on the abstract syntax tree (AST) of the parsed Lua program. There are about 40 or so different grammatical constructs in Lua, and so there are about 40 classes in the AST.

The visitors themselves act on each of these classes, but they frequently only have a few functions in them. For example, I've got one visitor that calls an aggregated visitor on each node in the abstract syntax tree.

The ForEach visitor has to know about several different ways that the classes can hold child nodes, but it can act on each of those techniques in a single function. For example, it's got a function to handle a vector of pointers to child nodes. It's got another one for a single pointer to a child node. It's got a few others, but it certainly doesn't have 40 member functions in it! I suspect that that's what the author of the wikipedia article would have ended up doing, and it would have been miserable.

I've got a bunch of visitors other than the ForEach visitor. Each of these describe an algorithm that acts on the AST. The algorithms end up almost completely separated from the data structure that the algorithm acts on. I can add a new algorithm (and visitor) without recompiling or editing the data structure code. This separation is what the visitor pattern is great at.

The reverse is certainly not true, however. If the data structure code changes, then the visitors will at least have to be recompiled. In my case, I've written a lot of the visitors generically enough that most of the code doesn't have to change, but a recompile of C++ code that contains a lot of template metaprogramming can be expensive even without having to edit the code. I wanted to be somewhat immune to changes in the data structures, though.

For example, Lua has a "repeat...until" statement as well as a "while" statement. (They do exactly what you'd expect.) Internally, I decided that the repeat...until statement should store the test and statement block in the reverse order of the while loop. I want to be able to change that without changing the ForEach visitor. I want the ForEach visitor to be mostly independent of the internal organization of the AST classes. Because ForEach uses template metaprogramming instead of manual coding to figure out how to traverse the AST, this works perfectly.

All of this works reasonably well with compile time polymorphism. Most of this would be impossible without it.

In my case, the disadvantages aren't all that bad. However, they do exist, so let's look at them.

The largest problem is that small amounts of template metaprogramming were required to pull this off. This means that compile times are longer than a Java programmer would think was possible. (It takes a 5 seconds or so per source file for me.)

It also means that you get some really ugly functions with lots of the "typename" keyword messing up the flow of the code. I suppose you're supposed to get used to that, but I hate it. This isn't really a problem with the visitor pattern, however. It's really a problem with C++. It may be cleaner in other languages.

Overall, however, there weren't many disadvantages to using this type of visitor. However, traversing an AST appears to be exactly what visitors are good for; The GoF book uses that as their example, and quite a few web pages have emulated them. In retrospect, it makes me feel a bit foolish for writing this article!

This originally appeared on PC-Doctor's blog.

Alternatives to Object Oriented Programming

Object oriented programming is extremely popular these days. It's so popular that some people aren't even aware of alternatives to it. The reason its popular is clear: OOP works well a lot of the time. Traditional object oriented programming styles have some significant disadvantages in some circumstances, however. Some of the alternatives are worth looking into when those disadvantages become awkward.

As part of the static analysis project that I'm working on, I'm trying to use alternatives wherever I can. In part, this is because the advantages are substantial for the project I'm working on, but I'm also curious how far it can go and what will happen when I push it too far.

So far, I love it. Inheritance is a relationship between classes that is only a small step from C++ friendship. The derived class becomes extremely tightly coupled to the base class, and, in some cases, the base class can even become coupled to the derived class. The temptation to add functionality to the base class can be large as well.

I'll talk about two ways of partially avoiding inheritance. Both of them rely on compile time polymorphism and cannot (or should not) be used if runtime polymorphism is required.

How often do OO programmers actually need the runtime polymorphism that they enjoy so much? Certainly it's needed some of the time, but it's not nearly as useful as it looks at first. For example, my static analysis project contains a different class for each of several dozen different Lua constructions. This sounds a lot like a case for inheritance, but it's only got one virtual function in the whole project, and that's there only to merge the C and C++ code. If the whole thing had been in C++, it wouldn't have been needed.

Free Functions

Many programmers in the OOP world will happily add new functions to the public interface of classes whenever they can. For example, the C++ string class that PC-Doctor uses has a public member function to return a uuencoded version of the string. This is an extreme case, but it's still worth looking at.

In this case, we've got some functionality that we want to give to a small number of users of a class. There aren't many places in our code base that use this function, and essentially everything uses the string class. Furthermore, the function can easily be implemented using the existing public interface to the string. In this case, it's pretty obvious that everyone would be happier if the function was a free function. As a free function, most users of the string won't ever have to see it. In fact, they don't have to include the appropriate header file or even link to the appropriate module. The interface to string becomes a bit easier to understand for the majority of string users, too.

Furthermore, if you wanted to implement uuencoding for multiple types of strings, you could still do it. For example, if you had different types for UTF-8 and UTF-16 strings, you could overload the free function for both them.

As I said, this was an extreme case. There are disadvantages to free functions, however. Let's look at the other extreme for a bit. For this, let's see what Herb Sutter has to say about free functions in Exceptional C++ Style. He claims that all you should construct your interface to classes by exposing the minimal set of public member functions with everything else as a free function. His argument in the book also looks at a string class, std::basic_string.

Strings, however, are a bad example to generalize from. Some of his arguments get much weaker if you look at a highly specialized class that will only be used in a few places in a single program. Since the vast majority of code that I write is exactly that, I'd rather study this case.

With that in mind, I've made the 40 classes that form the abstract syntax tree for Lua into classes that only have constructors, destructors, and access to internal data. This is, perhaps, a bit more than even Herb Sutter would recommend. I give public access to all of the member data so that there isn't a chance that any member functions could be needed.

I've written most of the code, and it works fine. However, it hasn't survived the maintenance phase, and it's not clear that it will.

However, I'm not going to change the way I write normal application code. At PC-Doctor, I'll write many more classes that get used by only a few other classes. I won't create many free functions for them. Why?

First, free functions clutter the namespace they reside in. I only want that function to be seen by the small number of programmers that might be interested in the class I just wrote. Considerably more people will see the namespace that the class is in.

Let's look at an example. Suppose I want to write a function that appends an alert to a collection of alerts that will be shown to the user. The function might be named "append". If this is a free function, it'll get lost. I want users of my AlertCollection class to know that they can append alerts to it. I don't want them to have to dig around. My users are not going to be using it for long, and they don't want to remember the details later. They just want a complete catalog of everything they can do so they can quickly do what needs to be done.

If a programmer at PC-Doctor decided that they were fans of handling alerts and they wanted to actually remember my API, then putting some of the functions as free functions is, again, a problem. There's one extra bit of information that the user has to remember for each function.

Finally, languages like Java and C# can't have free functions. This is their loss, unfortunately. While I can't give a hard and fast rule for when to use free functions, I can say that they are a valuable tool when developing some APIs.

Visitor Pattern

Well, that was more than I wanted to write in one week. (Remember, I'm a programmer!) The visitor pattern will have to wait until next week. It's probably worth a whole post on its own anyway. It's tricky to use effectively, but it's occasionally extremely effective.

This originally appeared on PC-Doctor's blog.

Mixing C and C++ Code

One of the early steps to my static analysis project is to parse the language that I'm going to analyze. I'd like to form a relatively clean Abstract Syntax Tree that I can play with later.

C++ has a lot of advantages over C for this sort of thing. It's got an enormous amount of machinery that can be used to build high level abstractions without sacrificing much more runtime overhead than you're willing to pay for.

However, C is also still a useful language. For example, it's extremely simple when compared to C++. Perhaps in part because of this, a lot of code generators out there still create C code. If you want to use one of these code generators with anything but C or C++, you can't do much to tweak the output they generate. C++ could well be unique in its ability to mix the old style C code with modern programming techniques.

I've been using some code generated by Flex and ACCENT recently, and it's nice being able to use a lot of C++'s features with the code. (ACCENT still uses K&R function definitions. It's pretty old and primitive, I guess.)

I've worked on a few projects that required a mix of the two, but I can't claim that I've ever thought about exactly how it should be done. Part of the problem is that it's extremely easy to mix the two. C++ allows much more integration with a C library than most languages. In C#, for example, about all you can do when dealing with a C library is make calls into it. C++ allows you to define types that can be used by the C library, you can play with #defines just like a C programmer can, and you can pass function pointers around without worrying whether they're from C or C++. (Though std::tr1::function is better!) You can even include random header files and make calls into the internals of a library.

Clearly, some thought should be done before mixing the two. I'm embarrassed to say that I haven't ever done this deliberately.

Currently, I'm using Flex, a library that exposes more than simply an API to call into. You are supposed to compile the code into your module, and it is possible to modify the code by creating some C compatible types and passing them in through #defines.

It's all very clunky and C-like, but it works great, and that's all I care about today. I'm happy to use it, but I don't really want to pass in a POD type. I want a full fledged C++ class to be used by Flex.

Mixing Flex's generated code and modern C++ code is entirely possible, but there is a lot of type safety that gets thrown away when going to the C code. For example, Flex thinks that each programming language construct and all of its parts are all made up of Symbol* objects. In C++, I don't want to have an abstract base class that defines all behavior that all of these constructs might ever need. Instead, each different type of construct is a different type in C++. This means, for example, that it's not possible to pass an object that isn't an expression to a function that requires an expression. I like that a lot, but Flex wants all of these objects to be the same.

The C code, therefore, will go through a facade that does a bunch of careful type conversions and then passes the resulting, typed objects through to the C++ code.

This is a frequent theme in other projects that I've done that involve mixing C and C++ code. C++ is capable of supporting a much richer set of types than C is, and there always seems to be some work involved in preserving C++'s type expressiveness in C.

Incidentally, the facade is incredibly ugly at the moment. There's lots of typeid operators and long if..then..else if...etc constructs generated by preprocessor macros. However, the rest of the C++ code always knows exactly what the type is, so it's all worth it.

Flex and Accent's generated code do have to perform operations on these objects, however. They end up using a long list of "member functions" that always take Symbol pointers. For example, there might be a function to create a C++ expression object by taking a unary operator and another expression object. In Accent's world, this looks like this:

Symbol* CreateUnaryOperation( Symbol* op, Symbol* expression );

That's essentially a constructor for the Symbol* object. (Flex doesn't have to worry about whether or not it's a pointer or a struct. To Flex, it's just a copyable typedef.)

This is another common theme of mixing C and C++. You don't have to stop object oriented programming in C. You just have to wrap it all up in a bunch of free functions. It's even possible to have virtual functions by manually creating the vtbl.

It also nicely displays another annoying theme of mixing C and C++. In C++, you always know what the rules for the lifetime of an object is. If a function returns a value directly, then the caller gets control over the lifetime. If the function returns a smart pointer, then the smart pointer gets control over the lifetime. (Smart pointers also come with a set of rules about what the caller is allowed to do with the object.) If a reference is returned, then the object's lifetime is controlled elsewhere. It's all wonderfully unambiguous.

In C you always get either a value or a pointer. It's true in my project, too. All of my fake constructor functions return a Symbol*. I really want it to be an auto_ptr, though! If Flex or Accent were to misplace a pointer, then it would be lost forever. So far, they don't seem to do that, but it's certainly filled with ambiguity.

I haven't come up with a satisfactory solution to this problem. My current solution is to always do exactly the same thing with all of the objects returned by my facade's functions. If the programmer has to remember a set of rules at all times, then it'd better be a simple set of rules.

It looks like this is the beginning of officially working on statically analyzing an untyped language. Next week, I might talk about what language I picked and why. Or perhaps I'll talk about my crazy idea for a completely clean decorator pattern implementation so that I don't have to mess up my nice, clean AST.

This originally appeared on PC-Doctor's blog.

Model View Controller and Beautiful Code

I've been reading Beautiful Code. It's a fun book with chapters written by a few well-known programmers about some beautiful code that they've written or seen. They all have wildly different views on what makes code beautiful, so it's pretty entertaining. It's fun seeing what programmers I've heard of think is beautiful. Brian Kernighan, for example, wrote about some string processing code. I found that extremely entertaining and ironic. Yukihiro Matsumoto had an impressively concise description of what it takes to make beautiful code. It was refreshingly in character with the Ruby philosophy.

Most of the authors were completely unknown to me. However, even though the most obscure is more famous than me, I'm still going to tell you about some beautiful code that I run into frequently.

The Model View Controller (MVC) design pattern is, perhaps, one of the oldest out there. It's been used for almost 30 years now, and it is still an extremely clean way to divide up the responsibilities of a user interface. When I see it implemented well, it's extremely satisfying.

For those of you who grew up using Microsoft's GUI frameworks, you may have never seen this done well. .NET Forms and MFC make it difficult to implement it correctly. (In fact, this topic was inspired by me working simultaneously on a Rails and a MFC project.) Ruby on Rails, by the way, revolves around the MVC design pattern.

What's so beautiful about the pattern?

It cleanly separates three different activities. The separation is clean enough that the pattern is relatively easy to use. It's usually easy to figure out whether some functionality should go in the model, the view, or the controller.

Furthermore, even in a highly interactive GUI, the program becomes easily testable. The view is the only part that interacts directly with the user, and, other than interacting with the user, the models and controllers have all of the functionality. (This statement becomes less true if you're writing custom controls such as a text editor.) If you test the models and controllers, then you've tested a large fraction of the code.

I always smile when I see some MVC code that works well.

This originally appeared on PC-Doctor's blog.