The Next JavaScript...

ECMAScript 4.0 (ES4) is on its way. This will be the next standard for JavaScript. It's not going to be usable on web pages for a while, though. In fact, I suspect I won't be using it on my web page for at least 5 years. The problem is simple: as long as people still use older browsers, you won't be able to assume that people have it.

However, the features in it are an interesting look at what the standards committee thinks is wrong with the current JavaScript. This is not a minor patch release. This is a dramatically overhaul of the current JavaScript (ES3). Oh, they've included a lot of minor things that are simply broken in ES3. These changes are certainly interesting, but today I'm going to talk about their major focus. They want to make it easier to develop large applications in JavaScript. Clearly, they understand that people are starting to develop large applications for web browsers, and they feel that there are problems with the currently available technologies for this.

I don't have any experience with developing what I'd call a large JavaScript application, but we are starting to develop an extension of PC-Doctor for Windows that uses JavaScript in numerous places to control its behavior. In my dreams, I imagine that it will eventually become a large application made up of plugins that run on JavaScript.

Let's go through the major features that the committee thinks I'll need as our technology gets bigger...

Classes and conventional OOP. I thought this was interesting. They're adding conventional classes as an alternative to the prototype based inheritance that ES3 supports. This comes with a bunch of normal OOP things like virtual functions, getter and setter functions for "virtual properties", interfaces, inheritance, etc.

I can certainly understand why they wanted this. Type safety is greatly enhanced if you can know exactly what type something has. ES4 makes it possible to create a "conventional" class who's members look a lot like Java's and C#'s. You can't write or delete arbitrary properties of one of these classes. With ES4 and a properly written program, the language will be able to determine that you're doing something unintended to some objects and fail at runtime.

Type safety. The new class system allows values to actually have a strict type. Most user created types in ES3 are simply Objects with certain properties and a specific value of the prototype member. That's close to a type, but it's only an approximation since it's possible to change it with some member assignments. The new classes will have a strict type that cannot be changed.

This is bigger than it sounds. This means that code can require certain types in certain variables. It means that a member function will remain a member function for the lifetime of that object. The runtime engine can enforce this, too. The standard doesn't hide the fact that inserting some extra asserts in the code will slow things down, however. They bring this up clearly, and they don't apologize for it.

This is called type safety, and it's something that I've been advocating for a while. The committee wanted it badly enough to be willing to slow programs down for it. They go beyond even that, though.

They've introduced another mode of execution called "strict mode". In this optional mode, the execution engine is supposed to perform some static analysis of the program before it begins. The engine is supposed to fail when a variety of illegal things are possible in the code. For example:

1. Writing to a const property.
2. Calling functions with the wrong number of arguments.
3. Referencing properties of class objects that don't exist.
4. Failing a type check.

The great thing is that all of this happens before the code is executed. It's done entirely with static analysis. If anyone ever bothers to implement this mode (it's optional), then JavaScript is going to become my new favorite scripting language!

I should point out that there are few guarantees about strict mode. Different browsers are going to check for different things in strict mode. Therefore, it is only useful in development. I predict that developers will run the most strict web browser on their development machines and publish without strict mode turned on.

I'm going to claim that this is strong support for my work on a Lua static analysis program. They are coming extremely close to saying that in order to write large applications, a scripting language requires strong type checking, and static analysis might be able to relieve the runtime of some of its need for that. Since Lua's runtime has no static type checking, it needs static analysis as well.

Yeah. I'm going to claim that many of my previous rants are backed up by this committee!

Function overloading. Function overloading is a trick that's only available to languages that can define the type of an object. ES4's version is slower than Java's version because it has to happen at runtime. However, the important thing is that it's possible.

Functions are values of variables in JavaScript just like strings or numbers. You might ask how it's possible to assign more than one function to a single variable name. They do it in two ways.

The first is something they call a generic function object. This is an object that can hold more than one function in it. It exists as a single value. It's a bit clunky, but I couldn't come up with anything better.

Speaking of ugly, there's another way to overload functions. They added a switch statement that works on the type of the value. Here's the example from their overview paper:

switch type (v) {
case (s: string) { ... }
case (d: Date) { ... }
}

That looks like a pathetic attempt at overloading to me, but it could well be how generic function objects are implemented internally. It may be that this could be used to create a more flexible generic function object. However, I predict that consultants 5 years from now will make lots of money going to companies to give talks condemning the use of it.

Packages and namespaces. This is used to hide the activities of different components in an application. This is clearly lacking in ES3 even with small scale libraries and applications. Library authors tend to go to great lengths to hide as much of their code as possible from the global namespace. With the small applications I've written, it's been working fine, but I can see its limitations.

Overall, I'm pleased with where ES4 is trying to go. They're trying to make it safe to build large applications with JavaScript. Given where we're seeing web applications going, this is a great thing. However, I can't imagine that it'll work anytime soon since Google Web Toolkit gives ES3 programmers compile time type safety right now.

Mostly, however, I'm pleased that the committee is trying to add type safety to an untyped language in almost exactly the same way that I would.

This originally appeared on PC-Doctor's blog.

Requirements of a Language to Statically Analyze

For those of you just tuning in, I'm working on a project to statically analyze an untyped code base to try to bring some of the advantages of typed languages to the code base.

The first step is to figure out which language I should write a static analysis tool for. This is obviously an important decision with quite a few implications both at the beginning, during the creation of the parser, and at the end when we try to find an actively developed code base to look at.

The first requirement is that it be a relatively popular, untyped language. Let's look at some languages:

1. JavaScript, ECMA Script, or JScript. This is used by millions.
2. Lua. This is a popular language to embed in applications. I know it extremely well, too.
3. Python. This is really popular, and it's a pretty clean language, too.
4. Ruby. This is getting to be a popular language. (I really like the language, too.)
5. PHP. This is also a popular language.
6. Perl. I've gone through life without having to learn COBOL, Fortran, or Perl. I'm a really happy guy. Do people still use it? I wouldn't know.

Yes, there are some others, but I'll limit my discussion to these.

I'll reject Perl immediately because it's designed to be complex. This is just a hobby project, and I'm not going to be able to spend lots of time dealing with unneeded complexities. I also have no interest in learning it.

I'm also going to reject PHP for similar reasons. I just have no interest in it. Now we're down to harder choices.

We're down to four languages: Ruby, Python, JavaScript, and Lua. These are all languages that I've enjoyed using, and some of them I know pretty well.

Let's start with Ruby. Ruby is a wonderful scripting language. I love it for its terse, expressive syntax. I'm a bit worried by the quantity of syntactic sugar in it, but most of that can be stripped away after parsing it.

However, Ruby code is frequently written in a different style than conventional, object oriented code. Ruby programmers are used to doing things that would be utterly opaque to any reasonable static analysis tool. Here's an extreme example of some Ruby on Rails code that scares me:

lang_code = 'en'
obj = MyModel.find_by_name 'name of obj'
result = obj.send :"language_#{lang_code}"

For the non-Ruby geeks out there, this pulls a MyModel object out of the my_models table in the database. The object is populated with some members based on the database schema. That's mildly scary all by itself, since I'd rather not spend my time dealing with DB schemas. However, the last line is even worse! This calls the get_language_en member function in obj, but the only way for a program like mine to figure that out is to look at the value of lang_code. I'd much rather ignore values and only look at types. (Many scripting languages treat functions as ordinary values that can be moved around. As we'll see in future installments, this isn't always a big problem.)

Other languages can do this, too, but Ruby programmers enjoy doing it a bit too much.

Python is another well designed language that would be fun to use. I can't claim that I'm an expert at it, so there may be other problems with it. However, the standard libraries for it scare me a bit. There are a huge number of them. As far as I can tell, Python programmers use whatever they feel like using from these, so an automated way of deducing the argument types and return values of functions would be required. I'm sure that a doxygen summary of the functions could be found somewhere, but I'd prefer to not rely too heavily on infrastructure that doesn't sound like much fun to work on.

Lua is an extremely simple language that's designed to be embedded in applications. As such, it avoids both the large amount of syntactic sugar in Ruby and, at least in most cases, the huge library of Python. However, it does mean that you'd have to find the correct application of Lua to look at. Let's assume that such an application exists.

Like most (or all?) of these languages, Lua treats functions as first class values. Because of this, it is possible to pick the function to call based on a variable. Here's an example:

selector = 'a'
obj = { a: function(num) return num+1 end, b: function(str) return str..'!' end }
obj[selector]('string')

The last line is the worrisome one. This calls a function, passing a string into a function that probably requires a number. (There are ways to modify the behavior of types in Lua. Off the top of my head, I don't remember if the latest versions of Lua allow strings to have overloaded operators.) If we assume that this isn't possible, we'd like to detect such an error. That's extremely hard.

Realistically, however, if we reject Lua because of this problem, then we'll also reject all other scripting languages that treat functions as normal values. As far as I know, that's all of the ones on my list. Fortunately, this behavior is not a frequent pattern in the Lua code that I've seen and written. There are other ways of accomplishing the same task in Lua that are significantly simpler, more powerful, and, therefore, more popular.

The last language on the list is JavaScript. JavaScript can be used in a lot of different ways, but by far the most common and exciting way is in client side scripting on web pages. This provides an enormous body of code that could be analyzed. In addition, there are other tools out there that create type safe JavaScript by cross compiling from another language. This code would make an excellent test case for my tool.

JavaScript doesn't suffer from the same problems that Ruby and Python have, either. While you can do what I described in the Lua example above in JavaScript as well, it's not especially common for exactly the same reason. (Both Lua and JavaScript support closures. Closures are much more powerful, and, even though Internet Explorer has some nasty bugs related to them, they are extremely popular.)

Web browsers do have a mildly large library embedded in them, but the security restrictions are so large that the library is inherently limited. ActiveX controls and Mozilla Plugins and extend the API in annoying ways, but the body of code out there is so large and varied that pages that use plugins that I don't want to support can be thrown out.

The one real disadvantage of JavaScript is the optional semicolon rule in the parser. The parser is required to insert semicolons in a bunch of places, and this does sound mildly annoying to parse. It can't be that bad, though. (Lua has a similar rule involving newlines, but it's much more obscure.)

Up until now, JavaScript looks like an exciting possibility. Unfortunately, I do a lot of work here at PC-Doctor using JavaScript, and I'd rather do something completely different. I really don't want a useless hobby project to feel like work! If I need another excuse, it's that others have already written limited tools like this for JavaScript.

Lua 5.1 is it.

This originally appeared on PC-Doctor's blog.

Static Analysis of Untyped Languages

A while ago I wrote about whether or not untyped languages were a good idea or a bad idea. I didn't come to any real conclusions at the time, and it's bothered me. I'd like to outline a way to gather some real conclusions now. I still won't be able to come to a conclusion, but I think this approach sounds interesting.

There are a number of advantages and disadvantages to untyped languages. One of the disadvantages is that, because variables have no type, you can assign the wrong type to them. A strongly typed language can avoid most of this problem. (You might still have problems involving type conversion, but a good programmer or a strict language can avoid those problems.)

I'd like to propose a tool that can analyze some untyped code without running it. (This is called static analysis.) The tool will try to deduce the types of variables. In theory, it is possible to analyze what possible types a value has at any given time. We couldn't be as strict as a typed language, but we can make some conclusions. For example, let's look at the following Python code:

class Duck:
  def _init_(self):
    self.hungerLevel = 0

daffy = Duck()
feed(daffy)

The first few lines define a type with a single numeric member variable. We can tell this before we run the program. The last two lines create a Duck and pass it to the feed function. We can tell at compile time that daffy is a variable of type Duck and feed is a function that will receive Ducks. Functions may receive different types at different times, but we can record all of the possible inputs and make sure the types are compatible at each point.

This cannot work perfectly, of course. For example, there might be code paths that are simply impossible to go through because of the value of variables rather than the type of variables. Determining all of these cases cannot be done without running the program. (My theoretical computer science is pretty weak, but I vaguely remember some theorems that could prove that.) However, if there's a program out there that is type safe but requires some of the code paths to be impossible to reach, it's probably not a particularly well written program.

I'd like to write something like this, but haven't fully justified it, yet. If I had such a tool, I'd try to run it on a body of existing, open source, untyped code.

I can imagine two possible ways to use this to strongly suggest that untyped languages are dangerous despite the claims of untyped language fans. First, real problems could be found in the code base. If the author(s) of the untyped code would admit that problems were there, then, almost by definition, it seems as though the problems exist. Second, the author(s) could become of a fan of the tool and use it during development. Perhaps the final code doesn't have type safety problems, but the code has those problems in its early stages. This could also be used as evidence that there's a problem with untyped languages.

Unfortunately, it's hard to imagine how this experiment could prove that untyped languages don't have a problem. Furthermore, it's possible that a tool like this works well enough that the problem largely goes away after programmers start using it. (The latter seems unlikely, but it's certainly a potential complication.)

Don't expect rapid progress here, but I'd guess that, at the very least, I'll have some follow up posts about this project as it progresses.

This originally appeared in PC-Doctor's blog.

Choosing a programming language: Is type safety worth it?

I'm a big fan of the strongly typed language vs weakly typed language debate. It's old, but it's also important.

I'm revisiting this topic because I'm trying to decide exactly that problem on a project that I'm working on. In my case, I'm torn between client side JavaScript on the web browser and Google Web Toolkit (GWT) compiled to JavaScript. The only reason I have to mention this is because I want to eliminate the quality of the libraries from the debate. GWT has fairly rudimentary libraries, but they're growing, and you can add JavaScript libraries if you really have to.

I'm going to try to have a debate with myself about it. I can't claim that I'm unbiased, and I'm going to ignore arguments that I don't consider significant. I'd love to hear further arguments from you in the comments.

Argument for weak typing

Weak typing allows you to write code faster. Well, that's what the proponents of it claim, anyway. I haven't seen any good measurements of this. There's certainly less typing to do, though. You don't have to declare the type of your variables. (In JavaScript, you do have to declare your local variables, however.) (It is worth pointing out that I have never written software and been limited by my typing speed. I'm skeptical of this argument.)

If you're designing a scripting language, then weak typing is easier to explain to inexperienced programmers. There's a whole step that doesn't have to be done. In some circumstances, this is a clear win. (In my case, this is irrelevant.)

Argument for strong typing

Strong typing allows the compiler to do more work for you. Essentially, you're constraining how variables can be used, and the compiler can use those constraints to detect errors or perform optimizations.

The optimization part is important for a few people. The error detection is great for everyone.

Counterargument from weak typing.

Unit testing is the weakly typed solution to error checking. (It's also the strongly typed solution, but for a different class of error.) It's either hard or impossible to write a unit test that checks that function arguments have the correct properties before going into a function. However, you can write a unit test that ensures that functions behave correctly in a variety of different circumstances. That's what programmers really care about, anyway.

Also, suppose we want to have a function that supports multiple types? In a strongly typed language like Java, we can only support different types if they share an interface. In a weakly typed language, we can support multiple types if they support the same operations. If I want a function that adds all the elements of a collection together, I can write one function that supports both strings and complex numbers correctly. (This is possible with generic programming as well in a strongly typed language, but that's not really what we're arguing about here.)

Rebuttal from strong typing

Unit testing is a good guarantee, and it's one that's required by strongly typed languages as well. However, it's a fundamentally different guarantee than you get by having the compiler check the types of variables as they're used by the code. First, a compiler checks every line of code even if it's hard to get to or you forgot to write one of the unit tests. Second, the combination of tests and strong typing is a better guarantee than tests alone.

Here's a Ruby example of some dangerous code:


function foo( a, b )
  if a < 0.0
    print( b )
  end
end
foo( -12, 'This string is printed' )
foo( 5 )


This code has two "overloads". One takes a string and a negative number. The other takes a non-negative number and might take a string. The problem is, there is no good way to tell this based on the function signature. You can only tell by looking at the implementation. This means that you can't safely change the implementation of your function.

Unit testing partially guarantees that functions have working implementations. Strong typing partially guarantees that they are used correctly.

Conclusion

I'm going to go with Google Web Toolkit, which imposes strong typing on top of JavaScript. The mediocre library support may bite me later, but I'll be happier knowing that my compiler knows a bit about what my code will do.

I'm hoping I get flamed by someone, so please post your comments!

This originally appeared on PC-Doctor's blog.