haXe improves

posted on 2006-01-30

I did a lot of changes and improvements to haXe this WeekEnd.

The typer was greatly improved. I fixed some minor issues and I added some lazy typechecking so for example when a method is used for the first time, its body is first infered before we start unifying the arguments. This is really more powerful since you don’t have to type arguments and return types of your methods so much, they’ll still be strictly typechecked. In previous cases for example unification was done even if the method was not yet typechecked, and later the unified arguments types where used to typecheck the method body. That was more error-prone since correct methods could fail because of previous erronous usage. This is now fixed.

Also, a lot of changes have been made in order to ease haXe usage. The upcoming beta 1 release is focused on maximizing easability. For example, haXe will come with a small Neko Web Server that can be run locally to develop websites, so you don’t have to install and configure Apache (you might still need to install MySQL if you don’t have an accessible database).

Also, on Windows, a small executable called haxesetup will be part of the distribution. It will setup the PATH environment variable correctly and register the .hxml files to be run with the haxe commandline compiler.

The HXML format is just a list of haXe parameters in a text file (one parameter or class name with optional argument per line). Here’s an HXML example :

-neko test.n
-main TestServer.hx
--next
-swf test.swf
-main TestClient.swf

Running the HXML is the equivalent of calling the following command :

haxe -prompt -neko test.n -main TestServer.hx --next -swf \
    test.swf -main TestClient.swf

The user simply doubleclick on the HXML file to run it. If an error occurs and -prompt was set, then the haXe compiler will wait for Enter before exiting. This way the errors are displayed in the newly opened Windows Console. In that example, it will compile the server and the client (--next separates two haXe batchs, like if haXe was called two times).

So what ? You could think that it looks like a poor Makefile system. True. It’s nothing fancy, but it’s just a good and easy way to enable Windows users to use haXe without opening an IDE or the commandline, and it doesn’t prevent using the haXe compiler directly from the commandline.

Neko 1.2

posted on 2006-01-09

Neko 1.2 have just been released on http://nekovm.org

This include several major changes :

  • runtime exceptions : several operations that were returning “null” before are now raising an exception. This is the case for invalid function calls (not a function or invalid number of arguments), object field access (for example null.x) , array access (still null if outside of bounds) and numerical operations.
  • linker : using “nekoc -link” you can now link a lot of .n bytecode files together into a single standalone .n file
  • nekoboot : this utility enable you to create standalone executables from a single bytecode file
  • renaming : the neko virtual machine is now named “neko” (instead of nekovm) and both neko and nekoml compilers are named “nekoc” and “nekoml”. Compilers are built using nekoboot and are then standalon executables (this is more easy to use, simply “nekoc myfile.neko”).
  • TCO : tail recursion optimizations in Neko
  • Object Prototypes : object can now have chained prototype (see Language Reference Documentation)
  • standard library : added UTF8 support, improved XML parser, and other useful primitives as well
  • licence change : Neko 1.2 is now LGPL while Neko 1.1 was GPL

There is also some experimental JIT but it’s not yet stable and is currently very slow (the JIT-generator, not the generated code).

I’m looking for people interested in using Neko as a target for compilers and research papers, contact me directly or on Neko mailing list if interested.

haXe Type System Extensions for Async Calls

posted on 2006-01-06

Here’s an interesting problem to tackle. haXe is a common language targeting different platforms (Flash, JavaScript and Neko) with different API. It would be nice to have some kind of integrated RPC so you can directly and transparently call JS code from Flash, or Neko (server side) from JS (client side).

For example, let’s say you want to access the server filesystem from Flash. First you’ll have to connect to the server and negociate the rights to do this, but that’s not the difficult part. Ideally, you would just use the File class directly and if you have the rights all calls would be forwarded to the server and responses sent back transparently.

class File {
    public function new( name : String ) {
         ...
    }
    public function write( data : String ) : Void {
         ...
    }
    public function close() : Void {
         ...
    }
    public function size() : Int {
         ...
    }
}

The problem here is that File is implemented using haXe Neko APIs which are not available from Flash. Actually, when compiling haXe for Flash program, you don’t even have access to the File class since it’s in a separate directory.

In that situation, you would endup normally to write a FileProxy class that would implement the methods in terms of crossplatform RPC calls and use this proxy class instead of the original one. The idea behind Type System Extension is to have theses proxy automatically generated and correctly handled by the type system.

In order to have this work, we need to introduce a special parametrized type named Proxy. For instance you will first instanciate a Proxy<File> and then manipulate it as a File. A Proxy have the same methods as its parameter but all calls are forwarded using RPC.

To be able to pass the typechecker, the Proxy need to look for more directories so we can find the File class. Then, instead of fully typing and compiling it, we only need to parse it and extract the types of public methods and fields. This will build the type of the Proxy. As a consequence you cannot rely on haXe type inference of the proxied class, all the File methods must be correctly typed (in the case they’re not, maybe just removing them from the Proxy type is a sound possibility).

But this is not enough. RPC sometimes assume that the calls are synchronous, so every call is “waiting” for the answer of the server before returning. This is not possible with Flash and JS, since all network calls are asynchronous. The Proxy need then to make all methods return values asynchronous, for example transforming the method :

public function size() : Int

into the following :

public function size() : Async<Int>

with Async having the following definition for example :

class Async<T> {
      public var onValue : T -> Void; // event callback
}

This way instead of using transparantly the File class, you’ll do somethink like :

      var f : Proxy<File> = ... // instanciate
      f.write("hello world!"); // transparent call
      var size : Async<Int> = f.size(); // async call
      size.onValue = function(s : Int) {
           trace("file size is " + s);
      }

(the local variable types here are optional, they’re not needed since there is local type inference but it helps clarifying the example).

There is several problems left :

  • Classes platform : right now there is no “platform” information for a class, so if the File had one method that would return another File, the Proxy can’t know it should turn it into Async<Proxy<File>> instead of just Async<File> which is not correct.
  • Object persistance : since you create an object on the “other side”, you need to keep references of them. This can be done by keeping a connection open. All objects are free when the connection close (with maybe some chances to reconnect). But when an object is no longer needed, the client need to explicitly free it (by adding free() to the Proxy type). This is a limitation of the platforms that don’t offer finalizers for objects.

After being able to specify “platform” information for a class, it should be possible to write an Asynchronous RPC engine directly in haXe. This would enable inter-platform Proxy-transparent communications, with only some extensions to the type system.

Neko Boot

posted on 2006-01-05

I took a few hours to work at modifying the NekoVM boot so it’s possible to create standalone binaries.

The idea is first to link the bytecode (using the -link neko compiler parameter). This will look for all $loader.loadmodule("constant string",$loader) calls inside the bytecode, and will replace it by the corresponding inlined module. It works very nice with NekoML and MotionTypes, and should work for all other languages that are using such statements for module resolution, which is the normal way of doing things.

neko -link myapp.n MyModule

The output is one big linked bytecode file that shouldn’t need any other .n bytecode library anymore. Only in the case there is unpredictable loadmodule calls, then theses are left as-it and the loaded bytecode files are needed.

Then a simple Neko program called nekoboot will concat the NekoVM boot with the linked bytecode, and modify at some place a special value that is storing the filesize so it can access to it. At runtime, since the filesize is defined, the boot will load the module directly by reading itself instead of taking an argument. You only need std.ndll that is required by the boot loader, and of course the VM library (neko.dll or libneko.so) and other ndll C libraries that you are using.

nekoboot myapp.n

This will build a myapp.exe (or without extension on Linux/OSX) standalone binary.