Parrot Speed and Size

posted on 2006-02-27

I’m following the progress of Parrot, the Perl6 VM. I’m interested in the subject since NekoVM is trying to tackle the same problem : one single VM for several languages. Parrot have been developed for years now and include megabytes of source code. Still, it hasn’t reached 1.0 yet, and there is no production compiler targeting it. Neko is version 1.2 and is targeted by several compilers : MotionTypes, NekoML and haXe. How does Parrot compare to NekoVM ?

I downloaded last Win32 binary release and tried to compare a small benchmark, the fibonnacci(30) calculus. That’s one of my favorites benchmarks because in a few instructions you mesure comparisons, integer calculus and function calls, which are three important things for overall application speed.

Results are the following : Parrot using -j (JIT core) took 10 seconds to compute fib(30). NekoVM (which does not have JIT yet) took only 2 seconds. That’s 5 time faster, for such a simple thing. I don’t know the differences between the semantics of the systems, but since the Parrot Fib function was strictly typed, while Neko fib is dynamicly typed, I would expect a lot more speed on the Parrot side.

At the same time, the libparrot.dll is more than 2MB while neko.dll is 64Ko (353Ko to get a fair comparison and include standard library, GC and Neko compiler which is separate).

That’s good news for Neko, isn’t it ?

Developing Open Source Tools

posted on 2006-02-05

I’ve been asking several times how I’m making a living from writing tools such as MTASC, Neko or haXe. The answer is simple : I don’t.

My main job at Motion-Twin is to write games and develop gaming websites. That is taking me around 70% of my daily work time. The other 30% are used to develop tools that we’re using in-house. Motion-Types and Neko are two examples of projects supported by my company. I first started working on theses on my own time and when they got stable and we started using it, most of the improvements and bugfixes could be done on my work time. Just to clarify things, I’m co-founder of MotionTwin so it’s not like I’m working for free.

MTASC

MTASC is a specific sample. It was supported by my company and is maintained on both my free and working time. And that might surprise a few people, but this is a tool that we are not using at all. We are using Motion-Types, which is MTASC elder brother. Motion-Types is more powerful, but a bit tricky to master. So how did I came to develop MTASC ?

My company is doing pretty well. We are now eleven people. We could setup a business with no cash because we’re massively using Open Source tools. Our Linux servers are running PHP Apache and MySQL. Without theses great tools available we couldn’t have afforded licenses for ASP IIS and Oracle (among other proprietary solutions). We are greatly thankful to the open source community for theses great tools.

After MotionTypes was done and working, we felt like giving back to the OS community, so we made an ActionScript2 version of MotionTypes that became MTASC. That was back in October 2004. MTASC is now widely use by Flash developers around the world, while my company actually never used it.

Developing MTASC was done quite quickly. Around 4-5 days after the project was started I could make a first Beta version. Then it was a lot of small fixes that I was doing when I was felling like, but none of theses took more than a few hours. Actually, most of the bugfixes in MTASC were done in less than 30 minutes, sometimes even more quickly. The code is pretty small and expressive, so it’s quickly easy to figure out the place that need to be fixed. Over the time, MTASC improved and is now very stable. I can’t exactly say how much time it took to develop entirely but it’s more a matter of weeks than months.

Neko

Back in October 2004, I started working on a website called Dinoparc. I was before working mostly in Flash so it was a long time since I didn’t wrote a website, and I never wrote such a big one before. I was looking at the different tools such as PHP, Python or Ruby, but was not very satisfied with thel. At this time I had a small Virtual Machine and a compiler for a scripting language called MotionScript. It was nothing fancy and was looking a lot like ActionScript 1.

I tried to put the MotionScript VM inside a “mod” for Apache and it was done in a few days. Developing some bindings for a MySQL database was done almost as quickly, and then I wrote a code generator from MotionTypes to MotionScript. This was pretty easy again because MotionTypes already had an ActionScript1 code generator so after a few tweaks it could generate MotionScript code as well (see this post on the subject).

In a few weeks of hardcore development, I was then able to write a website in MotionTypes, generate the corresponding MotionScript code, compile it and run it under Apache and access to the MySQL databases. It was quite something to see that everything went so quick and I was pretty satisfied with the result.

This work inspired me to rewrite the specification of MotionScript in order to get an intermediate language that could be targeted by MotionTypes and other languages as well. I knew I was not done with writing compilers yet and I wanted to have clear an easy language to target when I wanted to do so. I named this language Neko (which means “cat” in japanese). That was around January 2005 but I was busy with my daily work and another project so I only finished the first specifications draft in April.

The first release of Neko was in August. It was already working pretty well but the Neko compiler was still written in OCaml. I wanted to bootstrap the language so I worked hard on NekoML, then rewrote the Neko and NekoML compilers in NekoML as explained in this post. Neko 1.1 was released in November 2005 and 1.2 in January 2006. Most of the work I’m doing on Neko is on behalf of my company, with some things such as JIT (that should be in 1.3 btw) that are not priority goals so I’m having fun at home developing theses.

Neko is quite a big amout of work. Difficult again to say exactly how much since one part was made when it was still named MotionTypes but that’s several months with a few weeks of hardcore programming (what I’m calling hardcore is wakeup-code-eat-code-eat-code-sleep days). But so far, this is the project I’ve been the most enjoying working on. Maybe because there is low-level (VM and libs) and high-level (compilers) involved. I’m pretty sure that Neko will continue to improve and will be a great platform for compiler writers and language designers. I heard someone is working on a Ruby-to-Neko compiler, which is great news for VM adoption.

haXe

When ActionScript3 was announced, I decided that I will work on another language instead of making an AS3 version of MTASC which was not making sense (and I was actually proved right). haXe is coming from years of experience with Motion-Types while developing Flash and ServerSide content with one single language.

In its current state it’s already greatly usable, and it has some nice features that even MotionTypes doesn’t have (like debug infos). It also have a cleaner type system implementation. I can see that in 6 months or so, we will switch our development tools from MotionTypes to haXe. For us it’s very important to use a language that comes with a community of users, which MotionTypes is lacking. haXe also is cleaning some problems with MotionTypes such as putting a common standard library and separating Flash API from Neko ones. It will also add JavaScript support which will be great to develop AJAX-style websites.

So far haXe represent only a few days of work. I’m working on it on my free time, which is a two or three hours in the evening and one day in the week-end. Since Neko is getting more mature, most of my developments are now focused on haXe. And as soon as we start using it in-house, it will evolve even more quickly !

Conclusion

This is so far the history of the tools I’ve been writing. I’m a passionate developer and I really love to work on such project, introduce new idea and improve existing workflows. The most difficult part is not to develop the tools themselves, but to convince the users to try them and work with them. Even MTASC which is compiling an already “widely” used language such as ActionScript2 is not yet adopted by a large majority of Flash developers (while once adopted, nobody want to work again without it).

For haXe of course I’m expecting a lot more resistance, and even sometimes hostility. I’m getting some rare mails sounding like why to make a new language ? you stupid. It’s not an easy thing to create a new language. People can sometimes feel threaten by a new technology, feeling that if they can’t keep-up with learning all theses things, they might lose their job one day.

OTOH, there is some early enthousiasts that are supporting me with every new thing I’m coming with. I would like to say to them : thank you !

haXe Beta 1

posted on 2006-02-05

haXe Beta 1 has just been released on haxe.org. Check about it !