OSCON Slides

posted on 2005-07-12

I finished my OSCON 2005 Slides for MTASC talk so I’m publishing them here. If you have some comments or found some typos, don’t hesitate to send me an email.

Download MTASC OSCON 2005 Slides

Neko Modules Loader

posted on 2005-07-10

I’m done with implementing Modules Loader in NekoVM. The idea is to be able to dynamicly load a module and filter/secure the access it makes to other modules or C primitives. A module loader then have two functions (in pseudo-typing) :

  function loadmodule( modname : String, loader : Loader ) : Module
  function loadprim( primname : String, primargs : int ) : Function

The loadmodule method is called when the loaded module request the access to another module. In the default loader this function will load the module, initialize it, and returns its export table, which is an object whose fields have been set using : $exports.myvar = 1234 for example. Since the export table is an object, the user can write his own loadmodule function that will allow only some modules to be loaded (raising an expection or returning null either) or that can modify or wrap the export table in order to secure, add traces, filter, …. the module accesses. This is very powerful feature that opens a lot of different usages.

The loadprim method resolve and load a C primitive using its name and number of arguments as parameters, and returns a function. The default loader split the name of the primitive in two : “std_fileopen” with 3 args will load the DLL “std” and get the “std_fileopen__3″ C function, then call it in order to retreive the pointer to the real C functions (all theses registrations are done with some Neko C API Macros). Again, the user can define his own loader using another behavior for loadprim : he can filter the C primitives that are accessible, or return a wrapper function that does additionnal checks (for exemple in our “fileopen” sample, the user can ensure that the module will not try to access files outside a given directory).

This loading process is very useful for both debugging and security. It also adds dynamic loading of modules which is a nice feature. The default loader currently cache the modules after they’re loaded, so modules cannot be unloaded right now, but that might be a possibility later, since all module functions are keeping a reference to the module, we can be sure that as long as we keep an accessible function, the module will not be garbage-collected.

Also, there is no check for recursive modules loading. If a module A loads B which in turn loads A, then the A export table returned to B will be filled only with values that have been set at the time A loaded B. Since we return a copy of the export table (because we don’t want it to be modified by another module), at the end of both A and B initialization, B might have a broken A table. This is not a very predictable behavior, but if the user know what he’s doing with recursive modules, it should be ok. One workaround is to set an “api” object field in the export table and then fill this object, so mutations done in A will occur also in B (export tables are not copied recursively).

A module loader example :

    var myloader = $new(null);
    myloader.loadmodule = function(mname,curloader) {
        $print("loading module ",mname);
        if( mname == "system" ) $throw("Cannot access system");
        return $loader.loadmodule(mname,curloader);
    }
    myloader.loadprim = function(pname,pargs)  {
        $print("loading primitive "+pname+" with "+pargs+" arguments");
        if( $ssub(pname,0,7) == "system_" )
            $throw("Cannot access system");
        return $loader.loadprim(pname,pargs);
    }
    myloader.loadmodule("secure.n",myloader);

Please note that since we’re passing recursively “myloader” to “loadmodule” second parameter, the module “secure.n” as well as all modules it’s loading will use our custom loader.