Implementing Simplified Plugin System on AmigaOS4
by Álmos Rajnai on utilitybase
This article explains how to set up a simple plugin system using the advantages of the ELF format.
Extending applications from the outside via "plugged-in" code snippets is a common technique nowadays. Usually it is not really hard to implement such expansion capabilities in the applications, but building an easy-to-maintain and flexible plugin system is not as simple as it seems. Especially when you don't want to deal with the low level of the connection mechanics between the application and the plugin executable.
In AmigaOS4 the executable format has been changed to ELF (Executable and Linkable Format) from the previous hunk executable. Due to this change many possibilities have been added to the OS, which was could not be exploited before. Just to name a few: dynamic linking via symbol resolving on load, extensive debugging information with the help of the DWARF standard and tagging information and special attributes and sections inside the executable. The ELF format is a lot more flexible than the hunk format was.
Among the many great features there is the possibility of proper handling of symbols inside the ELF file, at compile-time we can specify which symbols should be present in a file. Later during run-time, as a late binding, the symbols can be located by an executable. A perfect way to find entry point and data inside the plugin file.
If you need more complex handling of shared objects, then have a look on a2solib package on OS4Depot.net, which implements symbol handling via libdl function.
Benefits of Our Brew
The usual way of implementing such "late binding" on AmigaOS 3.x was typically a virtual library system, with its library base and a jumptable. Jumptables are a set of addresses relative to the library base. By pre-defining this tables an application can know where in a file to jump for calling a function or similar. Why wouldn't it be a flexible choice?
A jump table is always a rigid form, you can extend it, but you can not remove obsolete addresses from it, or at least you have to take care of the proper versioning, which is not possible via the old library system, jumping to an empty position inside the jumptable will crash the caller application,
The jumptable is supposed to hold only function start addresses, but no data pointers (although many coders have just ignored this),
The mechanism of library opening trough Exec and the management functions are totally useless in this case, but yet needed to be implemented. There is no need for open, close and expunge code, yet needs to be implemented.
Compiling a library requires either tricky setup or support from the compiler directly.
These virtual libraries are not unloaded by the application, but the system, when an expunge request happens (means: they hog resources, mainly memory). The expunge function is called only on a low memory situation by the system itself.
Implementing the plugin system with the help of symbol resolving gives great opportunities:
No need for jumptable, symbols are looked up on run-time and you get data or instruction pointers belonging to it.
One can extend the features or remove any of them without the need of keeping its empty space, something goes wrong with the version checking, the worst thing which could happen is that the caller simply can't resolve the missing symbols, but won't crash,
You can either resolve function start addresses or static data start addresses, no need to hack.
The management functions only need to be implemented in the application, inside the plugins there is usually no need for special handling of the system.
Compiling of the plugins need no special tricks or direct support from the compiler, they are just plain ELF files.
You can unload the plugin at anytime because it is loaded by the main application and not Exec.
I have to admit, that there is one issue which might affect your choice over these two possibilities: if you want some kind of backward compatibility or support the OS3.x platform, then you cannot choose the ELF executables because of obvious reasons.
For using local variables inside the plugin I suggest a handler structure, which could be either obtained via plugin initializing function or could be allocated by the application itself, if there was no need for different structures for each plugin. Thus you can pass a state of the processing for each plugin independently, even when there were more than one task is involved in the plugin code execution.
After compiling a source into ELF executable the compiler will leave in all the symbol information. You can get rid of them with strip command, which is a wise idea, because of the file size and thus removing the helping hands for reverse-engineering your code. The strip command removes everything except the bare minimum (the AmigaOS4 symbol, just to ease the identification of the ELF file). Luckily we can instruct strip to keep symbols that are simply the names of the functions and data. All you need to do is specifying the -k or --keep-symbol command line parameter with your symbols.
Compiling of the plugin sources need a few extra arguments, but nothing special. For example:gcc -nostdlib -o myplug myplug.c
Thus there will be no startup code and usual C libraries linked to your plugin, which would be useless anyway. Stripping the startup code is not really necessary, but it is useless. The system won't call it on load, we could call it, but the init code of the startup must not be executed.
Next thing is the stripping:strip -o myplug.plugin -K KeptSymbol1 -K KeptSymbol2 myplug
This command will produce a file, called "myplug.plugin", which is ready to use.
Now, let's check what needs to be done in the main application for opening the plugin and resolving the symbols. At first we need to load the plugin, there is nothing special involved, just simply load it via the DOS/LoadSeg() function:seglist=IDOS->LoadSeg("myplug.plugin");
Now we have the segment list for the loaded code. This must be kept until unloading the plugin later on via DOS/UnLoadSeg(). (There is nothing else needs to be done for removing it from memory.)
Resolving symbols in memory is done by elf.library, but it cannot handle segment lists, so we need to ask DOS for an ELF handler first:IDOS->GetSegListInfoTags(seglist,GSLI_ElfHandle,&elfhandle,TAG_DONE);
The handler loaded into the elfhandle, now we can reload it from disk:filehandle = IElf->OpenElfTags(OET_ElfHandle,elfhandle,TAG_DONE);
Now everything is ready for the symbol query, which needs a special structure, called Elf32_SymbolQuery, filling it up with proper values we will be able to get the symbol addresses in the original loaded file, so we are in the home straight:query.Flags = ELF32_SQ_BYNAME;
query.Name = "KeptSymbol1";
IElf->SymbolQuery(filehandle, 1, &query);
The result will be in field: query.Value. As a final step, reopened ELF file needs to be closed:IElf->CloseElfTags(filehandle, CET_CloseInput, TRUE, TAG_DONE);
And we are ready to go...
As you can see the resolving might imply several file accesses, so running all the symbol resolving in a batch could be a lot faster, no need to reopen the ELF file everytime, it is enough to set up the ELF/SymbolQuery function for each symbol then copy the results from the query structure.
A Bare Example
You can find a very minimal application, which demonstrates the simplified plugin system, described above. All sources, Makefile, object codes and executables are included.
Download it from here: plugin_example.lha.
You can recompile it by simply running the make command in the same drawer.
There is not much to say about this small example, all it implements is a basic "calculator", which takes two numbers and the name of a an operator plugin as the arguments. The plugin performs an "operation" on the numbers and the result is printed. You can give it a try like:
plugmain 36 6 opplus
Then you will get the answer: 42. (What else?
However if you try multiplication (opmulti), then you will get an error message reporting: could not open plugin. Just to demonstrate how passing of static data from the plugins work with an "incompatible" version number of V2.
The included makefile is very clever, you can simply add your plugin source names after the rest and it will build them. (If you were not aware of the possibilities of the makefiles, then I strongly suggest to take a few hours and check the GNU make docs!)
What Needs Some Attention
There are a few things which might break your code, let's quickly run trough them:
Don't forget to strip the unused symbols, plugins will work with all the symbols, but why waste space for them?
Don't forget to keep the symbols which you want to resolve later, though.
Be careful with the common C functions in the plugins, usually need some startup code, and your plugin donesn't have one. You will get linking errors if you used some kind of C function, which need to be resolved by you. (You can provide functions from the main application via passing a collection of pointers to the plugin. Those will work.)
Make sure you apply __attribute__((used)) special GCC attribute on every function and data you want to get later, the optimizer in GCC might think that some of them are not used and "optimize away" to reduce size of the file.
Adding a function with the name of _start might be helpful, this will be the entry function when the plugin executable is started from command line or Workbench. Adding a print-out of text explaining the user that this file is not intended to run as stand-alone might be helpful for your users. Be careful! You cannot use normal C functions, such as printf(), but as you can see in the sources of the plugins among the _start function parameters you can find ExecBase, so you can easily get the IExec from here: ExecBase->MainInterface.
To Nicolas Mendoza for revising the text and for his patience, and to Colin Wenzel for pointing out that the ELF files must be closed at the end.
If you have questions, have found any errors in the article, or just simply want to tell me your opinion, don't hesitate in writing a mail:
Álmos Rajnai, racs at vipmail dot hu.