| View previous topic :: View next topic |
| Author |
Message |
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
 Posted: Sun Dec 04, 2005 11:39 am Post subject: Core Project: External libraries |
 |
|
As was discussed in the 'pre-implemented FFT routine' thread (how the heck do you "pre-implement something? :P (rhetorical question)), dynamically loadable libraries (as in compiled, C libraries, not lua libraries) would be real nice. Adding functionality without bloating the LuaPlayer codebase. We've already heard about the advantages (divx decoding, fft, webkit? you name it.). How'd one go about doing it though? there's | Code: | | loadlib (libname, funcname) |
, but that's just for individual c functions. hm, but that could be wrapped with a lua file...
It does seem entirely possible. I'm throwing this thread in here anyway, to gather thoughts in a specific place.
(Hm, I'm taking a course in compiler technologies and specifically runtime linking right now, perhaps I can convince my prof that psp hacking is really school work ;) )
(please reserve the thread for technical discussion only, and not "yeah that'd be cool do that!" or "with that, we could do x and y and neatIdea!" (unless it's a *very* neat idea :P))
Last edited by nevyn on Thu Dec 22, 2005 6:31 am; edited 1 time in total |
|
| Back to top |
|
 |
Shine
Joined: 03 Dec 2004
Posts: 728
Location: Germany
|
| Posted: Mon Dec 05, 2005 3:26 am Post subject: Re: Lua core dev: dynloadable C libraries |
|
|
| nevyn wrote: | there's | Code: | | loadlib (libname, funcname) |
, but that's just for individual c functions. hm, but that could be wrapped with a lua file...
|
This function could be standard entry point like our other functions for luagraphics, luawlan etc. Since it is a Lua function, the function is called with a lua_State pointer and it can register the rest. I think the best would be to load all those libraries like this, perhaps with the boot.lua at startup for the standard libs. Then Lua Player itself is a module loader, only and all the functionality is provided by the loaded modules (and can be replaces independently or not loaded, if not needed).
The module concept itself should use PSP PRX files (something like shared libraries). I think they have the same address space and the link between the other modules could be the lua_State pointer or some predefined functions (e.g. getLuaPlayerVersion() etc.).
The advanced feature would be to implement something like in the Eclipse IDE: Every module has its own version and has a dependency list, which other modules with which version are needed. This could be used e.g. for the lua3d lib, because it needs a bit of the luagraphics lib. But calling a function of another library from within a library could be interesting :-) |
|
| Back to top |
|
|
JoshDB
Joined: 05 Oct 2005
Posts: 87
|
| Posted: Mon Dec 05, 2005 9:46 am Post subject: |
|
|
| I'm pretty sure I posted a topic a while ago asking what the "Library" folder was for. I'm not a C coder, but I've made the Hello World program... Is there a way to drag/drop LUA libraries from LUA.org into there? |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Wed Jan 11, 2006 11:26 am Post subject: |
|
|
I've just been thinking about porting a library to use in LuaPlayer, and I'd say having a loadable module would be the best method.
As has already been mentioned, I agree that the best way to do this would be to use PRX files. I've been looking at the examples that show the C API layer at:
http://lua-users.org/wiki/SampleCode
All the examples have a function which will bind the additional library methods, classes, etc into Lua. If LuaPlayer called a specific export to perform this function then the library could basically install itself. So call a function like:
int registerLuaLibrary( lua_State * L );
That function then does everything it needs to bind the loaded library into Lua. The only question I have is how does the library access the required Lua C functions. Do these need to be exported from the elf, or does Lua itself need to be put into another library? Can anyone solve that one?
David. aka Oobles.
www.livemedia.com.au/Blog |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Wed Jan 11, 2006 3:37 pm Post subject: |
|
|
After doing some more research and looking at the pspsdk examples of loading a PRX I've got some additional thoughts. There are two main functions that need to be called:
sceKernalLoadModule
sceKernelStartModule
It would be easier to just pass in lua_State into the sceKernelStartModule than to use special export. It would nice to be able to identify Lua specific PRX's, however, I suspect that a naming convention is probably easiest for Lua modules.
I still need to do some more looking at if lua functions need to be exported. I suspect that if the lua library doesn't keep any static data then the required functions could be duplicated in the PRX and just modify the lua_State passed in?
David. aka Oobles.
www.livemedia.com.au/Blog |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Wed Jan 11, 2006 11:02 pm Post subject: |
|
|
Another quick update. I compiled a very simple PRX which linked in lua and lualib. It created a PRX that was 182kb. Seems like a high price to pay for statically linking in Lua.
Code looked something like this...
| Code: |
PSP_MODULE_INFO("SIMPLELRX", 0x1000, 1, 1);
#define WELCOME_MESSAGE "Simple LRX loaded\n"
static int lua_doSomething(lua_State *L)
{
if (lua_gettop(L) != 0) return luaL_error(L, "wrong number of arguments"
);
lua_pushstring(L, WELCOME_MESSAGE );
return 0;
}
static const luaL_reg Simple_functions[] = {
{"doSomething", lua_doSomething},
{0,0}
};
int main(int argc, char **argv)
{
lua_State * L;
if ( argc != 1 )
{
return 1;
}
L = (lua_State *) argv[0];
luaL_openlib(L, "Simple", Simple_functions, 0);
return 0;
}
|
|
|
| Back to top |
|
|
LuMo
Joined: 21 Aug 2005
Posts: 410
Location: Austria
|
| Posted: Thu Jan 12, 2006 12:22 am Post subject: |
|
|
we were talking about external "plugins" for a while...
if you plan to integrate prx files...
the post with GL OR GU is solved, cause you can have both in seperate PRX files then....
@the size of the prx files...
i think noone actually cares about the size of the data
main prob might be the memory usage (RAM not MS)
i guess noone uses 32mb sticks (i use it for dev. only)
but one question came to my mind...
will it be possible to eg. extend functions of a finished class?
you have eg. image:blit in your code
so is it possible to implement another fkt to image?
greets
lumo
_________________
"Good artists copy, great artists steal."
Pablo Picasso
go2lumo.com |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Thu Jan 12, 2006 3:04 pm Post subject: |
|
|
I've completed some more tests. This time I added a new System.loadModule function to see if I could load the PRX and install a function.
| Code: |
static int lua_loadModule(lua_State *L)
{
u32 loadResult;
u32 startResult;
int status;
char *argv[2];
const char *path = luaL_checkstring(L, 1);
if (!path) return luaL_error(L, "Argument error: System.loadModule(name)
takes a module name as string as argument.");
loadResult = sceKernelLoadModule(path,0, NULL);
if (loadResult & 0x80000000)
{
return luaL_error(L, "Argument error: Failed to load module");
}
argv[0] = (char *) L;
argv[1] = 0;
startResult = sceKernelStartModule( loadResult, 1, argv, &status, NULL );
if ( loadResult != startResult )
{
return luaL_error(L, "Module error: Failed to load module");
}
return 0;
}
|
This all works, however, when the lua_openLib function is called inside the PRX main, the PSP crashes requiring reboot. This either means that you can't pass pointers via main argv, or calling lua functions with statically linked in libraries is not going to work. Either way, I think it will require that lua itself is pushed into a PRX.
If anyone can see errors in the code, or suggest another method that doesn't involve putting lua into a PRX.. let me know.
Edit: Just found on this page to confirm that Lua will need to be placed into a PRX. http://lua-users.org/wiki/CreatingBinaryExtensionModules
"If those functions exist within host application ( like lua.exe), we cannot link our extension module. We must move the Lua API into a DLL, so that the host application and the extension DLL can share the same copy of the Lua API code."
David. aka Oobles.
www.livemedia.com.au/Blog |
|
| Back to top |
|
|
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
| Posted: Fri Jan 13, 2006 4:18 am Post subject: |
|
|
Nice work on the research there. I concur on the filesize not being a major problem. However, if the lualib is put externally in a prx, it's not a problem anyway. (This is not a difficult task, is it? I haven't done much dynamic library work)
Might have some term gap time next week. If so, I'll check it out. |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Fri Jan 13, 2006 10:50 am Post subject: |
|
|
I've created a lua.prx which contain just the lua libraries. I've exported all the required functions and got LuaPlayer to link with these instead of the static libraries. Everything is compiled, however, I'm having trouble loading the lua.prx from within LuaPlayer.
If you like I can commit these changes so you can have a look. I'll keep trying to get this to work though. Just don't want us duplicating work. |
|
| Back to top |
|
|
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
| Posted: Fri Jan 13, 2006 11:06 am Post subject: |
|
|
| Oobles wrote: | I've created a lua.prx which contain just the lua libraries. I've exported all the required functions and got LuaPlayer to link with these instead of the static libraries. Everything is compiled, however, I'm having trouble loading the lua.prx from within LuaPlayer.
If you like I can commit these changes so you can have a look. I'll keep trying to get this to work though. Just don't want us duplicating work. |
You're very welcome to commit, I'm pretty curious :) You go ahead with your code, I'll fail another course if I get into hack mode now anyways... |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Sat Jan 14, 2006 4:43 pm Post subject: |
|
|
Ok, I've committed the lua.prx in src/luaprx with its own makefile. I haven't modified any of the actual luaPlayer code. I've added the following to main.cpp in user_main
| Code: |
// change directory.
getcwd(path, 256);
chdir(path);
// load the Lua PRX
strcat(path, "/lua.prx" );
int retVal = sceKernelLoadModule( path, 0, NULL );
if (retVal < 0)
{
debugOutput("Error: failed sceKernelLoadModule load lua.prx\n",
31);
sceKernelSleepThread();
}
debugOutput( "LoadModule\n", 12 );
int fd;
retVal = sceKernelStartModule( retVal, 0, NULL, &fd, NULL );
if ( retVal < 0 )
{
debugOutput("Error: failed sceKernelStartModule lua.prx\n", 44 )
;
}
|
You will also need at the top of main.cpp the following to ensure that enough memory is available to load the prx.
PSP_HEAP_SIZE_KB(10000);
You also need to modify the makefile to include LuaLib.o in the objects. To create LuaLib.c you run in the luaprx directory:
psp-build-exports --build-stubs exports.exp
You also remove the lua libraries from the Makefile.
After all that is done, it still doesn't work. :) I'm not exactly sure why though. Anyone got any ideas?
[edit: more info] As far as I can tell it now loads the lua.prx correctly, but after that, the screen goes black and nothing happens. I'm using a very basic lua script to check if it works. I'm wondering if anything else needs to be done to link in the PRX exports with the imports of LuaPlayer?
David. aka Oobles.
www.livemedia.com.au/Blog |
|
| Back to top |
|
|
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
| Posted: Sat Jan 14, 2006 7:44 pm Post subject: |
|
|
| Oobles wrote: |
[edit: more info] As far as I can tell it now loads the lua.prx correctly, but after that, the screen goes black and nothing happens. I'm using a very basic lua script to check if it works. I'm wondering if anything else needs to be done to link in the PRX exports with the imports of LuaPlayer?
David. aka Oobles.
www.livemedia.com.au/Blog |
Great work :)
Can you printf from c (not lua) or is the black screen a complete crash/infinite loop or something? |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Sun Jan 15, 2006 12:54 am Post subject: |
|
|
I've had another go at it, but still not working. I've run it under PSPLINK and now know that there is an "Address load/inst fetch" exception happenning in LuaPlayer. I haven't had time to investigate it further than that yet.. will leave it for another day. :)
David. aka Oobles.
www.livemedia.com.au/Blog |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Thu Mar 16, 2006 11:05 am Post subject: |
|
|
Three months later and I'm still working on this problem. I finally loaded a module yesterday and got things working. However, there might be some issues with keeping full compatibility with the last version of LuaPlayer.
The current solution is to put luaplayer into a user mode PRX file and use a small bootstrap code in kernel mode to load it. The luaplayer exports all the Lua functions. A module is loaded and in its main function it registers the module with luaplayer.
I still need to check if there are any kernel mode functions that can no longer be called? Can anyone tell me what parts of luaplayer requrie kernel mode?
Thanks,
David. aka Oobles.
http://www.livemedia.com.au/Blog |
|
| Back to top |
|
|
Shine
Joined: 03 Dec 2004
Posts: 728
Location: Germany
|
| Posted: Thu Mar 16, 2006 3:28 pm Post subject: |
|
|
| Oobles wrote: | Can anyone tell me what parts of luaplayer requrie kernel mode?
|
The USB support, SIO and stdout/stderr handler requires kernel mode functions, but Lua Player itself works without these functions. |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Mon Mar 20, 2006 9:55 pm Post subject: |
|
|
Well, I did it. I committed version v0.18alpha to Subversion. It is a bit of a re-arrangement of LuaPlayer, so it needs some testing. The big change is that luaplayer is now in a user mode PRX which is loaded from the kernel mode bootstrap. The net effect of all this is that luaplayer can now load other PRX modules.
I've added two example modules. The first named simple is mostly just a template for how to extend LuaPlayer. The second named sound is extracting the sound module out from the main LuaPlayer. I haven't actually removed sound from the main luaplayer, however, it could be done using this method. The only problem is that there would be issues with backward compatibility and require developers to update their code to load the sound or whatever other module.
The changelog has a few other bits and pieces that were changed a long the way. The main one that comes to mind is the debug message output.
Please someone test this and let me know how they go compiling and running code. There may still be some issues.
I would like to modify the module loading a little and make it more inline with the Lua loadlib function. I hope to get time to do this soon.
Regards,
David. aka Oobles.
http://www.livemedia.com.au/Blog |
|
| Back to top |
|
|
LuMo
Joined: 21 Aug 2005
Posts: 410
Location: Austria
|
| Posted: Mon Mar 20, 2006 11:22 pm Post subject: |
|
|
skip v0.17?
:D
great work, again ;)
greets
lumo
_________________
"Good artists copy, great artists steal."
Pablo Picasso
go2lumo.com |
|
| Back to top |
|
|
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
| Posted: Mon Mar 20, 2006 11:36 pm Post subject: |
|
|
| This kills post-1.5 version PSP support due to the kernel requirement, I s'pose? Personally I don't mind (since i have 1.5 myself :P), I love to see this progress. |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Tue Mar 21, 2006 8:04 am Post subject: |
|
|
LuMo, I skipped v0.17 because it was already in the ChangeLog.
Nevyn, yes, module loading does require kernel support. However, having said that, it should be easier to build LuaPlayer for post-1.5 because all the kernel code has been split out into the bootstrap.
I did some more testing before and found a major bug. I didn't put the exports file into the luaplayer makefile. This stops module loading from working. Oops. I will commit the fixes in a few hours during my lunch break. I don't have access to commit code right now.
Regards,
David. aka Oobles. |
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Tue Mar 21, 2006 12:23 pm Post subject: |
|
|
As promised, I've committed the fixes. At some stage I'll add an example for module loading. If you want to give it a go you just need.
| Code: |
System.loadModule( "simple" )
txt = Simple.doSomething()
-- then print text somewhere on screen to receive the secret message :)
|
The current behaviour for load module is that it will load modules from the application boot path and add .lrx on the end of the name.
David. aka Oobles. |
|
| Back to top |
|
|
daurnimator
Joined: 11 Dec 2005
Posts: 38
Location: melbourne, australia
|
| Posted: Sun Mar 26, 2006 2:12 pm Post subject: |
|
|
love it :D
coding with it now; but that bug is extremly annoying, just a hack to get around it for now would help... |
|
| Back to top |
|
|
daurnimator
Joined: 11 Dec 2005
Posts: 38
Location: melbourne, australia
|
|
| Back to top |
|
|
Oobles
Site Admin
Joined: 17 Jan 2004
Posts: 362
Location: Melbourne, Australia
|
| Posted: Tue Apr 04, 2006 9:10 pm Post subject: |
|
|
I've just committed an update to the module loading to Subversion. I've replaced the System.loadModule with the more traditional loadlib function. The way you write modules and load them has changed a little. Check the modules in subversion to see how. It now works like this in lua.
| Code: |
simple = loadlib( "simple", "init" )
simple()
|
The module name is passed in as the first parameter and the function to call to initialise the module as the second. It does not automatically call the init routine but instead returns it. You must call it yourself.
The PRX module name must be the same as the first parameter.
Let me know if you have any problems.
David. aka Oobles. |
|
| Back to top |
|
|
DiabloTerrorGF
Joined: 15 Jul 2005
Posts: 64
|
| Posted: Thu Apr 06, 2006 6:53 am Post subject: |
|
|
| Layman's terms please? |
|
| Back to top |
|
|
romero126
Joined: 24 Dec 2005
Posts: 200
|
| Posted: Wed Apr 12, 2006 10:10 am Post subject: |
|
|
You can make your own elf files. and run them in lua player ontop of your LUA script (So it can be used for display purposes).
You can load modules by useing the command
| Code: |
simple = loadlib( "simple", "init" )
simple()
|
|
|
| Back to top |
|
|
be2003
Joined: 20 Apr 2006
Posts: 144
|
| Posted: Thu Apr 20, 2006 3:04 pm Post subject: |
|
|
Would it make more sense to not add .lrx to the end?
Why not let the user(me) define it? (for more flexebility)
| Code: |
test = loadlib("test.lrx","init") -- or test.prx?
test()
|
Also, can lua load prx, dll, so?
| Oobles wrote: |
The current behaviour for load module is that it will load modules from the application boot path and add .lrx on the end of the name.
David. aka Oobles. |
_________________
- be2003
blog |
|
| Back to top |
|
|
be2003
Joined: 20 Apr 2006
Posts: 144
|
| Posted: Sat Apr 22, 2006 3:06 am Post subject: |
|
|
an lrx is just a renamed prx, right
_________________
- be2003
blog |
|
| Back to top |
|
|
nevyn
Joined: 31 Jul 2005
Posts: 136
Location: Sweden
|
| Posted: Sat Apr 22, 2006 3:18 am Post subject: |
|
|
| be2003 wrote: | Would it make more sense to not add .lrx to the end?
Why not let the user(me) define it? (for more flexebility)
|
That'd decrease portability; a Lua Player app would then require different versions for PSP / Windows / Linux / Mac versions (as windows uses .dll, mac .dylink, linux .so) |
|
| Back to top |
|
|
be2003
Joined: 20 Apr 2006
Posts: 144
|
| Posted: Sat Apr 22, 2006 5:15 am Post subject: |
|
|
but cant a prx be a library too, in one makefile it executes cp ./effect.prx ,/effect.lrx. so in that sense an lrx = prx
never mind, i am an idiot...
_________________
- be2003
blog |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
