| View previous topic :: View next topic |
| Author |
Message |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Sun Feb 24, 2008 4:49 am Post subject: Introducing SPE Execution Management System (spexms) |
|
|
So people have been wondering why the activity has been low in spu-medialib.
The reason is that i have been working on a framework for easy execution of multiple application across the spe's
The project is listed here
http://sourceforge.net/projects/spexms
And i am still looking for members
Please note this is still in development and that im working on extending the buildt in functions however the code below is not very much compared to what one achives.
The example code to make spexms load two spe binaries and execute them in random order is currently something like this
| Code: |
#include <spexms/spexms.h>
#include <spexms/spexms_types.h>
#include <stdlib.h>
#include <stdio.h>
int main (int nArg, char* cArg[]) {
int number_of_tasks=0;
exms_t *xms=init_spes();
spu_program_info_t *spi[2];
spi[0]=register_program("spe_hello_world");
spi[1]=register_program("spe_goodbye_world");
//printf("First command in the binary is %x",&spi->
if (nArg > 1)
number_of_tasks=atoi(cArg[1]);
///lets make spexms first task ever;
spexms_task_header_t xms_th[2];
xms_th[0].program_id=spi[0]->id;
xms_th[0].program_location=spi[0]->location;
xms_th[0].program_size=spi[0]->size;
xms_th[0].no_of_dma=0;
xms_th[1].program_id=spi[1]->id;
xms_th[1].program_location=spi[1]->location;
xms_th[1].program_size=spi[1]->size;
xms_th[1].no_of_dma=0;
int i;
int id;
for (i=0;i < number_of_tasks;i++)
{
id=rand()%2;
register_task(xms,spi[id]->id,0,&xms_th[id]);
}
sleep(1);
}
|
The code needed in the binary is something like this
| Code: |
#include <spexms/spexms_buildtin.h>
int main (spexms_buildtin_t *spexms,spexms_pointers_t *pointers)
{
int value=55;
//char *mychar="Hello World this is my int";
spexms->printInt("Goodbye world",value);
return value;
} |
_________________ Don't do it alone. |
|
| Back to top |
|
 |
ouasse
Joined: 30 Jul 2007 Posts: 90 Location: Paris, France
|
Posted: Sun Feb 24, 2008 6:27 am Post subject: |
|
|
I've noticed using libspe that it was already possible to run more that 6 spe threads. Some kind of trivial (non-preemptive ?) multithreading across spe's is then performed.
What does your library provide that is not already possible ? |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Sun Feb 24, 2008 9:18 am Post subject: |
|
|
it moves the code as data and double buffer the code on the spe so techically you can use 65536 programs without doing a context switch. and without any wait*. provided this and that ofc
*one will wait for the few things in the kernel to execute.
The other thing it provides is a fifo scheduler of your tasks so you fire and forget on the ppu and the spu's execute your task as soon as possible.
In addition to that it provides and abstraction layer making it easy to develop things for the spu.
As you can se you dont need to worry about pthread etc and libspe and mbox commands you simply schedule a job and it gets done voila.
Now when it comes to knowing a job is done etc im currently working on that but my priority thus far has been getting fire and forget working. _________________ Don't do it alone. |
|
| Back to top |
|
 |
ps2devman
Joined: 09 Oct 2006 Posts: 265
|
Posted: Sun Feb 24, 2008 7:21 pm Post subject: |
|
|
| Great! Thanks a lot! |
|
| Back to top |
|
 |
ouasse
Joined: 30 Jul 2007 Posts: 90 Location: Paris, France
|
Posted: Mon Feb 25, 2008 6:08 am Post subject: |
|
|
this sounds really interesting. Have you planned support for task dependencies, which should schedule all spawned tasks in order to respect some data dependence between them ?
It should be great to be able to specify that task X's input is the output of tasks Y and Z, for instance. Thus, tasks Y and Z must be executed before task X. |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Mon Feb 25, 2008 8:22 am Post subject: |
|
|
There is a flag for dependency buildt into it as well as a register to keep track of executed tasks in current and "previous" fifo.
But i have not yet buildt that part in ..
This is very complicated and any help will be of interest. _________________ Don't do it alone. |
|
| Back to top |
|
 |
cchalpha
Joined: 24 Feb 2008 Posts: 9
|
Posted: Tue Feb 26, 2008 6:59 pm Post subject: |
|
|
unsolo, it's a great idea to make this project.
I believe in homebrews game programming, the spe scheduling should be setup manually for best efficiency. But under linux, it is another case, linux is a general proposed OS, it is necessary to have such a scheduling mechanism to make the spe code executing as easy as possible.
I just co the project, and go through some of them, not finish yet. Here is my questions,
SPE scheduling is known by the process which loaded spexms. But if spexms lib are loaded by many processes, is there any conflict among each other?
I don't know how much I can help, but at least I can add some comments to the code while I read through them. |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Wed Feb 27, 2008 4:07 am Post subject: |
|
|
Spexms is compatible with the current kernel spu task scheduler so provided tasks are small it should automatically swap.
This would be good even for otheros applications..
so using spu-top + two programs using spexms one should be able to se it even interleaving the two.
Havin said that im trying to work out some good regulation algo (pd regulator in its own pthread) that makes sure sufficient spe's are running to complete your queued tasks as soon as possible _________________ Don't do it alone. |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Mon Mar 03, 2008 9:41 am Post subject: |
|
|
small video made of spexms + draw_rect (v2) in action from spu.medialib
http://www.customweb.no/unsolo/DSC00055.3GP
Its doing about 1.1GPixels/second read modify and write of 200x200 squares mode is fire and forget so overlapping may cause conflict _________________ Don't do it alone. |
|
| Back to top |
|
 |
Chambers1
Joined: 25 Jan 2008 Posts: 7 Location: US
|
Posted: Mon Mar 03, 2008 12:16 pm Post subject: |
|
|
Wow! I don't know what this means but it's getting closer to a fully working x264 media player for ps3 so I'm happy :) Great job unsolo!! _________________ You mean people actually do that? |
|
| Back to top |
|
 |
ouasse
Joined: 30 Jul 2007 Posts: 90 Location: Paris, France
|
Posted: Mon Mar 03, 2008 6:35 pm Post subject: |
|
|
unsolo, this looks very good ! there should be no overlapping problem for drawing code where spes share pre-defined independent parts of the screen memory for drawing stuff.
The question I asked before was mainly for handling 3d code, where spe's would first be used for computing triangles, and then for drawing them.
The dependencies should simply be handled with a global barrier : all computing tasks must be finished before the drawing tasks. this should be easier to implement than dependencies between specific tasks. |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Mon Mar 03, 2008 11:46 pm Post subject: |
|
|
Im going to look at a global Spexms barrier task that waits untill all previous tasks are completed. One also has the dma sync alternatives.. but anyways this is how its currently set up _________________ Don't do it alone. |
|
| Back to top |
|
 |
cchalpha
Joined: 24 Feb 2008 Posts: 9
|
Posted: Tue Mar 04, 2008 1:01 pm Post subject: |
|
|
| According to your example code in spexms, the call to spu is asynchronous currently, but synchronous calls should be also provided as another choice of programming model/pattern. I think the complete I/O API model should be a good reference, in simple, it calls your handle funtion with the parameter you provided when things are done. |
|
| Back to top |
|
 |
unsolo
Joined: 16 Apr 2007 Posts: 155 Location: OSLO Norway
|
Posted: Tue Mar 04, 2008 5:50 pm Post subject: |
|
|
Spexms is open source GPL2 + exception anyone that feels that they can contribute feel free do join the project. _________________ Don't do it alone. |
|
| Back to top |
|
 |
|