forums.ps2dev.org Forum Index forums.ps2dev.org
Homebrew PS2, PSP & PS3 Development Discussions
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Introducing SPE Execution Management System (spexms)

 
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PS3 Linux Development
View previous topic :: View next topic  
Author Message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Sun Feb 24, 2008 4:49 am    Post subject: Introducing SPE Execution Management System (spexms) Reply with quote

So people have been wondering why the activity has been low in spu-medialib.

The reason is that i have been working on a framework for easy execution of multiple application across the spe's

The project is listed here
http://sourceforge.net/projects/spexms

And i am still looking for members

Please note this is still in development and that im working on extending the buildt in functions however the code below is not very much compared to what one achives.

The example code to make spexms load two spe binaries and execute them in random order is currently something like this

Code:
 

#include <spexms/spexms.h>
#include <spexms/spexms_types.h>

#include <stdlib.h>

#include <stdio.h>


int main (int nArg, char* cArg[]) {

   int number_of_tasks=0;
   exms_t *xms=init_spes();
   
   spu_program_info_t *spi[2];
   
   spi[0]=register_program("spe_hello_world");   
   spi[1]=register_program("spe_goodbye_world");
   //printf("First command in the binary is %x",&spi->

   if (nArg > 1)
      number_of_tasks=atoi(cArg[1]);

   ///lets make spexms first task ever;

   spexms_task_header_t xms_th[2];

   xms_th[0].program_id=spi[0]->id;
   xms_th[0].program_location=spi[0]->location;
   xms_th[0].program_size=spi[0]->size;
   xms_th[0].no_of_dma=0;

   xms_th[1].program_id=spi[1]->id;
   xms_th[1].program_location=spi[1]->location;
   xms_th[1].program_size=spi[1]->size;
   xms_th[1].no_of_dma=0;
      
   int i;
   int id;
   for (i=0;i < number_of_tasks;i++)
   {
      id=rand()%2;
      register_task(xms,spi[id]->id,0,&xms_th[id]);
   }
   sleep(1);

}



The code needed in the binary is something like this

Code:

#include <spexms/spexms_buildtin.h>

int main (spexms_buildtin_t *spexms,spexms_pointers_t *pointers)
{
   int value=55;
   //char *mychar="Hello World this is my int";
   spexms->printInt("Goodbye world",value);
   return value;

}

_________________
Don't do it alone.
Back to top
View user's profile Send private message
ouasse



Joined: 30 Jul 2007
Posts: 90
Location: Paris, France

PostPosted: Sun Feb 24, 2008 6:27 am    Post subject: Reply with quote

I've noticed using libspe that it was already possible to run more that 6 spe threads. Some kind of trivial (non-preemptive ?) multithreading across spe's is then performed.

What does your library provide that is not already possible ?
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Sun Feb 24, 2008 9:18 am    Post subject: Reply with quote

it moves the code as data and double buffer the code on the spe so techically you can use 65536 programs without doing a context switch. and without any wait*. provided this and that ofc

*one will wait for the few things in the kernel to execute.

The other thing it provides is a fifo scheduler of your tasks so you fire and forget on the ppu and the spu's execute your task as soon as possible.

In addition to that it provides and abstraction layer making it easy to develop things for the spu.
As you can se you dont need to worry about pthread etc and libspe and mbox commands you simply schedule a job and it gets done voila.

Now when it comes to knowing a job is done etc im currently working on that but my priority thus far has been getting fire and forget working.
_________________
Don't do it alone.
Back to top
View user's profile Send private message
ps2devman



Joined: 09 Oct 2006
Posts: 265

PostPosted: Sun Feb 24, 2008 7:21 pm    Post subject: Reply with quote

Great! Thanks a lot!
Back to top
View user's profile Send private message
ouasse



Joined: 30 Jul 2007
Posts: 90
Location: Paris, France

PostPosted: Mon Feb 25, 2008 6:08 am    Post subject: Reply with quote

this sounds really interesting. Have you planned support for task dependencies, which should schedule all spawned tasks in order to respect some data dependence between them ?

It should be great to be able to specify that task X's input is the output of tasks Y and Z, for instance. Thus, tasks Y and Z must be executed before task X.
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Mon Feb 25, 2008 8:22 am    Post subject: Reply with quote

There is a flag for dependency buildt into it as well as a register to keep track of executed tasks in current and "previous" fifo.

But i have not yet buildt that part in ..

This is very complicated and any help will be of interest.
_________________
Don't do it alone.
Back to top
View user's profile Send private message
cchalpha



Joined: 24 Feb 2008
Posts: 9

PostPosted: Tue Feb 26, 2008 6:59 pm    Post subject: Reply with quote

unsolo, it's a great idea to make this project.

I believe in homebrews game programming, the spe scheduling should be setup manually for best efficiency. But under linux, it is another case, linux is a general proposed OS, it is necessary to have such a scheduling mechanism to make the spe code executing as easy as possible.

I just co the project, and go through some of them, not finish yet. Here is my questions,
SPE scheduling is known by the process which loaded spexms. But if spexms lib are loaded by many processes, is there any conflict among each other?

I don't know how much I can help, but at least I can add some comments to the code while I read through them.
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Wed Feb 27, 2008 4:07 am    Post subject: Reply with quote

Spexms is compatible with the current kernel spu task scheduler so provided tasks are small it should automatically swap.

This would be good even for otheros applications..

so using spu-top + two programs using spexms one should be able to se it even interleaving the two.

Havin said that im trying to work out some good regulation algo (pd regulator in its own pthread) that makes sure sufficient spe's are running to complete your queued tasks as soon as possible
_________________
Don't do it alone.
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Mon Mar 03, 2008 9:41 am    Post subject: Reply with quote

small video made of spexms + draw_rect (v2) in action from spu.medialib

http://www.customweb.no/unsolo/DSC00055.3GP


Its doing about 1.1GPixels/second read modify and write of 200x200 squares mode is fire and forget so overlapping may cause conflict
_________________
Don't do it alone.
Back to top
View user's profile Send private message
Chambers1



Joined: 25 Jan 2008
Posts: 7
Location: US

PostPosted: Mon Mar 03, 2008 12:16 pm    Post subject: Reply with quote

Wow! I don't know what this means but it's getting closer to a fully working x264 media player for ps3 so I'm happy :) Great job unsolo!!
_________________
You mean people actually do that?
Back to top
View user's profile Send private message
ouasse



Joined: 30 Jul 2007
Posts: 90
Location: Paris, France

PostPosted: Mon Mar 03, 2008 6:35 pm    Post subject: Reply with quote

unsolo, this looks very good ! there should be no overlapping problem for drawing code where spes share pre-defined independent parts of the screen memory for drawing stuff.

The question I asked before was mainly for handling 3d code, where spe's would first be used for computing triangles, and then for drawing them.

The dependencies should simply be handled with a global barrier : all computing tasks must be finished before the drawing tasks. this should be easier to implement than dependencies between specific tasks.
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Mon Mar 03, 2008 11:46 pm    Post subject: Reply with quote

Im going to look at a global Spexms barrier task that waits untill all previous tasks are completed. One also has the dma sync alternatives.. but anyways this is how its currently set up
_________________
Don't do it alone.
Back to top
View user's profile Send private message
cchalpha



Joined: 24 Feb 2008
Posts: 9

PostPosted: Tue Mar 04, 2008 1:01 pm    Post subject: Reply with quote

According to your example code in spexms, the call to spu is asynchronous currently, but synchronous calls should be also provided as another choice of programming model/pattern. I think the complete I/O API model should be a good reference, in simple, it calls your handle funtion with the parameter you provided when things are done.
Back to top
View user's profile Send private message
unsolo



Joined: 16 Apr 2007
Posts: 155
Location: OSLO Norway

PostPosted: Tue Mar 04, 2008 5:50 pm    Post subject: Reply with quote

Spexms is open source GPL2 + exception anyone that feels that they can contribute feel free do join the project.
_________________
Don't do it alone.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.ps2dev.org Forum Index -> PS3 Linux Development All times are GMT + 10 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group