| View previous topic :: View next topic |
| Author |
Message |
anmabagima
Joined: 01 Oct 2009
Posts: 96
|
 Posted: Thu Dec 17, 2009 2:42 am Post subject: out of core data storage |
 |
|
Hi there,
I'm not sure whether this post has the right subject or not. However, I've seen articles on the web discussing "out of core data storage" which some PC OS/applications are capable of. This mean that in the case the application need to process huge ammount of data it is not loaded into the RAM (core) but left in the file. The file access in this scenario is like accessing memory using a pointer to the data once the file was opened.
As the PSP has no harddisk but only SD card I question myself whether the PSP is capable of supporting this approach or not. And now I would like to ask you.... As on a common PC the access speed would be very much slow due to limmitations of the hardware but the SD card access on PSP should work like accessing RAM from a performence point of view, shouldn't it ?
The sceIO operations just gives file handles.You need to use them with READ methods to get data from the file into a buffer. I would like to have something like a pointer to the raw data of the file and beeing able to directly access this data I'm interested in, as I would do using them within a buffer in RAM. Is that possible ?
Any thoughts and Ideas are welcome and much appriciated.
Regards
AnMaBaGiMa |
|
| Back to top |
|
 |
a_noob
Joined: 17 Sep 2006
Posts: 97
Location: _start: jr 0xDEADBEEF
|
| Posted: Thu Dec 17, 2009 4:34 am Post subject: |
|
|
I am not to sure about via the sceIo, I think it would need to use some reversing because it just returns a handle no actual data, but if you where to use stdio maybe, unless it was just hacked to store the handle and then accessed, but it it is properly used/coded then here is the FILE datatype wich has the info you need.
| Code: | typedef struct {
int level; /* fill/empty level of buffer */
unsigned flags; /* File status flags */
char fd; /* File descriptor */
unsigned char hold; /* Ungetc char if no buffer */
int bsize; /* Buffer size */
unsigned char *buffer; /* Data transfer buffer */
unsigned char *curp; /* Current active pointer */
unsigned istemp; /* Temporary file indicator */
short token; /* Used for validity checking */
} FILE; |
_________________
| Code: | .øOº'ºOø.
'ºOo.oOº' |
|
|
| Back to top |
|
|
Raphael
Joined: 17 Jan 2006
Posts: 646
Location: Germany
|
| Posted: Thu Dec 17, 2009 6:28 am Post subject: |
|
|
What you talk about is not possible on the PSP, because it does not have an MMU, especially not one that is capable of memmapping files.
So there is no direct way of pointer->file mapping, unless you write your own software layer, which pipes an smart pointer abstraction through sceIo function calls.
Unless you use C++ and operator overloading and templates, it wouldn't change anything about the feel of having to use functions to write data to files and even then, you still won't be able to write directly to constant addresses as with memory: *((int*)0xabcdef) = somevalue;
_________________
<Don't push the river, it flows.>
http://wordpress.fx-world.org - my devblog
http://wiki.fx-world.org - VFPU documentation wiki
Alexander Berl |
|
| Back to top |
|
|
anmabagima
Joined: 01 Oct 2009
Posts: 96
|
| Posted: Thu Dec 17, 2009 6:59 am Post subject: |
|
|
Hi,
thanks. I will try this. But "my" FILE structure looks pretty much different:
| Code: |
typedef __FILE FILE;
struct __sFILE {
unsigned char *_p; /* current position in (some) buffer */
int _r; /* read space left for getc() */
int _w; /* write space left for putc() */
short _flags; /* flags, below; this FILE is free if 0 */
short _file; /* fileno, if Unix descriptor, else -1 */
struct __sbuf _bf; /* the buffer (at least 1 byte, if !NULL) */
int _lbfsize; /* 0 or -_bf._size, for inline putc */
|
and some more stuff ... The _bf is defined as
| Code: |
struct __sbuf {
unsigned char *_base;
int _size;
}; |
I would assume the pointer to _base will do the job as well. I'll give it a try. |
|
| Back to top |
|
|
m0skit0
Joined: 02 Jun 2009
Posts: 226
|
| Posted: Thu Dec 17, 2009 7:12 am Post subject: |
|
|
| anmabagima wrote: | | but the SD card access on PSP should work like accessing RAM from a performence point of view, shouldn't it ? |
Abolutley NOT. DDR2 SDRAM offers 3200 MB/s, when best SD card you could find can only perform at 45 MB/s. Also SD Cards (as flash memory) have a latency measured in miliseconds, while main memory latency is usually measured in nanoseconds. So no, no way you can use SD as virtual RAM and expect close performance. RAM wins high hand.
| anmabagima wrote: | | I would like to have something like a pointer to the raw data of the file and beeing able to directly access this data I'm interested in, as I would do using them within a buffer in RAM. Is that possible ? |
It is possible, but as I stated before, you would have a big drop in performance, not to speak about the software layer you'd have to set up as commented by last answer.
_________________
| The Incredible Bill Gates wrote: | | The obvious mathematical breakthrough would be development of an easy way to factor large prime numbers. |
|
|
| Back to top |
|
|
anmabagima
Joined: 01 Oct 2009
Posts: 96
|
| Posted: Thu Dec 17, 2009 5:39 pm Post subject: |
|
|
Hi,
thanks to all for the valuable statements.
Regards
AnMaBaGiMa |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
