Logged in as Guest Fri, Mar. 12th, 4:27 AM.
|
|
|
|
|
Well, I had hoped to write more on NVFS after I had a lot more to say, however, I have received several emails (including one from Ben) that have a common misunderstanding about the NVFS implementations on the Tungsten T5 and the Treo 650, and I figured it might be helpful to try and clear it up.
By far, the issue that is most misunderstood is the DBCache that the devices utilize, and how it is flushed once it is full. Before going too quickly into it, I should layout a little backstory on the NVFS implementation (especially regarding what is known as the DBCache.) Let's start with the physical layout of the T5. According to the docs, the memory footprint on the T5 looks like this:
RAM: 32MB total
16MB – OS image copied from flash at boot
10MB – storage heap cache
6MB – dynamic heap
FLASH: 256MB total
160MB – VFS flash drive
55MB – flash-based storage heap
41MB – compressed OS image, hidden storage space,
and blocks for remapping bad sectors
As you can see, there is a 10 MB storage heap cache. According to the SDK, while an application is running, any active/open databases are allocated out of this 10 MB. This allocation occurs in almost exactly the same way that it is allocated on pre-NVFS devices. (This 10 MB is what is commonly referred to as the DBCache, and I will henceforth refer to it as such.) Also according to the SDK, there are certain times when that DBCache is "flushed" to the VFS Flash drive for permanent storage:
1. Application exit
2. Power cycle
3. Database closing
4. Explicity by the user with the DmSyncDatabase function
The next time that a database is opened, it is copied from the VFS drive to the DBCache for quicker usage. To better understand NVFS, you have to understand that the VFS memory is what is known as NAND memory. One of the implications of NAND memory is that it is quick to read, but slow to write. , and if you do have to write to NAND, it's better to do it in big chunks. (For a really good explanation of NAND RAM, see Ed Nisley's article in the March 2005 issue of Dr. Dobb's Journal). Because of this limitation, palmOne's NVFS implementation is optimized to minimize the number of writes that occur to the NAND memory. This of course, is where the DBCache comes into play: to minimize the number of writes to NAND. To date, this information has been understood by all of the people that have written me about my previous T5 NVFS article. The part that has not been understood is the runtime usage of the DBCache. As pointed out earlier, it is only 10 MB in size. For normal things, that is a fairly acceptable size. However, in my sample loop, I was allocating approximately 40 MB worth of data. If the OS only flushed the DBCache during those 4 stated conditions above, no program could ever allocate a DB at runtime that was more than 10 MB without having to either close the database or explicitly call DmSyncDatabase. For a 256 MB device, that would be quite a crippling limitation! Fortunately, there is one more condition when the devices flush the DBCache: when the DBCache itself becomes full. It is slightly buried in a very short paragraph in the SDK, but the docs do mention that when the DBCache becomes full, the OS will "intelligently purge" it. If you take the code from my earlier article and run it on the T5, you can actually see this in action. Once the allocations hit around 8 MB, the T5 starts crawling. This slow, chunking slowdown is occuring as the OS flushes the DBCache, and it is paying a penalty for the slow write speed of the NAND RAM. I can't say exactly what algoritm the T5 and the 650 use to do the purging, but I have determined that it takes a fairly minimalist approach. As my loop keeps allocating, the DBCache fills and begins flushing, and it seems to keep having to flush until it is finished. Because of this, I suspect that it is only flushing enough memory for the requested allocation. Perhaps it should flush more, perhaps not; it probably depends on what you are doing. My biggest complaint is that there isn't any exposed functionality to perform this flushing from the application's perspective. If you are trying to squeeze every last bit of performance out of your application and you're running on an NVFS device, you're at the mercy of the OS in those regards. The only function that is given to you as a developer is DmSyncDatabase, but that just writes it to the VFS RAM; it doesn't empty ANYTHING out of the DBCache. Ugh. That's about it for my explanation of the DBCache. I hope that it helps someone. It certainly cost me a lot of blood sweat and mostly tears to figure it out. As always, please let me know if you think I've gotten anything wrong. -Jon |
|
| Submitted by bosshogg on Thursday the 24th of February 2005, at 09:07 pm | |
|
|
|
|
Anybody who has ever tried to create a copy protection scheme for the PalmOS ends up with the fundamental problem of trying to uniquely identify a device. Unfortunately, there is no single, unified way to do this, and what the developer is left doing is putting together a mish-mash of different methods for different devices. Here, I present a couple different ways to do that, based on device type.
Devices from palmOne: Many palmOne devices have what is known as flash ROM, which contains a serial number that is "baked" into the hardware. Fortunately, palmOne provides an easy interface for accessing that ROM serial number:
UInt8* ptr;
UInt16 size;
Err err = SysGetROMToken(0,sysROMTokenSnum, &ptr,&size);
if (!err && size && ptr && *ptr != 0xFF)
MemMove(dest,ptr,size);
Handspring Devices: Handspring devices (including the Treo) also utilize Flash ROM, but Handspring provided a separate API than palmOne did for accessing the ROM serial number. Now that palmOne owns the Treo line, it is possible that this method will be deprecated in favor of SysGetROMToken. Until that time, however, you can use the following: char dest[64]; UInt16 size = sizeof(dest); HsGetVersionString(hsVerStrSerialNo, dest, &size);Bluetooth Devices: Some devices ship with a Bluetooth radio built in. For those devices, one option is to use the Mac address of the Bluetooth radio. You can use the Bluetooth API provided by PalmSource to do that:
BtLibDeviceAddressType address;
Err err = BtLibGetGeneralPreference(btLibRef,
btLibPref_LocalDeviceAddress,
&address,
sizeof(address));
if (!err)
MemMove(&address.address, dest, sizeof(address));
Other Devices:
Unfortunately, the above three methods do not work on all devices. In fact, on certain devices, like the TungstenE, there is nothing that can be used that is tied specifically to the hardware. In those scenarios, it is possible to fall back to using the Hotsync user name. While this is not a perfect solution, it does allow the user to more easily move between multiple devices. To obtain the Hotsync user name, you can use the following code: #include SD/MMC Card Unique Identifier: Sometimes, an application needs to provide an identifier that is generalized across multiple devices, but less breakable than the Hotsync user name. In those cases, the unique identifier can be tied to an expansion card which then functions as a sort of "dongle" that is required to run the application. The downside of this is that the device must actually have an expansion card slot, and that the SD/MMC card must actually be inserted to run the application. (These applications tend to fall into vertical markets, although they definitely aren't limited to that. ) Fortunately, the Expansion Manager provides a fairly painless way to get the unique identifier of the card:
ExpCardInfoType info;
Err err = ExpCardInfo(slotRef, &info);
if (!err)
MemMove(dest, info.deviceUniqueIDStr, sizeof(info.deviceUniqueIDStr));
There are other methods to retrieve a unique identifier on the PalmOS, but the above listed methods are the 'big hitters'. Now please understand that a unique identifier does not equal copy protection, but generally it is the first step down that road. As an aside, if you are contemplating implementing a copy protection scheme, I highly recommend you read Aaron Ardiri's white paper on the subject, available here. And as always, if you find any of the above code helpful, I'd love to hear about it. -Jon |
|
| Submitted by bosshogg on Tuesday the 22nd of February 2005, at 11:41 am | |
|
|
|
|
Over the past few months, I have seen the requests for help retrieving the phone number on a Treo increase dramatically. In the vain hope that maybe this entry will get correctly indexed by Google (and therefore reduce the number of requests for help on the forums) I release this code to assist you however it may. As always, a quick note to let me know it helped you is appreciated but in no way required. Happy coding!
-Jon ///////////////////////////////////////////////////////
Char* number = NULL;
PhnAddressList list;
PhnAddressHandle address;
PhnLibOpen(phnLibRef);
if (IsCDMA())
{
Err err = PhnLibGetOwnNumbers(phnLibRef, &list);
if (!err)
{
err = PhnLibAPGetNth(phnLibRef, list, 1, &address);
if (!err && address)
{
number = PhnLibAPGetField(phnLibRef,
address,
phnAddrFldPhone);
MemHandleFree(address);
}
}
}
else
{
Err err = PhnLibGetOwnNumbers(phnLibRef, &list);
if (!err)
{
err = PhnLibGetNth(phnLibRef, list, 1, &address);
if (!err && address)
{
number = PhnLibGetField(phnLibRef,
address,
phnAddrFldPhone);
MemHandleFree(address);
}
}
}
if (number)
{
StrCopy(destNumber,number);
MemPtrFree (number);
}
|
|
| Submitted by bosshogg on Friday the 18th of February 2005, at 09:31 am | |
|
|
|
|
Have you ever had one of those moments where you get so excited about finding something out that you totally ignore all of your past experiences? Well, that happened to me when I wrote about my experience with the T5, and now I shall eat crow. Sort of. After posting about the T5 and a loop that I wrote that caused it to crash, I received an email from a developer who pointed out to me that the T5 is simply running out of memory, and the call to DmNewRecord is simply failing. I took a look, and sure enough, he was right. Ugh. I hate being wrong... So, the short story is that I was wrong and the T5 doesn't crash in that loop if you check DmNewRecord for failure. However, this brings me to a question that is harder to answer: Why is a device with 256 MB of RAM running out of space when I try to allocate approximately 40 MB? I don't have a good answer to this yet, but I figured I should at least rectify my posting about the T5 crashing. Hopefully I'll have more time soon to finish the investigation. More to come... |
|
| Submitted by bosshogg on Tuesday the 15th of February 2005, at 12:06 pm | |
|
|
|
|
Recently, I have heard rumblings that the new Tungsten T5 is a bit on the unstable side, but until recently, I dismissed most of them. That is, until I was forced to come face to face with the harsh reality that is NVFS. Now, just to be clear, I don't work for palmOne or PalmSource so I really have no idea other than intuitive guesses why it crashes, but if you want to see the reported instability in action, throw this code into a small PRC and run it on a T5:
//NOTE: Obviously, you need to make sure that the database
//exists, but for the sake of simplicity, I have left error
//checking and the standard 'boilerplate' DB code out.
LocalID id = DmFindDatabase(0, "____TEMP_DB____");
DmOpenRef db = DmOpenDatabase(0, id, dmModeReadWrite);
for (UInt16 i = 0; i < 40000; i++)
{
UInt16 recordId = 0xFFFF;
MemHandle handle = DmNewRecord(db, &recordId, 1024);
DmReleaseRecord(db, recordId, true);
//This part is optional, but it lets you at least see
//that something is happening...
if (! (i % 1000))
{
char msg[64];
StrPrintF(msg, "Records created: %u", i);
ErrAlertCustom(0, msg, "", "");
}
}
You should find that after what seems like an inexorable amount of time, it will crash with either a generic "Fatal Alert" or a "DataMgr.c, Line:11321, Index out of range" fatal alert. Quite frankly, I was shocked by how easy it was to make this happen. It seems to me like a pretty simple/obvious unit test when you implement a new file system to create a large number of generic files (or in this case records) in a loop and make sure everything works acceptably. Obviously, no one stopped to do that here. I have a lot more to write about my experiences with NVFS, but I'll save that for later. |
|
| Submitted by bosshogg on Tuesday the 08th of February 2005, at 08:51 pm | |
