Setting HD block size to 64KB

[mcbrems]

Hey,

Sage recommends formatting the video storage drive to 64KB block size for best performance. It isn't so obvious how to do this in Mac OSX.

I've scoured the Apple support site, and could only find reference to specifically setting allocation block size in relation to Xsan drives using Xsan Admin. The only other reference to formatting a drive to optimize for video playback was in relation to Apple's iMovie, stating that formatting the drive to HFS Extended would yield the best result. It's not clear how any of this relates to journalled formatting either.

This is another avenue I'm considering to eliminate the occasional video hiccup. Does anybody have experience with this?

Thanks,
MB

[karljayne]

I read somewhere on the internet that the default block size on Leopard is 4Kb. As far as I can tell there is no way to change that when partitioning a device from OS X, other than with RAID tools. Of course, that doesn't mean you can't partition the drive with another OS, perhaps there's a utility that allows block sizing for Linux?

[iconic]

Originally Posted by mcbrems

This is another avenue I'm considering to eliminate the occasional video hiccup. Does anybody have experience with this?

I wouldn't bother, the Unified Buffer Cache should smooth out mass storage bottlenecks. Is the hiccup while recording, playing back, or both (in which case it has absolutely nothing to do with mass storage)?

[mcbrems]

Only when playing back... usually 1080HD material. After the hiccup (dropped frames) I am able to skip back to just before the occurance and verify the integrity of the stream - in other words, I immediately play it again and no frames are dropped in the exact same place, confirming that all of the frames were recorded, and that the issue is with playback.

[sandor]

I'm going to look into it a little further, but part of the 64 K block issue is with NTFS itself. I use ext3 formatted hard drives (default block size) as recording drives, and have no issues with fragmentation. I am assuming that HFS+ with journaling will have just as few problems with fragmentation as ext3 - NTFS just flat out stinks in many ways.




***edit***

forgot to add, HFS+, as well as utilizing delayed allocations (in order to find contiguous blocks), has built-in, on-the-fly, auto-defragmentation for files under 20 MB in size.


So, basically, the way HFS+ was created and how it deals with allocating blocks for data goes a tremendously long way in preventing the fragmentation of data in the first place.

[iconic]

Originally Posted by mcbrems

Only when playing back... usually 1080HD material. After the hiccup (dropped frames) I am able to skip back to just before the occurance and verify the integrity of the stream - in other words, I immediately play it again and no frames are dropped in the exact same place, confirming that all of the frames were recorded, and that the issue is with playback.

I'm guessing you don't get the same issue with VLC? That doesn't sound like it's mass-storage related. You don't have Apple Mail running at the same time do you?

[mcbrems]

Originally Posted by sandor

forgot to add, HFS has built-in, on-the-fly, auto-defragmentation for files under 20 MB in size.

How does HFS+ auto-defrag with files under 20MB help with 8GB mpg files? As I understand, the larger the block allocation, the less blocks of data, and therefore less "fragments." With huge video files, aren't we concerned with fragmentation at sizes higher than 20MB?

Thx

[mcbrems]

Originally Posted by iconic

I'm guessing you don't get the same issue with VLC? That doesn't sound like it's mass-storage related. You don't have Apple Mail running at the same time do you?

VLC plays the same files fine. In fact, SageTV V6.4.0 stutters on archived files that 6.3.5 plays just fine, so I know it's not storage-related, and I know the issue is 6.4.0-centric when it happens with nothing else open but Sage.

[sandor]

Originally Posted by mcbrems

How does HFS+ auto-defrag with files under 20MB help with 8GB mpg files? As I understand, the larger the block allocation, the less blocks of data, and therefore less "fragments." With huge video files, aren't we concerned with fragmentation at sizes higher than 20MB?

Thx

its just one of the many underlying benefits of a file system such as HFS+ or ext3 - these file systems work in ways that keep fragmentation from happening in the first place, unlike NTFS.

I haven't done any testing wrt to HFS+, but, as i said, my Sage system records everything to an ext3 NAS, and i have done extensive testing with it, and block size makes nary a difference in performance.

[CadErik]

Originally Posted by sandor

its just one of the many underlying benefits of a file system such as HFS+ or ext3 - these file systems work in ways that keep fragmentation from happening in the first place, unlike NTFS.

This is totally wrong, there are even defrag tools for ext3.
My guess is that the file usage on Linux is different enough and doesn't cause inherently so much fragmentation - which is why this is less an issue -.
There are no secret miracles when you need to read and write multiple streams of undefined length of data at the same time, fragmentation happens.