64K cluster size?

[kandieman101]

Can anyone quickly summarize why the 64K is the advantage, and why? Seems like very small clusters and I don't know if that makes it faster/slower/what...

For resulting in better formance playback, is this only when watching "live tv" or recordings, or both?

[stanger89]

Normally clusters are 4k, 64k is actually quite a bit larger than normal. Larger clusters help reduce fragmentation, and reduce/eliminate playback issues (whether liveTV or not). Highly recommended.

[dvd_maniac]

64K is actually large for a cluster size.when dealing with larger files, like videos, it is better to use larger cluster sizes for fragmentation purposes. It is much easier and faster for the hard drive to find a smaller number of larger clusters then a larger number of smaller ones. The latter might have problems with stuttering when searching for the next cluster.
I have 4x400GB drives in a raid-5 with 64K clusters size and have never needed to defragment it and have never had any stuttering issues. Before I had the Raid-5 card installed I was recording directly to a raid-o set but with 8K clusters and I did have some problems with stuttering when the drive was fragmented.

[Blkhrtd]

Thinking along the line of "More is better", why not go with a larger number, say: 512k cluster size? I understand that it'd be a little wasteful, but I was just wondering.

-Shadrach

[zz5]

The reason is that 64K is the largest you can do with the NTFS file system. If we could do larger, we would. TIVO uses 1MB clusters.

[briands]

I have my raid array formated with 64k clusters. I have also have my music and Photo shares on this drive. Should I store these on this array? What about those little .xml and .edl files. Will those cause problems other than wasted space.

[stevech]

TiVo and 1MB clusters - the MCE gang at Micro$oft couldn't influence a change to NTFS? Or maybe that was in the infamous new file system that didn't make it to Vista.

BTW: Was there to be an MCE-like version of Vista?

[Ponchera]

Originally Posted by stevech

BTW: Was there to be an MCE-like version of Vista?

Yep, the new MCE is - Windows Vista ULTIMATE.

Tried it, not too thrilled with the MCE portion of it. a change here and there UI wise, haven't noticed anything really beneficial to it. I'll stick to Sage which seems to be running fine on Vista ultimate aside from the timezone issue (fixed, see post about updating JRE)

[ke6guj]

Originally Posted by Ponchera

Yep, the new MCE is - Windows Vista ULTIMATE.

Actually MCE is included in Vista Home Premium and Vista Ultimate. The main difference is that Ultimate has all the Vista Business features as well as the Home features.

The main thing I don't like is that if you want MCE and Remote Desktop, you have to go to $$$ Ultimate, its not in the cheaper Home Premium. RDP is become a consumer feature for many people, not just a business feature.

[dvd_maniac]

Originally Posted by briands

I have my raid array formated with 64k clusters. I have also have my music and Photo shares on this drive. Should I store these on this array? What about those little .xml and .edl files. Will those cause problems other than wasted space.

You could partition the drive and create a small enough partition for your music and photos and use the smaller cluster size. But if you do not have a fairly large collection of them then it might be a waste of time.

[GTwannabe]

Originally Posted by ke6guj

Actually MCE is included in Vista Home Premium and Vista Ultimate. The main difference is that Ultimate has all the Vista Business features as well as the Home features.

The main thing I don't like is that if you want MCE and Remote Desktop, you have to go to $$$ Ultimate, its not in the cheaper Home Premium. RDP is become a consumer feature for many people, not just a business feature.

You could get Business... it's basically Ultimate without MCE functionality.

[ke6guj]

Yup, you just have to decide what features you what. Look at the feature matrix and find the version that best suits you for the least $$$. I was just saying that if you want RDP with MCE, or in a "home" version, you had to go Ultimate.

For me, I've got a couple copies of Ultimate on the way from Microsoft gratis, so I'm good for now. And I'll probably just use Business on the rest, since RDP is more important than the MCE features.

[rfutscher]

Originally Posted by Blkhrtd

Thinking along the line of "More is better", why not go with a larger number, say: 512k cluster size? I understand that it'd be a little wasteful, but I was just wondering.

-Shadrach

Video file systems are one of the hardest file systems to create. There are many tricks used to get it operating correctly. Here are some of the things to watch out for.

Block size is one of the most important. As the input buffers fill up they need to be emptied to disk. If the block size is too small you will need several blocks to empty the input queue. This leads to sequential writes to several blocks. If the block size is too large the input buffer will not be able to fill the block. The same with playback, the output queue has to be kept filled. If you have several tuners and playback clients there are a lot of queues to be serviced one by one in random order. The disk heads will always have to move for every queue. Sequential reads will kill the file system performance.
If you have to defragment the drive to keep it working you have other problems than fragmentation.

Offline transcoding can cause problems because you have to both read and write to the disk. You have twice the disk I/O. The heavy use causes the disk to heat up and the platters will expand. During idle times the disks do a thermal recal and align the heads over the tracks. If the transcoding pushes the file system to the max there is no time to do a thermal recal and you start to read and write to the wrong track. You may need to off load the transcoding to another computer.

As more and more record and play out ports are used the needed bandwidth adds up quickly. One trick is to software Raid over many hardware Raids. Spread the data over more disk drives.

Hardware Raids can be a big problem. The block size is determined by the hardware when the Raid set is created. This collection of disks then emulates one large disk, which is what the operating system thinks it has. By the time you format the drive in the operating system the damage may have already been done. Some hardware Raids just don’t work for a video file system.

The bit rate for video clips that professional video servers use is twice the bit rate needed by the following device. This is to reduce transcoding errors. For example for ATSC the HD bit rate approaches 20 Mb. The video clips need to be stored at twice that rate or 40 Mb. For SD the transmission the rate is 4Mb, the storage rate is 8Mb. If you edit the show several times you need to store it at a much higher rate. A rate of 50Mbs is some times used. If there are 20 – 30 record or play out ports, the needed bandwidth adds up quickly. Professional video servers can have as many as 96 disk drives in 12 hardware Raids.
You have to be careful with FTP file transfers to and from other servers, tape archives, and transcoders. You have to limit the data transfer rate of the FTP server. This limit varies depending on the bandwidth needed for the video ports, which have the highest priority. The trick here is to transfer to a disk cache first and then that cache can spool the clip at the constant bit rate that a tape drive needs. It can also speed up and slow down when transferring over a Wide Area Network.

[hemicuda]

will reiserfs do 64k clusters? the only spec I've seen says 512-8k.