SageTV performance tweaks: What do you do to gain extra SageTV performance?

[Slipshod]

One more tip...

If you're running Sage as a server only (no local video), use a server-class motherboard with onboard Video. I've got one with a built-in 32MB video card, and running under WHS I've got almost my entire 4GB of memory available for use in a 32bit Windows install.

[sandyj]

Hi could someone please explain were the below setting is and what would the recommended setting be for use with extenders.

Quote Plucky
"The timeout setting isn't as important with background loading but with the default ui and smm it uses that setting for inactivity time to cache ahead."

Is it this one ui/inactivity_timeout_for_full_thumb_load=1500 and should it be changed to =1000

[mkanet]

I'm curious if there are people who use RAM disks to enhance any part(s) of sageTV performance.

[jorton]

Hey MKANET,

I really like Andy VT's (Babgvant) Sagetv Registry Tool that can set your java heap (amoung other things) without have to bust out regedit etc...

Here's a link,

http://babgvant.com/files/folders/sa...ntry12640.aspx

PS - You've got 4 pages of posts to skim through already, any plans to summarize all this somewhere?

Cheers,

Jeremy

[mkanet]

Thanks for the info for the registry editing tool. Yeah, I'll definitely update the first post to include all tips so far.

-Michael

Originally Posted by jorton

Hey MKANET,

I really like Andy VT's (Babgvant) Sagetv Registry Tool that can set your java heap (amoung other things) without have to bust out regedit etc...

Here's a link,

http://babgvant.com/files/folders/sa...ntry12640.aspx

PS - You've got 4 pages of posts to skim through already, any plans to summarize all this somewhere?

Cheers,

Jeremy

[nwboater]

Originally Posted by mkanet

Thanks for the info for the registry editing tool. Yeah, I'll definitely update the first post to include all tips so far.

-Michael

Great thread - thanks for starting it! And I'll be looking forward to the first post summary.

Could I suggest that any more pro/con post on SSDs be made in a new thread.

Thanks,
Rod

[davephan]

Originally Posted by mkanet

I'm curious if there are people who use RAM disks to enhance any part(s) of sageTV performance.

Here's some review of RAMDisk products. Some are free. It looks like some perform much better than others.

http://www.raymond.cc/blog/archives/...d-write-speed/

It's been decades since I experimented with a RAM Disk. The last time I used a RAM Disk, the speed difference was absolutely shocking. I think I may try experimenting again on my general purpose computer, taking an image of the system before the testing so I can completely remove the demo software after the testing.

I don't know if only the SageTV directory structure would need to be copied to the RAM disk, or maybe the Java application would need to be installed there too.

Perhaps another use might be to temporarily copy video recordings to a RAM disk for Comskip processing.

The RAM disk might make video editing, especially with h.264 files, a lot faster.

Dave

[GKusnick]

Can I voice a dissenting opinion about RAMdisks? Or are only happy thoughts allowed here?

The conventional wisdom is that if you have extra RAM that's addressable by the OS, you should let the OS manage it as part of its page cache. Taking RAM away from the OS to make a RAMdisk basically means you're manually taking over responsibility for deciding what data to cache in RAM. The OS has logic for doing that automatically and dynamically, using actual performance-based metrics, instead of just guessing at a static set of files to cache (which is what you'd be doing). So you're better off leaving it to the OS.

The only time a RAMdisk might make sense is if you have a chunk of RAM that the OS is unable to use for some reason and that would go totally unused if you don't make a RAMdisk of it. Even then, mirroring your entire Sage installation directory into it would require that you have some way of writing back out the files that have changed (like Wiz.bin!) so you don't lose data on reboot.

I'm not looking to spoil anyone's fun but it seems to me that any responsible discussion of tweaking should include both pros and cons. Not every tweak is for everyone (otherwise it would be the default), so it's important to understand what problem a tweak is meant to address and what its limits are.

[mkanet]

Dave, thanks so much for this. What I'm curious about; and, think maybe some others might as well, is what specific components of sageTV can be put on a ram disk without risk of corrupting data/config files. It would be great to see who has something working reliably; while, taking advantage of noticeable performance gains. For example, personally, I would be very nervous to put files like wiz.bin, sageTV properties files, and the like on a RAM disk.

I've never tried a ram disk for sageTV itself. However, I've had very disappointing experiences in the past when trying to gain overall OS performance by forcing the OS and it's respective components to stay in RAM. I've tried various methods; such as, using the Windows DisablePagingExecutive option (to prevent OS and respective Windows services from disk swapping) and also putting temp folders, pagingfile, etc onto a RAM disk. I've tried this 3 different times (starting from NT 3.5 to Window XP); every time, I ended up having to eventually restore my OS partition from image based backup. I finally gave up the idea of putting temp files, pagefiles, and any kind of data files in general on RAM disk. This was even with a UPS and stable hardware on 3 different setups.

I eventually compromised with a more conservative method by storing files that are frequently written/read on non-volatile solid state media (SSD) for OS, Apps, and respective data files, temp files, etc.

Anyway, maybe someone might post an practical/safe solution with RAM disks that could directly/indirectly speed up some parts their sageTV setup.

-Michael

Originally Posted by davephan

Here's some review of RAMDisk products. Some are free. It looks like some perform much better than others.

http://www.raymond.cc/blog/archives/...d-write-speed/

It's been decades since I experimented with a RAM Disk. The last time I used a RAM Disk, the speed difference was absolutely shocking. I think I may try experimenting again on my general purpose computer, taking an image of the system before the testing so I can completely remove the demo software after the testing.

I don't know if only the SageTV directory structure would need to be copied to the RAM disk, or maybe the Java application would need to be installed there too.

Perhaps another use might be to temporarily copy video recordings to a RAM disk for Comskip processing.

The RAM disk might make video editing, especially with h.264 files, a lot faster.

Dave

[Fuzzy]

Originally Posted by mkanet

I've had very disappointing experiences in the past when trying to gain overall OS performance by forcing the OS and it's respective components to stay in RAM. I've tried various methods; such as, using the Windows DisablePagingExecutive option (to prevent OS and respective Windows services from disk swapping) and also putting temp folders, pagingfile, etc onto a RAM disk. I've tried this 3 different times (starting from NT 3.5 to Window XP); every time, I ended up having to eventually restore my OS partition from image based backup. I finally gave up the idea of putting temp files, pagefiles, and any kind of data files in general on RAM disk. This was even with a UPS and stable hardware on 3 different setups.

Goes back to my recommendation to disable paging altogether (on a dedicated sagetv server). Easiest way to force things to stay in ram, is to remove an alternative.

[nwboater]

Originally Posted by GKusnick

Can I voice a dissenting opinion about RAMdisks? Or are only happy thoughts allowed here? ........

I'm not looking to spoil anyone's fun but it seems to me that any responsible discussion of tweaking should include both pros and cons. Not every tweak is for everyone (otherwise it would be the default), so it's important to understand what problem a tweak is meant to address and what its limits are.

Greg,

Not sure if you were directing this at my comments on moving SSD talk to another thread. It just seemed that much of the thread was directed towards a debate on SSDs and that it might be worthwhile for it to have it's own thread. Sometimes a thread can veer so far OT that it loses it's original intent. When people in the future are doing research it can be very difficult for them to find what they are looking for.

In any case you are one of the very experienced and helpful posters on this forum and have helped me in the past.

Very sorry if my post was taken as wanting to stifle discussion.

Rod

[mkanet]

Actually its the complete opposite of what you said. I had catastrophic issues with forcing the server OS and respective services to NOT swap to disk. However, in all cases, I had IIS and many other server based services/apps installed that needed to OS swap file. There are a significant number of people who use their servers for more than basic SageTV features. That being said, I know there are minimalistic Windows PC/SageTV setups that would really benefit from disabling the pagefile.

Originally Posted by Fuzzy

Goes back to my recommendation to disable paging altogether (on a dedicated sagetv server). Easiest way to force things to stay in ram, is to remove an alternative.

[Fuzzy]

eliminating the swap file will work on any system.. as long as you have enough ram for what you want to do with he system. In days where you can relatively easily run systems with 16GB of RAM, the swap file is just a waste of time. (especially since, even WITH 16GB, windows will still try to swap things to disk).

[scoful]

Originally Posted by Fuzzy

eliminating the swap file will work on any system.. as long as you have enough ram for what you want to do with he system. In days where you can relatively easily run systems with 16GB of RAM, the swap file is just a waste of time. (especially since, even WITH 16GB, windows will still try to swap things to disk).

This isn't entirely true - some applications have required a swap file to operate at all. A trick we used to use was set the swap file minimum and maximum to the smallest allowable size (3MB?). This would satisfy the applications requirement that a swap file exists (Internet Explorer was one such application) but minimize the amount of swapping. This can yield some strange results. A non system managed pagefile of a larger fixed size can also cause stability issues - specifically on 2003 server.

Also in some situations XP, while configured without a swap file still has and uses one. If you go it the settings even though all drives are set to have no pagefile, it still shows an overall size and one does in fact exist - usually on your 'C' drive.

I have used a ramdisk utility on a machine with 8 Gigs of RAM and 32 bit XP. Setup Ramdisk to use the inaccessible upper 4 gigs and put the pagefile there. Mixed results. Performance improved, but stability was reduced. This may be attributed to the ramdisk utility itself. Microsoft has a ramdisk utility that seems to work great - but it won't access the above 4 Gig RAM...

[doncote0]

I used to use a standard mouse, standard chair and standard keyboard. Straight rows of keys on the keyboard and the mouse had a ball in the bottom and I moved it around. The chair was a POS that no one else wanted. My keyboard was titled upwards and was on top of the desk.

I know what works better for me now. Trackball or joystick mouse, ergonomic keyboard (slanted down), ergonomic chair and a keyboard tray that puts the keyboard at a comfortable leve just above my legs. Ergonomic chair angle is slightly angled forward to neutral. Works for me. No more early signs of carpal tunnel, shoulder or lower back pain. I will not go back.

I am using SSD's now. I really won't use anything else anymore since the improvement has made such a difference. My system is probably faster than most, but the addition of SSD's help put my system 6th on the list of the top 20 fastest systems ever tested @ passmark.com. Crazy and not bad when compared to over a quarter of a million other systems.

Since then, my system with a performance mark of 5688 has fallen off the list giving way too faster processors, more cores, more processors and more heavily overclocked systems. (Of course that means those systems are probably more expensive as well.) I am fine with that - I initially tested to check my system's ability and stability following a complete computer upgrade.

Those top systems have some basic things in common:
1. Fast and usually overclocked processor
2. Fast video card
3. SSD Drives
4. 64 bit operating system
5. A lot of RAM
6. A power supply that can handle the demands of all of the above
7. Not cheap, unless you plan on using that system for 5 years or more (which I do--I do not like continual upgrades).

SSD's work for me, and I am not going back. I'll explain why in the next post.

[doncote0]

1. Responsiveness. No latency. None--as in zero or so close to zero it doesn't matter.
2. Seek time is 0.1 msec. A fast standard hard drive can have seek times around 4 msec, but it also has latency.
3. IOPS (inputs/outputs per second). This is the number of read and write actions that can occur per second without problems. For my system, each individual SSD supports 50000 IOPS. A good standard hard drive will support 200 IOPS. This means that the system will effectively never wait or have to buffer information going to the harddrive. Have 10 read and write activites that all are scheduled at the same time, no problem.
4. Reliability and dependability. A good standard hard drive is not supposed to fail for about 60 years (average) given optimum conditions. Optimum conditions are the key to that average. Most standard hard drives will have problems within 7.5 years. A SSD is not supposed to fail for about 230 years (average) given optimum conditions. Testing has revealed that normal household optimum condition variations do not have a significant effect on SSD performance or longevity.
So if I use a similar ratio as that used for standard hard drives, I guess I can expect that they will last at least 30 years but will probably last longer.
5. Quiet. My system is quiet and these do not generate any noise.

Does that mean you cannot keep your standard hard drive to boot with? No.
Does that mean that I do not backup the information on my boot drive? No.
Do I hate standard hard drives? No, I have two that I use for data.
Are standard hard drives cheaper? Yes.

I am responsible for the upgrade path for the computers in our networked enviroment where I work. That path now includes SSD's, because of the increase in performance, reliability, because the important information is saved on the network (instead of the local hard drive) and because the effective life of the hard drive will not place extra risks on the time between upgrades (like it does now). So in a down turned ecomomy, SSD's will allow us to get extra years out of our existing systems and reduce the overall cost despite the extra cost of the SSD technology.

[GKusnick]

Originally Posted by doncote0

Those top systems have some basic things in common:

Another thing many of those top-performing systems have in common is owners who spend a lot of effort tweaking performance just to get on that list. If that's your thing, fine, but that's a completely different discussion from what makes a good SageTV server. Overclocking in particular is not something I would recommend for a stable, reliable, power-sipping media server.

[korben_dallas]

I recently rebuilt my SageTV server using a 32GB ssd and Windows 7 Ultimate x64. After Windows setup, SageTV, Arcsoft (for the HD PVR), and just a few other programs, I have 12GB free space. All SageTV recordings, media storage, and backup files are on 3 x 500GB hdds. While I don't logon locally very often, startup from cold boot is about 30 seconds, with more than half of that bios & POST. When I remote desktop to the machine, it logs in under 5 seconds.

I am highly impressed with stability. Not a single HD PVR usb error in 3 weeks, though that could also be the new motherboard.

So I highly recommend Win 7 x64 on an SSD for SageTV server usage.

[blade]

Originally Posted by GKusnick

The only time a RAMdisk might make sense is if you have a chunk of RAM that the OS is unable to use for some reason and that would go totally unused if you don't make a RAMdisk of it. Even then, mirroring your entire Sage installation directory into it would require that you have some way of writing back out the files that have changed (like Wiz.bin!) so you don't lose data on reboot.

I agree completely. That's exactly what I was hoping someone would have some experience with. The only use I can see for a RAM disk is making use of more than 3.x GB of ram with XP 32 bit by putting the swap file and maybe some temp files (if there are any that can safely be nuked with the occasional reboot) on the RAM disk. I don't think there would be any real benefit to using a RAM disk any other way.

[blade]

Originally Posted by scoful

I have used a ramdisk utility on a machine with 8 Gigs of RAM and 32 bit XP. Setup Ramdisk to use the inaccessible upper 4 gigs and put the pagefile there. Mixed results. Performance improved, but stability was reduced. This may be attributed to the ramdisk utility itself. Microsoft has a ramdisk utility that seems to work great - but it won't access the above 4 Gig RAM...

That's the exact situation I was considering trying with my Sage server though if it can't be done reliably then it's definitely a no go.