SageTV & Threads ( Using multiple Cores )

[joe123]

Hi.

I searched but did not find much. Does SageTV (V6) uses threads?

I upgraded my hardware to a quad core AMD system and when SageTV starts, SageTVservice.exe is at 25% (using 1 of the 4 CPUs at 100%), so it looks like SageTV does not take advantage of multiple cores, but I may be wrong.

Anyone knows?

[GKusnick]

This is a bit like asking whether SageTV uses files. Threads are an essential component of modern software architecture. All apps use them to some degree or other, simply by virtue of running on top of a multitasking OS.

That said, there are still going to be cases where some long computations are single-threaded, either because they're inherently sequential or because it's not a good use of developer time to redesign them for parallel execution. Starting up the service probably falls into the latter category: it's just not relevant to the steady-state performance of the system, and so not a high priority for multi-threading.

[paulbeers]

Technically Sage is multi-threaded, but I wouldn't call it the most optimized for multi-core environments. Actually though, this works a bit to our advantage as it allows for whole Cores to be used by other processes outside of Sage (i.e. ComSkip, SageTranscoder, PlayOn, etc.) without Sage getting in the way and slowing down their processes.

[joe123]

Originally Posted by GKusnick

This is a bit like asking whether SageTV uses files. Threads are an essential component of modern software architecture. All apps use them to some degree or other, simply by virtue of running on top of a multitasking OS.

That said, there are still going to be cases where some long computations are single-threaded, either because they're inherently sequential or because it's not a good use of developer time to redesign them for parallel execution. Starting up the service probably falls into the latter category: it's just not relevant to the steady-state performance of the system, and so not a high priority for multi-threading.

I should have said if SageTV runs threads in parallel, ie making use of multiple core CPUs.

I would not put SageTV in the a dim light not being able to do startup things in parallel because of resources in programming. Even much lower support apps like Showanalyzer do a great job of running is parallel.

As has been noted, SageTV does do some things in parallel. I would be nice to know what those things are. Generally speaking, I don't think SageTV takes great advantage of running things in parallel thus using multiple cores.

SageTV Start-up is the most obvious.

[stanger89]

Originally Posted by joe123

As has been noted, SageTV does do some things in parallel. I would be nice to know what those things are. Generally speaking, I don't think SageTV takes great advantage of running things in parallel thus using multiple cores.

What difference does it make? In general Sage uses very little CPU, even on my ancient server, and Athlon XP 1800+ (single core, single thread), Sage uses only a fraction of the CPU power available.

[joe123]

Originally Posted by stanger89

What difference does it make? In general Sage uses very little CPU, even on my ancient server, and Athlon XP 1800+ (single core, single thread), Sage uses only a fraction of the CPU power available.

It depends on how much you have. I have 40+ Favorites with 8TB of movies and 15k songs and that wiz.bin file is mighty big.

Any time SageTV has to update that wiz.bin file, things really slow down.

Now that I upgraded to a quad core AMD phenom II system, it's a lot better in terms of speed. The circle of death is now more reasonable as the waits are much shorter.

[stanger89]

Originally Posted by joe123

It depends on how much you have. I have 40+ Favorites with 8TB of movies and 15k songs and that wiz.bin file is mighty big.

Any time SageTV has to update that wiz.bin file, things really slow down.

Hm... I've got 133 favorites, a 30MB wiz.bin, and probably 4TB of movies and haven't noticed anything like that.

[tmiranda]

Sage does a lot of things in parallel. To list a few that I've seen:
- generate thumbnails
- check for STV updates
- check for online properties updates
- refreshing the UI (that takes a lot of CPU)
- keeping track of the used and unused disk space
- optimizing the recording schedule
- updating the clock

Yes, these are all small items but this list is far from complete. I'm not saying that Sage makes the best use of all the cores, but it does a decent job.

Also remember that Sage is built on java, which I strongly suspect makes good use of multiple cores.

[blade]

The only time I have any complaint about multi-threading is when running placeshifter. My server has an old Opteron 175 overclocked to 2.8 ghz and it's just not quite enough for placeshifting HD. One core will be maxed out while the other is doing practically nothing. I don't use placeshifter much so I can't justify upgrading the system. I don't know much about the technical aspects of transcoding and multi-threading so it may not be possible for this to be improved, but until someone tells me nothing can be done I'm holding out hope that at some point the transcoder will do a better job of multi-threading.

[tmiranda]

Yes, the transcoder is a good example of where Sage could do a much better job of using multiple cores.

[GKusnick]

Originally Posted by joe123

Any time SageTV has to update that wiz.bin file, things really slow down.

But what makes you think that's a CPU issue? Even if it is, that doesn't necessarily mean it can be made faster by multithreading, or that there's anything about the update process that can be meaningfully parallelized.

My guess is that any circle-of-death issues you're seeing in the UI have more to do with the add-ons you're running than with a lack of multithreading in the core.

Originally Posted by tmiranda

Yes, the transcoder is a good example of where Sage could do a much better job of using multiple cores.

It already does simultaneous transcoding jobs in parallel. So if you have multiple Placeshifter ot MVP clients, you're probably not going to see a dramatic improvement from multithreaded transcoding.

[joe123]

Originally Posted by stanger89

Hm... I've got 133 favorites, a 30MB wiz.bin, and probably 4TB of movies and haven't noticed anything like that.

My wiz.bin is about that size too and I saw a huge and I mean HUGE speed up difference now with my new PVR system.

Went from duo cure Intel 3GHz to Quad Phenom II 3.2GHz with DD3 memory.

My circle of of death went from 30-50 seconds to ~10 seconds.

I am amazed that with your size of wiz.bin, you don't see any slowness. Do you have a music library and if so, how big is it? I got about 15k songs.

[ehfortin]

Hi,

I'm running SageTV Server on an atom 330 (dual core, dual thread which shows as 4 processors in Windows Server 2003) and I don't have any problem with recording or doing playback on all of my client/extenders (total of 3) even if the machine is also my dhcp/dns/firewall (under a virtualbox VM) and MSSQL server.

However, I'm seeing the famous circle-of-death (COD) on the extenders and client. The first thing I looked for was the CPU usage. Even while the COD appears, the CPU is far from spiking. I've also noticed on my SageTV client while monitoring the java heap when I got the COD that once some heap reduction occur (probably after some house cleaning), things are becoming nearly instant. BTW, my SageTV client is a dual-core 2.2 Ghz Intel processor so it is not as limited as the atom 330 (which is still doing great in my setup as a server)

BTW, I'm only seeing COD when changing context (moving from videos to TV is an example or getting back to the TV menu after stopping a playback is another one).

I'm using SageMC so it may be related or not. However, it seems like there is something that is not CPU intensive that is waiting or looping for a while when there is context change.

Coming back to SageMC, I'm willing to test the performance without it but my wife and kids don't want the standard SageTV interface. Is there another interface that is more friendly (like SageMC) but less prone to the COD?

I felt that it could be useful to put that in the mix as I'm seeing more and more people that are putting high performance quad-core CPU as their SageTV server even if it is not required. Like I said, outside of the COD which may be irritating sometime, I have no issue recording HD program through my HD-PVR with such a small setup (files are residing on an external NAS that runs on OpenSolaris 2010.03 (well, what was supposed to be it at the time). And as a supplemental note, this SageTV server was running on a dual-core D920 before and I had the same COD (can't comment on if they were shorter or not but they didn't felt like shorter).

Hope it is useful information.

ehfortin

[stanger89]

Originally Posted by joe123

My wiz.bin is about that size too and I saw a huge and I mean HUGE speed up difference now with my new PVR system.

Went from duo cure Intel 3GHz to Quad Phenom II 3.2GHz with DD3 memory.

My circle of of death went from 30-50 seconds to ~10 seconds.

I am amazed that with your size of wiz.bin, you don't see any slowness. Do you have a music library and if so, how big is it? I got about 15k songs.

I've probably got about 1/3 the songs, even though I never really use it.

But remember this is on a Athlon XP 1800+ (1.5GHz). I seriously doubt your problem was CPU, you had 3-4x the horsepower I've got before you updated, probably like 6-8x now.

I'm guessing your issue was solved by something else you did, cleaning out customizations, or maybe the HDD?

[joe123]

Originally Posted by stanger89

I'm guessing your issue was solved by something else you did, cleaning out customizations, or maybe the HDD?

No, that's the thing. My old setup was in use for some ~6 years. I upgraded my motherboard, cpu and memory, but the disks are all the same and the same wiz.bin file.

I did re-install XP OS, but I did that about a year ago with my old setup hoping to fix things and the Circle of Death did not go away.

[joe123]

Originally Posted by ehfortin

However, I'm seeing the famous circle-of-death (COD) on the extenders and client. The first thing I looked for was the CPU usage. Even while the COD appears, the CPU is far from spiking. I've also noticed on my SageTV client while monitoring the java heap when I got the COD that once some heap reduction occur (probably after some house cleaning), things are becoming nearly instant. BTW, my SageTV client is a dual-core 2.2 Ghz Intel processor so it is not as limited as the atom 330 (which is still doing great in my setup as a server)

BTW, I'm only seeing COD when changing context (moving from videos to TV is an example or getting back to the TV menu after stopping a playback is another one).

I'm using SageMC so it may be related or not. However, it seems like there is something that is not CPU intensive that is waiting or looping for a while when there is context change.

For many years, I have asked here about COD ( Circle of Death ) and every reply was that if the CPUs were not maxing out during the COD, then it was not a system performance problem.

COULD NOT BE FURTHER FROM THE TRUTH! I had the same EXACT problems with COD in that when I hit stop after a playback, I got the Circle of Death - sometimes for 60 seconds. Watching the CPUs revealed little CPU usage, so I never had an excuse to upgrade my system.

Due to a different issue, I recently upgraded my PVR to a Fry's Special with quad AMD Phenom II 955 Black edition, MSI 890GXM-G65 Motherboard and 4GB of CosAir DD3 memory. Kept all same disks.

My Circle of Deaths went from ~60 seconds to ~10 seconds and in some cases, COD went away all together. Yes, the CPUs are still mostly idle. I wish I would have done this a long time ago as the COD were making SageTV pretty much useless.

I also was having skips with my HD-PVR which have gone away 100% to date with my new setup. I could not be happier.

In short, ignore all of the advice here about NOT upgrading and upgrade. With Fry's electronics, you have 15 days to test drive it and return if you don't want it. A lot of work, but you spend nothing to test a new system. I spent a lot more time tinkering and playing around trying to fix the COD than the time it took me to re-do my system.

[paulbeers]

I don't agree with your assessment. As Stanger stated, there was most likely something wrong with your previous setup. Until very recently, I ran a system with an AMD Athlon 4850e processor and never saw the COD in day to day operations*. I have over 30K of MP3's, 10K? in pictures, and over 500 DVD's ripped and never see the COD you do. I am running the Stock GUI with 768mb given to the Java Heap. No issues.


* I say day to day, because when I add a DVD to my collection I will get the COD for about 10-15 seconds as the Meta Data and Far Art are downloaded and updated.

[paulbeers]

Originally Posted by stanger89

even on my ancient server, and Athlon XP 1800+ (single core, single thread), .

I can't believe you still haven't upgraded your server! You have won even more respect from me, as I wouldn't be able to restrain myself from upgrading my server!

[blade]

I have to agree with paulbeers and stanger89. There are far too many people, myself included, running Sage on much slower hardware that don't experience the "COD".

[stanger89]

Originally Posted by joe123

COULD NOT BE FURTHER FROM THE TRUTH! I had the same EXACT problems with COD in that when I hit stop after a playback, I got the Circle of Death - sometimes for 60 seconds. Watching the CPUs revealed little CPU usage, so I never had an excuse to upgrade my system.

My Circle of Deaths went from ~60 seconds to ~10 seconds and in some cases, COD went away all together. Yes, the CPUs are still mostly idle. I wish I would have done this a long time ago as the COD were making SageTV pretty much useless.

Have you tried looking at CPU usage with "Show Kernel Times" enabled? If you don't have that enabled it can "hide" CPU usage in the OS/drivers for example.

I also was having skips with my HD-PVR which have gone away 100% to date with my new setup. I could not be happier.

What else did you do? Reinstall the OS?

Originally Posted by paulbeers

I can't believe you still haven't upgraded your server! You have won even more respect from me, as I wouldn't be able to restrain myself from upgrading my server!

Yeah it's odd, probably the only thing in my entire HT that I haven't touched. But probably the biggest reason is "it works". I am pondering an upgrade, but not because of performance issues, because I need more storage and want to consolidate my equipment.

Just as an experiment I've installed Sage on my i7 920 desktop and imported all my media and recordings, and there's no appreciable difference in performance between the two, what little I've played.