Spinning Circle - somewhat fedup

[Islander]

Posting this under SageMC because I am using it and to avoid being called out if posted under the default ui. I have not used the default ui because honestly I find it downright ugly, so I cannot tell if this would go away of I were to move to the stock ui.

I can live with the occasional 5 seconds spinning circle, but not with the 15+ seconds and at times several minutes spinning circle. I have no doubt that the new server built from the ground up as a fresh install is more than sufficient to not have to deal with this when no extenders are on, no medial is being played, BMT web is not being used, webserver not being used, sjq is not moving any files, and I am simply trying to navigate from one main menu to another.

Where can I look to see what is causing this? I am somewhat of a pack rat and my wiz bin has over 1,300 medial files plus about 2K plus dvds which I just finished ripping a few weeks ago, but that being a database which I am hoping the system deals with efficiently I do not think should be a major issue, right?

Any tips or suggestion on what has worked for other will be appreciated.

[tmiranda]

The first thing you should check is the Java Heap size. If that is set too small you will get a lot of sluggishness. Sorry but I don't know where to look for the Java Heap size in SageMC.

To change the Heap Size you need to edit the registry. Search and you will find a lot of info on Java Heap size.

Tom

[Islander]

Thanks Tom. JavaHeap is setup as 1,200. In the previous server I had experimented with various value and anything much higher than that gives errors on SageTV startup (could have been 1500 or 2000 cannot remember). Unless there is a way to set it up higher?

Thanks.

[tmiranda]

That should be plenty high. How much is actually used? What version of java are you using?

Now that I actually looked at your sig, that hardware configuration should seriously fly. Something is really amiss if you are seeing spinning circles.

I know how you feel about the default STV, but you really should try it out so you can isolate your problems. If the default STV is snappy you know the issue lies with SageMC. If the default STV is also really slow you either have a problem with your Sage install, your network, or your hardware.

I know very little about SageMC but if you post more details about your config I am sure people will chime in with possible solutions.

If you have not done so already, use a bare bones version of SageMC with no STVi's and turn off all of the fancy stuff (fanart, auto compression, dynamic menus, etc) to see what happens.

Tom

[Islander]

Originally Posted by tmiranda

That should be plenty high. How much is actually used? What version of java are you using?

Now that I actually looked at your sig, that hardware configuration should seriously fly. Something is really amiss if you are seeing spinning circles.

I know how you feel about the default STV, but you really should try it out so you can isolate your problems. If the default STV is snappy you know the issue lies with SageMC. If the default STV is also really slow you either have a problem with your Sage install, your network, or your hardware.

I know very little about SageMC but if you post more details about your config I am sure people will chime in with possible solutions.

If you have not done so already, use a bare bones version of SageMC with no STVi's and turn off all of the fancy stuff (fanart, auto compression, dynamic menus, etc) to see what happens.

Tom

Thanks Tom, yes that is what I thought about the hardware versus the requirements. The previous server was an atom 330 4gb memory and it just wasn't that great for playback, I could it see the video tearing up a little. Everything on the new server runs silky smooth including vista startup and other non-video related software, and of course playback is just smooth.

I guess that if no one has a sort of magic bullet then I will start by running the default ui and see how that goes. I will also try disabling auto bmt and will look around where to disable dynamic menus (I will try disabling the few animations options I have on but again I am not sure how that could be taking the system this much). I am not using auto compression, need to get into that soon because even 20tb gets eaten rather quick at 5mg per hour plus DVD rips plus imports, but I wanted to sort out the spinning circle issue before adding yet more layers.

Insofar as setup and what not:

- CPU usage about 6% may go as high as 35% when playing an mpeg2 recording highest quality memory always seems to hover around 55%. When the spinning circle tantrum starts CPU usage fluctuates from 6% to 18% all the way to 35% but again I would not expect those numbers to be that taxing.

- Java Heap Size Used/Total/Max while playing the same video as above on a small window 249/489/1216 again those numbers do not look like they would tax the system that much.

- Running Java 1.6.0.19 (although same behavior was noticed under all previous versions of Java), SageMC 6.3.9a (latest official version as far as I know I guess I could try the betas but did not want to add just another variable to the mix), SageMC jar 2.2, Phoenix jar 1.71.

Any further comments or advice will be greatly appreciated.

[wayner]

I have similar problems to Islander - my Sage can become very unresponsive and at time I cannot shut down (or restart) the SageTV service. I am running Sage for WHS. I have increased my JVM to 768. I run SageMC on my four HD-200s but on my server I run the default UI and I still get the problems there. I will try disabling some of the addins to see if that helps.

Anyone have any other ideas?

[toricred]

@wayner, WHS isn't really meant to use as a client so that might be your problem. Also with that many HD200s you might want to raise your JVM to at least 1024.

@Islander, It looks like your storage is network attached and not local to the server. In my experience this works fine for SD, but HD just doesn't seem to quite work. Others may have had success, but that's been my experience.

[wayner]

Originally Posted by toricred

@wayner, WHS isn't really meant to use as a client so that might be your problem. Also with that many HD200s you might want to raise your JVM to at least 1024.

I realize this and I don't/can't use my WHS server for playback. When I say client I mean for doing stuff like setup or scheduling recordings, etc.

What I mean is that tonight the Sage UI would not respond to any menu selections - I just got ths spinning circle for a very long time.

[Islander]

Originally Posted by toricred

@wayner, WHS isn't really meant to use as a client so that might be your problem. Also with that many HD200s you might want to raise your JVM to at least 1024.

@Islander, It looks like your storage is network attached and not local to the server. In my experience this works fine for SD, but HD just doesn't seem to quite work. Others may have had success, but that's been my experience.

Actually I do not have HD its all SD but for some of the imports. The DVDs are 99% non-blu ray as well. The recording is being done to the local velociraptor then moved over to the WHS by sjq. The funny thing is that I can use MPC or even WMP to play the videos and all start immediately without nay a hitch, something is just causing SageTv to be unresponsive and I guess I need to find out what it is, but I have seen other threads in the past with people complaining about the spinning circle so I know I am not alone.

If the issue was limited to the playing of the media then I would think some of it is related to the media residing on the WHS but with 1gig network all around that should not be such a big issue. I experience spinning even when going down from one menu to another (just scrolling down) on the main page, I just refuse to believe this is how the system is intended to behave even with add ins installed.

For the avoidance of doubt, the spinning circle is not being experience, for the most part or at least not the long duration annoying kind, when starting a media or during play just in all other places.

[wayner]

By the way, when I changed JVM size to 1024 I got an error saying JVM wouldn't start. Once I changed it back to 768 all was well.

[Islander]

"I know how you feel about the default STV, but you really should try it out so you can isolate your problems. If the default STV is snappy you know the issue lies with SageMC. If the default STV is also really slow you either have a problem with your Sage install, your network, or your hardware."

Test one

Tried the stock ui which should have taken out of the equation SageMC, the dynamic menus etc. The only thing that could still possibly be there is BMT auto and I will try disabling that while still running stock and see how it goes.

Counting the many times that I reinstalled SageTv in the previous server to get rid of this issue I have installed it probably 6-7 times so by now I am almost certain that there are no issues with the install.

I do not experience any issues with the network even when it is heavily taxed. I experience issues with Sage all the time. I would have thought that the network would have come into play mostly when accessing media stored in the WHS but my issues are even moving from one menu to another, not even clicking to go into the menus.

Hardware - this is the second server. A few months out of desperation I ran Sagetv out of the $5k+ gaming rig that I put together and I was having similar issues.

I guess another option/test would be to try to get my whs and system to run another media software such as XBMC or what have you (will have to research this a little) and see how my system/whs/etc works in that environment. I really do not even want to entertain that as I have already spent $1k on extenders for Sage and was ready to plunge for two more but the spinning circle issue is giving me gray hairs....

[Fuzzy]

You should turn on debug logging. In there it should log when it is doing the spinning circle. See what actions are before and after the spinning circle instance, and it might lead you to the culprit. It could be as simple as a corrupt media file trying to be processed.

[Islander]

Good idea. I actually have debug turned on (could that be causing issues?) and I happen to notice the time on a long spinning circle and this is what it says in summary: [I think I see some sort of scanning in between not necessarily the cause for thee HANG but I have my scanning set as manual /shrug.]

Sun 4/11 23:06:42.594 [AWT-EventQueue-0@97eded] Found hover_my tv.png in C:\Program Files (x86)\SageTV\SageTV\STVs\SageTV3/SageMCE/Themes\BlackMyst Mod/MenuIcons
Sun 4/11 23:06:42.781 [AWT-EventQueue-0@97eded] Found hover_my tv.png in C:\Program Files (x86)\SageTV\SageTV\STVs\SageTV3/SageMCE/Themes\BlackMyst Mod/MenuIcons
Sun 4/11 23:06:42.829 [ReProcessHook@cbb301] Setting Focus on net.sf.sageplugins.sagemenu.SageMenu@18806c4 to net.sf.sageplugins.sagemenu.InternalScreenMenuItem@5c358b
Sun 4/11 23:08:44.554 [AWT-EventQueue-0@97eded] processOptionsMenu optionsMenu=default:PRQST-1212722|OptionsMenu:MyTVPopup
Sun 4/11 23:08:46.625 [Ministry@1ffd008] Ministry is waiting for 180 sec
Sun 4/11 23:08:51.459 [AWT-EventQueue-0@97eded] setUI(sage.b4@b5ed8e[Schedule]) histIdx=2 uiHistory=[sage.b4@2248c5[Setup Menu], sage.b4@1184531[Dynamic Menu by nielm], sage.b4@b5ed8e[Schedule]]
Sun 4/11 23:08:52.583 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 750

Note: Pasting only the start and end of HANG with anything else that happens in between:

Sun 4/11 23:10:06.087 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 74254
Sun 4/11 23:10:06.761 [Fork-PRQST-1215135@b90a40] Cleaning up orphaned files...
Sun 4/11 23:10:06.762 [Fork-PRQST-1215135@b90a40] Scanning video directory \\HPSERVER1\Recorded TV
Sun 4/11 23:10:06.837 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 75004

Sun 4/11 23:10:27.088 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 95255
Sun 4/11 23:10:27.193 [Fork-PRQST-1215135@b90a40] Scanning video directory D:\Recorded TV
Sun 4/11 23:10:27.193 [Fork-PRQST-1215135@b90a40] Scanning video directory \\HPSERVER1\Videos
Sun 4/11 23:10:27.441 [Fork-PRQST-1215135@b90a40] Scanning video directory \\HPSERVER1\Downloaded Content
Sun 4/11 23:10:27.470 [Fork-PRQST-1215135@b90a40] MyFile generation waiting 5 minutes ...
Sun 4/11 23:10:27.839 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 96006

Sun 4/11 23:11:46.593 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 174760
Sun 4/11 23:11:46.625 [Ministry@1ffd008] Ministry is waiting for 180 sec
Sun 4/11 23:11:47.343 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 175510

Sun 4/11 23:12:28.595 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 216762
Sun 4/11 23:12:29.294 [Seeker@f84b89] Diskspace checking is running
Sun 4/11 23:12:29.295 [Seeker@f84b89] Verifying existence of all TV media files in database fixDurs=false avoidArchive=true
Sun 4/11 23:12:29.346 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 217512

Sun 4/11 23:13:05.348 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 253515
Sun 4/11 23:13:05.525 [Seeker@f84b89] Enforcing keep at most limits for the files...
Sun 4/11 23:13:05.527 [Seeker@f84b89] Seeker clearing unwanted and partial files...
Sun 4/11 23:13:06.098 [AWTThreadWatcher-SAGETV_PROCESS_LOCAL_UI@1368aae] EventThread-SAGETV_PROCESS_LOCAL_UI Hang Detected - hang time = 254265
Sun 4/11 23:13:06.197 [Seeker@f84b89] needFreeSize = 2.925 GB currFreeSize=83.103318016 GB
Sun 4/11 23:13:06.198 [Seeker@f84b89] Seeker waiting for 11 mins.
Sun 4/11 23:13:06.198 [PowerManagement@724f11] Enabled SS

Above is last block then the HANG ended.

Any ideas of what I should check next? From now on I am making a point of writing down spinning circles episodes (I am assuming that even in the best of scenarios once has them right but only short duration?) and checking the log and see if I can detect a trend.

I apologize if the above copy and paste comes out wrong. How do I make the nice looking scrollable windows I see some people using?

Thanks!

[Fuzzy]

Don't see anything obvious, though as previously suggested, you should run with the stock STV for a few days and see if it occurs there. As for the scrollable window, you need to include your text in CODE blocks. (The # button in the advanced editor).

[davephan]

Could it be due to scaling up to 20 TB, most of which could be full? I scaled up to 9 TB, I sometimes get the spinning circle, and my 9 TB is mostly full. Maybe the database chokes when it gets to a certain size. You could test that by shutting down SageTV. Copy your SageTV directory. Then remove most of your video recordings by changing your recordings and libraries to other library names within the SageTV configuration. After you have significantly less items stored, according to SageTV, the SageTV database should then be much smaller. Maybe SageTV would run faster with a smaller database. I would also take an image of the C drive before doing the test. I don't know what the practical limit is for storage is with SageTV. I have never seen any file limits posted. I assume at some port, the database will choke when there are too many recordings.

Also, I tried installing Vista Ultimate 64-bit that I had laying around on a new i-7 950 quad core 3.06 gig with 3 gig 1333 DDR 3 RAM computer. The Vista operating system was about 1/3 the speed of XP. Vista was so slugglish and dog slow, with just the operating system, that I abandoned it can installed XP Pro, which ran 3 time faster than Vista Utimate. I didn't bother trying the 32-bit Vista Ultimate since my experience with the 64-bit Vista Ultimate was so bad.

Dave

[tmiranda]

I'm with Dave. It may be a problem with the database size or something strange with the wiz.bin. I'd try the following:

- Remove all of the import paths.
- Stop Sage completely.
- Move all of the wiz.bin's (backups included) to some safe place.
- Restart Sage and let it rebuild the wiz.bin from scratch. All of the recorded TV will show up as imported media but that should not be a problem since you are only testing.
- Let Sage complete its scan after startup.

If your system speeds up you know it's the wiz.bin database. You'll then need to isolate if it's a problem with a corrupt wiz.bin, too many imported files, too large or a storage array, etc.

When you're done testing stop Sage, restore your old wiz.bin, and all of your recordings should be back.

Tom

[Islander]

Ok I will try that tonight and let it run overnight.

"- Remove all of the import paths." I am assuming that this so that whenever the scan is done it is only dealing with the "recorded" media. Wouldn't it be best to leave the imported paths in (whether initially or after the recorded scan is done and the system tested) to see if it is the size of the DB that is choking it?

This is what the Db looks like:

Wizbin size 22,413kb
# of dvds and imported movie files as reports by SMW = 1,806
Music = 8,006 songs
Photos = 4,700 or so
Recorded TV 2,100 recorded shows and movies, add possibly another 200 or so imported shows

Are those numbers likely to choke the db?

Thanks.

[wayner]

Pls keep us posted on your progress as it may help others.

[PLUCKYHD]

Do you see the lag in smw? That is a large database and can take time to group and build, but shouldn't see the lag you are seeing in those logs.

[Islander]

Plucky I am glad to report that I am seeing no lag at all. I may get a few seconds spinning circle going from the main menu into SMW but that is all. Scrolling through the library using the default of 9x6 (right?) is very smooth. After the install on the new server I noticed that if I scrolled a page down then the next time I did it it was even faster (maybe some background caching going on?) so I scrolled page by page until I got to the end of my collection.

What worries me about these HANGS in the ui is that they happen even when no one is using the interface either on the server or the extenders. I just looked at the log since last night and there were too many hangs to count and I was not able to find a smoking gun.

Will report hopefully sometime tomorrow.