SageTV Server Linux with Comskip for Docker (and unRAID) Support Thread

[Fuzzy]

Hmmm.. looks like I'll need that version tag as well. Restarted the docker tonight, and the new sage version is not liking my wiz.bin at all.

[EnterNoEscape]

Originally Posted by Fuzzy

Hmmm.. looks like I'll need that version tag as well. Restarted the docker tonight, and the new sage version is not liking my wiz.bin at all.

Before you restarted and upgraded, did you have the version just prior (9.0.4.287) installed?

[stuckless]

Originally Posted by Fuzzy

Hmmm.. looks like I'll need that version tag as well. Restarted the docker tonight, and the new sage version is not liking my wiz.bin at all.

I'm wondering if this is perhaps related to the shutdown process. I'm not sure how unRAID shuts down the docker container, but if it just kills it, then maybe that is causing the corruption since it's not getting a chance to flush writes to the files.

That being said I've restarted my docker container many times and I've never had an issue... and event his time I didn't have an issue. It wasn't until I started to install the updated plugins that for some reason sagetv failed to install/upgrade all the required files (something that I've seen many times)

[Fuzzy]

Originally Posted by stuckless

In the sagetv/server/ directory there is a file, .SAGE_CUR_VERSION, that holds the currently installed version. If you edit this file and set the version to 9999.0.0.0 it will never update (at least not for a very long time).

Later if you do want to take an update, just delete the file, and it will pull down the latest version.

and FWIW, setting my .SAGE_CUR_VERSION TO 9999.0.0.0 did not prevent downloading the current version. I had to leave it set to 9.0.7.362, and extract the 9.0.4.287 overtop the install to downgrade. Looking at the install script, it looks like it is only comparing the versions, and if they don't match, it downloads the current version. it doesn't do any sort of greater-than comparison.

[starfire]

Yes, I found that also reported here

[Fuzzy]

Originally Posted by stuckless

I'm wondering if this is perhaps related to the shutdown process. I'm not sure how unRAID shuts down the docker container, but if it just kills it, then maybe that is causing the corruption since it's not getting a chance to flush writes to the files.

That being said I've restarted my docker container many times and I've never had an issue... and event his time I didn't have an issue. It wasn't until I started to install the updated plugins that for some reason sagetv failed to install/upgrade all the required files (something that I've seen many times)

Not sure. Rolling back the old version accesses the wiz.bin just fine, no matter how many time it restarts, so I'm not thinking it's a shutdown issue - if anything, it seems more like it's an issue in the database upgrade process. Don't know. The whole wiz.bin part of the sage code is still a mysterious black box to me.

[stuckless]

Originally Posted by Fuzzy

and FWIW, setting my .SAGE_CUR_VERSION TO 9999.0.0.0 did not prevent downloading the current version. I had to leave it set to 9.0.7.362, and extract the 9.0.4.287 overtop the install to downgrade. Looking at the install script, it looks like it is only comparing the versions, and if they don't match, it downloads the current version. it doesn't do any sort of greater-than comparison.

Yeah, I noticed that... The new script does do a greater check... and support supplying a specific version.

Originally Posted by Fuzzy

Not sure. Rolling back the old version accesses the wiz.bin just fine, no matter how many time it restarts, so I'm not thinking it's a shutdown issue - if anything, it seems more like it's an issue in the database upgrade process. Don't know. The whole wiz.bin part of the sage code is still a mysterious black box to me.

The last release was in June... and I noticed there are LOTS of changes since june. When my changes were pretty small and were around the plugin system... I'm not sure what all the other changes were for. That being said... I don't get any corruption even on the latest version when I restart my containers... in unRAID or in my test environment (which also uses docker containers).

[BobPhoenix]

Originally Posted by Fuzzy

Hmmm.. looks like I'll need that version tag as well. Restarted the docker tonight, and the new sage version is not liking my wiz.bin at all.

Me too. Probably did the same thing you did. Had to figure out how to extract the files in the .deb package for SageTV 9.0.4.287 and then copied them over the top replacing existing files of the server directory.

[seaverd]

Same issue here...wiz.bin corrupt. Any tips on how you extracted the .deb would be greatly appreciated.

Dan

[seaverd]

Thought I figured this out...with docker started I downloaded the older version to a windows pc and extracted the deb to a tar file then extracted the .tar file and copied the folder over the top of the server folder in the sagetv appdata. When I start the sagetv miniclient I get unable to connect failed to connect at IP address (port 31099) after 30000ms: is connected failed: Econnrefused (connection refused)

Any suggestions?

[BobPhoenix]

Originally Posted by seaverd

Same issue here...wiz.bin corrupt. Any tips on how you extracted the .deb would be greatly appreciated.

Dan

This is what I did:

1. Copied the DEB file to the /appdata/sagetv directory on my cache drive
2. Opened up a putty screen to my unRAID server
3. Started the SageTV docker
4. Issued the following commands
5. First: "docker exec -ti stuckless-sagetv-server-java8 bash"
6. Then: "dpkg-deb -x /opt/sagetv/sagetv-server_9.0.4_amd64.deb /opt/sagetv/unzipped"
7. Now exit Docker bash shell with exit
8. Stop the SageTV docker
9. Copy the the /appdata/sagetv/unzipped/opt/sagetv/server directory over the top of the existing /appdata/sagetv/server directory with MC on unRAID replacing ALL files
10. Edit the ".SAGE_CUR_VERSION" in MC and make sure the version is 9.0.7.362 inside it
11. Exit the editor and MC
12. Start the SageTV docker and watch to make sure .corrupt0/1/etc... isn't created.

[seaverd]

Bob,

Thanks for the detailed reply. It has me up and running. Just before this update I also updated phoenix core and I notice that I have several plugins that say failed i.e. Gemstone, Jetty Web Server, Pushbullet, sagetv-addons License Server.

[BobPhoenix]

Originally Posted by seaverd

Bob,

Thanks for the detailed reply. It has me up and running. Just before this update I also updated phoenix core and I notice that I have several plugins that say failed i.e. Gemstone, Jetty Web Server, Pushbullet, sagetv-addons License Server.

Sorry don't use those so can't help with that. Hopefully someone else can help with that.

When I started the updated version of SageTV with NO wiz.bin file it looked like it would work but I didn't have the hours and hours it would take to get through all of my media so I looked for a way to downgrade and came up with my method above.

[seaverd]

Thanks again Bob. I removed plugins and reinstalled and everything seems to be functional.

Dan

[Fuzzy]

The alternative method I did, which seemed easier, was to, with the docker shut down, ssh into my unRAID, and cd to the sagetv/server folder. There I used wget to pull down the .tar.gz from bintray, and then used tar to extract it to the server folder. Since it had already updated to the 9.0.7 version, the .SAGE_CUR_VERSION file was already correct. Then rename the .corrupt copy back to Wiz.bin, and start up the docker.

[stuckless]

Originally Posted by Fuzzy

The alternative method I did, which seemed easier, was to, with the docker shut down, ssh into my unRAID, and cd to the sagetv/server folder. There I used wget to pull down the .tar.gz from bintray, and then used tar to extract it to the server folder. Since it had already updated to the 9.0.7 version, the .SAGE_CUR_VERSION file was already correct. Then rename the .corrupt copy back to Wiz.bin, and start up the docker.

Yeah, that's what I would have done as well... The bigger question on my mind is why did the new sagetv binary corrupt wiz.bin for so many, but not everyone. ie, while I experienced a plugin catastrophe after updating, my wiz.bin was in tact... and has been in tact after serveral restarts, etc.

While this thread is related to unRAID, I'm not sure ALL the corrupt wiz.bin issues happened on unRAID... if it was, then maybe unRAID is the key here... I know from testing, I run sagetv in a Docker (have multiple dockers setup with multiple different configurations) and none of those have corrupt wiz.bin issues.

There were 3 separate issues that I've found with this latest release...
1. corrupt wiz.bin
2. massive plugin failures (which I think is related to a a corrupt Sage.properties..ie plugins like sagex that can't be easily uninstalled because of dependencies was basically showing as not install)
3. I noticed this the other day... but a client that I use a lot in testing, when connecting to the server, was forced to go through the client setup, again... which leads me to thing that perhaps the client properties somehow got corrupted??

I've made a change to the Docker container to accept a VERSION variable to prevent upgrades, etc... but I think I'm going to add an auto-backup to the upgrade as well, so that everytime it does an upgrade it backs up the core sagetv files, jars, Sage.properties and wiz.bin, to make it easier to restore it, should you/we need to do so.

While I'm still running the latest.. these corruption issues has me a little concerned.

[Fuzzy]

Originally Posted by stuckless

Yeah, that's what I would have done as well... The bigger question on my mind is why did the new sagetv binary corrupt wiz.bin for so many, but not everyone. ie, while I experienced a plugin catastrophe after updating, my wiz.bin was in tact... and has been in tact after serveral restarts, etc.

While this thread is related to unRAID, I'm not sure ALL the corrupt wiz.bin issues happened on unRAID... if it was, then maybe unRAID is the key here... I know from testing, I run sagetv in a Docker (have multiple dockers setup with multiple different configurations) and none of those have corrupt wiz.bin issues.

There were 3 separate issues that I've found with this latest release...
1. corrupt wiz.bin
2. massive plugin failures (which I think is related to a a corrupt Sage.properties..ie plugins like sagex that can't be easily uninstalled because of dependencies was basically showing as not install)
3. I noticed this the other day... but a client that I use a lot in testing, when connecting to the server, was forced to go through the client setup, again... which leads me to thing that perhaps the client properties somehow got corrupted??

I've made a change to the Docker container to accept a VERSION variable to prevent upgrades, etc... but I think I'm going to add an auto-backup to the upgrade as well, so that everytime it does an upgrade it backs up the core sagetv files, jars, Sage.properties and wiz.bin, to make it easier to restore it, should you/we need to do so.

While I'm still running the latest.. these corruption issues has me a little concerned.

Yeah, when I get some time, I was going to try to assess the wiz.bin corruption a bit. It was a specific error that was showing, but i need to research what that error means. I'm not really sure what changes were made in Wizard.java since the last version, and how that might interact. It could also very well be that my (and other's) Wiz.bin has long had a corruption issue that is just now showing due to changes in Wizard, and it might not be that Wizard is actually corrupting something.

FWIW, the error I'm getting in loading the database is:

ERROR DB RECORD LENGTH VIOLATION was 2195768 should be 2228856 fp=17281899 length=22928153 opcode=5 typecode=0

[stuckless]

Originally Posted by Fuzzy

Yeah, when I get some time, I was going to try to assess the wiz.bin corruption a bit. It was a specific error that was showing, but i need to research what that error means. I'm not really sure what changes were made in Wizard.java since the last version, and how that might interact. It could also very well be that my (and other's) Wiz.bin has long had a corruption issue that is just now showing due to changes in Wizard, and it might not be that Wizard is actually corrupting something.

FWIW, the error I'm getting in loading the database is:

ERROR DB RECORD LENGTH VIOLATION was 2195768 should be 2228856 fp=17281899 length=22928153 opcode=5 typecode=0

So, the previous build was done on June 23rd.... I see all the rewrite of the SageTV file handling was chagned after that (in july)... so this is the first build that introduces that... and it's the only changeset that I can see that changes the Wizard.java.

Out of curiosity... for the people that have failures... is unicode filenames enabled... Mine is not (and by default I think it is turned off).

[tmiranda]

Seems to me we need some type of Beta testing process.

[Narflex]

I was very careful about reviewing the filesystem changes that affected reading/writing the Wiz.bin...but of course I could have missed something. If somebody wants to get me a Wiz.bin file that's loading as corrupt I'm more than happy to look into exactly why that's occurring...that's one of the worst things that can go wrong with SageTV and I always want to understand why it happened when it does so it can be fixed.