Scripts: SageTV backup scripts (dailly config, full weekly, full manual)

[robogeek]

I thought I'd share the backup scripts I'm using to create zip archive backups that are based on the script Enormous posted here. These scripts create daily backups of the SageTV configuration files, full weekly SageTV directory backups, and a script to create manual backups prior to upgrades or major configuration changes. The daily config and weekly full backup scripts will also auto-delete old backups. The only additional requirement for auto-delete is that your Windows Command Line must include the forfiles command. To check this requirement, open a command prompt and type in:

forfiles

If it gives you an error, then these daily/weekly backup scripts won't auto-delete old backups for you. If the command gives you a list of files in quotes, then you should be good to go.

If you have enough space on the C: drive to hold SageTV backups then download the attached zip file and extract to the root of the C: drive. This will give you a directory structure:

C:\SageTV Backups
----C:\SageTV Backups\Config (daily auto backups)
----C:\SageTV Backups\Full (manual backups)
----C:\SageTV Backups\Full (weekly auto backups)
---sagetv-config-backup.bat
---sagetv-manual-backup.bat
---sagetv-weekly-backup.bat
---zip.exe

The zip.exe is freely redistributable and was obtained from here: http://stahlforce.com/dev/index.php?tool=zipunzip

If you don't want to put the backups on the C: drive, extract the zip to where you want the backups stored and then you'll have to modify some of the variables in the batch files to point to the correct backup location. Read on for more information.

sagetv-config-backup.bat
-----------------------------
This script will save you if something corrupts your wiz.bin database or if you make some changes to your .properties files that cause your SageTV setup to crash or misbehave. This script will backup wiz.bin, wiz.bak, sage.properties, sage.properties.autobackup, sageclient.properties, sageclient.properties.autobackup, SageTV\client\*.properties, SageTV\client\*.properties.autobackup, and the Windows irremote.ini files. And it also backs up the SageTV\Common\RemoteCodes directory. By default, it will keep 14 days of backups and delete older backups. The number of backups kept is configured with the DaysToKeep variable in the batch file. Simply set this up as a daily scheduled task in Windows to backup your configuration files automatically each day. If you have other configuration files that you want backed up, you'll have to add the appropriate commands.

sagetv-weekly-backup.bat
-----------------------------
This script is very much like the sagetv-config-backup script, with only a couple of exceptions. This script will backup your entire SageTV root (C:\Program Files\SageTV\SageTV) and SageTV Common (C:\Program Files\SageTV\Common) directories and the Windows irremote.ini file. Instead of running daily, set this script up to run as a weekly scheduled task in Windows. By default it will keep four weekly backups (4 weeks x 7 days/week = 28 days for DaysToKeep variable). Caveat regarding full backups...if you have BMT/BMI and/or the Phoenix API installed, you may want to keep your fan art and the Phoenix cache directories located outside of the SageTV root directory to keep backup sizes reasonable. You can change the fan art directory from within the Fan Art configuration in SageTV. To change the cache location, you'll have to shutdown SageTV (and SageTV Service) and locate the lines in the sage.properties file that contain cache\. There should be three of those lines. Just change the locations to someplace else and then move the cache directory from the SageTV root directory to the new location and restart SageTV. If you have other SageTV related items that are not located in the SageTV root directory or the SageTV Common directory, you'll have to add the appropriate commands to capture those items as well.

sagetv-manual-backup.bat
-----------------------------
This script is useful for making full backups before upgrading to a new SageTV release or before making significant changes to your SageTV configuration. This script is like the sagetv-weekly-backup script except you can call this script with a version number or short description (be sure spaces are replaced with _ or -) to yield backup names that can be more easily identified. So, before you upgrade from SageTV v6.6.1 to v6.6.2, you can call it like this:

sagetv-manual-backup v6.6.1

which will give you a backup named SageTV v6.6.1 Backup (mm-dd-yyyy hhmmXM).zip.

Or, before upgrading SageTV v6.6.2 with Jetty v1.5 to Jetty v1.6, you can call it like this:

sagetv-manual-backup v6.6.2_Jetty_v1.5

which will give you a backup named SageTV v6.6.2_Jetty_v1.5 Backup (mm-dd-yyyy hhmmXM).zip.

The manual backup script doesn't auto-delete any backups, so you will have to clean up these backups on your own. And again, if you have other SageTV related items that are not located in the SageTV root directory or the SageTV Common directory, you'll have to add the appropriate commands to capture those items as well.




If installing to someplace other than the root of the C: drive or if your SageTV installation is not in the default C:\Program Files\SageTV location, open each script with notepad and follow the instructions in the REM statements to configure the variables. If you have a default SageTV installation in C:\Program Files\SageTV and you keep the backups in C:\SageTV Backups then you shouldn't need to make any changes.

There are a couple of errors that you may see that are normal and are probably not an indication of any problems. Not everybody will have an irremote.ini file in their Windows directory. If you don't, the scripts will give you a 'The System cannot find the file specified' error when backing up the irremote.ini file. I believe the irremote.ini file is only used in special circumstances when you use a remote to control applications other than SageTV on your PC, so if you don't have it, don't worry about it. The other common error you'll see is when there are no backups ready to be deleted. The forfiles command will give an 'ERROR: No files found with the specified search criteria' error if no backup files are ready to be deleted. Niether of these errors are critical and they should not halt the batch file process.

These scripts copy the SageTV files to a new directory, then zips the contents of the new directory, then deletes the directory, and only the zipped backup remains. Because the scripts copy the SageTV files, you should be sure the drive with the backups has enough free space to make the copies and the zips. I mention this because of the caveat I mentioned above regarding fan art and the Phoenix API cache. My Phoenix cache was well over 1GB! I didn't feel it was necessary to backup the cache, so I moved it to the root of my fan art directory. I had already relocated the fan art when I set it up initially. I set it up in the root of one of my video import directories, and now the cache is there as well.

I'm running these scripts with no issues under Windows Home Server. They should work on XP and Vista (with UAC disabled) as well. I have never run SageTV under Vista with UAC, so the scripts probably will need to be altered to run correctly under this setup. Any questions or problems, post here and I'll try to help out.

[ADDED 8/9/09] - Backup scripts specifically for WHS.
I have attached a set of scripts that are geared toward WHS setups. Basically the same, except you should extract the zip to the Software share. This will give you a SageTV Backups directory structure in Software share. This allows you to store the backups in the storage pool and take advantage of folder duplication to add redundancy if desired. You should be sure to have enough free space on the D: drive to temporarily hold a copy of all of the files from the SageTV root and SageTV Common directories, plus a copy of the full zip file backup. To keep WHS from wasting time trying to possibly duplicate the temporary files, they will be put into a temporary area until zipped, then the zip will be copied to the final location in the Software share, and then the temporary area will be cleaned up. The temp directory doesn't have to be on the D: drive...you can modify the backupTempDir variable in the batch scripts to change the location.

[ADDED 8/9/09] - Updated both sets of scripts to allow easy configuration of additional files and directories to be backed up. If you have other SageTV/HTPC related files or directories you want to backup with the main SageTV files, you can now add them in fairly easily without having to code all of the copy commands yourself. Simply define the source pathname(s) and destination directory name(s) and set the appropriate variables to tell the script how many you have defined and you're all set. You can see more instructions and examples of how it works by opening the .bat files with a text editor. If you have problems or need help setting it up, post a message here.

[ADDED 8/11/09] - Updated both sets of scripts to fix problem when adding additional files and directories to the backup using the #filePath/#fileDestination, #dirPath/#dirDestination definitions. The extra quotes didn't seem to cause problems on my WHS machine, but may cause problems for other Windows OS versions, so I cleaned up the variables and copy commands. The #filePath/#fileDestination, #dirPath/#dirDestination variables should now be defined WITHOUT quotes if the quotes are causing errors for you. Also fixed a typo in the sagetv-config-backup script that failed to copy the sageclient.properties.autobackup file.
[EDIT] If you were running the WHS scripts, you only need to replace the sagetv-config-backup script. Fixed variable error in SageTV Backups.zip that affected manually added files.

[gplasky]

Nice job. I'm going to test these out tonight on my WHS.

Gerry

[peternm22]

Here's the forfiles program for those who need it: ftp://ftp.microsoft.com/reskit/y2kfix/x86/forfiles.exe

Just drop it into you Windows directory (usually C:\Windows\ ) and it should work.


Nice scripts by the way, I'll try them out when I get a chance.

[robogeek]

Originally Posted by gplasky

Nice job. I'm going to test these out tonight on my WHS.

Gerry

Thanks. If there are any other SageTV related config files I've missed, please let me know. I've also added a new set of scripts that are basically the same, but may play a little better on WHS is you want to put the backups in the storage pool instead of on the c: or d: drive. I'm also working on adding an easy set of variables to modify to add your own files and directories to the backup without having to fully code it in yourself...just set the pathnames to the files and directories and the script will back them up.

If you want to convert to the new scripts, just move your SageTV Backups directory to the Software share, download the new attachment, and extract the three updated batch files over the existing ones. Make any changes to the batch file variables you may need and that should be it. You'll now have the backups going to the storage pool and be able to take advantage of WHS folder duplication for added security

[robogeek]

Originally Posted by peternm22

Here's the forfiles program for those who need it: ftp://ftp.microsoft.com/reskit/y2kfix/x86/forfiles.exe

Just drop it into you Windows directory (usually C:\Windows\ ) and it should work.


Nice scripts by the way, I'll try them out when I get a chance.

Thanks for posting forfiles.exe link.

[robogeek]

I have posted updated scripts that now allow you to fairly easily (if you can follow simple .batch file programming) add more files and directories to the backup:

[ADDED 8/9/09] - Updated both sets of scripts to allow easy configuration of additional files and directories to be backed up. If you have other SageTV/HTPC related files or directories you want to backup with the main SageTV files, you can now add them in fairly easily without having to code all of the copy commands yourself. Simply define the source pathname(s) and destination directory name(s) and set the appropriate variables to tell the script how many you have defined and you're all set. You can see more instructions and examples of how it works by opening the .bat files with a text editor. If you have problems or need help setting it up, post a message here.

If you installed the previous scripts, just extract the new .bat files over the old ones. If you made any edits to any of the variables in the old scripts, you'll have to re-edit the new scripts as well.

[wrems]

Nice work! I like this very much. My approach was much less savvy and not nearly as efficient.

I just had a thought for adding redundancy for those not running raid or WHS storage pools or other schemes. I use a free program called DropBox quite a bit for synching files/file access on the internet. It is a pretty slick program. One could set up DB to hold your backup folders/batch files and have the zips that get created automatically uploaded to the internet as they get created by your batch files. That way in the event of a catastrophic event the files would be stored remotely, and accessible from any computer. This provides one more layer of protection.

All DB does is transfer whatever file(s) get placed in its folder into online storage. They start you off with 2gigs for free, which would be enough at least for the daily backups. The weekly backups, on the other hand, would probably exceed that size depending on how big the folder structure is.

The combination of your scripts and DropBox could be an eloquent way to protect these files.

Just a thought.

[phareous]

I'm having trouble with the config backup script when I try to add additional files

First, I have to to take the quotes off of 1fileDestination because it is including the quotes in the copy line... something like
copy path1 "path2"file""

But more than that, it looks like backupTempDir is never defined...so its throwing the copied files into my C:\ root

[robogeek]

Originally Posted by phareous

I'm having trouble with the config backup script when I try to add additional files

First, I have to to take the quotes off of 1fileDestination because it is including the quotes in the copy line... something like
copy path1 "path2"file""

Thanks for catching that. I have fixed that and will have updated scripts posted shortly.

But more than that, it looks like backupTempDir is never defined...so its throwing the copied files into my C:\ root

Not sure why it would do that. The WHS scripts have backupTempDir set near the top of the script. By default it is set to D:\SageTV Backup Temp. The script will create the directory if it doesn't exist, but the script will also need to run with high enough privileges to create the directories. Double check the backupTempDir variable to be sure it's set to a pathname that can be created on your setup and if you changed it from the default, be sure that the pathname doesn't contain any illegal characters.

[phareous]

This is the non-WHS scripts, and I don't see backupTempDir defined in these scripts (did a search)

[robogeek]

Originally Posted by phareous

I'm having trouble with the config backup script when I try to add additional files

First, I have to to take the quotes off of 1fileDestination because it is including the quotes in the copy line... something like
copy path1 "path2"file""

But more than that, it looks like backupTempDir is never defined...so its throwing the copied files into my C:\ root

Also, which OS are you running and which set of scripts are you running? I just double checked again and the extra quotes that were in the previous version of the scripts didn't cause any problems for me. I'm running Windows Home Server (WHS) and the SageTV Backup for WHS scripts. If you still run into problems, paste a copy of the top variable section of the .bat file so I can see how the variables are defined.

[phareous]

I'm running under plain Vista (no WHS)

SET SageTVDir=C:\Program Files\SageTV
SET backupDir=C:\SageTV Backups\Config (daily auto backups)
SET windowsDir=C:\WINDOWS
SET zipexeDir=C:\SageTV Backups

SET numFiles=1

SET 1filePath="c:\ProgramData\LM Gestion\LM Remote KeyMap\*.xml"
SET 1fileDestination="LMRemoteKeymap"

SET numDirs=0

[robogeek]

[EDIT] If you were running the WHS scripts, you only need to replace the sagetv-config-backup.bat file to correct the failed sageclient.properties.autobackup copy failure. Or find that line in the .bat file and replace the backupDir variable with backupTempDir. The other changes regarding quotes on file and directory names are more for consistency and don't affect the operation of the script.

If you were running the other set of scripts from SageTV Backups.zip, you only need to download the new scripts if you are adding files to the backup with the #filePath/#fileDestination, #dirPath/#dirDestination variables. Or open up each of the .bat files and find and replace all instances of backupTempDir with backupDir.
[EDIT]


[ADDED 8/11/09] - Updated both sets of scripts to fix problem when adding additional files and directories to the backup using the #filePath/#fileDestination, #dirPath/#dirDestination definitions. The extra quotes didn't seem to cause problems on my WHS machine, but may cause problems for other Windows OS versions, so I cleaned up the variables and copy commands. The #filePath/#fileDestination, #dirPath/#dirDestination variables should now be defined WITHOUT quotes if the quotes are causing errors for you. Also fixed a typo in the sagetv-config-backup script that failed to copy the sageclient.properties.autobackup file.

The drawback to .bat files... if you download the updated scripts, you'll have to copy any modifications you've made to the previous scripts to the new scripts. You should be able to copy and paste the variable block from the old scripts to the new ones if you've made extensive changes. The only thing you may need to change is deleting the quotes from the file/dir definitions of the additional files and directories you have added to the backup...but you probably only need to do this if the extra quotes are causing errors when copying.

[robogeek]

Originally Posted by phareous

I'm running under plain Vista (no WHS)

SET SageTVDir=C:\Program Files\SageTV
SET backupDir=C:\SageTV Backups\Config (daily auto backups)
SET windowsDir=C:\WINDOWS
SET zipexeDir=C:\SageTV Backups

SET numFiles=1

SET 1filePath="c:\ProgramData\LM Gestion\LM Remote KeyMap\*.xml"
SET 1fileDestination="LMRemoteKeymap"

SET numDirs=0

OK, I'm retarded I found the problem with the SageTV Backups script. Just change all instances of backupTempDir to backupDir and you should be good to go. The SageTV Backups for WHS script are OK, but with the standard SageTV Backups scripts I didn't change that variable after I copied/pasted.

[scat]

robogeek

Awesome work, I will save this for future reference

Thanks
Scat

[ChaOConnor]

Jason, just got WHS setup and Sage Running on it. I want to set these backups up, but everytime I run them, I get permission errors when it goes to zip the files up. I've tried running them directly from the console, but no luck. Any ideas? Thanks!

[dstanley]

Hmmmm ... Strange happening for me on my WHS - The BAT file was compressing my Windows folder instead of the backed up folder!?
I had to 'unquiet" the zip.exe command to see what was happening.
Then I had to add a line that would chnage to my D: drive before the zip command for it to work properly:

This is the what I had to do to 'fix' it on my system:

ECHO Compressing backup: %folderName%
REM add change to D: drive
D:
REM now the proper folder gets zipped
CD "%backupTempDir%\%folderName%"
"%zipexeDir%\zip.exe" -r -9 -q -S "%backupTempDir%\%folderName%".zip *.*
CD "%zipexeDir%"

After I did this to all the scripts then it worked properly for me ...

[robogeek]

Originally Posted by ChaOConnor

Jason, just got WHS setup and Sage Running on it. I want to set these backups up, but everytime I run them, I get permission errors when it goes to zip the files up. I've tried running them directly from the console, but no luck. Any ideas? Thanks!

Did you grab the WHS scripts or the non-WHS script? Where did you extract them?

[robogeek]

Originally Posted by dstanley

Hmmmm ... Strange happening for me on my WHS - The BAT file was compressing my Windows folder instead of the backed up folder!?
I had to 'unquiet" the zip.exe command to see what was happening.
Then I had to add a line that would chnage to my D: drive before the zip command for it to work properly:

This is the what I had to do to 'fix' it on my system:

ECHO Compressing backup: %folderName%
REM add change to D: drive
D:
REM now the proper folder gets zipped
CD "%backupTempDir%\%folderName%"
"%zipexeDir%\zip.exe" -r -9 -q -S "%backupTempDir%\%folderName%".zip *.*
CD "%zipexeDir%"

After I did this to all the scripts then it worked properly for me ...

In the task properties Task tab, put the full path to the directory where the scripts are located in the 'Start in' text box. If you put the scripts in the Software share as suggested, then the 'Start in' box should have:

"D:\shares\Software\SageTV Backups"

The mods you made accomplish the same thing, so there is no real need to do anything else on your part

[ChaOConnor]

Originally Posted by robogeek

Did you grab the WHS scripts or the non-WHS script? Where did you extract them?

I grabbed the WHS ones and put them in the Software folder.