XXL Sage Install - Page 5

ToxMox · #81 06-02-2006, 07:38 PM

I understand your point but what would the amount of tuners have any bearing on how often to back up the wiz.bin?

As long as you have a backup within a few days of the corruption you can pretty much get back show info for all your recorded shows.

EDIT: Just another point of reference. I think I've been using the same wiz.bin since September 2003 and have pretty much installed almost every single release, rc, beta, private beta since then and have never had a corruption in my file. This includes migrating my Sage server to new hardware a few times along the way.

The problem with corrupted may exist but I would imagine it is a rather small percentage of users experiencing this otherwise we would see many more posts about it than we currently do. I suppose this would make for a good poll topic.

Deadbolt · #82 06-02-2006, 08:00 PM

Quote:

Originally Posted by ToxMox

I understand your point but what would the amount of tuners have any bearing on how often to back up the wiz.bin?

As long as you have a backup within a few days of the corruption you can pretty much get back show info for all your recorded shows.

EDIT: Just another point of reference. I think I've been using the same wiz.bin since September 2003 and have pretty much installed almost every single release, rc, beta, private beta since then and have never had a corruption in my file. This includes migrating my Sage server to new hardware a few times along the way.

The problem with corrupted may exist but I would imagine it is a rather small percentage of users experiencing this otherwise we would see many more posts about it than we currently do. I suppose this would make for a good poll topic.

I believe the 20 tuner reference was because more tuners = more shows being recorded (potentially) = more changes to wiz.bin over a the same amount of time ... even if you get to it within a few days of the last backup there could be many changes lost if the file is corrupt. Even worse is the case where you are away for a week or two on vacation and your daily backup that is on a weekly rotation is overflowed with corrupt wiz.bin files.

ToxMox · #83 06-02-2006, 08:13 PM

All valid points except the weekly rotation one. When recording gigs and gigs of data every day on 20+ tuners surely he can spare the space to keep 2 months worth of wiz.bin backups.

Even if the data was kept in an open xml format or something doesn't mean it will never corrupt. Any database or any file in fact is open to being corrupted.

If it is an issue I think the people having it should start support tickets and post to each other to see if there is a common thread with the problem.

As for more changes to the wiz.bin potentially causing issues I don't seem to have a problem there either. I regularly record up to 7 shows at a time and around 3-5 times a week add and remove import directories in Sage for various reasons which is basically adding and removing ~15,000 songs and hundreds of video files to and from the wiz.bin constantly so its not like I'm a light user.

I know if I had the corrupting wiz.bin issue I would be the first one moaning about it and wanting a fix but now that I know there are rock solid configurations possible I'm not too worried. If I had the issue I'd probably get so frustrated that if I had the problem for more than a couple of months I'd either A) find another PVR app B) Format my Sage server and keep it very pristine and see if my prob goes away and if that didn't work I'd C) Start removing one piece of hardware at a time over the course of a month or so and finally if none of those work then I'd probably build a new Sage server.

Deadbolt · #84 06-02-2006, 08:29 PM

Quote:

Originally Posted by ToxMox

All valid points except the weekly rotation one. When recording gigs and gigs of data every day on 20+ tuners surely he can spare the space to keep 2 months worth of wiz.bin backups.

Even if the data was kept in an open xml format or something doesn't mean it will never corrupt. Any database or any file in fact is open to being corrupted.

If it is an issue I think the people having it should start support tickets and post to each other to see if there is a common thread with the problem.

As for more changes to the wiz.bin potentially causing issues I don't seem to have a problem there either. I regularly record up to 7 shows at a time and around 3-5 times a week add and remove import directories in Sage for various reasons which is basically adding and removing ~15,000 songs and hundreds of video files to and from the wiz.bin constantly so its not like I'm a light user.

I know if I had the corrupting wiz.bin issue I would be the first one moaning about it and wanting a fix but now that I know there are rock solid configurations possible I'm not too worried. If I had the issue I'd probably get so frustrated that if I had the problem for more than a couple of months I'd either A) find another PVR app B) Format my Sage server and keep it very pristine and see if my prob goes away and if that didn't work I'd C) Start removing one piece of hardware at a time over the course of a month or so and finally if none of those work then I'd probably build a new Sage server.

Hey, I've never had a problem either (not that it means SageTV doesn't ever cause a corruption)...I'm just thinking that is what he was getting at. I certainly don't think you should have to maintain a two month daily history of wiz.bin files to feel safe. People need to bug these problems to support. Wether it is a closed file format or an open database format...if the SageTV is the cause of the corruption it should be fixed.

GKusnick · #85 06-02-2006, 08:56 PM

Quote:

Originally Posted by src666

...a corrupted WIZ.BIN file is 100% non-recoverable...

Quote:

Originally Posted by src666

Yes, there are ways to minimize the damage of a corrupted WIZ.BIN, but there's not a way to FIX the file. And if you don't catch the damage soon enough...then you are totally screwed.

I'm not following this argument. If you have backups, then the worst case is you lose any additions since the last backup. That's not what I'd call "100% non-recoverable" or "totally screwed".

And as I noted before, you don't need daily snapshots of wiz.bin to be backed up. Nielm's XML file generator exports all the important show info as it's generated, so if you're running that plugin, then you have up-to-the-minute backups without any wiz.bin snapshots. (It's true that Niel hasn't yet written the code to re-import those XML files back into a clean wiz.bin, but in principle anybody could write that code, so it's not accurate to say there's no way to recover. There is a way; it just involves writing some code.)

If you have no backups of any kind, then yes, you're screwed if wiz.bin gets corrupted, or if your hard drive crashes, or if your house burns down. That's why backups were invented, and anybody who doesn't have some sort of backup strategy for data they care about (including but not limited to wiz.bin) is taking needless risks. That's nobody's fault but their own.

blade · #86 06-02-2006, 08:59 PM

I don't think anyone is saying that corruptions don't happen, but restoring the wiz.bin 5 times in 6 months isn't typical. The people that are going to post are usually the ones who are having problems. For the handful that have the problem there is likely hundreds or thousands of others that have never had a corrupt wiz.bin, but you're not going to see all of them post to say everything is working perfectly.

I totally agree that regular backups should be kept, but at the same time I don't think it's something that everyone should expect to be dealing with on a regular basis.

Opus4 · #87 06-02-2006, 09:41 PM

If you do get a corrupted wiz.bin, save it so the developers can see if they might be able to figure out what happened to it. There have been fixes in the past, so any remaining errors should be reported.

(I've been uisng my original wiz.bin since around Oct 2003.)

- Andy

src666 · #88 06-02-2006, 10:59 PM

OK, once more for clarity.

If you can't recover any data from a corrupt WIZ.BIN file, then that file is 100% non-recoverable. Period.

If you can recover most of your data by reverting to a backed up WIZ.BIN, then the pain is minimized. This doesn't change the fact that WIZ.BIN corruption is non-recoverable.

If you are configuring a system that requires 20+ tuners (I believe the original requirement was more than 40 SD, 10 HD, ?? Satellite), then you are probably configuring a system that does a TON of recording. Not to mention a ton of watched flags, a ton of like/don't like settings, favorites, etc., etc.. All of which go into the WIZ.BIN file.

Regarding backups - if I have to go back a week to find a good WIZ.BIN, that means that I have lost between 20 (off season) and 40 (in season) shows. That hurts. I can't imagine how many shows, flags, etc. that this guys system would lose.

Talking about running 2 months of WIZ.BIN backups as a solution is absolutely ridiculous. How many shows do you record, watch, discard, flag like/don't like, etc. in 1 month, much less 2? How many do you think would pass through the system under discussion in this thread? You would probably recover most of your favorites, and your watched history (which isn't inconsequential), but it does very little to ameliorate the pain of having all of your current show data disappear.

Yes, having a good backup strategy for the WIZ.BIN file is essential. It's so essential that many of the people who have never lost one, and feel confident that it's not a problem, have such a strategy.

But that's nowhere near an adequate replacement for having the ability to recover data from a corrupt file. And poo-pooing the valid concerns of those of us who have lost our WIZ.BIN files doesn't really add much weight to the argument.

Frankly, I really can't see why you guys have so much invested in dismissing the issue. If it never happened to you, then great - enjoy your good luck. But crashes happen. Bugs happen. Just plain strange things happen. And there are quite a few of us who lost our WIZ.BIN files, and warning this fellow of that possibility seems to be a pretty rational and frankly neighborly thing to do. Telling him not to worry about it seems like telling your friends not to wear their seatbelts just because you've never been in an accident. And dismissing the experiences of those of us who have needed seatbelts at one time or another certainly doesn't add to the value of the discussion.

GKusnick · #89 06-03-2006, 02:32 AM

Quote:

Originally Posted by src666

But that's nowhere near an adequate replacement for having the ability to recover data from a corrupt file.

This is the part I don't get. Why is it so important to be able to recover data from a corrupt file? Wouldn't you rather recover it from an uncorrupted backup, or an external XML file? If my hard drive crashes, I don't insist on being able to get data off it. I do insist on having recent copies of the data in other places so that the loss is minimized. If my car crashes, I don't insist that it be repaired at any cost. At some point, it makes more sense to just junk it and let the insurance company buy me a new one.

Nobody is saying that you shouldn't worry about wiz.bin corruption -- just that you shouldn't obsess about it. Tools are available to ensure that little or no data is permanently lost as a result of a wiz.bin crash, either by backing up the file itself at regular intervals, or by exporting the data it contains to a different file format. Warning people of the risks might be the neighborly thing to do, but so is showing them how to minimize those risks.

blade · #90 06-03-2006, 05:13 AM

Quote:

Originally Posted by src666

Frankly, I really can't see why you guys have so much invested in dismissing the issue. If it never happened to you, then great - enjoy your good luck. But crashes happen. Bugs happen. Just plain strange things happen. And there are quite a few of us who lost our WIZ.BIN files, and warning this fellow of that possibility seems to be a pretty rational and frankly neighborly thing to do. Telling him not to worry about it seems like telling your friends not to wear their seatbelts just because you've never been in an accident. And dismissing the experiences of those of us who have needed seatbelts at one time or another certainly doesn't add to the value of the discussion.

When the majority of people never had the problem, I don't consider it "good luck". More like you and a few others had bad luck. You're making it sound like we're the lucky few that never had the problem, when that's simply not the case.

I don't think anyone is dismissing the fact that some have had problems, but I wouldn't say it's a wide spread problem that only the lucky few don't encounter. Your example goes both ways. Just because your car breaks down doesn't mean everyone that owns that model should be afraid to go for a drive.

There's nothing wrong with warning the guy to make regular backups, but to try and make it sound like he's going to be constantly reverting back to an older wiz.bin and losing information isn't true. If someone is having those types of problems on a regular basis they have a problem somewhere because that's not typical.

src666 · #91 06-03-2006, 09:22 AM

Quote:

Originally Posted by GKusnick

This is the part I don't get. Why is it so important to be able to recover data from a corrupt file? Wouldn't you rather recover it from an uncorrupted backup, or an external XML file? If my hard drive crashes, I don't insist on being able to get data off it. I do insist on having recent copies of the data in other places so that the loss is minimized. If my car crashes, I don't insist that it be repaired at any cost. At some point, it makes more sense to just junk it and let the insurance company buy me a new one.

If the corruption is minor, then in an open format (XML, whatever) you can save everything but the part that got corrupted. Maybe that's just one show of data, maybe it's just some configuration data, maybe it guide data. Whatever. But if you can save everything else, and transplant the data into a backup or new WIZ.BIN, then you are ahead of the game. Heck, maybe it's a major corruption and you can't use any of it. Having the opportunity to even try is the important part. Do you really want to have to "revert" to your second car when a taillight goes out?

Quote:

Originally Posted by GKusnick

Tools are available to ensure that little or no data is permanently lost as a result of a wiz.bin crash, either by backing up the file itself at regular intervals, or by exporting the data it contains to a different file format. Warning people of the risks might be the neighborly thing to do, but so is showing them how to minimize those risks.

Yes, backup is an obvious bandaid, if you know that you need to do it. The first time my WIZ.BIN was lost I didn't know. I just wanted to let the guy know of the issue. I think if you go back and read the original postings you will see that all I was doing was informing him that this file is critical, and he would be putting unusally high stress on it.

The simple fact is that we do not know what causes WIZ.BIN corruption. I hope that it's not a bug in SageTV. If it is, I hope it is incredibly rare. Ideally, it is caused by hardware problems, not software. But we don't know. Without knowing, you have to be aware of the _possibility_ that it is a bug, and that someone building a mondo absurdo system is going to be thrashing that file hard. Heck, even if it isn't a bug, that file is still the most likely point of corruption in a system working as hard as his will be.

Quote:

Originally Posted by blade

When the majority of people never had the problem, I don't consider it "good luck". More like you and a few others had bad luck. You're making it sound like we're the lucky few that never had the problem, when that's simply not the case.

I don't think anyone is dismissing the fact that some have had problems, but I wouldn't say it's a wide spread problem that only the lucky few don't encounter. Your example goes both ways. Just because your car breaks down doesn't mean everyone that owns that model should be afraid to go for a drive.

There's nothing wrong with warning the guy to make regular backups, but to try and make it sound like he's going to be constantly reverting back to an older wiz.bin and losing information isn't true. If someone is having those types of problems on a regular basis they have a problem somewhere because that's not typical.

As I mentioned above, go back and read the posts. It wasn't until people started saying "don't worry, it's not a problem" and "I never had any problems" that the conversation took a left turn. I never once claimed or implied that he would be "constantly reverting back to an older WIZ.BIN". It happened to me twice in the past year+, and the first time I didn't have any backups. The second time was my own damn fault, for not watching where I was installing things. Some have had better luck, some have had much, much worse.

My intention was to provide valid information, and that information is still valid:

- WIZ.BIN is 100% proprietary, and you can't fix it if something goes wrong
- WIZ.BIN is 100% necessary to proper operation of SAGE
- A system with 20-60 tuners is going to be thrashing WIZ.BIN
- BE CAREFUL WITH WIZ.BIN

What's so wrong with any of that?

blade · #92 06-03-2006, 10:42 AM

Quote:

Originally Posted by src666

As I mentioned above, go back and read the posts. It wasn't until people started saying "don't worry, it's not a problem" and "I never had any problems" that the conversation took a left turn. I never once claimed or implied that he would be "constantly reverting back to an older WIZ.BIN". It happened to me twice in the past year+, and the first time I didn't have any backups. The second time was my own damn fault, for not watching where I was installing things. Some have had better luck, some have had much, much worse.

My intention was to provide valid information, and that information is still valid:

- WIZ.BIN is 100% proprietary, and you can't fix it if something goes wrong
- WIZ.BIN is 100% necessary to proper operation of SAGE
- A system with 20-60 tuners is going to be thrashing WIZ.BIN
- BE CAREFUL WITH WIZ.BIN

What's so wrong with any of that?

Mark SS stated that he had restored his wiz.bin 5 times since December. That's when he was told that he had a problem somewhere. Restoring a wiz.bin that often is not typical. Some of us spoke up to say we have not had those problems. Should we all have remained silent and let readers believe that it's normal to restore the file on a monthly basis? There's nothing wrong with you pointing out that some people have problems and there's nothing wrong with others posting to say the problem isn't as widespread as one might be lead to believe from the posts you and MarkSS made. Personally I see it as bringing a little balance to the discussion so that it doesn't incorrectly appear that everyone has the problem and is just living with it.

You are the one claiming we are dismissing the problem because we disagree with you as to the overall severity of it. I don't think anyone has said that it's not an issue and not to worry about the file.

src666 · #93 06-03-2006, 01:50 PM

Quote:

Originally Posted by blade

You are the one claiming we are dismissing the problem because we disagree with you as to the overall severity of it. I don't think anyone has said that it's not an issue and not to worry about the file.

That's the strange part - we don't disagree about the severity of the problem. I don't think this happens to everyone. I don't think it happens to most users. But it happens to enough users that everyone should know about it, and prepare for it. For you and me (and 99.999% of all Sage users), a good backup strategy is all that's needed to prevent it from being a crippling problem.

But for THIS system, being proposed on THIS thread, a single WIZ.BIN corruption can be a crippling event, even with a reasonable backup strategy.

Think of it this way - in how many other threads have I been posting about WIZ.BIN corruption? Very few, for the most part only those where it is brought up by someone else, and not for a long time. That's because for most people it's not that big a deal, or that likely to happen.

Quote:

Originally Posted by blade

Some of us spoke up to say we have not had those problems. Should we all have remained silent and let readers believe that it's normal to restore the file on a monthly basis?

The system being proposed here is exceptional in many ways, and it's vulnerability to WIZ.BIN corruption is likewise exceptional. Is your system in ANY way comparable to the one being discussed? Do you really think that your lack of problems directly translates to this system? I don't. Nor do I think that my lack of recent, unself-inflicted, problems translates to this system.

But I do think that he needs to be aware of anything that can knock his system out. And a bad WIZ.BIN rates right up there as far as crippling events.

GKusnick · #94 06-03-2006, 01:53 PM

Quote:

Originally Posted by src666

WIZ.BIN is 100% proprietary

To be accurate, the file format is undocumented, but the APIs for reading and writing it are public. Anybody with reasonable Java skills can write code to extract data from wiz.bin (perhaps even a partially damaged one; I haven't tried it) into an arbitrary external format (such as XML) and import that data back into a clean wiz.bin. You don't need to understand the binary file format to make humanly-readable copies of the data it contains.

If it's the closed file format that bothers you, then I recommend (again) Nielm's XML file generator, which exports show info in XML form, dynamically, so it's always current with wiz.bin.

blade · #95 06-03-2006, 04:03 PM

Quote:

Originally Posted by src666

The system being proposed here is exceptional in many ways, and it's vulnerability to WIZ.BIN corruption is likewise exceptional. Is your system in ANY way comparable to the one being discussed? Do you really think that your lack of problems directly translates to this system? I don't. Nor do I think that my lack of recent, unself-inflicted, problems translates to this system.

Do you really think even a fraction of the stuff mentioned in this thread will actually be doing anything? Whether you have 4 or 40 tuners there is still only so much content to be recorded and only so much TV that one family can watch.

You've said that even losing a few days worth of wiz.bin info would be disastrous for this system. How do you know? Is he really going to be recording that much more than some of the Sage users with 4-12 tuners? If it were going in a dorm or other shared housing, then I might agree with you; however, I just don't believe there is any way in a single residence home that the system is going to see anywhere near that much activity. Unless of course he's going to turn on IR and let it record everything that airs just for the heck of it.

As I've already said, I agree that he should take precautions and should be aware of the possibility. I don't really understand what else you expect people to say. We pretty much all agree that it's a possible problem, but that the majority of people don't have problems with corrupt wiz.bin files.

ToxMox · #96 06-03-2006, 04:15 PM

Just wanted to mention. If you lose the wiz.bin and restore a backup from three days before the only thing you'd lose is any watched status or recordings scheduled or favorites added since the backup was made. If you have a wiz.bin from 3 days ago the data existed in the EPG back then for the shows recorded over the next three days so all those recordings would/should import automatically and show up like normal recorded shows with all of their data because of the ID number in the file names.

src666 · #97 06-03-2006, 04:46 PM

Quote:

Originally Posted by blade

Do you really think even a fraction of the stuff mentioned in this thread will actually be doing anything? Whether you have 4 or 40 tuners there is still only so much content to be recorded and only so much TV that one family can watch.

If he needs 40-60 tuners (which I personally think is ludicrous), then he must think he will be using a large proportion of those tuners at any given time, or at least enough of the time to make it economically worthwhile.

For all we know, he's going to be producing the next version of "OutFoxed", and missing even a single airing would be a critical failure. The fact remains, we just don't know what real load he intends to place on the system. While I personally think that you are right, and there just isn't that much to watch, I'm not going to assume that he has no real use for such a system - he seems to think he does.

Quote:

Originally Posted by blade

As I've already said, I agree that he should take precautions and should be aware of the possibility. I don't really understand what else you expect people to say. We pretty much all agree that it's a possible problem, but that the majority of people don't have problems with corrupt wiz.bin files.

I didn't really expect people to say much of anything. I pointed out that the WIZ.BIN file _has_ been known to become corrupt (for whatever reason), that it was critical to the system, and that he needed to take appropriate steps. Others felt that this was an alarmist position, and sought to reassure him that they have never had any problems, so he shouldn't worry about it too much. True, he may never have a problem, but if he does, and he's not prepared, then it will be a big problem.

In general, I don't think there is a serious disagreement among us. But when applied to this specific configuration, I believe that (1) WIZ.BIN is more likely to have trouble than a normal system (how much more I have no idea) and (2) in a system of this scale, and WIZ.BIN problem can be a BIG problem.

src666 · #98 06-03-2006, 04:53 PM

Quote:

Originally Posted by ToxMox

Just wanted to mention. If you lose the wiz.bin and restore a backup from three days before the only thing you'd lose is any watched status or recordings scheduled or favorites added since the backup was made. If you have a wiz.bin from 3 days ago the data existed in the EPG back then for the shows recorded over the next three days so all those recordings would/should import automatically and show up like normal recorded shows with all of their data because of the ID number in the file names.

This has not been my experience. Towards the middle of last year I performed an software upgrade and the Sage system ended up being installed under the \Program Files\SageTV tree, instead of the \Program Files\Frey Technologies tree where my existing setup lived. I thought I had done something terribly wrong and lost everything. I set up a couple of recordings so I wouldn't miss that night's episodes, and assumed that everything else would be stuck in "Archived Recordings".

Once I found out what had happened, I performed the upgrade on the correct directory, and upon restarting all of my new recordings dropped into "Archived" status. Even though the guide information was pretty current (it was the next day) in the original WIZ.BIN file, it never hooked things up with the new files. I ended up with a couple of files that didn't have information - not all that big a deal for me, but certainly inconvenient.

Maybe this was an aberration, or maybe they have changed things so it works better now, or maybe I just did something odd. But as I said, my direct experience with the problem doesn't support your statement.

ToxMox · #99 06-03-2006, 05:03 PM

That may have been before Sage was changed to automatically try and match files in the recordings directory with items in its database. I feel like it was around the middle of last year that it changed.

src666 · #**100** 06-03-2006, 05:10 PM

Quote:

Originally Posted by ToxMox

That may have been before Sage was changed to automatically try and match files in the recordings directory with items in its database. I feel like it was around the middle of last year that it changed.

Good to know that they addressed that one. So much has changed since then that it's kind of tough keeping track of it all.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)