Tutorial: SageTV on WHS with 64K clusters in the drive pool...

[S_M_E]

You're right, I divided by 64 not 65536 but that is still a lot (3,276,800) of tombstones.

Good catch...

EDIT:
Just based on my file system (using number of files per total TB) I'd need a ~72TB WHS to get that many tombstones and I'd still have enough free space on my D to add ~800,000 more tombstones. That's pushing it a bit though, if a person used a 500G or larger system drive that number goes up even more. I think it's safe to say that 64K clusters aren't that big of a deal, space wise, for most people.

[crarbo1]

S_M_E,
What programs are known to format the D partition with data already on it? I'm considering formatting the D partition to 64K but don't know the programs that can do it while D is active.

Oh, did you ever buy that Norco case? If so, how do you like it?

Thanks,
Chuck

Originally Posted by S_M_E

You're right, I divided by 64 not 65536 but that is still a lot (3,276,800) of tombstones.

Good catch...

EDIT:
Just based on my file system (using number of files per total TB) I'd need a ~72TB WHS to get that many tombstones and I'd still have enough free space on my D to add ~800,000 more tombstones. That's pushing it a bit though, if a person used a 500G or larger system drive that number goes up even more. I think it's safe to say that 64K clusters aren't that big of a deal, space wise, for most people.

[S_M_E]

Originally Posted by crarbo1

S_M_E,
What programs are known to format the D partition with data already on it? I'm considering formatting the D partition to 64K but don't know the programs that can do it while D is active.

There is a slight difference in format vs convert. The "format method" outlined in the tutorial and discussed in later pages is destructive. It involves backing up the D partition fully, formatting and restoring. A number of utilities will work for that, including the built-in Disk Management utility. I'm not a fan of that option.

The "convert method" is non-destructive (although you still might want to fully backup D just in case (I didn't)) and I've only verified that Acronis Disk Director Server 10.0 works for converting but other utilities that support win2k3 server may also convert. Paragon Partition Manager 8.5 Enterprise Server Edition, for example might also work. There might also be free trials of those programs. There are a number of drive utilities that are freeware that MIGHT also have the convert option but I haven't tested any of them. It's reported that "Parted Magic" doesn't have a convert option but it can resize C (another tutorial) at install time. Acronis DD is expensive but I know it works. You can try other freeware utilities if you want to experiment.

I put the Norco project on hold, the deal that Newegg is offering isn't the deal I thought it was. I do still plan on going with that case though.

[crarbo1]

I would only go with the convert method. I saw that you had used Acronis for that but I thought you may have had others tell you that other programs converted it fine as well. I may hold off until other proven methods exist. Don't know if I want to spend that kind of money just for this one job.

What was wrong with NewEgg's deal? When I need to upgrade to a new case, I really think that that is the one I will be going with. NewEgg is really my choice when it comes to hardware, so that is why I wonder what is wrong with their deal.

Thanks,
Chuck

Originally Posted by S_M_E

There is a slight difference in format vs convert. The "format method" outlined in the tutorial and discussed in later pages is destructive. It involves backing up the D partition fully, formatting and restoring. A number of utilities will work for that, including the built-in Disk Management utility. I'm not a fan of that option.

The "convert method" is non-destructive (although you still might want to fully backup D just in case (I didn't)) and I've only verified that Acronis Disk Director Server 10.0 works for converting but other utilities that support win2k3 server may also convert. Paragon Partition Manager 8.5 Enterprise Server Edition, for example might also work. There might also be free trials of those programs. There are a number of drive utilities that are freeware that MIGHT also have the convert option but I haven't tested any of them. It's reported that "Parted Magic" doesn't have a convert option but it can resize C (another tutorial) at install time. Acronis DD is expensive but I know it works. You can try other freeware utilities if you want to experiment.

I put the Norco project on hold, the deal that Newegg is offering isn't the deal I thought it was. I do still plan on going with that case though.

[S_M_E]

Originally Posted by crarbo1

I would only go with the convert method. I saw that you had used Acronis for that but I thought you may have had others tell you that other programs converted it fine as well. I may hold off until other proven methods exist. Don't know if I want to spend that kind of money just for this one job.

What was wrong with NewEgg's deal? When I need to upgrade to a new case, I really think that that is the one I will be going with. NewEgg is really my choice when it comes to hardware, so that is why I wonder what is wrong with their deal.

The only other person to "convert" was heffe2001 and he used Acronis too but I don't know if it was a trial or the full version.

The price on the case is fine but I was going to place a large order (4K-5K) and not everything was that cheap at Newegg. That and they used to have no payments for 12 months but that went away too. I'll wait and see...

[rmac321]

Perhaps I'm seeing one of the issues relating to SATA vs IDE. I finally set up a second server to play with WHS so I can take my time and not disturb my working XP SageServer.

First issue. I found a good deal the other day for WD Blue 640's, so I grabbed three. (Of course Newegg had a coupon code for them 2 days later, but that's a different story.) I can see why it would be nice to have a distinct size HDD for the first/system/primary drive. Since these HDD's all have the same model number, they all look the same in the drive management utilities.

WHS install: I decided that since this was an experiment, why not experiment and since the BIOS saw them as SATA's I left them SATA's. WHS installed just fine relative to the HDD's. The only significant driver issue was needing to add a LAN driver since the onboard LAN didn't show initially. Everything installed fine, and WHS worked as expected. I added the DiskManagement Add-In, which added and seems to work, but then comes my 2nd issue.

2nd Issue: The Disk Management Add-In shows all 3 drives, but the two secondary drives show no files on them. It occurs to me that maybe I'm looking in the wrong place. On the Disk Management screen in the console, both disks show "0% used". If I pull up the details for either drive, they both show "0GB used". I decided that since this is an experiment and I have no data to lose, and since the add-In showed no data on either drive, I just quick formatted both of them with 64k clusters. I can confirm that at least one of them did have data on it, since the format did corrupt the files in the software share. I then added a bunch of photos to the photo share to see if I screwed everything up and will note that all the files copied ok and can be read from my laptop from the photo share directory. But, even after adding many MB's of files to the Photo share, the DiskManagement Add-In still reported that the 2 data drives have no usage. Next I removed and re-added each drive, but DiskManagement Add-In still shows no usage. Any thoughts? Is this part of the SATA/IDE issue?

I note that WHS and the Disk Management Add-In are now showing my disks as ATA/IDE, not SATA, so I don't know if that means that my BIOS needs to be changed to match this or not.

[S_M_E]

1) If you didn't have to specify RAID/AHCI drivers during install then your controller must be supported in WHS, which is good, or your BIOS just calls IDE mode SATA. Either way, if you don't have to specify drivers at install, it will make a future recovery, server reinstall more likely to succeed.

2) The Disk Management add-in has a wireframe option so you can see which drives go where in your case. The sooner you make a wireframe the easier it is to maintain when you add more drives.

3) It sounds like you didn't follow the 64K tutorial...

4) How much data did you copy over to the pool and did you copy to the UNC path?

5) The ATA/IDE thing is normal and should just be ignored. All of my SATA drives are reported as "Internal (ATA)" or IDE drives. I think your BIOS just calls IDE mode SATA, what you want to avoid on WHS is AHCI or RAID modes because those complicate things.

[rmac321]

Originally Posted by S_M_E

1) If you didn't have to specify RAID/AHCI drivers during install then your controller must be supported in WHS, which is good, or your BIOS just calls IDE mode SATA. Either way, if you don't have to specify drivers at install, it will make a future recovery, server reinstall more likely to succeed.

OK, so it sounds like as far as BIOS is concerned, functionally I'm ok for now, but I still wonder if it has anything to do with my free space reporting issue.

Originally Posted by S_M_E

2) The Disk Management add-in has a wireframe option so you can see which drives go where in your case. The sooner you make a wireframe the easier it is to maintain when you add more drives.

I made a simple wireframe, but since the drives are given pseudo IDE addresses, I have to trust that Disk 0 in WHS = Sata 1 on MB, Disk 1 in WHS = SATA 2 on MB, and Disk 2 in WHS = SATA 3 on MB. This is where the 3 equal size disks caused some consternation. I knew which order I installed, but since they are now renamed, I have to trust that they are booting in that order.

Originally Posted by S_M_E

3) It sounds like you didn't follow the 64K tutorial...

I did follow the tutorial up to the point that I couldn't tell which drive had the data on it in the Disk Management Add-In. I did step 5 before step 4 since that is done from the Shared Folders tab. My problems started at step 7. At step 6 I was showing no activity on any drive, so I went to 7, but the free space on both drives was exactly the same and was exactly the same as their total capacity. Just to clarify terminology, the only place Disk Management Addin reports "free space" is in the details window. In the main tab window it displays "% used" and in my case that is 0 for both drives. In the details window for both drives "free space" is exactly the same as "capacity". Even right now, I can see activity on disk 2, but it still reports 0% used and in the details shows the entire contents as free. OTOH, if I open the Computer Management\Disk Management (the one used in step 8) I can see a small difference in free space reported, but not in the Add-In.

FWIW, the addin also reports that my system drive has exactly 20GB used, which is what I would expect, correct?

So, yes, I followed the tutorial up to the point that it no longer corresponded to what I was seeing. That's when I decided to goof around and see what happened.

Originally Posted by S_M_E

4) How much data did you copy over to the pool and did you copy to the UNC path?

The Shared Folders tab reports that I have 53.73MB in the photo share. I do not have duplication enabled on that share at the moment.

Yes, I use UNC paths. I've had an MSS for over a year, so I always do copying/moving/transferring from My Network Places. In the present case I did the transfer in File Explorer on my Laptop, not in the Remote Desktop or directly on the server.

Originally Posted by S_M_E

5) The ATA/IDE thing is normal and should just be ignored. All of my SATA drives are reported as "Internal (ATA)" or IDE drives. I think your BIOS just calls IDE mode SATA, what you want to avoid on WHS is AHCI or RAID modes because those complicate things.

I may still play with the BIOS to see how it affects the addin issue I'm seeing.

You wondered the other day what the MSS uses. You can hot swap with the MSS and the drives are reported as SCSI, so they must have some form of AHCI implemented, no?.

[S_M_E]

1) I wouldn't worry about the BIOS, that has nothing to do with your free space issue.

2) You can shut down, unplug the pooled drives, start up, note position of the system drive, shut down, reconnect 1 pooled drive, start up, add to wire frame and repeat for the rest of the pool...to be sure. Also, the add-in should (not always though) give the serial number of the drive, which is unique. That would be easier.

3) If you had followed it you wouldn't have been able to format the data on the SW share.

4) 53M out of 640G is probably not enough to report in the add-in or the drive extender didn't have time to run, add a couple of gigs, let the drive extender run (or force it) and see what it says.

5) You can play with the BIOS setting but you'll have to reinstall each time for it to see the drives.

I have no idea what the HP does. I assumed it was IDE mode to make it easier but the reporting as SCSI might just be a different controller too. They might have slipstreamed their drivers into their recovery DVD. *shrug*

[mangriotis]

Originally Posted by S_M_E

2) You can shut down, unplug the pooled drives, start up, note position of the system drive, shut down, reconnect 1 pooled drive, start up, add to wire frame and repeat for the rest of the pool...to be sure. Also, the add-in should (not always though) give the serial number of the drive, which is unique. That would be easier.

Speaking of temperature / serial number reporting (pardon the thread jack), I now have 5 HDD in the pool and one outside for backing up the system drive. This is on a Gigabyte GA-P965-DS3 mobo. 4 of the HDD are on the natively supported SATA controller and are recognized no problem by WHS (as 'Internal (ATA)'). The system drive and backup drive are on attached via IDE cable (PATA), and do not display temperature or serial information. In fact, it seems as though SMART is not enabled on these drives (even though I checked in BIOS and SMART is enabled). Any ideas? (Also, if you prefer, I can take my partly related questions to another forum, just point me to a more applicable place please...)

[S_M_E]

Originally Posted by mangriotis

Speaking of temperature / serial number reporting (pardon the thread jack), I now have 5 HDD in the pool and one outside for backing up the system drive. This is on a Gigabyte GA-P965-DS3 mobo. 4 of the HDD are on the natively supported SATA controller and are recognized no problem by WHS (as 'Internal (ATA)'). The system drive and backup drive are on attached via IDE cable (PATA), and do not display temperature or serial information. In fact, it seems as though SMART is not enabled on these drives (even though I checked in BIOS and SMART is enabled). Any ideas? (Also, if you prefer, I can take my partly related questions to another forum, just point me to a more applicable place please...)

Not every question here is related to the tutorial but I really can't help much with PATA drive questions. Also, I don't recommend PATA drives for the system nor the pool. I'd suggest getting a SATA card to add more ports before using a PATA system/pool drive. PATA drives are fine for backups, although I'd connect them externally.

You might ask at the forum for the DM add-in though:
http://forum.wegotserved.com/index.php?showforum=68

[rmac321]

Originally Posted by S_M_E

2) You can shut down, unplug the pooled drives, start up, note position of the system drive, shut down, reconnect 1 pooled drive, start up, add to wire frame and repeat for the rest of the pool...to be sure. Also, the add-in should (not always though) give the serial number of the drive, which is unique. That would be easier.

I did similar. I removed from the pool and let the add-in confirm my locations.

S/N:The add-in gives me 1 out of 3 serial numbers. There is some discussion regarding that on the We Got Served site. Basically out of the authors control.

Originally Posted by S_M_E

3) If you had followed it you wouldn't have been able to format the data on the SW share.

You'll have to clarify that a little. WHS will let you format any data drive. It asks if you're sure, but there is no prevention from formatting a drive containing data. I acknowledge I went off-script, but as I said, I didn't go off the tutorial until the add-in didn't give me any indication which of the 2 data drives contained the data. As I said earlier, it was clear something was already wrong, so I decided to goof around a little before I started over.

Originally Posted by S_M_E

4) 53M out of 640G is probably not enough to report in the add-in or the drive extender didn't have time to run, add a couple of gigs, let the drive extender run (or force it) and see what it says.

I guess this is why I am confused by your responses. The tutorial uses 2 1TB drives. I used 640GB drives. The tutorial would indicate that just the files in the software share should be enough for the add-in to indicate that there is data on one drive. If that is true of 1TB drives, it should work for 640GB.

This would also be a good time to ask a question about something else in the tutorial. Perhaps I misunderstand and this will straighten me out. Why do you leave duplication on for the software share? Doesn't the software share act like any other share saving its main copy to one secondary drive and then duplicating to a second secondary drive? Wouldn't it be better to turn off duplication on the software share just like all the others so that DE doesn't attempt to start writing those duplicates just as you start to format a re-added drive?

Originally Posted by S_M_E

5) You can play with the BIOS setting but you'll have to reinstall each time for it to see the drives.

Isn't that part of the fun? Seriously though, that's why I posted in the first place. I was using the tutorial, this is the tutorial thread and when I got to step 7, what I saw was not what the tutorial told me I would see. I was just trying to find out if I was misreading something or if the add-in was not behaving as it should.

FWIW, I just copied an HD mpg to the whs server and now I finally have an indication of some useage on Drive 2, so the add-in is working, it just needs more data than what is preloaded in the software share to show any filespace being used. In light of that, wouldn't it be better to change step 7 of the tutorial so you remove and re-add the drive like you do for the other drive in steps 12-14 before formatting instead of trying to determine which has files on it?

While the add-in is a cool bit of software, it doesn't seem to add any value in terms of the tutorial when it comes to large drives. Is it possible that an earlier version of the add-in reported usage and/or free space in smaller increments?

[S_M_E]

2) True, the WGS forums mentions the serial issue, which is why I said not always.

3) Right, but if you remove the drive from the pool first and let DE finish the migration. Then when you add the drive back to the pool and then format it *immediately*, there can't be any data on it. I suggesting leaving the SW share duplicated so the data would be safe. Not following the tutorial, however, can put the data at risk so perhaps it's better to turn off all duplication. I'll have to think about it for a while...

4) DE runs on a schedule, if you didn't wait for it to balance (or force it to balance), then the DM add-in may not have up to date info but the Computer Management\Disk Management utility will. When I wrote the tutorial I already had .msi files in the software share too so maybe that made a difference. The reason I suggested adding *more* to the pool is that you had already went off the path of the tutorial and added data.

5) The trial is the time to play with it, I'm not saying you shouldn't just warning that it would be needed...

You always remove and re-add existing drives before formatting them.

I don't know if version changes made a difference in the add-in, I'm not sure how sensitive it is.

[rmac321]

Originally Posted by S_M_E

The reason I suggested adding *more* to the pool is that you had already went off the path of the tutorial and added data.

I'm glad you did push to add more data. I almost didn't and assumed that the add-in wasn't working. It's nice to know it is working.

Originally Posted by S_M_E

2) You always remove and re-add existing drives before formatting them.

Actually, my gut told me to remove and re-add before the first format because that would seem like the best way to insure an empty drive, but I wanted to follow the tutorial precisely the first time through and the tutorial does not instruct you to remove and re-add the first drive you format. It only does it for the second drive.

step 7, determine empty drive using add-in
step 8, open Computer management
step 9, open disk management
step 10, Format the empty drive

[S_M_E]

Interesting, I'll have to update that to make sure other people remove/re-add the drive first to make sure that no data is left on the drive. Optionally, I could have them turn off all duplication and remove all pooled drives, adding and formatting one at a time to 64K...

Here's what I think happened:
My software share must have been on D and one of the pooled drives and that's why only one of my pooled drives showed empty. Evidently, your software share was on both of your pooled drives and when you formatted it, it hurt the software share. Funny how nobody else had this same problem though.

[rmac321]

Originally Posted by S_M_E

Interesting, I'll have to update that to make sure other people remove/re-add the drive first to make sure that no data is left on the drive. Optionally, I could have them turn off all duplication and remove all pooled drives, adding and formatting one at a time to 64K...

Here's what I think happened:
My software share must have been on D and one of the pooled drives and that's why only one of my pooled drives showed empty. Evidently, your software share was on both of your pooled drives and when you formatted it, it hurt the software share.

It's possible that the only reason I really damaged it is because I intentionally went off the tutorial and didn't remove the 2nd drive before formatting it so I could see what would happen. Since you have duplication on in the tutorial, it's possible that people could accidentally kill the software on the first data drive but when they remove the second data drive, the duplicate would be preserved and transferred to the already reformatted first drive. Perhaps the duplication has made it unnecessary to remove and re-add the first drive as long as DE has had time to duplicate the software share. If WHS has the tombstone and one of the two copies of the file, would it just automatically recreate the 2nd file without any user input?

Originally Posted by S_M_E

Funny how nobody else had this same problem though.

Just to be clear, my problem never was the damaged software share. That was an anticipated result. My only problem was that the Disk Management Add-In didn't report as I had expected it to.

[S_M_E]

Originally Posted by rmac321

It's possible that the only reason I really damaged it is because I intentionally went off the tutorial and didn't remove the 2nd drive before formatting it so I could see what would happen. Since you have duplication on in the tutorial, it's possible that people could accidentally kill the software on the first data drive but when they remove the second data drive, the duplicate would be preserved and transferred to the already reformatted first drive. Perhaps the duplication has made it unnecessary to remove and re-add the first drive as long as DE has had time to duplicate the software share.

I've updated the tutorial, just in case. Now the used/free space reported by the add-in is irrelevant but you can still use it to see disk activity.

[Peter_h]

I'm plannning on doing a server reinstall very soon.

I'm planning to use the Xfiles installer from Wegotserved to do the reinstall. It includes a partition tool "Easeus Partition Manager" that can be loaded after the PE to resize the C: partition.

Would this be a good time to go ahead and format the d: partition to 64k cluster?

[S_M_E]

Originally Posted by Peter_h

I'm plannning on doing a server reinstall very soon.

I'm planning to use the Xfiles installer from Wegotserved to do the reinstall. It includes a partition tool "Easeus Partition Manager" that can be loaded after the PE to resize the C: partition.

Would this be a good time to go ahead and format the d: partition to 64k cluster?

I've seen it but never used it.

If that tool will "convert" the D partition to 64 clusters after install, then yes, that's a good time. At the time that the C partition is stretched (using my other tutorial) the D partition doesn't even exist yet.

EDIT:
Wait...
If you're doing a "server reinstall" instead of a "new install," it's too late to stretch the C partition. The only time it's safe to resize the C partition is during a "new install" so it depends on how you plan on reinstalling.

[Peter_h]

Originally Posted by S_M_E

I've seen it but never used it.

If that tool will "convert" the D partition to 64 clusters after install, then yes, that's a good time. At the time that the C partition is stretched (using my other tutorial) the D partition doesn't even exist yet.

EDIT:
Wait...
If you're doing a "server reinstall" instead of a "new install," it's too late to stretch the C partition. The only time it's safe to resize the C partition is during a "new install" so it depends on how you plan on reinstalling.

Well I want to reinstall WHS but I would like to keep the data on the pool so I assume that is a reinstall?