SageTV in a school system

[JBurgin]

Originally Posted by ToxMox

Well Jack says he is using a quality that records around 1GB/hr per tuner. So the drive system needs to be able to handle 6GB/hr for the tuners plus 25GB/hr for the clients for a total of 31GB/hr. Divide that by 3600 and you get 8.61MB/second that the drive system would need to be able to randomly seek.
Double that to around 17MB/s if you want to use a better quality. I would imagine that using 64K clusters, using the drives individually per tuner and using large buffers all around would most likely handle his problem as long as not all the clients were viewing the same recording or happened to all be viewing different shows that happen to reside on the same drive.
It would be nice if one could assign a RAM cache for Sage to use.

I can't remember but is there a way to make the clients read the files directly from the recording folders rather than getting them streamed through the Sage service? I seem to remember something about UNC paths that could help out.

I would imagine there is some raid out there that could handle the throughput. Could take the seperate drive theory further and store to seperate RAIDs.

Have you contacted Sage support yet? I'm curious what they have to say regarding this.

update:

We did see an unique increase in performance after did the single drive to single tuner solution. The hard drive write rates went down to an average of 26% per drive, with a spike to 65% every 30 to 60 seconds. We ended up pegging our 100Mb connection again, so we put the server back on a Gig, the video began stuttering again (not sure why, as all performance tests did not point to an overload) so I'm doing another test today to see what the deal was. I'm also gonna try the UNC path option. I remember doing that for stored videos (It was the only way to get ASF files to play through SageTV), so I think I'll try that with the source video directories to see what I can do.

We've actually decided that we cannot get the performance we want out of our current hardware. We're trying our best to keep it down to one server, although it looks like we will be unable to. After the UNC testing (should that fail). I'm going to attempt a server split. Once machine being the network encoder, the other being the server/storage solution.

[JBurgin]

Howdy guys, thought you might like an update on our school's SageTV system and how the testing is going. Thanks for all your help so far, it has been most appreciated.

We decided to order some new hardware. We needed something with enough SATA ports and the right amount of PCI slots to keep us running well. We decided that if we go our RAID card out of the system and went with onboard SATA, that would help divide up the bandwidth issues. Now the PCI bus drives the tuner cards, and the SATA drives are on their own. Here's our specs now:

ASUS M2N-SLI Deluxe motherboard
1GB Dual Channel RAM
AMD Socket AM2 3800+ Dual Core Processor

We have removed the 3ware card from the equation alltogether. As the motherboard has 6 SATA ports on it that can handle 3.0Gb/s transfer rates.

The other selling point of this motherboard was the NVidia chipset and their new networking features. The motherboards come with two gigabit network ports, and it does support teaming. However, I don't think we will be utilizing the teaming feature. We have made use of their TCP/IP acceleration features, which allows all network traffic from specified applications to be offloaded from the CPU, thereby creating faster throughput.

Anyways, let me stop talking and list the changes I've done.

#1 - I goofed. I said that we were originally doing a 1.0GB/h quality rate. That was not true, the system was configured to do a 1.8GB/h quality rate. I have since adjusted that to the lowest DVD quality possibly, which was 1.4GB/h. This is the best quality setting we were satisfied with.

#2 - The TV tuner and recording scenarios remain the same. Each tuner gets it's own hard drive to record to.

#3 - I enabled TCP/IP acceleration on the sagetv.exe and sagetvservice.exe programs with our new chipset.


The results are impressive to say the least.

With the same amount of clients connected (23 is the max I can run with, right now) Network utilization had gone from 99% on a 100Mbps connection, to 75% (the avg). [Come to find out, our lines here in this particular school I'm testing are not true Gig lines. So even when I was previously testing, I was hitting bottlenecks. Even if the server was on a 1Gb connection]

Disk write rates are down by substantial rates. Where they were once at 25% with an occasional spike at 65%, they're now down to 10% with a spike to 25% every hour or so (when the program changes and files must be deleted)


Regardless, I have 6 different channels being recorded right now. 23 clients tapping into those 6 channels, and have yet to have a stutter, or glitch, or hiccup. Granted it takes about...oh....3 grand in hardware to make it run. But we're still saving money versus buying TV distribution equipment.

[cslatt]

Originally Posted by JBurgin

Howdy guys, thought you might like an update on our school's SageTV system and how the testing is going. Thanks for all your help so far, it has been most appreciated.

Awesome read, thanks for posting!

[teknubic]

Congratulations
I gotta look into that TCP/IP acceleration thingie more. I just don't see how you could get that much difference from turning it on. Must be a combination of that and the RAID card removal.

[Dekard]

If I can, I'd suggest the real performance difference here has to be using 1 hd to 1 tuner... If you wanted to get a higher performance disk array you'd want to go to SCSI since it has a TCQ function that will greatly optimize your disk access. You of course would want a proper raid card in a pci-x or pci-e slot to take advantage of these drives. To be able to achieve 1 terabyte of storage would be significantly more costly, and the main advantage would be the ability to have a redundant storage system.

Running standard drives the way you are is just asking for failures. I would consider a near-line level drive, like the Western Digital RE2's, which are designed for server level 24\7 access unlike standard drives. You may be using this level of drive already, I just didn't notice.

Also consider the effect that a drive failure would have on the ability of the system to keep operating? Try disconnecting a drive during operation to simulate a failure and see how the server responds. Using a desktop level board, as you are, means that they are *not* tolerant of failing disks. It will likely cause a BSOD or worse just hang the system. ( I would suggest something like the SUPERMICRO PDSG4 which should meet all your needs and provide a proper backbone to build a system like this on. For harddrives the HITACHI Ultrastar 10K300 would be ideal, though costly.)

[dagar]

Jburgin

How is your network segmented? Are all 23 clients hitting the same server IP and are they all in the same network segment (IP network and VLAN)?

[Homebuilder]

Hi, Really enjoyed your post!

I was curious,

How do you set up UNC direct paths in a sage server for the clients to access directly content?

Also, how do you set up a tuner dedicated to one hard drive?

[phenixdragon]

Pretty cool idea, but I really hope this all isn't funded by tax payers.

But just skimming the thread I beleive your problem is just system resources, you are going to max out your HDD and your NIC, both jsut depends on how many clients are accessing the files at one time. I don't think anything else is going to be a problem.

[dflachbart]

Originally Posted by phenixdragon

Pretty cool idea, but I really hope this all isn't funded by tax payers.

=>

Originally Posted by JBurgin

But we're still saving money versus buying TV distribution equipment.

[phenixdragon]

Originally Posted by flachbar

=>

Which then I could through in that I'm not sure why a school needs cable in the first place.

[blade]

Originally Posted by phenixdragon

Which then I could through in that I'm not sure why a school needs cable in the first place.

There are lots of educational programs. When I was in school we watched a news show every morning that was specifically designed to be shown in school to children. We also watched programs on history, literature, etc...

Schools having cable is nothing new. We had it when I was in school and that was over 15 years ago.

[phenixdragon]

So did I (20 years ago) but only the short period that I went to private school and we watched shows like Reading Rainbow, and what not, so that was early Elm. school.

But in HS? There's is no need for it. It's just time to take a nap and slack off from the little work thats already being done. Id rather see reading and teaching going on. In fact, the classes we had movies in was the classes I did the least amount of work to no work at all. But really, there is nothing educational that is being produced. It's a gimmick and nothing more.

But that's all besides the support on it is what I also posted earlier where it sounds like a bandwidth issue on the HDDs or network.

Ohh yes another thought on the CPU. Just because it might only be hitting a certain amount, lets go with 30%, it doesn't mean that it's not the cause of the problem. If there are multiple streams happening at the same time it's going to have problems processing regardless if it hits 30% or 100%. But I don't think thats the issue, but an idea. My setup at home I do see problems now that I added an HD card, which only uses 30% of the CPU when nothing else is running. So just a thought.

[blade]

Originally Posted by phenixdragon

So did I (20 years ago) but only the short period that I went to private school and we watched shows like Reading Rainbow, and what not, so that was early Elm. school.

But in HS? There's is no need for it. It's just time to take a nap and slack off from the little work thats already being done. Id rather see reading and teaching going on. In fact, the classes we had movies in was the classes I did the least amount of work to no work at all. But really, there is nothing educational that is being produced. It's a gimmick and nothing more.

I went to public school and we had it in HS. Before we had cable, teachers would tape programs at home and bring them for us to watch. We even watched movies and TV programs in college. Tests were given on the programs we watched so if you slept or slacked off your grade was going to reflect it. Really it wasn't much different than slacking off during a lecture.

If you think there is nothing educational being produced you must not be watching PBS, Disc, History, A&E, etc... Many students find literature and history boring. Personally I enjoyed literature, but was bored to death in history. Now I watch more programs about history than anything else. Actually seeing the places or recreations of events is much more interesting than reading about it in a text book. I'm not saying it should replace reading and traditional teaching, but to think it can't be a valuable aid makes no sense.

We watched CNN and other news programs and had classroom discussions about current events. I still remember the books we watched movie versions of (in addition to reading) more vividly than the stories I only read.

I agree some teachers abuse it and treat it as a substitution for traditional teaching, but if used in addition to traditional methods students can learn far more than if restricted to traditional teaching. Visual stimulus is very powerful and including it in the teaching process can go a long way toward helping students learn.

[cbhatt]

Hi all,
I just found this thread and its very interesting. I work in a school hose too and we have an inhouse cable TV distribution plant which we are upgrading. We want to do something similar to JBurgin but we never thought about using Sage to do it. We went with a slightly more expensive solution, Cisco IPTV. We connect our cable feed to tuners, and tuners to the Content Servers which pipe everything through our network. We use a client program on the PC to receive the stream.

We encode our stream to mpeg-2 (not sure about the bitrates) but you can read the CNN ticker when projected onto an 120 inch screen. Our network bandwidth is 100Mbps to the desktop and 1000Mbps backbone. We offer 6 channels.

I wonder if you looked into either the Cisco IPTV system or the VBrick system for you school?

[Menehune]

The version of Cisco IPTV (3400 series?) we demoed a few years ago needed an expensive client license and viewer installed on each computer that wanted to view the stream. Too expensive for our needs and how we wanted to use the system.