Connection options between storage server and SageTV server?

[Konig]

Hypothetical situation: I have separate WHS (or NAS) and SageTV server systems, with most of my media stored (archived) on my WHS (or NAS). How can I ensure that I have fast enough connection between them? I'm concerned about lag when browsing my media from an HD200.

-Could I have a dedicated Gigabit ethernet connection between the two using a crossover cable?
-Could I use firewire networking? Or USB2?
-Could I somehow have an eSATA connection between them?

[razrsharpe]

how about just hook up both servers to the same gigabit switch and see if there is a problem? A saturated gigabit connection should easily be able to handle 15 full bitrate bluray streams (40 Mbps) or 40 full bitrate hdtv streams (19Mbps).

[Fuzzy]

Also, browsing won't be affected, as the browsing is just accessing the database on the sage server. the only time sage will be accessing the storage server, is when it is refreshing the library, or you are actually playing back from it, or recording to it.

[Konig]

Originally Posted by Fuzzy

Also, browsing won't be affected, as the browsing is just accessing the database on the sage server. the only time sage will be accessing the storage server, is when it is refreshing the library, or you are actually playing back from it, or recording to it.

Will I see more (anoying) delay when FF and REV'ing if the media is stored on the separate server?

I'm doubtful that I'll ever get anywhere near Gigabit speed thru my cheap d-link router. I also have a lot of slower components hooked into it.

Instead of blowing a lot of money on a more expensive router I figured a cat6 crossover cable and two gig NICs would be cheap and would be a dedicated pipe from server to server. I just don't know how to set up this connection. would it mess up the rest of the network?

[KarylFStein]

As razrsharpe said, try the simplest thing first and see how it performs. But, it's always good to have a backup plan ...

I have a "smart" switch. It's s little more expensive than just a switch, (and amazingly loud--good thing it's in the basement), but no where near the cost of a "managed" switch especially the "refurbished" one I bought. Anyway, the switch has link aggregation or trunking (combining multiple Ethernet wires into one connection), per-port "priority" settings, (give priority to traffic on certain ports), and VLAN, (a separate network).

If I switch to the separate WHS and SageTV setup and have performance problems, my thoughts are to try one or more of the following depending on what the problem seems to be:

1) Set up traffic from SageTV to have top priority.
2) Put a dual-NIC card that supports trunking in the WHS server and trunk it on the switch.
3) Put an extra NIC in each server and dedicate that to communication between SageTV and WHS, (straight connection that may require a cross-over cable--I know I used to need one for this, but not sure if NICs these days can figure that out on their own). If I put dual-NIC cards in each, I could trunk that as well. Also, I could possibly turn on jumbo frames for that link. If I can't do the bonding with a straight connection, then I could do it through the switch and set that channel up as a VLAN.
4) Create a VLAN for SageTV and extenders, and add a separate link on the WHS server to that VLAN.

Anyway, I haven't tried any of these, so take it for what it's worth. I expect that I won't have any issues using the single LAN/single Ethernet set-up, but then our recording/viewing is pretty light and I don't think anyone will be watching TV when the WHS backup stuff kicks in.

[razrsharpe]

Originally Posted by Konig

Will I see more (anoying) delay when FF and REV'ing if the media is stored on the separate server?

no idea i dont have separate servers... although ive thought about it

I'm doubtful that I'll ever get anywhere near Gigabit speed thru my cheap d-link router. I also have a lot of slower components hooked into it.

I would set up my network so only internet traffic went through the router and all internal network traffice went through a gigabit switch (like this one from hp). ie cable/dsl modem -> router/dhcp server -> gigabit switch -> all internal connected devices. I personally have not used the hp router but here very good things about it... much better then that dlink/linksys/smc consumer nonsense. Consumer routers make crappy switches hence i only use it for internet traffic and as an dhcp server.

.. and two gig NICs

go intel nics all the way and if you want to add more bandwith between the servers use the dual nics and they can team/work together as KarylFStein suggested

[KarylFStein]

Originally Posted by Konig

Instead of blowing a lot of money on a more expensive router I figured a cat6 crossover cable and two gig NICs would be cheap and would be a dedicated pipe from server to server. I just don't know how to set up this connection. would it mess up the rest of the network?

Technically the connection will work fine between the servers. The question I have is how to make this work within SageTV and all the available extender options.

I *think* what you'd need to do is assign static IP addresses on the new NICs and put them on a different subnet than the rest of your LAN. Then, you'd need to set up a static route on each box so traffic to the other's LAN IP will go over the private link. You'll want to use UNC paths in SageTV for recording and media directories.

It will look like this:

192.168.0.100 (WHS) ---- LAN ----- 192.168.0.101 Sage TV
192.168.1.100 (WHS) --- XOVER --- 192.168.1.101 SageTV

On the WHS box, set a route for 192.168.0.101 to use gateway 192.168.1.100 and on the SageTV box, set a route for 192.168.0.100 to use gateway 192.168.1.101.

This way, if SageTV tries to write a recording to \\WHS\Recorded TV, it will actually pass that over the private network. However, extenders/clients will hit \\WHS\Recorded TV over the regular LAN IP. You can't do something like set the TV recording directory to \\192.168.1.100\Recorded TV in SageTV because your extenders/clients won't work, (although I think Placeshifter would), unless you put a second router in for the 192.168.1.x network and set up a route to it in your Internet router. But I don't think you want that...

Again, not sure if this will work and I've only done this sort of routing stuff on Linux boxes, so believe at your own risk .

[Islander]

I am using an HP 48x as storage device and the SageTV server is an Atom 330.

My experience:

- The Atom is underpowered so some of this could be related to that fact as I was using it as playback and recording device. Recording directly to the HP over gigabyte connection was not that great even with only one tuner.

- What has worked for me is to record to the Atom local drive and then use SJQ to move the recording automatically when finished. I am currently using this system with 100% success using: two tuners quite frequently recoding at the same time, 5 extenders plus the Atom for viewing content (several of which are quite frequently used to view content at the same time while also recording) and I am at times copying over downloaded content to the HP while this all goes on without a hitch. No fancy setup, a regular giga switch DGS-2208 to where the atom, the extenders, other computers (6), and the router connect to (that is, the only thing connected to the router is the adsl modem).

[tmiranda]

I'll throw in my 2 cents. I used to have my recordings and imported videos spread over two NAS servers. When drives got a lot cheaper I consolidated everything into the Sage server. I saw no difference in speed or responsiveness of the UI but my electric bill did go down

[Fuzzy]

Really, I think the root here is WHY do you have them on separate servers? What are you gaining from the more complicated setup? I can think of no reason to split them up, and everything from power savings to performance would lead me to believe that a single 24/7 machine is enough for ANYONE's needs. You pretty much have unlimited storage expansion options available on pretty much any OS, so it's not like there is an upper limit on what a single system can do. (Even SATA port multipliers work perfectly well in a sage system, especially if it's for the archived media, as you're only really accessing one or maybe two of the drives at a time anyways).

[KarylFStein]

Originally Posted by Islander

- What has worked for me is to record to the Atom local drive and then use SJQ to move the recording automatically when finished.

That is pretty neat; I hadn't seen that feature in SJQ before. So, record to local disk, run commercial skip, then move the recording and EDL files to the NAS for longer-term storage. As a benefit, you can still at least record TV if the NAS is down, (or watch Live TV, but who does that).

Does the location that the file gets moved to have to be listed as a "recording directory" and do the shows still show up under Recorded TV? What happens when someone's watching a recording when SJQ decides to move it to the NAS?

[KarylFStein]

Originally Posted by Fuzzy

Really, I think the root here is WHY do you have them on separate servers? What are you gaining from the more complicated setup? I can think of no reason to split them up, and everything from power savings to performance would lead me to believe that a single 24/7 machine is enough for ANYONE's needs. You pretty much have unlimited storage expansion options available on pretty much any OS, so it's not like there is an upper limit on what a single system can do. (Even SATA port multipliers work perfectly well in a sage system, especially if it's for the archived media, as you're only really accessing one or maybe two of the drives at a time anyways).

The reason I've been kicking it around is so I could have WHS pretty much stock for faster restores and be able to image my SageTV/Web/Email/etc. server, which I'd move to the second box. Of course, I could get rid of WHS, but I really do like the pool and the client backups...

[Fuzzy]

While I don't really see MUCH need for the pool... it doesn't do much for sage itself, as it can span multiple volumes perfectly fine, and most other things I'm keeping (photos and docs) are small enough that they don't need to span multiple volumes.. The system backup of other systems seems of SOME use.. still, as that's not really something that would require a FULL OS, I'd think about running WHS in a virtualPC on your sage server. This would allow you to have WHS's backup features, but not really require a separate system, while also eliminating some of the drier issues that arrise from trying to run Sage directly on WHS.

[Islander]

Originally Posted by KarylFStein

That is pretty neat; I hadn't seen that feature in SJQ before. So, record to local disk, run commercial skip, then move the recording and EDL files to the NAS for longer-term storage. As a benefit, you can still at least record TV if the NAS is down, (or watch Live TV, but who does that).

Does the location that the file gets moved to have to be listed as a "recording directory" and do the shows still show up under Recorded TV? What happens when someone's watching a recording when SJQ decides to move it to the NAS?

What you describe in the first paragraph is about it. I do not use com skip yet but it should be no problem adding that step.

Yes, both the local Atom and the "NAS" location have to be listed as recording directories. Not to worry, the system will never record to the NAS provided that you amend your properties file to force each tuner to record to the local drive. As an example this is what mine looks like:

mmc/encoders/-1341936237/forced_video_storage_path_prefix=C\:\\

mmc/encoders/1499460733/forced_video_storage_path_prefix=C\:\\

As to your last question I am not entirely sure so it is best to ask in the SJQ thread. This is what the log from a move looks like:

Sat Mar 27 08:01:27 GMT-05:00 2010
Source: C:\MyVideos\MeetDave-655076-0.mpg
Target: \\HPSERVER1\Recorded TV
Renaming source file... DONE!
Copying source file to target directory... DONE!
Deleting source media object in SageTV... DONE!
Renaming target file... DONE!
Creating SageTV media object for target file... DONE!
Relinking target file to SageTV airing... DONE!
Deleting source file... DONE!
Copying artifacts... DONE!
Archive operation completed SUCCESSFULLY!
Sat Mar 27 08:20:35 GMT-05:00 2010

[Robert Wise]

Originally Posted by KarylFStein

Technically the connection will work fine between the servers. The question I have is how to make this work within SageTV and all the available extender options.

I *think* what you'd need to do is assign static IP addresses on the new NICs and put them on a different subnet than the rest of your LAN. Then, you'd need to set up a static route on each box so traffic to the other's LAN IP will go over the private link. You'll want to use UNC paths in SageTV for recording and media directories.

It will look like this:

192.168.0.100 (WHS) ---- LAN ----- 192.168.0.101 Sage TV
192.168.1.100 (WHS) --- XOVER --- 192.168.1.101 SageTV

On the WHS box, set a route for 192.168.0.101 to use gateway 192.168.1.100 and on the SageTV box, set a route for 192.168.0.100 to use gateway 192.168.1.101.

This way, if SageTV tries to write a recording to \\WHS\Recorded TV, it will actually pass that over the private network. However, extenders/clients will hit \\WHS\Recorded TV over the regular LAN IP. You can't do something like set the TV recording directory to \\192.168.1.100\Recorded TV in SageTV because your extenders/clients won't work, (although I think Placeshifter would), unless you put a second router in for the 192.168.1.x network and set up a route to it in your Internet router. But I don't think you want that...

Again, not sure if this will work and I've only done this sort of routing stuff on Linux boxes, so believe at your own risk .

I believe you, but why won't the below work? If Sage knows about the recording UNC and I'm accessing Sage through 192.168.0.101 as the default NIC in my Sage box, where's the weak link?

You can't do something like set the TV recording directory to \\192.168.1.100\Recorded TV in SageTV because your extenders/clients won't work

I was going to try something like this with ZFS, but I wondered about Sage liking it.

[Fuzzy]

Originally Posted by Robert Wise

You can't do something like set the TV recording directory to \\192.168.1.100\Recorded TV in SageTV because your extenders/clients won't work

Extenders/clients don't need to have direct access to the media files, they only need to communicate to the server. Clients will, initially, try to access the files directly, but if they cannot get access, they will be proxied through the server.

[Robert Wise]

Originally Posted by Fuzzy

Extenders/clients don't need to have direct access to the media files, they only need to communicate to the server. Clients will, initially, try to access the files directly, but if they cannot get access, they will be proxied through the server.

Translation, it should work?

[Fuzzy]

Originally Posted by Robert Wise

Translation, it should work?

Not sure specifically waht you are trying to acomplish, as you had responded to a year old thread. I was merely correctng the wrong assumption that the extenders/clients need direct access to the media files. They only need access to the server, and only the server needs access to the media itself.

This is for playback. Other items may need access to the media location, such as ComSkip playback (which the clients would need access to the .EDL files).

Also, the location the files are moved to doesn't need to be a recording directory, in fact, I'd recommend NOT setting it as a recording folder, unless you want to record there. Moving a recording to an import folder keeps it as a recording. The only REAL difference, as far as sage is concerned, between an import folder, and a recording folder, is whether it will record to it. Files found in either are treated the same, and since it will recognize the file as a recording, it will still be treated as such.

Personally, I'm still in the camp that there is no compelling reason to move recordings, and just keep everything in my sage server.

[Robert Wise]

Originally Posted by Fuzzy

Not sure specifically waht you are trying to acomplish, as you had responded to a year old thread. I was merely correctng the wrong assumption that the extenders/clients need direct access to the media files. They only need access to the server, and only the server needs access to the media itself.

This is for playback. Other items may need access to the media location, such as ComSkip playback (which the clients would need access to the .EDL files).

Also, the location the files are moved to doesn't need to be a recording directory, in fact, I'd recommend NOT setting it as a recording folder, unless you want to record there. Moving a recording to an import folder keeps it as a recording. The only REAL difference, as far as sage is concerned, between an import folder, and a recording folder, is whether it will record to it. Files found in either are treated the same, and since it will recognize the file as a recording, it will still be treated as such.

Personally, I'm still in the camp that there is no compelling reason to move recordings, and just keep everything in my sage server.

I tried it with Unraid (single NIC) and it was OK, except when other things were happening on the UnRaid box. I'm moving to ZFS for other reasons, so I'd like to keep the Sage/Video activities in one pool and the important stuff in another pool. I should be able to saturate Gig-E with ZFS, so that's why the dual NIC crossover idea. Good to know about the import folder, in case I have to go that route and use the SJQ mover script.