problem: memories/videos

[sflamm]

Thanks - this looks very similar to the solution that I created above.

I see only a few differences:

1. Mine: <option name="children-only" value="true"/>
Yours: <option name="children-only" value="false"/>

Can you explain the meaning of "children-only"? I assumed it meant to not keep the parent folder in the view... but it is a little confusing as "HomeVideos" is the parent that is being selected by the <bookmark> tag and you have it set to "false"...

2. Mine: <option name="mediamask" value="BDV"/>
Yours: <option name="mediamask" value="V"/>

I assume that the HomeVideos can still have blu-ray and DVD source materials... these will not be picked up by "V" alone, correct?

3. Mine: <option name="root" value="HomeVideos"/>
Yours: <option name="bookmark" value="/HomeVideos/"/>

I use a "root" instead of a bookmark...there is no practical difference here, correct? You pointed out that with "bookmark" you can navigate up and "root" is the top of the tree... however it does not seem to be any different for the "Memories->Videos" screen...

4. Mine:
<presentation>
<sort by="title">
<option name="folders-first" value="true"/>
</sort>
</presentation>


Yours: None

Is the default to present the folders by name first? Is that why you do not use a <presentation> tag? If so - it would be faster to remove the <presentation> tag as I assume it is re-sorting what is already sorted, correct?

Another question I have - is there any performance disadvantage to using a raw source? Most views seem to use other views which in turn use raw sources... I would expect that creating this 'layering' would perform worse than a raw source, is that correct?

As for the directory structure, one of the values of SageTV imports is that it does not need a single 'root'... so source material can be anywhere on your network. I think it would be a good idea if you allow the syntax to take that into account as many folks will not have a single tree hierarchy for their imports. It

[bialio]

There isn't much if any penalty performance wise on raw sources vs view-sources. You can use whatever gets the job done and not worry too much about performance. There are a few sorts that take longer than others (favorite priority is one), and one view that takes a while (it's labeled 'slow' already), but any others should be pretty quick no matter how you build them.

btl.

[stuckless]

Originally Posted by sflamm

Thanks - this looks very similar to the solution that I created above.

I see only a few differences:

1. Mine: <option name="children-only" value="true"/>
Yours: <option name="children-only" value="false"/>

Can you explain the meaning of "children-only"? I assumed it meant to not keep the parent folder in the view... but it is a little confusing as "HomeVideos" is the parent that is being selected by the <bookmark> tag and you have it set to "false"...

2. Mine: <option name="mediamask" value="BDV"/>
Yours: <option name="mediamask" value="V"/>

I assume that the HomeVideos can still have blu-ray and DVD source materials... these will not be picked up by "V" alone, correct?

3. Mine: <option name="root" value="HomeVideos"/>
Yours: <option name="bookmark" value="/HomeVideos/"/>

I use a "root" instead of a bookmark...there is no practical difference here, correct? You pointed out that with "bookmark" you can navigate up and "root" is the top of the tree... however it does not seem to be any different for the "Memories->Videos" screen...

4. Mine:
<presentation>
<sort by="title">
<option name="folders-first" value="true"/>
</sort>
</presentation>


Yours: None

Is the default to present the folders by name first? Is that why you do not use a <presentation> tag? If so - it would be faster to remove the <presentation> tag as I assume it is re-sorting what is already sorted, correct?

Another question I have - is there any performance disadvantage to using a raw source? Most views seem to use other views which in turn use raw sources... I would expect that creating this 'layering' would perform worse than a raw source, is that correct?

As for the directory structure, one of the values of SageTV imports is that it does not need a single 'root'... so source material can be anywhere on your network. I think it would be a good idea if you allow the syntax to take that into account as many folks will not have a single tree hierarchy for their imports. It

I guess, all I'm trying to get you to do, is make a view where you actually see HomeVideos listed as view... and then worry about the bookmark|root. There's no point in troubleshooting why "root" doesn't work, if you view doesn't contain a specific folder to begin with. Once the base view looks like you'd expect, then getting bookmark|root working is pretty trivial.

The children-only=false means to preserve the root folder rather than just taking the "children" of the root folder.

The mediamask can be whatever you want... typically I don't have DVDs or BluRays in my HomeVidoes so I just use the V.

As for the single root... Phoenix doesn't require a single root, I'm just trying to show you how you can get what you want In fact I have multiple video import directories from multiple drives but they all get shown as a single merged filesystem (combined).

At this point, given that sagetv is dead, I'd try to work with what's there for Phoenix, because I don't see much development happening in the next while So, if a solution requires
a. you change YOUR import structure
b. I make changes to Phoenix code
(Ii'd go with option 'a', since that's the one that you can control -- and it's most likely the most time effective solution for both of us )

[sflamm]

all I'm trying to get you to do, is make a view where you actually see HomeVideos listed as view... and then worry about the bookmark|root.

Agreed that is the approach I took as well. I found that by using the either the raw source or using default.videofolders as a source I was able to see the top level folders "Videos", "HDVideos" & "HomeVideos" (leveraging BMT UI directly made this easy )

It was when I tried to apply the bookmark|root to take only the hierachy under "HomeVideos" that the problem arose. It only would work if I used the raw source.

If there is a way to get it to work with a non-raw source I'd like to know what I am doing wrong...

The children-only=false means to preserve the root folder rather than just taking the "children" of the root folder.

Makes sense - which is why I'm confused about your block:

<option name="bookmark" value="/HomeVideos/"/>
			<option name="children-only" value="false"/>

If the bookmark is set, shouldn't children-only be "true" (we only want the children and do not want the 'root' which is "HomeVideos"?

At this point, given that sagetv is dead, I'd try to work with what's there for Phoenix, because I don't see much development happening in the next while So, if a solution requires
a. you change YOUR import structure
b. I make changes to Phoenix code

I'm of course happy to make whatever adjustments are necessary - but it seems like this is not really an issue with SageTV... it is Phoenix code that applies the "root|bookmark", "filter" and "presentation"....

As I pointed out above, both the raw and default.videofolders gave me the correct starting point - it was refining that starting point for the view which is the issues, correct? And that is only a question of Phoenix code not SageTV code as I understand it...

Btw, with all the great work you and the Phoenix team have done... I think that SageTV will live on for many years in reality. ( ty! )

There isn't much if any penalty performance wise on raw sources vs view-sources. You can use whatever gets the job done and not worry too much about performance. There are a few sorts that take longer than others (favorite priority is one), and one view that takes a while (it's labeled 'slow' already)

I read through the x-vfs.xml as I worked on the issue. FYI, the 'slow' view "sagevideoimports" is not in Phoenix 409 (but is in Phoenix 408).

The real issue is the phoenix.view.bmt.sagevideoimports view is based on the raw source <item name="sageimports" label="Sage Import Folders" class="sagex.phoenix.vfs.sources.SageSourcesFactory"> which is the ONLY method I have working currently... (it is a bit slow)

I'm sure it would be faster if I could base it on the phoenix.view.default.videofolders source but I cannot seem to get root|bookmark to work with that source... Any way to fix this?

[bialio]

I think the slow one is still there, but it's not in x-vfs.xml anymore. I'm not sure where it is now, but it shows up in BMT at least.

			<source name="sagefilesystem">
				<option name="content" value="VideosByFolder"/>
			</source>

This source is the fast one for the 'folder' view - the only issue with this one is that it returns the equivalent of the children-only="true" - this is hard coded into the Sage core and there is no way around it. So if you want to see a top level folder from this source, you have to introduce an extra level of folders into the hierarchy on disk.

			<source name="sageimports">
				<option name="combine" value="false"/>
				<option name="mediamask" value="BDV"/>
			</source>

This is the 'slow' one - it basically walks the tree and finds all your media - this is what makes is slow. It's like doing a library scan each time you create that view.

<option name="bookmark" value="/HomeVideos/"/>
			<option name="children-only" value="false"/>

If the bookmark is set, shouldn't children-only be "true" (we only want the children and do not want the 'root' which is "HomeVideos"?

You want the 'HomeVideos' folder to be there, because that's what the bookmark / root is looking for. If you just have the children, then the bookmark that you are setting doesn't get found and is ignored.

[bialio]

Think of the root / bookmark as 'after the view gets built, this is where we will navigate to'. It doesn't impact how the view is built, which is why getting the view built where it shows the HomeVideos folder without any bookmarks is the way to go about solving this.

[sflamm]

Think of the root / bookmark as 'after the view gets built, this is where we will navigate to'

Ok - that is probably where the issue lies. I interpreted it differently.

When there is an XML block like this:

<view name="phoenix.view.default.homevideos" label="Home Videos" flat="false">
			<option name="bookmark" value="/HomeVideos"/>
			<option name="children-only" value="true"/>

by convention that is usually interpreted to mean there are two unordered options that are applied to the view.
The way it would typically be read is:

1. Take the original output of the view (which in this case is a set of folders "Videos", "HDVideos", "HomeVideos") and apply each of the two options.

2. The "children-only" option excludes any top-level root node (the prefix).

3. The "bookmark" option selects "one of the folder in the hierarchy to serve as the root node" (which may or may not be excluded).


If I have understood what you are saying correctly, the "bookmark" doesnt really act as an 'option' - it is more of a 'filter' on the result set of the view (like the other filters)... but currently I believe only 1 filter is allowed. Filters are usually ordered...

Just some food for thought...

In summary, based on the current design I need to change 'children-only' to 'true' (I had 'false' set).
Now that I understand the design I'll try it out when I get home and report the results.

Thanks for all the help/patience. Let me know if I am misunderstanding still.

[sflamm]

That should have been:

In summary, based on the current design I need to change 'children-only' to 'false' (I had 'true' set).
Now that I understand the design I'll try it out when I get home and report the results.

[stuckless]

bookmark is an "option" for 2 reasons

1. It's not a filter, since it's not filtering the view, it's just positing the view's state to be at given level... just like when you bookmark a folder location in windows, or bookmark a page on a site in a browser.

2. A filter at the top level implicitly flattens your view. ie, if there was a <filter by="bookmark" value="HomeFolder"/> then it would never work since a top level filter on a view flattens the view before a filter is applied. That's an important thing to know about filters, since, if you are using a structured view source and then add a top level filter, then the results are no longer structured.

Given that there's very little documentation on the vfs, these types of confusions are bound to happen. Perhaps someday, someone (other than us) will understand the VFS enough to produce some documentation

[sflamm]

an important thing to know about filters, since, if you are using a structured view source and then add a top level filter, then the results are no longer structured

I have observed this empirically - but I would suggest an option on the <filter> tag to preserve the structure (i.e. flat="false") as it is not necessary that a filter would flatten the structure and there are many cases where it is it is not desired - the filter should only 'filter' items in or out (walk and prune the tree).


I tried the changes last night.... the approach does not work.
The fundamental issue is that the view phoenix.view.default.videofolders creates an output that is a list of folders from the top level of everyone of the source locations instead of just the top level import source nodes "Videos", "HDVideos", "HomeVideos".

ie. if the source files look like //nas/Videos/video1, //nas/HDVideos/video2, //nas/HomeVideos/video3

expected output: "Videos", "HDVideos", "HomeVideos"
actual output: "video1", "video2", "video3"

As a result there is nothing to use the root|bookmark option on.

I would think that I need to <filter> out the non-"HomeVideos" folders but that does not seem to be able to be done because:

1) I cannot seem to get the filter by filepath to work... tried to do this by regex but did not succeed.

What is the syntax methodology to use this?

2) Based on your comments above the filter would flatten the folder hierarchy which is exactly what I want to preserve


The phoenix.view.default.videofolders view is based on:

<source name="sagefilesystem">
				<option name="content" value="VideosByFolder"/>
			</source>

I tested this raw source and it appears to remove all of the top level import nodes ... is there a way to provide an option to this source to leave the top level nodes?

As another note, I found that the "sageimports" source actually adds an additional top level "Sage File Imports" node above the source paths. The structure becomes:

/Sage File Imports/Videos/...
/Sage File Imports/HDVideos/...
/Sage File Imports/HomeVideos/...

The root|bookmark works perfectly in this case once you include the right path of the view "/Sage File Imports/HomeVideos" but as was pointed out this is the slowest view.

Is there a way to fix this with the "sagefilesystem" source?

I tried to use the "filesystem" source but I could not figure out the syntax for using it ...

[bialio]

As I said above :

			<source name="sagefilesystem">
				<option name="content" value="VideosByFolder"/>
			</source>

This source is the fast one for the 'folder' view - the only issue with this one is that it returns the equivalent of the children-only="true" - this is hard coded into the Sage core and there is no way around it. So if you want to see a top level folder from this source, you have to introduce an extra level of folders into the hierarchy on disk.

Your best best would be to add a level of folders so that the fast view works for you.
btl.

[stuckless]

Originally Posted by sflamm

I have observed this empirically - but I would suggest an option on the <filter> tag to preserve the structure (i.e. flat="false") as it is not necessary that a filter would flatten the structure and there are many cases where it is it is not desired - the filter should only 'filter' items in or out (walk and prune the tree).

You do this now by moving the <filter> to the <presentation>. By not flattening a view before applying a root filter, you'd end up with folders with 0 items... that's why the filter flattens a view. When you put a filter at the presentation level, then it will apply the filter and behave as you said, but you may end up with zero item folders.

Something you can try in your <view> is to use a source like this

<source name="sagefilesystem">
	<option name="content" value="Filesystem"/>
        <option name="subdir" value="HomeVideos"/>
       	<option name="grouping" value="Folder"/>
</source>

the subdir may need to be the actual full path including HomeVideos, not sure, since I've never used it.

And if this doesn't get you where you need to be, then I'd seriously consider restructuring your videos

[stuckless]

Originally Posted by sflamm

I tested this raw source and it appears to remove all of the top level import nodes ... is there a way to provide an option to this source to leave the top level nodes?

This source simply proxies the sagetv MediaNodeAPI... so if it's not showing it, then it suspect no.

As another note, I found that the "sageimports" source actually adds an additional top level "Sage File Imports" node above the source paths. The structure becomes:

/Sage File Imports/Videos/...
/Sage File Imports/HDVideos/...
/Sage File Imports/HomeVideos/...

This is probably because of the children=false on the <view>. Try changing that value and it shoudl remove the Sage File Imports path.

[sflamm]

By not flattening a view before applying a root filter, you'd end up with folders with 0 items... that's why the filter flattens a view

Just to clarify, that is because the folders were empty to begin with...correct? So it is a good reflection of the real structure on the drive, yes?

Something you can try in your <view> is to use a source like this
Code:
<source name="sagefilesystem">
<option name="content" value="Filesystem"/>
<option name="subdir" value="HomeVideos"/>
<option name="grouping" value="Folder"/>
</source>

Will try tonight and let you know.

This is probably because of the children=false on the <view>. Try changing that value and it shoudl remove the Sage File Imports path.

Correct - I discovered that as well... but as you know I was trying children=false as the goal last try...

[sflamm]

I tried the following with limited success:

<view name="phoenix.view.default.AAA" label="AAA" flat="false">

			<option name="children-only" value="false"/>

			<description>Home Videos</description>
			<tag value="video"/>

			<source name="sagefilesystem">
				<option name="content" value="Filesystem"/>
				<option name="subdir" value="//nas/HomeVideos"/>
				<option name="grouping" value="Folder"/>
			</source>


	</view>

It list all the directories correctly for HomeVideos - unfortunately all the folders display as "0" contents. It has the folder hierarchy absolutely correct but there are no files in any folder.

Ideas?

[stuckless]

Originally Posted by sflamm

I tried the following with limited success:

<view name="phoenix.view.default.AAA" label="AAA" flat="false">

			<option name="children-only" value="false"/>

			<description>Home Videos</description>
			<tag value="video"/>

			<source name="sagefilesystem">
				<option name="content" value="Filesystem"/>
				<option name="subdir" value="//nas/HomeVideos"/>
				<option name="grouping" value="Folder"/>
			</source>


	</view>

It list all the directories correctly for HomeVideos - unfortunately all the folders display as "0" contents. It has the folder hierarchy absolutely correct but there are no files in any folder.

Ideas?

I don't know much about the underlying API (it was new added in Sage7) but maybe try without the Grouping option to see if that makes a difference.

[sflamm]

I did try variants including eliminating the grouping and content options.

The only variants that made any change:

1) The view must be have "flat=false" or else no items populate at all
2) The subdir must be a full unc path or no items populate at all

Other than that the folders always showed with 0 items.

[stuckless]

Originally Posted by sflamm

I did try variants including eliminating the grouping and content options.

The only variants that made any change:

1) The view must be have "flat=false" or else no items populate at all
2) The subdir must be a full unc path or no items populate at all

Other than that the folders always showed with 0 items.

Looks like we've hit the end of the road on this

[sflamm]

looks that way... oh well.

At least I got to play with the views...

[Dargason]

I spent some time today trying to learn how views work, and figuring out a way to organize home videos seemed like a good place to start. I've had some success, so I though I should share.

The first method I used was to assign a genre to my home videos through BMT. So I have an "Africa" genre for our trip to Africa last summer, and an "Alaska" genre for some videos I've take while fishing in Alaska... you get the idea. Then I used the following view:

		<view name="phoenix.view.default.homevideos" label="Home Movies by Genre" flat="true">
			<description>Home Videos</description>
			<tag value="homevideos"/>			
			<tag value="default"/>
			<view-source name="phoenix.view.primary.homevideos"/>
			
			<presentation level="1">			
				<group by="genre">
					<option name="empty-foldername" value="Miscellaneous"/>
				</group>						
				<sort by="title"/>				
			</presentation>
		</view>

This works great, but requires a bit of maintenance. So I created the following one, which basically shows the videos based on the folder structure:

		<view name="HomeMoviesByFolder" label="Home Movies by Folder" flat="false">
			<option name="children-only" value="true"/>
			<description>View of All Home Movies</description>
			<tag value="video"/>
			<tag value="default"/>
			<source name="sagefilesystem">
				<option name="content" value="VideosByFolder"/>
			</source>
			<filter by="homevideos" scope="include"/> 
			<presentation>
				<sort by="title">
					<option name="folders-first" value="true"/>
				</sort>
			</presentation>
		</view>

One thing I noticed while testing. These views work fine and perform as expected in BMT and on my HD200, but I don't get items returned when I try to view them using the Sage client on my pc (which happens to be the server, if that matters.) I assume this is a bug.'

Edit: Found out why I wasn't seeing home videos in the PC client. It was because I never bothered to configure the phoenix "Home Videos" directory since I hardly ever use it.