General Overview of SMM, MM, Sage & Collection management For PluckyHD

[wjones]

Hi PluckyHD,

This post is really aimed at you, but could be useful for everyone.

Before I start, I just want to say what a fantastic job you have done on SMM, wow factor just gets better and better with each new release ( for the Multi disc stuff!).

I am struggling to understand some of the top level concepts around MM, Collection manager & SMM – and was hoping you could fill in some of the gaps.

-I have MM installed on my WHS, this also installs SQL server – which I assume contains the database to store the collection.
-I have Collection Manager installed on a PC on my network, which connects to the SQL Server on my WHS (or MM?).
-I have SMM plugin installed on the SageTV server which is also installed on the WHS.

So far so good!

Eye Candy:
Under d:\My Movies\Filestorage\Backdrops, Covers etc. are a load of images (Note, this is not under d:\Shares\My Movies as stated in the wiki)
The fact this is stored directly under the D: folder means the files are stored on the D: drive of the WHS not in the pool (Not a problem for me as my D: is a SSD which makes the artwork loading nice and quick). Where is this location defined & how can I change it?
I think this folder is managed by the MM installation on the WHS, but collection manager seems to be able to update this.

Movie Metadata:
In each movie folder are a few files :
- mymovies.xml file, this contains all the metadata for the movie, plus the path & filename for the eye candy stored above, it is in this format:
\\server\MyMoviesFileStorage$\Covers\medfd3996e7-8eea-47ab-a594-497e2edcf387.jpg
- moviename.dvdid.xml
- Images : mymovies-front,jpg, fanart.jpg, folder.jpg etc.

Who / What maintains these .XML file & images?

Moviename.Properties and Moviename.jpg file
In the “root” folder of the movies (e.g. under d:\Shares\Movies\DVDs) are 2 files which represent a movie subdirectory.
In MM (Both on WHS and Collection management), there is an option to write sagetv .properties files.
Is this used at all by SMM, Or only by SageTv Clients?

SMM
I understand that SMM has it’s own database, and that if I rescan my collection from within Sage it will trigger a refresh of the SMM database.
Where is this database stored? Does it use the same SQL Server as MM on the WHS?

SageTV
Sage also has a complete metadata store (wiz.bin?), where all the file and folder locations are stored.
As I understand it, wiz.bin does not store any covers / fanart etc.. images come from the moviename.jpg in the video root folder.


Questions:
1) I have 5 options for refreshing movies & metadata:

- Sage : Setup, Scan imported Media
- Sage : Setup, Detailed Setup, Advanced, Rescan Existing Library files
- Sage SMM: Options, Database, Refresh Import Library
- Sage SMM: Options, Database, Rebuild DataFile
- MM (Both Collection manager and WHS): Maintenance, Update Stored Meta-Data for all titles in collection database

I find myself just randomly selecting one or more of these to attempt to resolve issues. What is the correct sequence?

2) I regularly have issues with missing covers for movies (mostly top level of multi disc sets)

When I look more closely, I see that the file specified in the .XML of the first movie of the multi disk set does not exist in the \\server\MyMoviesFileStorage$\Covers\ folder.
I can see the cover in collection management.. but the link to it in SMM seems to be broken.
Brian at MM recommends running “optimize database” to refresh covers, but this did not help for me.
After performing optimize database, surely I need to sync with SMM / Sage?
Any ideas how to fix this?

3) Moving movies

I have a “new” folder, where all my newly ripped movies are created.
This folder is monitored by MM so the metadata is created automatically & the entry is added to the MM database.
Later on, I move this entire folder to my “main movies” folder and update the location in Collection management, Disks section,
I suspect this is a really bad idea.. and could be the source of lots of issues.
Is there a correct way to do this?

[PLUCKYHD]

Originally Posted by wjones

Eye Candy:
Under d:\My Movies\Filestorage\Backdrops, Covers etc. are a load of images (Note, this is not under d:\Shares\My Movies as stated in the wiki)
The fact this is stored directly under the D: folder means the files are stored on the D: drive of the WHS not in the pool (Not a problem for me as my D: is a SSD which makes the artwork loading nice and quick). Where is this location defined & how can I change it?
I think this folder is managed by the MM installation on the WHS, but collection manager seems to be able to update this.

D:\ is the storage location on the WHS but it is also a share to access on other computers as the wiki states, but I see the confusion.

There is no way to change where it is saving currently I dont control that. You can edit a registry value but it sometimes causes issues so I don't recomend that.

Originally Posted by wjones

Movie Metadata:
In each movie folder are a few files :
- mymovies.xml file, this contains all the metadata for the movie, plus the path & filename for the eye candy stored above, it is in this format:
\\server\MyMoviesFileStorage$\Covers\medfd3996e7-8eea-47ab-a594-497e2edcf387.jpg
- moviename.dvdid.xml
- Images : mymovies-front,jpg, fanart.jpg, folder.jpg etc.

Who / What maintains these .XML file & images?

Mymovies.xml is a backup incase you nuke your database or move the folder this is the current data it is using so you can easily reimport and get the same data.

MOVIENAME.DVDID.XML is used in media center only you can turn off the option to write those.

Images are backups just like the xml and are what are used to scale and such that way the fanart can stay with your movie again if you nuke it by accident.

Originally Posted by wjones

In the “root” folder of the movies (e.g. under d:\Shares\Movies\DVDs) are 2 files which represent a movie subdirectory.
In MM (Both on WHS and Collection management), there is an option to write sagetv .properties files.
Is this used at all by SMM, Or only by SageTv Clients?

Only used if you want the information in the OSD player or default videos views I recommend writing them.

Originally Posted by wjones

SMM
I understand that SMM has it’s own database, and that if I rescan my collection from within Sage it will trigger a refresh of the SMM database.
Where is this database stored? Does it use the same SQL Server as MM on the WHS?

in the share\sagetv\database it is a single database file. It uses the mymovies sql data but the file allows quicker access in Sagetv then quering data everytime.

Originally Posted by wjones

SageTV
Sage also has a complete metadata store (wiz.bin?), where all the file and folder locations are stored.
As I understand it, wiz.bin does not store any covers / fanart etc.. images come from the moviename.jpg in the video root folder.

Correct I don't write any information to the default Db. however if you write the property files they do.

Originally Posted by wjones

Questions:
1) I have 5 options for refreshing movies & metadata:

- Sage : Setup, Scan imported Media
- Sage : Setup, Detailed Setup, Advanced, Rescan Existing Library files
- Sage SMM: Options, Database, Refresh Import Library
- Sage SMM: Options, Database, Rebuild DataFile
- MM (Both Collection manager and WHS): Maintenance, Update Stored Meta-Data for all titles in collection database

I find myself just randomly selecting one or more of these to attempt to resolve issues. What is the correct sequence?

There is no sequence and nothing is necessary if you are patient everything runs automatically at library refresh (default is every 2 hours)

Rebuild datafile should be used with extreme caution and only when there are issues no related to no seeing new movies.

Originally Posted by wjones

2) I regularly have issues with missing covers for movies (mostly top level of multi disc sets)

When I look more closely, I see that the file specified in the .XML of the first movie of the multi disk set does not exist in the \\server\MyMoviesFileStorage$\Covers\ folder.
I can see the cover in collection management.. but the link to it in SMM seems to be broken.
Brian at MM recommends running “optimize database” to refresh covers, but this did not help for me.
After performing optimize database, surely I need to sync with SMM / Sage?
Any ideas how to fix this?

You could try rebuilding the database file.

Originally Posted by wjones

3) Moving movies

I have a “new” folder, where all my newly ripped movies are created.
This folder is monitored by MM so the metadata is created automatically & the entry is added to the MM database.
Later on, I move this entire folder to my “main movies” folder and update the location in Collection management, Disks section,
I suspect this is a really bad idea.. and could be the source of lots of issues.
Is there a correct way to do this?

I still don't understand people moving movies and not just copying it to the location they want makes no since and seems redundant.

That being said everything will auto update to the changes once a scan is performed

SMM is not instant that is your biggest confusion please read the wiki it runs with the default sage import scan that defaults to every 2 hours. People are updating MyMovies collection manager and expecting to instantly see the results in SMM and that is not how it works. They then assume they have an issue and start rebuilding things.

If patient everything happens at the next scan.

cheers

[wjones]

Ok, thankyou very much for your rapid response PluckyHD.

You have clarified enough of my points to keep me quiet for a while

I am still not 100% sure for the missing files in d:\Mymovies, but I will leave the rescan to do its thing.. and wait for the result.

One of the main reasons I do the whole "moving movies" thing.. is that if I just chuck all my new movies into a folder with the old ones, it makes offsite backup much harder to manage.
Normally, I rip my newly purchased movies to HDD, and once I have a batch, I copy them to an external HDD. After they are archived, I move them into the standard folder with all the other movies.

I know people will question why I am archiving movies when I have the original.. but after I lost my first 2tb disk and had to re-rip 200+ movies, I made the decision.. "never again". Now, I have duplicated folders and offsite backup!
I live in a high risk lightning strike zone - so have multiple UPS, network cable surge protection, belt & braces!

Cheers
Will