Profiler Multi-threading

[Taddeusz]

Not having any knowledge of the Profiler code I was just wondering if there is a possibility of having it multi-threaded? The reason I ask is the slow startup. I realize there is the option to postpone the Profiler initialization to get SageTV up and running faster but then the upcoming recordings don't get populated until it is eventually run at some future time.

I notice that when Profiler is initializing a single CPU of my system is pegged until it is completed. Meaning the code runs a single blocking thread until complete. I don't believe that simply postponing this task is enough. Certainly there are other tasks that can only run once the Profiler has completed initialization but is there a reason for completely blocking startup until that point?

Not just that but is it possible for the Profiler initialization code itself to be multi-threaded?

[Fuzzy]

Originally Posted by Taddeusz

Not having any knowledge of the Profiler code I was just wondering if there is a possibility of having it multi-threaded? The reason I ask is the slow startup. I realize there is the option to postpone the Profiler initialization to get SageTV up and running faster but then the upcoming recordings don't get populated until it is eventually run at some future time.

I notice that when Profiler is initializing a single CPU of my system is pegged until it is completed. Meaning the code runs a single blocking thread until complete. I don't believe that simply postponing this task is enough. Certainly there are other tasks that can only run once the Profiler has completed initialization but is there a reason for completely blocking startup until that point?

Not just that but is it possible for the Profiler initialization code itself to be multi-threaded?

It's probably possible, since it is sort of job based, building a giant queue of airings, files, and agents, that doesn't really look back in on it's own results - but at the same time, 'startup' is not something that happens all that often in most uses, and maintaining the profiler updates when a show is watched or flagged don't like is a very quick action.

Regarding it blocking, you can't watch any live tv or do any recording until the scheduler is up and running, which is very dependent on carny getting done with what it's doing, so it does need to block. If you are looking to get in and watch something right away, that's exactly what the limited init flag is for.

[stanger89]

Originally Posted by Taddeusz

Not having any knowledge of the Profiler code I was just wondering if there is a possibility of having it multi-threaded? The reason I ask is the slow startup. I realize there is the option to postpone the Profiler initialization to get SageTV up and running faster but then the upcoming recordings don't get populated until it is eventually run at some future time.

I haven't paid attention for a while, but I'm pretty sure the recording schedule is available pretty much immediately on startup, it's just the intelligent recordings that don't show up for a while.

[Taddeusz]

Originally Posted by stanger89

I haven't paid attention for a while, but I'm pretty sure the recording schedule is available pretty much immediately on startup, it's just the intelligent recordings that don't show up for a while.

Not sure what changed but I used to have carny init disabled and the recording schedule was always there right away. Now since some time before I upgraded to v9 the profiler init must complete before my upcoming recordings will populate. Disabling carny init leaves it blank till it decides to run.

[Narflex]

You want to use carny_limited_init=true to have the fast startup, which means it only calculates Favorite recordings before startup is completed and then does the Intelligent recordings in the background after that.

There's also carny_disable_init=true...which if you set that, then you'll likely have no scheduled recordings at all after startup.

And yes, this code could definitely be multi-threaded...but this would only benefit startup as it already does lots of sleep calls in it to prevent using too much CPU when it's running normally.

I am very much inclined to have it force carny_limited_init to be true if there are a certain number of Agents in the DB so people don't need to worry about setting that....or just do it all the time since I think it's a much bigger complaint to say "Why does it take an hour for SageTV to startup?" rather than "Why does it take hours for Intelligent recording entries to show up?".

Any opinions on that?

[Taddeusz]

Originally Posted by Narflex

You want to use carny_limited_init=true to have the fast startup, which means it only calculates Favorite recordings before startup is completed and then does the Intelligent recordings in the background after that.

There's also carny_disable_init=true...which if you set that, then you'll likely have no scheduled recordings at all after startup.

And yes, this code could definitely be multi-threaded...but this would only benefit startup as it already does lots of sleep calls in it to prevent using too much CPU when it's running normally.

I am very much inclined to have it force carny_limited_init to be true if there are a certain number of Agents in the DB so people don't need to worry about setting that....or just do it all the time since I think it's a much bigger complaint to say "Why does it take an hour for SageTV to startup?" rather than "Why does it take hours for Intelligent recording entries to show up?".

Any opinions on that?

Well crud, looks like I've been mixing up those two options for a long time. I bet I mixed them up when I did a clean install on Windows quite a while ago. Probably a year and a half ago at least. That would explain why I all of a sudden had a problem.

BTW, they are actually disable_carny_init and limited_carny_init.

I would have to agree with you that the limited_carny_init should be defaulted to true. People that have a problem with it can always set it to false. Probably making an Advanced option available in the STV with wording such as "Full Startup Initialization" or "Fast Startup Initialization" so users don't have to dig in the properties file and get confused about those two carny options like I did. Also, since the names of the options are in many cases such as this named after the different internal components of SageTV they aren't necessarily self explanatory to new or nontechnical users.

[Fuzzy]

I see no problem with delaying full processing until after up and running. Certainly would minimize worry about really long startups, leading to people forcing close the app. Perhaps it might warrant an icon when the full processing is going on, similar to the media scan icon, to perhaps help explain to users why their intelligent recordings are all of a sudden not happening after they just restarted to update a plugin, etc.

[Narflex]

OK, I changed the default in the code...this will only affect new installs though since it won't override an existing setting.