What's this?

[texneus]

Came home from work tonight to find one CPU core of my server pegged at 100%. Traced it to my Ubuntu VM that hosts the SageTV server via Docker. No idea how long it's been actually doing this (for all I know it was doing it last night). SageTV seemed to be working OK, in fact it did a guide update earlier and just finished recording two programs. All outward apperences are normal. Nothing odd looking in the logs either.

So what is it? Google says kswapd0 has to do with virtual memory, but it seems nothing is using all that much RAM at this point. Interestingly at about 7:00 this evening Java was at 80%, now down to ~31%. Could this be Java garbage collection? Is this typical after only approx. a day and a half of uptime? I've never encountered it before.

Anyway, I rebooted the VM, all's fine again (Java at 33.5% memory).

[stuckless]

likely given how much you are showing in swap that you are running out of physical memory (or memory allocated to your vm in this case). And when that happens, your hard drive becomes memory (swap). Of course this is slower, and, and result in high cpu and IO if it has to keep swapping stuff in and out of memory.

I would give more memory to your VM.

In terms of java, it will consume as much as you give it before really trying to reclaim more. So, if you have a VM with 5gig memory and give java 4, then yeah, you are going to swap alot.

As a rule of thumb, maybe try setting your VM memory to be 2x Java memory, that should allow the OS and Java to run without swapping. (you can obviously fine tune this, and, in an ideal world, you'd have 0 in swap. I just checked my desktop that runs a lot of stuff, and have been up for about 3 months... it has 0 in swap.

[texneus]

Well, I know what virtual memory is. What I don't understand is why the process supposedly only responsible for managing it freaked out as the Java RAM footprint was shrinking. Shouldn't be any virtual memory used based on that list of processes, but yah, I see now it swapped quite a bit out. But also what is swapped would fit within available RAM. So mystery prevails. Memory leak somewhere maybe? Don't know, probably a retorical question at this point and for this forum. All looks fine for now. 42.3% RAM used by Java, no swap used.

[KarylFStein]

I agree to increase your VM memory. But just a note on why swap may be used when there is plenty of free physical memory. On one of my VMs I installed a package that required mysql-server. Now I have my own MySQL cluster so used that in the configuration. But when I removed the mysql-server package or even just disabled it on the VM there were problems with "apt upgrade" calls. So I left mysql-server installed and running on the VM although with no external access. The kernel I believe put that mysql-server stuff into swap because mysql-package asked for memory but over time was really not using that memory so it was safe to move to swap. The other four VMs I run basically use no swap and I think all would be fine with all if there was no swap available. Not an answer, but just a note on why the kernel may move "unused" stuff to swap as it needs to keep as much cache stuff as possible in physical RAM.

[stuckless]

Originally Posted by texneus

Well, I know what virtual memory is. What I don't understand is why the process supposedly only responsible for managing it freaked out as the Java RAM footprint was shrinking. Shouldn't be any virtual memory used based on that list of processes, but yah, I see now it swapped quite a bit out. But also what is swapped would fit within available RAM. So mystery prevails. Memory leak somewhere maybe? Don't know, probably a retorical question at this point and for this forum. All looks fine for now. 42.3% RAM used by Java, no swap used.

Sometimes I never know what "memory" people are referring to when they say java and memory

For my main server, I have a JVM memory set to 2048m (2gig). Right now when I look at sagetv, it says that "java" is using 900m of allocated memory. My "top" process shows it using 1.5gigs. All this makes sense. Java as a process will require at least how much memory I give it for inside the JVM plus what is needed by the OS to run it. So let's say that java need half a gig to run, if I tell it to use a 2g jvm, then at some point it will need to allocate 2.5gigs of physical OS memory. And, it may never release it, even if the memory allocation INSIDE the jvm decreases.

As for swap. The algorithms for when an OS will put things into swap is complex, and, might now always be related to running out of memory. For linux, the used to be (and maybe still is) a kernel parameter for 'swapiness' that was simply a value that dictated how agressively the OS would put things into swap. I used to set mine so that it would rarely use swap. Some things like unRAID won't even install a swap device.

Swap, made sense back in the days when OSes were growing in size and memory was expensive. Today, with 32g machines being the norm, even things like windows, rarely needs swap. And for a VM, I would never turn on swap. Why allow something that is being virtualized to also swap? just give it the memory it needs.

If you have a sagetv VM setup... set the memory to be JVM max size + .5g (for java) + 1g (for the os) +1g gig if you are running a windowing enviroment +more if you are running more services (docker, opendct, etc, etc).

Give the vm as much memory as you think it will need to run smoothly. I'd also turn off services that you don't need. I see docker running. Are you using docker?