I was wondering if there’s a way to have a Perl process share some commonly used modules (such as DBI, DBIx::Class and Template) amongst different users? I’ve looked at FCGI::Spawn but it didn’t seem to me it was capable of it. What I’d like to do is run a single Perl daemon which has some modules preloaded and is able to spawn other processes (e.g. a Catalyst application), sharing the same memory block for loaded modules. I know that’s possible, Starman does it for example, but that’s meant for a single user.
Instead of a single user I want this process to do what suexec does for Apache. suexec Will only launch a CGI/FastCGI script if the owner and group are defined correctly. Once launched the process will launched by the owner of the script, a normal user.
My current problem (well, not current but in the future it’ll be) is that through suexec you can run many webapplications, like Catalyst, Dancer and Mojolicious. The thing is, the commonly used modules will be loaded by each process. So if there are 3 users with a Catalyst website using Moose, DBIx::Class and Template that means for every user process these modules need to be loaded into memory. They can’t share the same module space (for some modules this would be bad, of course). When running a few dozen of smallish websites this will eat up RAM quickly.
I’ve came across Plack::App::Apache::ActionWrapper which partly solves this problem for a single user with multiple PSGI application. So far I haven’t been able to make it work for a single user, let alone multiple. But I had hoped it would be possible to use a single wrapper like this, preload some modules, and use this single wrapper for all users.
I suppose I have to ponder a bit more about it. Although I wonder if it’s even possible though. Yes, mod_perl can preload modules but this means executing Perl as the user running the webserver process. I prefer PSGI or FastCGI.
I’m assuming that you’re doing this for either the startup time savings or memory savings. As for memory, you might not get as much savings as you’d think because of the way Perl manages memory. On systems with copy-on-write memory you can definitely get savings, but also it’s impossible to tell when them memory that those modules occupies becomes “dirty” and needs to be unshared simply because the way Perl manages those things and what causes things to be written to memory pages isn’t obvious.
Native programs written in native compiled code typically use mmap() to load libraries. Because these are marked as MMAP_SHARED, many programs, even as different users, can all share the RAM pages to load the code just once.
The same technique cannot easily be used by Perl. It could, for example, transform an optree in memory into a position-independent contiguous block of data, which could then be written to disk, and mmap()ed by all the other programs. Trust would be an issue here – root would have to pre-compile those optrees. You then start to lose benefits of instant edit-and-reload. Plus it makes for a more fragile system, and one that’s quite a departure from where we currently are.
All to save some memory? RAM is cheap, programmers are expensive.
A bit of both really. In a recent small Mojolicious project starting up the dynamic FastCGI server can take from 1 to 4,5 seconds, whilst the CGI variant executes in about 400-600ms. Once the FastCGI server is loaded requests are extremely fast (16-50ms).
True, memory is cheap but there’s always a limit of course.
Thanks for this info as it has given me a bit better understanding on the topic.