Getting the content lighttpd's returning from a request

One of the things I want to do with the mod_mcpage plugin for lighttpd is have it be able to place pages into memcached itself, along with being able to serve pages out of memcached, but it was never very clear how to get a hold of it. It was clearly in con->write_queue, but it took me a while to figure out how. Once I found it, though, it's not very hard.

Note: Here, I'm working with content being sent back by mod_proxy, which is sent back in chunks in memory. Files are chunked in a different fashion, but can easily be added. This code is adapted from network_write_chunkqueue_linuxsendfile in network_linux_sendfile.c.

First, the plugin needs a handler attached to handle_subrequest; for mod_mcpage, I added  p->handle_subrequest    = mod_mcpage_handle_subrequest; to mod_mcpage_plugin_init. This subrequest handler will run as mod_proxy sends content back.

Then, there needs to be a handler_ctx struct for the connection specific buffers for storing the content type and the page.

Now, as the request continues running, the subrequest handler continues running.  Each time it's called, it checks to see if the file is finished and mod_proxy is done. When it is, it walks through the chunks of the file and appends them to a char pointer, which is then placed into the handler_ctx struct.

char *cret = malloc(0);
size_t crlen = 0;
chunk *c;
if(con->file_finished == 1){
           for(c = con->write_queue->first; c; c = c->next){
                   crlen += c->mem->used;
                   cret = realloc(cret, crlen);
                   strcat(cret, c->mem->ptr);
                   }
           }
if(crlen != 0)
           buffer_copy_string(hctx->outpg, cret);
free(cret);

Now the page returned by mod_proxy is in hctx->outpg, where you could do whatever you want with it.

Amusingly enough, while I was writing this up, I realized that I was doing it wrong by not waiting until the file was finished to get the file chunks, yet managed to get it right by screwing up and using strcpy instead of strcat. Oops.  In fact, now I see that a bunch of stuff can be shifted around and moved inside of that conditional that checks if the file is finished. Fortunately I'm still at a point where I only just figured out how to get at the page being returned, so I'm still filling out the rest of the pieces.

Update -- Further thoughts: Having the handler_ctx struct isn't actually necessary, now that the page data doesn't have to persist across calls to the subrequest handler. It could just be a pair of char pointers to hold the content type and the page. I'm a little torn, though, because using the handler_ctx struct might work better for memory management since it has its own init and free functions to handle taking it down properly, and the free function can be called if the connection gets reset. I'm also wondering now if using memcpy would be better than strcat. I don't have any particular plan to put non-text files into memcached, but someone else might. Bears investigating.

comments powered by Disqus