Caching Open Output Filehandles (Perl Cookbook, 2nd Edition)

## 7.7. Caching Open Output Filehandles

### 7.7.1. Problem

You need more output files open simultaneously than your system allows.

### 7.7.2. Solution

Use the standard FileCache module:

>     use FileCache;
>     cacheout ($path);         # each time you use a filehandle
>     print $path "output";

### 7.7.3. Discussion

FileCache's `cacheout` function lets you work with more output files than your operating system lets you have open at any one time. If you use it to open an existing file that FileCache is seeing for the first time, the file is truncated to length zero, no questions asked. However, in its opening and closing of files in the background, `cacheout` tracks files it has opened before and does not overwrite them, but appends to them instead. This does not create directories for you, so if you give it /usr/local/dates/merino.ewe to open but the directory /usr/local/dates doesn't exist, `cacheout` will `die`.

The `cacheout` function checks the value of the C-level constant `NOFILE` from the standard system include file sys/param.h to determine how many concurrently open files are allowed on your system. This value can be incorrect on some systems and even missing on a few (for instance, on those where the maximum number of open file descriptors is a process resource limit that can be set with the limit or ulimit commands). If `cacheout` can't get a value for `NOFILE`, set `$FileCache::cacheout_maxopen` to be four less than the correct value, or choose a reasonable number by trial and error.

Example 7-1 splits an xferlog file (created by most FTP servers nowadays) into separate files, each named after the authenticated user. Fields in xferlog files are space-separated, with the fourth field from the last holding the authenticated username.

#### Example 7-1. splitwulog

>       #!/usr/bin/perl
>       # splitwulog - split wuftpd log by authenticated user
>       use FileCache;
>       $outdir = "/var/log/ftp/by-user";
>       while (<>) {
>           unless (defined ($user = (split)[-4])) {
>              warn "Invalid line: $.\n";
>              next;
>           }
>           $path = "$outdir/$user";
>           cacheout $path;
>           print $path $_;

>       }

### 7.7.4. See Also

Documentation for the standard FileCache module (also in Chapter 32 of Programming Perl); the `open` function in perlfunc(1) and in Chapter 29 of Programming Perl

Copyright © 2003 O'Reilly & Associates. All rights reserved.