Storing Data Structures to Disk (Perl Cookbook, 2nd Edition)

## 11.13. Storing Data Structures to Disk

### 11.13.1. Problem

You want to save your large, complex data structure to disk so you don't have to reconstruct it from scratch each time your program runs.

### 11.13.2. Solution

Use the Storable module's `store` and `retrieve` functions:

>     use Storable;
>     store(\%hash, "filename");

>     # later on...
>     $href = retrieve("filename");        # by ref
>     %hash = %{ retrieve("filename") };   # direct to hash

### 11.13.3. Discussion

The Storable module uses C functions and a binary format to walk Perl's internal data structures and lay out its data. It's more efficient than a pure Perl and string-based approach, but it's also more fragile.

`The` `store` and `retrieve` functions expect binary data using the machine's own byte-ordering. This means files created with these functions cannot be shared across different architectures. `nstore` does the same job `store` does, but keeps data in canonical (network) byte order, at a slight speed cost:

>     use Storable qw(nstore);
>     nstore(\%hash, "filename");
>     # later ...
>     $href = retrieve("filename");

No matter whether `store` or `nstore` was used, you need to call the same `retrieve` routine to restore the objects in memory. The producer must commit to portability, but the consumer doesn't have to. Code need only be changed in one place when the producer has a change of heart; the code thus offers a consistent interface for the consumer, who does not need to know or care.

The `store` and `nstore` functions don't lock the files they work on. If you're worried about concurrent access, open the file yourself, lock it using Recipe 7.18, and then use `store_fd` or its slower but machine-independent version `nstore_fd`.

Here's code to save a hash to a file, with locking. We don't open with the `O_TRUNC` flag because we have to wait to get the lock before we can clobber the file.

>     use Storable qw(nstore_fd);
>     use Fcntl qw(:DEFAULT :flock);
>     sysopen(DF, "/tmp/datafile", O_RDWR|O_CREAT, 0666)
>             or die "can't open /tmp/datafile: $!";
>     flock(DF, LOCK_EX)                   or die "can't lock /tmp/datafile: $!";
>     nstore_fd(\%hash, *DF)
>         or die "can't store hash\n";
>     truncate(DF, tell(DF));
>     close(DF);

Here's code to restore that hash from a file, with locking:

>     use Storable qw(retrieve_fd);
>     use Fcntl qw(:DEFAULT :flock);
>     open(DF, " < /tmp/datafile")      or die "can't open /tmp/datafile: $!";
>     flock(DF, LOCK_SH)                or die "can't lock /tmp/datafile: $!";
>     $href = retrieve_fd(*DF);
>     close(DF);

With care, you can pass large data objects efficiently between processes using this strategy, since a filehandle connected to a pipe or socket is still a byte stream, just like a plain file.

Unlike the various DBM bindings, Storable does not restrict you to using only hashes (or arrays, with DB_File). Arbitrary data structures, including objects, can be stored to disk. The whole structure must be read in or written out in its entirety.

### 11.13.4. See Also

The section on "Remote Procedure Calls (RPC)" in Chapter 13 of Advanced Perl Programming, by Sriram Srinivasan (O'Reilly); Recipe 11.14

Copyright © 2003 O'Reilly & Associates. All rights reserved.