Alternately, you can create a DBM::Deep handle by using Perl's built-in
tie() function. The object returned from tie() can be used to call methods,
such as lock() and unlock(). (That object can be retrieved from the tied
-variable at any time using tied() - please see L<perltie/> for more info.
+variable at any time using tied() - please see L<perltie> for more info.
my %hash;
my $db = tie %hash, "DBM::Deep", "foo.db";
=over 4
-=item * C<DBM::Deep-E<gt>TYPE_HASH>
+=item * C<<DBM::Deep->TYPE_HASH>>
-=item * C<DBM::Deep-E<gt>TYPE_ARRAY>.
+=item * C<<DBM::Deep->TYPE_ARRAY>>
=back
This only takes effect when beginning a new file. This is an optional
-parameter, and defaults to C<DBM::Deep-E<gt>TYPE_HASH>.
+parameter, and defaults to C<<DBM::Deep->TYPE_HASH>>.
=item * locking
As with hashes, you can treat any DBM::Deep object like a normal Perl array
reference. This includes inserting, removing and manipulating elements,
and the C<push()>, C<pop()>, C<shift()>, C<unshift()> and C<splice()> functions.
-The object must have first been created using type C<DBM::Deep-E<gt>TYPE_ARRAY>,
+The object must have first been created using type C<<DBM::Deep->TYPE_ARRAY>>,
or simply be a nested array reference inside a hash. Example:
my $db = DBM::Deep->new(
push @$db, "bar", "baz";
unshift @$db, "bah";
- my $last_elem = pop @$db; # baz
- my $first_elem = shift @$db; # bah
- my $second_elem = $db->[1]; # bar
+ my $last_elem = pop @$db; # baz
+ my $first_elem = shift @$db; # bah
+ my $second_elem = $db->[1]; # bar
my $num_elements = scalar @$db;
$db->clear(); # hashes or arrays
-=item * lock() / unlock()
+=item * lock() / unlock() / lock_exclusive() / lock_shared()
-q.v. Locking.
+q.v. L</LOCKING> for more info.
=item * optimize()
-Recover lost disk space. This is important to do, especially if you use
-transactions.
+This will compress the datafile so that it takes up as little space as possible.
+There is a freespace manager so that when space is freed up, it is used before
+extending the size of the datafile. But, that freespace just sits in the datafile
+unless C<optimize()> is called.
-=item * import() / export()
+=item * import()
-Data going in and out.
+Unlike simple assignment, C<import()> does not tie the right-hand side. Instead,
+a copy of your data is put into the DB. C<import()> takes either an arrayref (if
+your DB is an array) or a hashref (if your DB is a hash). C<import()> will die
+if anything else is passed in.
+
+=item * export()
+
+This returns a complete copy of the data structure at the point you do the export.
+This copy is in RAM, not on disk like the DB is.
=item * begin_work() / commit() / rollback()
These are the transactional functions. L</TRANSACTIONS> for more information.
+=item * supports( $option )
+
+This returns a boolean depending on if this instance of DBM::Dep supports
+that feature. C<$option> can be one of:
+
+=over 4
+
+=item * transactions
+
+=back
+
=back
=head2 Hashes
Enable or disable automatic file locking by passing a boolean value to the
C<locking> parameter when constructing your DBM::Deep object (see L<SETUP>
- above).
+above).
my $db = DBM::Deep->new(
file => "foo.db",
=head2 Explicit Locking
You can explicitly lock a database, so it remains locked for multiple
-actions. This is done by calling the C<lock()> method, and passing an
-optional lock mode argument (defaults to exclusive mode). This is particularly
-useful for things like counters, where the current value needs to be fetched,
-then incremented, then stored again.
+actions. This is done by calling the C<lock_exclusive()> method (for when you
+want to write) or the C<lock_shared()> method (for when you want to read).
+This is particularly useful for things like counters, where the current value
+needs to be fetched, then incremented, then stored again.
- $db->lock();
+ $db->lock_exclusive();
my $counter = $db->get("counter");
$counter++;
$db->put("counter", $counter);
# or...
- $db->lock();
+ $db->lock_exclusive();
$db->{counter}++;
$db->unlock();
-You can pass C<lock()> an optional argument, which specifies which mode to use
-(exclusive or shared). Use one of these two constants:
-C<DBM::Deep-E<gt>LOCK_EX> or C<DBM::Deep-E<gt>LOCK_SH>. These are passed
-directly to C<flock()>, and are the same as the constants defined in Perl's
-L<Fcntl/> module.
+=head2 Win32/Cygwin
- $db->lock( $db->LOCK_SH );
- # something here
- $db->unlock();
+Due to Win32 actually enforcing the read-only status of a shared lock, all
+locks on Win32 and cygwin are exclusive. This is because of how autovivification
+currently works. Hopefully, this will go away in a future release.
=head1 IMPORTING/EXPORTING
encryption. Filtering applies to the base DB level, and all child hashes /
arrays. Filter hooks can be specified when your DBM::Deep object is first
constructed, or by calling the C<set_filter()> method at any time. There are
-four available filter hooks, described below:
+four available filter hooks.
+
+=head2 set_filter()
+
+This method takes two paramters - the filter type and the filter subreference.
+The four types are:
=over
$db->set_filter( "filter_store_value", undef );
-=head2 Real-time Encryption Example
-
-Here is a working example that uses the I<Crypt::Blowfish> module to
-do real-time encryption / decryption of keys & values with DBM::Deep Filters.
-Please visit L<http://search.cpan.org/search?module=Crypt::Blowfish> for more
-on I<Crypt::Blowfish>. You'll also need the I<Crypt::CBC> module.
-
- use DBM::Deep;
- use Crypt::Blowfish;
- use Crypt::CBC;
-
- my $cipher = Crypt::CBC->new({
- 'key' => 'my secret key',
- 'cipher' => 'Blowfish',
- 'iv' => '$KJh#(}q',
- 'regenerate_key' => 0,
- 'padding' => 'space',
- 'prepend_iv' => 0
- });
+=head2 Examples
- my $db = DBM::Deep->new(
- file => "foo-encrypt.db",
- filter_store_key => \&my_encrypt,
- filter_store_value => \&my_encrypt,
- filter_fetch_key => \&my_decrypt,
- filter_fetch_value => \&my_decrypt,
- );
-
- $db->{key1} = "value1";
- $db->{key2} = "value2";
- print "key1: " . $db->{key1} . "\n";
- print "key2: " . $db->{key2} . "\n";
-
- undef $db;
- exit;
-
- sub my_encrypt {
- return $cipher->encrypt( $_[0] );
- }
- sub my_decrypt {
- return $cipher->decrypt( $_[0] );
- }
-
-=head2 Real-time Compression Example
-
-Here is a working example that uses the I<Compress::Zlib> module to do real-time
-compression / decompression of keys & values with DBM::Deep Filters.
-Please visit L<http://search.cpan.org/search?module=Compress::Zlib> for
-more on I<Compress::Zlib>.
-
- use DBM::Deep;
- use Compress::Zlib;
-
- my $db = DBM::Deep->new(
- file => "foo-compress.db",
- filter_store_key => \&my_compress,
- filter_store_value => \&my_compress,
- filter_fetch_key => \&my_decompress,
- filter_fetch_value => \&my_decompress,
- );
-
- $db->{key1} = "value1";
- $db->{key2} = "value2";
- print "key1: " . $db->{key1} . "\n";
- print "key2: " . $db->{key2} . "\n";
-
- undef $db;
- exit;
-
- sub my_compress {
- return Compress::Zlib::memGzip( $_[0] ) ;
- }
- sub my_decompress {
- return Compress::Zlib::memGunzip( $_[0] ) ;
- }
-
-B<Note:> Filtering of keys only applies to hashes. Array "keys" are
-actually numerical index numbers, and are not filtered.
+Please read L<DBM::Deep::Manual> for examples of filters.
=head1 ERROR HANDLING
by specifying the 'pack_size' parameter when constructing the file.
DBM::Deep->new(
- filename => $filename,
+ file => $filename,
pack_size => 'large',
);
instead of 32-bit longs. After setting these values your DB files have a
theoretical maximum size of 16 XB (exabytes).
-You can also use C<pack_size =E<gt> 'small'> in order to use 16-bit file
+You can also use C<<pack_size => 'small'>> in order to use 16-bit file
offsets.
B<Note:> Changing these values will B<NOT> work for existing database files.
parameters are per-file, meaning you can access 32-bit and 64-bit files, as
you choose.
-B<Note:> We have not personally tested files larger than 4 GB -- all my
-systems have only a 32-bit Perl. However, I have received user reports that
+B<Note:> We have not personally tested files larger than 4 GB -- all our
+systems have only a 32-bit Perl. However, we have received user reports that
this does indeed work.
=head1 LOW-LEVEL ACCESS
data in this structure (be wary of name collision), which is then accessible from
any child hash or array.
-=head1 CUSTOM DIGEST ALGORITHM
-
-DBM::Deep by default uses the I<Message Digest 5> (MD5) algorithm for hashing
-keys. However you can override this, and use another algorithm (such as SHA-256)
-or even write your own. But please note that DBM::Deep currently expects zero
-collisions, so your algorithm has to be I<perfect>, so to speak. Collision
-detection may be introduced in a later version.
-
-You can specify a custom digest algorithm by passing it into the parameter
-list for new(), passing a reference to a subroutine as the 'digest' parameter,
-and the length of the algorithm's hashes (in bytes) as the 'hash_size'
-parameter. Here is a working example that uses a 256-bit hash from the
-I<Digest::SHA256> module. Please see
-L<http://search.cpan.org/search?module=Digest::SHA256> for more information.
-
- use DBM::Deep;
- use Digest::SHA256;
-
- my $context = Digest::SHA256::new(256);
-
- my $db = DBM::Deep->new(
- filename => "foo-sha.db",
- digest => \&my_digest,
- hash_size => 32,
- );
-
- $db->{key1} = "value1";
- $db->{key2} = "value2";
- print "key1: " . $db->{key1} . "\n";
- print "key2: " . $db->{key2} . "\n";
-
- undef $db;
- exit;
-
- sub my_digest {
- return substr( $context->hash($_[0]), 0, 32 );
- }
-
-B<Note:> Your returned digest strings must be B<EXACTLY> the number
-of bytes you specify in the hash_size parameter (in this case 32).
-
-B<Note:> If you do choose to use a custom digest algorithm, you must set it
-every time you access this file. Otherwise, the default (MD5) will be used.
-
=head1 CIRCULAR REFERENCES
-B<NOTE>: DBM::Deep 0.99_03 has turned off circular references pending
-evaluation of some edge cases. I hope to be able to re-enable circular
-references in a future version after 1.00. This means that circular references
-are B<NO LONGER> available.
-
-DBM::Deep has B<experimental> support for circular references. Meaning you
+DBM::Deep has full support for circular references. Meaning you
can have a nested hash key or array element that points to a parent object.
This relationship is stored in the DB file, and is preserved between sessions.
Here is an example:
print $db->{foo} . "\n"; # prints "bar"
print $db->{circle}->{foo} . "\n"; # prints "bar" again
+This also works as expected with array and hash references. So, the following
+works as expected:
+
+ $db->{foo} = [ 1 .. 3 ];
+ $db->{bar} = $db->{foo};
+
+ push @{$db->{foo}}, 42;
+ is( $db->{bar}[-1], 42 ); # Passes
+
+This, however, does I<not> extend to assignments from one DB file to another.
+So, the following will throw an error:
+
+ my $db1 = DBM::Deep->new( "foo.db" );
+ my $db2 = DBM::Deep->new( "bar.db" );
+
+ $db1->{foo} = [];
+ $db2->{foo} = $db1->{foo}; # dies
+
B<Note>: Passing the object to a function that recursively walks the
object tree (such as I<Data::Dumper> or even the built-in C<optimize()> or
C<export()> methods) will result in an infinite loop. This will be fixed in
-a future release.
+a future release by adding singleton support.
=head1 TRANSACTIONS
-New in 0.99_01 is ACID transactions. Every DBM::Deep object is completely
+As of 1.0000, DBM::Deep hass ACID transactions. Every DBM::Deep object is completely
transaction-ready - it is not an option you have to turn on. You do have to
specify how many transactions may run simultaneously (q.v. L</num_txns>).
Transactions in DBM::Deep are done using a variant of the MVCC method, the
same method used by the InnoDB MySQL engine.
-=head1 PERFORMANCE
-
-Because DBM::Deep is a conncurrent datastore, every change is flushed to disk
-immediately and every read goes to disk. This means that DBM::Deep functions
-at the speed of disk (generally 10-20ms) vs. the speed of RAM (generally
-50-70ns), or at least 150-200x slower than the comparable in-memory
-datastructure in Perl.
-
-There are several techniques you can use to speed up how DBM::Deep functions.
-
-=over 4
-
-=item * Put it on a ramdisk
-
-The easiest and quickest mechanism to making DBM::Deep run faster is to create
-a ramdisk and locate the DBM::Deep file there. Doing this as an option may
-become a feature of DBM::Deep, assuming there is a good ramdisk wrapper on CPAN.
-
-=item * Work at the tightest level possible
-
-It is much faster to assign the level of your db that you are working with to
-an intermediate variable than to re-look it up every time. Thus
-
- # BAD
- while ( my ($k, $v) = each %{$db->{foo}{bar}{baz}} ) {
- ...
- }
-
- # GOOD
- my $x = $db->{foo}{bar}{baz};
- while ( my ($k, $v) = each %$x ) {
- ...
- }
-
-=item * Make your file as tight as possible
-
-If you know that you are not going to use more than 65K in your database,
-consider using the C<pack_size =E<gt> 'small'> option. This will instruct
-DBM::Deep to use 16bit addresses, meaning that the seek times will be less.
-
-=back
-
=head1 MIGRATION
As of 1.0000, the file format has changed. Furthermore, DBM::Deep is now
=head2 Caching
-If a user is willing to assert upon opening the file that this process will be
+If a client is willing to assert upon opening the file that this process will be
the only consumer of that datafile, then there are a number of caching
possibilities that can be taken advantage of. This does, however, mean that
DBM::Deep is more vulnerable to losing data due to unflushed changes. It also
single-process. I have no idea how I'd specify this, though. Suggestions are
welcome.
-=head2 Importing using Data::Walker
-
-Right now, importing is done using C<Clone::clone()> to make a complete copy
-in memory, then tying that copy. It would be much better to use
-L<Data::Walker/> to walk the data structure instead, particularly in the case
-of large datastructures.
-
=head2 Different contention resolution mechanisms
Currently, the only contention resolution mechanism is last-write-wins. This
=item * CODE
-L<Data::Dump::Streamer/> provides a mechanism for serializing coderefs,
+L<Data::Dump::Streamer> provides a mechanism for serializing coderefs,
including saving off all closure state. This would allow for DBM::Deep to
store the code for a subroutine. Then, whenever the subroutine is read, the
code could be C<eval()>'ed into being. However, just as for SCALAR and REF,
=back
+=head2 External references and transactions
+
+If you do C<<my $x = $db->{foo};>>, then start a transaction, $x will be
+referencing the database from outside the transaction. A fix for this (and other
+issues with how external references into the database) is being looked into. This
+is the skipped set of tests in t/39_singletons.t and a related issue is the focus
+of t/37_delete_edge_cases.t
+
=head2 File corruption
The current level of error handling in DBM::Deep is minimal. Files I<are> checked
on DBM::Deep, as every element has to be fetched from disk, then stored again in
a different location. This will be addressed in a future version.
+This has been somewhat addressed so that the cost is constant, regardless of
+what is stored at those locations. So, small arrays with huge data structures in
+them are faster. But, large arrays are still large.
+
=head2 Writeonly Files
-If you pass in a filehandle to new(), you may have opened it in either a readonly or
-writeonly mode. STORE will verify that the filehandle is writable. However, there
-doesn't seem to be a good way to determine if a filehandle is readable. And, if the
-filehandle isn't readable, it's not clear what will happen. So, don't do that.
+If you pass in a filehandle to new(), you may have opened it in either a
+readonly or writeonly mode. STORE will verify that the filehandle is writable.
+However, there doesn't seem to be a good way to determine if a filehandle is
+readable. And, if the filehandle isn't readable, it's not clear what will
+happen. So, don't do that.
=head2 Assignments Within Transactions
=head1 CODE COVERAGE
-B<Devel::Cover> is used to test the code coverage of the tests. Below is the
-B<Devel::Cover> report on this distribution's test suite.
+L<Devel::Cover> is used to test the code coverage of the tests. Below is the
+L<Devel::Cover> report on this distribution's test suite.
- ----------------------------------- ------ ------ ------ ------ ------ ------
- File stmt bran cond sub time total
- ----------------------------------- ------ ------ ------ ------ ------ ------
- blib/lib/DBM/Deep.pm 94.4 85.0 90.5 100.0 5.0 93.4
- blib/lib/DBM/Deep/Array.pm 100.0 94.6 100.0 100.0 4.7 98.8
- blib/lib/DBM/Deep/Engine.pm 97.2 85.8 82.4 100.0 51.3 93.8
- blib/lib/DBM/Deep/File.pm 97.2 81.6 66.7 100.0 36.5 91.9
- blib/lib/DBM/Deep/Hash.pm 100.0 100.0 100.0 100.0 2.5 100.0
- Total 97.2 87.4 83.9 100.0 100.0 94.6
- ----------------------------------- ------ ------ ------ ------ ------ ------
+ ------------------------------------------ ------ ------ ------ ------ ------
+ File stmt bran cond sub total
+ ------------------------------------------ ------ ------ ------ ------ ------
+ blib/lib/DBM/Deep.pm 97.2 90.9 83.3 100.0 95.4
+ blib/lib/DBM/Deep/Array.pm 100.0 95.7 100.0 100.0 99.0
+ blib/lib/DBM/Deep/Engine.pm 95.6 84.7 81.6 98.4 92.5
+ blib/lib/DBM/Deep/File.pm 97.2 81.6 66.7 100.0 91.9
+ blib/lib/DBM/Deep/Hash.pm 100.0 100.0 100.0 100.0 100.0
+ Total 96.7 87.5 82.2 99.2 94.1
+ ------------------------------------------ ------ ------ ------ ------ ------
=head1 MORE INFORMATION
or send email to L<DBM-Deep@googlegroups.com>. You can also visit #dbm-deep on
irc.perl.org
-The source code repository is at L<http://svn.perl.org/modules/DBM-Deep>
+The source code repository is at L<http://github.com/robkinyon/dbm-deep>
=head1 MAINTAINERS
projects / dbsrgits/DBM-Deep.git / blobdiff