1 package Hash::Util::FieldHash;
6 use Scalar::Util qw( reftype);
9 our @ISA = qw(Exporter);
21 our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
23 our $VERSION = '1.01';
27 my %ob_reg; # private object registry
28 sub _ob_reg { \ %ob_reg }
29 XSLoader::load('Hash::Util::FieldHash', $VERSION);
34 return unless ref() && reftype( $_) eq 'HASH';
35 return $_ if Hash::Util::FieldHash::_fieldhash( $_, 0);
36 return $_ if Hash::Util::FieldHash::_fieldhash( $_, 2) == 2;
43 return unless ref() && reftype( $_) eq 'HASH';
44 return $_ if Hash::Util::FieldHash::_fieldhash( $_, 0);
45 return $_ if Hash::Util::FieldHash::_fieldhash( $_, 1) == 1;
50 sub fieldhashes { map &fieldhash( $_), @_ }
51 sub idhashes { map &idhash( $_), @_ }
58 Hash::Util::FieldHash - Support for Inside-Out Classes
62 ### Create fieldhashes
63 use Hash::Util qw(fieldhash fieldhashes);
65 # Create a single field hash
68 # Create three at once...
69 fieldhashes \ my(%foo, %bar, %baz);
71 fieldhashes @hashrefs;
73 ### Create an idhash and register it for garbage collection
74 use Hash::Util::FieldHash qw( idhash register);
76 my $obj = \ do { my $o };
77 # register the idhash for garbage collection with $object
78 register( $obj, \ %name);
79 # the following entry wil be deleted when $obj goes out of scope
80 $name{ $obj} = 'John Doe';
82 ### Register an ordinary hash for garbage collection
83 use Hash::Util::FieldHash qw( id register);
85 my $obj = \ do { my $o };
86 # register the hash %name for garbage collection of $obj's id
87 register $obj, \ %name;
88 # the following entry wil be deleted when $obj goes out of scope
89 $name{id $obj} = 'John Doe';
94 C<Hash::Util::FieldHash> offers a number of functions in support of
95 L<the inside-out technique> of class construction.
103 Returns the reference address of a reference $obj. If $obj is
104 not a reference, returns $obj.
106 This function is a stand-in replacement for
107 L<Scalar::Util::refaddr|Scalar::Util/refaddr>, that is, it returns
108 the reference address of its argument as a numeric value. The only
109 difference is that C<refaddr()> returns C<undef> when given a
110 non-reference while C<id()> returns its argument unchanged.
112 C<id()> also uses a caching technique that makes it faster when
113 the id of an object is requested often, but slower if it is needed
120 If C<$id> is the id of a registered object (see L</register>), returns
121 the object, otherwise an undefined value. For registered objects this
122 is the inverse function of C<id()>.
127 register($obj, @hashrefs)
129 In the first form, registers an object to work with for the function
130 C<id_2obj()>. In the second form, it additionally marks the given
131 hashrefs down for garbage collection. This means that when the object
132 goes out of scope, any entries in the given hashes under the key of
133 C<id($obj)> will be deleted from the hashes.
135 It is a fatal error to register a non-reference $obj. Any non-hashrefs
136 among the following arguments are silently ignored.
138 It is I<not> an error to register the same object multiple times with
139 varying sets of hashrefs. Any hashrefs that are not registered yet
140 will be added, others ignored.
142 Registry also implies thread support. When a new thread is created,
143 all references are replaced with new ones, including all objects.
144 If a hash uses the reference address of an object as a key, that
145 connection would be broken. With a registered object, its id will
146 be updated in all hashes registered with it.
152 Makes an idhash from the argument, which must be a hash.
154 An I<idhash> works like a normal hash, except that it stringifies a
155 I<reference used as a key> differently. A reference is stringified
156 as if the C<id()> function had been invoked on it, that is, its
157 reference address in decimal is used as the key.
161 idhashes \ my( %hash, %gnash, %trash)
164 Creates many idhashes from its hashref arguments. Returns those
165 arguments that could be converted or their number in scalar context.
171 Creates a single fieldhash. The argument must be a hash. Returns
172 a reference to the given hash if successful, otherwise nothing.
174 A I<fieldhash> is, in short, an idhash with auto-registry. When an
175 object (or, indeed, any reference) is used as a fieldhash key, the
176 fieldhash is automatically registered for garbage collection with
177 the object, as if C<register $obj, \ %fieldhash> had been called.
181 fieldhashes @hashrefs;
183 Creates any number of field hashes. Arguments must be hash references.
184 Returns the converted hashrefs in list context, their number in scalar
191 A word on terminology: I shall use the term I<field> for a scalar
192 piece of data that a class associates with an object. Other terms that
193 have been used for this concept are "object variable", "(object) property",
194 "(object) attribute" and more. Especally "attribute" has some currency
195 among Perl programmer, but that clashes with the C<attributes> pragma. The
196 term "field" also has some currency in this sense and doesn't seem
197 to conflict with other Perl terminology.
199 In Perl, an object is a blessed reference. The standard way of associating
200 data with an object ist to store the data inside the object's body, that is,
201 the piece of data pointed to by the reference.
203 In consequence, if two or more classes want to access an object they
204 I<must> agree on the type of refrerence and also on the organization of
205 data within the object body. Failure to agree on the type results in
206 immediate death when the wrong method tries to access an object. Failure
207 to agree on data organization may lead to one class trampling over the
210 This object model leads to a tight coupling between subclasses.
211 If one class wants to inherit from another (and both classes access
212 object data), the classes must agree about implementation details.
213 Inheritance can only be used among classes that are maintained together,
214 in a single source or not.
216 In particular, it is not possible to write general-purpose classes
217 in this technique, classes that can advertise themselves as "Put me
218 on your @ISA list and use my methods". If the other class has different
219 ideas about how the object body is used, there is trouble.
221 For reference L<Name_hash> in L<Example 1> shows the standard implementation of
222 a simple class C<Name> in the well-known hash based way. It also demonstrates
223 the predictable failure to construct a common subclass C<NamedFile>
224 of C<Name> and the class C<IO::File> (whose objects I<must> be globrefs).
226 Thus, techniques are of interest that store object data I<not> in
227 the object body but some other place.
229 =head2 The Inside-out Technique
231 With I<inside-out> classes, each class declares a (typically lexical)
232 hash for each field it wants to use. The reference address of an
233 object is used as the hash key. By definition, the reference address
234 is unique to each object so this guarantees a place for each field that
235 is private to the class and unique to each object. See L<Name_id> in
236 L<Example 1> for a simple example.
238 In comparison to the standard implementation where the object is a
239 hash and the fields correspond to hash keys, here the fields correspond
240 to hashes, and the object determines the hash key. Thus the hashes
241 appear to be turned I<inside out>.
243 The body of an object is never examined by an inside-out class, only
244 its reference address is used. This allows for the body of an actual
245 object to be I<anything at all> while the object methods of the class
246 still work as designed. This is a key feature of inside-out classes.
248 =head2 Problems of Inside-out
250 Inside-out classes give us freedom of inheritance, but as usual there
253 Most obviously, there is the necessity of retrieving the reference
254 address of an object for each data access. It's a minor inconvenience,
255 but it does clutter the code.
257 More important (and less obvious) is the necessity of garbage
258 collection. When a normal object dies, anything stored in the
259 object body is garbage-collected by perl. With inside-out objects,
260 Perl knows nothing about the data stored in field hashes by a class,
261 but these must be deleted when the object goes out of scope. Thus
262 the class must provide a C<DESTROY> method to take care of that.
264 In the presence of multiple classes it can be non-trivial
265 to make sure that every relevant destructor is called for
266 every object. Perl calls the first one it finds on the
267 inheritance tree (if any) and that's it.
269 A related issue is thread-safety. When a new thread is created,
270 the Perl interpreter is cloned, which implies that all reference
271 addresses in use will be replaced with new ones. Thus, if a class
272 tries to access a field of a cloned object its (cloned) data will
273 still be stored under the now invalid reference address of the
274 original in the parent thread. A general C<CLONE> method must
275 be provided to re-establish the association.
279 C<Hash::Util::FieldHash> addresses these issues on several
282 The C<id()> function is provided in addition to the
283 existing C<Scalar::Util::refaddr()>. Besides its short name
284 it can be a little faster under some circumstances (and a
285 bit slower under others). Benchmark if it matters. The
286 working of C<id()> also allows the use of the class name
287 as a I<generic object> as described L<further down|Generic Object>.
289 The C<id()> function is incorporated in I<id hashes> in the sense
290 that it is called automatically on every key that is used with
291 the hash. No explicit call is necessary.
293 The problems of garbage collection and thread safety are both
294 addressed by the function C<register()>. It registers an object
295 together with any number of hashes. Registry means that when the
296 object dies, an entry in any of the hashes under the reference
297 address of this object will be deleted. This guarantees garbage
298 collection in these hashes. It also means that on thread
299 cloning the object's entries in registered hashes will be
300 replaced with updated entries whose key is the cloned object's
301 reference address. Thus the object-data association becomes
304 Object registry is best done when the object is initialized
305 for use with a class. That way, garbage collection and thread
306 safety are established for every object and every field that is
309 Finally, I<field hashes> incorporate all these functions in one
310 package. Besides automatically calling the C<id()> function
311 on every object used as a key, the object is registered with
312 the field hash on first use. Classes based on field hashes
313 are fully garbage-collected and thread safe without further
318 Another problem that occurs with inside-out classes is serialization.
319 Since the object data is not in its usual place, standard routines
320 like C<Storable::freeze()>, C<Storable::thaw()> and
321 C<Data::Dumper::Dumper()> can't deal with it on their own. Both
322 C<Data::Dumper> and C<Storable> provide the necessary hooks to
323 make things work, but the functions or methods used by the hooks
324 must be provided by each inside-out class.
326 A general solution to the serialization problem would require another
327 level of registry, one that that associates I<classes> and fields.
328 So far, he functions of C<Hash::Util::Fieldhash> are unaware of
329 any classes, which I consider a feature. Therefore C<Hash::Util::FieldHash>
330 doesn't address the serialization problems.
332 =head2 The Generic Object
334 Classes based on the C<id()> function (and hence classes based on
335 C<idhash()> and C<fieldhash()>) show a peculiar behavior in that
336 the class name can be used like an object. Specifically, methods
337 that set or read data associated with an object continue to work as
338 class methods, just as if the class name were an object, distinct from
339 all other objects, with its own data. This object may be called
340 the I<generic object> of the class.
342 This works because field hashes respond to keys that are not refrences
343 like a normal hash would and use the string offered as the hash key.
344 Thus, if a method is called as a class method, the field hash is presented
345 with the class name instead of an object and blithely uses it as a key.
346 Since the keys of real objects are decimal numbers, there is no
347 conflict and the slot in the field hash can be used like any other.
348 The C<id()> function behaves correspondingly with respect to non-reference
351 Two possible uses (besides ignoring the property) come to mind.
352 A singleton class could be implemented this using the generic object.
353 If necessary, an C<init()> method could die or ignore calls with
354 actual objects (references), so only the generic object will ever exist.
356 Another use of the generic object would be as a template. It is
357 a convenient place to store class-specific defaults for various
358 fields to be used in actual object initialization.
360 Usually, the feature can be entirely ignored. Calling I<object
361 methods> as I<class methods> normally leads to an error and isn't used
362 routinely anywhere. It may be a problem that this error isn't
363 indicated by a class with a generic object.
365 =head2 How to use Field Hashes
367 Traditionally, the definition of an inside-out class contains a bare
368 block inside which a number of lexical hashes are declared and the
369 basic accessor methods defined, usually through C<Scalar::Util::refaddr>.
370 Further methods may be defined outside this block. There has to be
371 a DESTROY method and, for thread support, a CLONE method.
373 When field hashes are used, the basic structure reamins the same.
374 Each lexical hash will be made a field hash. The call to C<refaddr>
375 can be omitted from the accessor methods. DESTROY and CLONE methods
378 If you have an existing inside-out class, simply making all hashes
379 field hashes with no other change should make no difference. Through
380 the calls to C<refaddr> or equivalent, the field hashes never get to
381 see a reference and work like normal hashes. Your DESTROY (and
382 CLONE) methods are still needed.
384 To make the field hashes kick in, it is easiest to redefine C<refaddr>
387 sub refaddr { shift }
389 instead of importing it from C<Scalar::Util>. It should now be possible
390 to disable DESTROY and CLONE. Note that while it isn't disabled,
391 DESTROY will be called before the garbage collection of field hashes,
392 so it will be invoked with a functional object and will continue to
395 It is not desirable to import the functions C<fieldhash> and/or
396 C<fieldhashes> into every class that is going to use them. They
397 are only used once to set up the class. When the class is up and running,
398 these functions serve no more purpose.
400 If there are only a few field hashes to declare, it is simplest to
402 use Hash::Util::FieldHash;
404 early and call the functions qualified:
406 Hash::Util::FieldHash::fieldhash my %foo;
408 Otherwise, import the functions into a convenient package like
409 C<HUF> or, more general, C<Aux>
413 use Hash::Util::FieldHash ':all';
418 Aux::fieldhash my %foo;
422 =head2 Garbage-Collected Hashes
424 Garbage collection in a field hash means that entries will "spontaneously"
425 disappear when the object that created them disappears. That must be
426 borne in mind, especially when looping over a field hash. If anything
427 you do inside the loop could cause an object to go out of scope, a
428 random key may be deleted from the hash you are looping over. That
429 can throw the loop iterator, so it's best to cache a consistent snapshot
430 of the keys and/or values and loop over that. You will still have to
431 check that a cached entry still exists when you get to it.
433 Garbage collection can be confusing when keys are created in a field hash
434 from normal scalars as well as references. Once a reference is I<used> with
435 a field hash, the entry will be collected, even if it was later overwritten
436 with a plain scalar key (every positive integer is a candidate). This
437 is true even if the original entry was deleted in the meantime. In fact,
438 deletion from a field hash, and also a test for existence constitute
439 I<use> in this sense and create a liability to delete the entry when
440 the reference goes out of scope. If you happen to create an entry
441 with an identical key from a string or integer, that will be collected
442 instead. Thus, mixed use of references and plain scalars as field hash
443 keys is not entirely supported.
447 The examples show a very simple class that implements a I<name>, consisting
448 of a first and last name (no middle initial). The name class has four
455 An object method that initializes the first and last name to its
456 two arguments. If called as a class method, C<init()> creates an
457 object in the given class and initializes that.
461 Retrieve the first name
465 Retrieve the last name
469 Retrieve the full name, the first and last name joined by a blank.
473 The examples show this class implemented with different levels of
474 support by C<Hash::Util::FieldHash>. All supported combinations
475 are shown. The difference between implementations is often quite
476 small. The implementations are:
482 A conventional (not inside-out) implementation where an object is
483 a hash that stores the field values, without support by
484 C<Hash::Util::FieldHash>. This implementation doesn't allow
485 arbitrary inheritance.
489 Inside-out implementation based on the C<id()> function. It needs
490 a C<DESTROY> method. For thread support a C<CLONE> method (not shown)
491 would also be needed. Instead of C<Hash::Util::FieldHash::id()> the
492 function C<Scalar::Util::refaddr> could be used with very little
493 functional difference. This is the basic pattern of an inside-out
496 =item * C<Name_idhash>
498 Idhash-based inside-out implementation. Like L<Name_id> it needs
499 a C<DESTROY> method and would need C<CLONE> for thread support.
501 =item * C<Name_id_reg>
503 Inside-out implementation based on the C<id()> function with explicit
504 object registry. No destructor is needed and objects are thread safe.
506 =item * C<Name_idhash_reg>
508 Idhash-based inside-out implementation with explicit object registry.
509 No destructor is needed and objects are thread safe.
511 =item * C<Name_fieldhash>
513 Fieldhash-based inside-out implementation. Object registry happens
514 automatically. No destructor is needed and objects are thread safe.
518 These examples are realized in the code below, which could be copied
519 to a file F<Example.pm>.
523 use strict; use warnings;
526 package Name_hash; # standard implementation: the object is a hash
530 my ( $first, $last) = @_;
531 # create an object if called as class method
532 $obj = bless {}, $obj unless ref $obj;
533 $obj->{ first} = $first;
534 $obj->{ last} = $last;
538 sub first { shift()->{ first} }
539 sub last { shift()->{ last} }
543 join ' ' => $n->first, $n->last;
550 use Hash::Util::FieldHash qw( id);
556 my ( $first, $last) = @_;
557 # create an object if called as class method
558 $obj = bless \ my $o, $obj unless ref $obj;
559 $first{ id $obj} = $first;
560 $last{ id $obj} = $last;
564 sub first { $first{ id shift()} }
565 sub last { $last{ id shift()} }
569 join ' ' => $n->first, $n->last;
582 use Hash::Util::FieldHash;
584 Hash::Util::FieldHash::idhashes( \ my ( %first, %last));
588 my ( $first, $last) = @_;
589 # create an object if called as class method
590 $obj = bless \ my $o, $obj unless ref $obj;
591 $first{ $obj} = $first;
592 $last{ $obj} = $last;
596 sub first { $first{ shift()} }
597 sub last { $last{ shift()} }
601 join ' ' => $n->first, $n->last;
614 use Hash::Util::FieldHash qw( id register);
620 my ( $first, $last) = @_;
621 # create an object if called as class method
622 $obj = bless \ my $o, $obj unless ref $obj;
623 register( $obj, \ ( %first, %last));
624 $first{ id $obj} = $first;
625 $last{ id $obj} = $last;
629 sub first { $first{ id shift()} }
630 sub last { $last{ id shift()} }
634 join ' ' => $n->first, $n->last;
639 package Name_idhash_reg;
640 use Hash::Util::FieldHash qw( register);
642 Hash::Util::FieldHash::idhashes \ my ( %first, %last);
646 my ( $first, $last) = @_;
647 # create an object if called as class method
648 $obj = bless \ my $o, $obj unless ref $obj;
649 register( $obj, \ ( %first, %last));
650 $first{ $obj} = $first;
651 $last{ $obj} = $last;
655 sub first { $first{ shift()} }
656 sub last { $last{ shift()} }
660 join ' ' => $n->first, $n->last;
665 package Name_fieldhash;
666 use Hash::Util::FieldHash;
668 Hash::Util::FieldHash::fieldhashes \ my ( %first, %last);
672 my ( $first, $last) = @_;
673 # create an object if called as class method
674 $obj = bless \ my $o, $obj unless ref $obj;
675 $first{ $obj} = $first;
676 $last{ $obj} = $last;
680 sub first { $first{ shift()} }
681 sub last { $last{ shift()} }
685 join ' ' => $n->first, $n->last;
691 To exercise the various implementations the script L<below|Example 2> can
694 It sets up a class C<Name> that is a mirror of one of the implementation
695 classes C<Name_hash>, C<Name_id>, ..., C<Name_fieldhash>. That determines
696 which implementation is run.
698 The script first verifies the function of the C<Name> class.
700 In the second step, the free inheritablility of the implementation
701 (or lack thereof) is demonstrated. For this purpose it constructs
702 a class called C<NamedFile> which is a common subclass of C<Name> and
703 the standard class C<IO::File>. This puts inheritability to the test
704 because objects of C<IO::File> I<must> be globrefs. Objects of C<NamedFile>
705 should behave like a file opened for reading and also support the C<name()>
706 method. This class juncture works with exception of the C<Name_hash>
707 implementation, where object initialization fails because of the
708 incompatibility of object bodies.
712 use strict; use warnings; $| = 1;
718 use base 'Name_id'; # define here which implementation to run
722 # Verify that the base package works
723 my $n = Name->init( qw( Albert Einstein));
724 print $n->name, "\n";
727 # Create a named file handle (See definition below)
728 my $nf = NamedFile->init( qw( /tmp/x Filomena File));
729 # use as a file handle...
734 # ...and as a Name object
735 print "...brought to you by ", $nf->name, "\n";
739 # Definition of NamedFile
746 my ( $file, $first, $last) = @_;
747 $obj = $obj->IO::File::new() unless ref $obj;
748 $obj->open( $file) or die "Can't read '$file': $!";
749 $obj->Name::init( $first, $last);
757 To make C<Hash::Util::FieldHash> work, there were two changes to
758 F<perl> itself. C<PERL_MAGIC_uvar> was made avaliable for hashes,
759 and weak references now call uvar C<get> magic after a weakref has been
760 cleared. The first feature is used to make field hashes intercept
761 their keys upon access. The second one triggers garbage collection.
763 =head2 The C<PERL_MAGIC_uvar> interface for hashes
765 C<PERL_MAGIC_uvar> I<get> magic is called from C<hv_fetch_common> and
766 C<hv_delete_common> through the function C<hv_magic_uvar_xkey>, which
767 defines the interface. The call happens for hashes with "uvar" magic
768 if the C<ufuncs> structure has equal values in the C<uf_val> and C<uf_set>
769 fields. Hashes are unaffected if (and as long as) these fields
770 hold different values.
772 Upon the call, the C<mg_obj> field will hold the hash key to be accessed.
773 Upon return, the C<SV*> value in C<mg_obj> will be used in place of the
774 original key in the hash access. The integer index value in the first
775 parameter will be the C<action> value from C<hv_fetch_common>, or -1
776 if the call is from C<hv_delete_common>.
778 This is a template for a function suitable for the C<uf_val> field in
779 a C<ufuncs> structure for this call. The C<uf_set> and C<uf_index>
780 fields are irrelevant.
782 IV watch_key(pTHX_ IV action, SV* field) {
783 MAGIC* mg = mg_find(field, PERL_MAGIC_uvar);
784 SV* keysv = mg->mg_obj;
785 /* Do whatever you need to. If you decide to
786 supply a different key newkey, return it like this
793 =head2 Weakrefs call uvar magic
795 When a weak reference is stored in an C<SV> that has "uvar" magic, C<set>
796 magic is called after the reference has gone stale. This hook can be
797 used to trigger further garbage-collection activities associated with
798 the referenced object.
800 =head2 How field hashes work
802 The three features of key hashes, I<key replacement>, I<thread support>,
803 and I<garbage collection> are supported by a data structure called
804 the I<object registry>. This is a private hash where every object
805 is stored. An "object" in this sense is any reference (blessed or
806 unblessed) that has been used as a field hash key.
808 The object registry keeps track of references that have been used as
809 field hash keys. The keys are generated from the reference address
810 like in a field hash (though the registry isn't a field hash). Each
811 value is a weak copy of the original reference, stored in an C<SV> that
812 is itself magical (C<PERL_MAGIC_uvar> again). The magical structure
813 holds a list (another hash, really) of field hashes that the reference
814 has been used with. When the weakref becomes stale, the magic is
815 activated and uses the list to delete the reference from all field
816 hashes it has been used with. After that, the entry is removed from
817 the object registry itself. Implicitly, that frees the magic structure
818 and the storage it has been using.
820 Whenever a reference is used as a field hash key, the object registry
821 is checked and a new entry is made if necessary. The field hash is
822 then added to the list of fields this reference has used.
824 The object registry is also used to repair a field hash after thread
825 cloning. Here, the entire object registry is processed. For every
826 reference found there, the field hashes it has used are visited and
827 the entry is updated.
829 =head2 Internal function Hash::Util::FieldHash::_fieldhash
831 # test if %hash is a field hash
832 my $result = _fieldhash \ %hash, 0;
834 # make %hash a field hash
835 my $result = _fieldhash \ %hash, 1;
837 C<_fieldhash> is the internal function used to create field hashes.
838 It takes two arguments, a hashref and a mode. If the mode is boolean
839 false, the hash is not changed but tested if it is a field hash. If
840 the hash isn't a field hash the return value is boolean false. If it
841 is, the return value indicates the mode of field hash. When called with
842 a boolean true mode, it turns the given hash into a field hash of this
843 mode, returning the mode of the created field hash. C<_fieldhash>
844 does not erase the given hash.
846 Currently there is only one type of field hash, and only the boolean
847 value of the mode makes a difference, but that may change.
851 Anno Siegel, E<lt>anno4000@zrz.tu-berlin.deE<gt>
853 =head1 COPYRIGHT AND LICENSE
855 Copyright (C) 2006 by (Anno Siegel)
857 This library is free software; you can redistribute it and/or modify
858 it under the same terms as Perl itself, either Perl version 5.8.7 or,
859 at your option, any later version of Perl 5 you may have available.