4 use vars qw($VERSION @ISA @EXPORT_OK %EXPORT_TAGS $warn $dangle);
12 @EXPORT_OK = qw(size total_size);
14 # This allows declaration use Devel::Size ':all';
15 %EXPORT_TAGS = ( 'all' => \@EXPORT_OK );
19 XSLoader::load( __PACKAGE__);
22 $dangle = 0; ## Set true to enable warnings about dangling pointers
31 Devel::Size - Perl extension for finding the memory usage of Perl variables
35 use Devel::Size qw(size total_size);
37 my $size = size("A string");
39 my @foo = (1, 2, 3, 4, 5);
40 my $other_size = size(\@foo);
42 my $foo = {a => [1, 2, 3],
45 my $total_size = total_size($foo);
49 This module figures out the real size of Perl variables in bytes, as
50 accurately as possible.
52 Call functions with a reference to the variable you want the size
53 of. If the variable is a plain scalar it returns the size of
54 this scalar. If the variable is a hash or an array, use a reference
61 The C<size> function returns the amount of memory the variable
62 returns. If the variable is a hash or an array, it only reports
63 the amount used by the structure, I<not> the contents.
65 =head2 total_size($ref)
67 The C<total_size> function will traverse the variable and look
68 at the sizes of contents. Any references contained in the variable
69 will also be followed, so this function can be used to get the
70 total size of a multidimensional data structure. At the moment
71 there is no way to get the size of an array or a hash and its
72 elements without using this function.
76 None but default, but optionally C<size> and C<total_size>.
78 =head1 UNDERSTANDING MEMORY ALLOCATION
80 Please note that the following discussion of memory allocation in perl
81 is based on the perl 5.8.0 sources. While this is generally
82 applicable to all versions of perl, some of the gory details are
83 omitted. It also makes some presumptions on how your system memory
84 allocator works so, while it will be generally correct, it may not
85 exactly reflect your system. (Generally the only issue is the size of
86 the constant values we'll talk about, not their existence)
90 It's important first to understand how your OS and libraries handle
91 memory. When the perl interpreter needs some memory, it asks the C
92 runtime library for it, using the C<malloc()> call. C<malloc> has one
93 parameter, the size of the memory allocation you want, and returns a
94 pointer to that memory. C<malloc> also makes sure that the pointer it
95 returns to you is properly aligned. When you're done with the memory
96 you hand it back to the library with the C<free()> call. C<free> has
97 one parameter, the pointer that C<malloc> returned.
98 There are a couple of interesting ramifications to this.
100 Because malloc has to return an aligned pointer, it will round up the
101 memory allocation to make sure that the memory it returns is aligned
102 right. What that alignment is depends on your CPU, OS, and compiler
103 settings, but things are generally aligned to either a 4 or 8 byte
104 boundary. That means that if you ask for 1 byte, C<malloc> will
105 silently round up to either 4 or 8 bytes, though it doesn't tell the
106 program making the request, so the extra memory can't be used.
108 Since C<free> isn't given the size of the memory chunk you're
109 freeing, it has to track it another way. Most libraries do this by
110 tacking on a length field just before the memory it hands to your
111 program. (It's put before the beginning rather than after the end
112 because it's less likely to get mangled by program bugs) This size
113 field is the size of your platform integer, Generally either 4 or 8
116 So, if you asked for 1 byte, malloc would build something like this:
120 +------------------+ <----- the pointer malloc returns
126 As you can see, you asked for 1 byte but C<malloc> used 8. If your
127 integers were 8 bytes rather than 4, C<malloc> would have used 16 bytes
128 to satisfy your 1 byte request.
130 The C memory allocation system also keeps a list of free memory
131 chunks, so it can recycle freed memory. For performance reasons, some
132 C memory allocation systems put a limit to the number of free
133 segments that are on the free list, or only search through a small
134 number of memory chunks waiting to be recycled before just
135 allocating more memory from the system.
137 The memory allocation system tries to keep as few chunks on the free
138 list as possible. It does this by trying to notice if there are two
139 adjacent chunks of memory on the free list and, if there are,
140 coalescing them into a single larger chunk. This works pretty well,
141 but there are ways to have a lot of memory on the free list yet still
142 not have anything that can be allocated. If a program allocates one
143 million eight-byte chunks, for example, then frees every other chunk,
144 there will be four million bytes of memory on the free list, but none
145 of that memory can be handed out to satisfy a request for 10
146 bytes. This is what's referred to as a fragmented free list, and can
147 be one reason why your program could have a lot of free memory yet
148 still not be able to allocate more, or have a huge process size and
149 still have almost no memory actually allocated to the program running.
153 Perl's memory allocation scheme is a bit convoluted, and more complex
154 than can really be addressed here, but there is one common spot where Perl's
155 memory allocation is unintuitive, and that's for hash keys.
157 When you have a hash, each entry has a structure that points to the
158 key and the value for that entry. The value is just a pointer to the
159 scalar in the entry, and doesn't take up any special amount of
160 memory. The key structure holds the hash value for the key, the key
161 length, and the key string. (The entry and key structures are
162 separate so perl can potentially share keys across multiple hashes)
164 The entry structure has three pointers in it, and takes up either 12
165 or 24 bytes, depending on whether you're on a 32 bit or 64 bit
166 system. Since these structures are of fixed size, perl can keep a big
167 pool of them internally (generally called an arena) so it doesn't
168 have to allocate memory for each one.
170 The key structure, though, is of variable length because the key
171 string is of variable length, so perl has to ask the system for a
172 memory allocation for each key. The base size of this structure is
173 8 or 16 bytes (once again, depending on whether you're on a 32 bit or
174 64 bit system) plus the string length plus two bytes.
176 Since this memory has to be allocated from the system there's the
177 malloc size-field overhead (4 or 8 bytes) plus the alignment bytes (0
178 to 7, depending on your system and the key length)
179 that get added on to the chunk perl requests. If the key is only 1
180 character, and you're on a 32 bit system, the allocation will be 16
181 bytes. If the key is 7 characters then the allocation is 24 bytes on
182 a 32 bit system. If you're on a 64 bit system the numbers get even
187 Since version 0.72, Devel::Size uses a new pointer tracking mechanism
188 that consumes far less memory than was previously the case. It does this
189 by using a bit vector where 1 bit represents each 4- or 8-byte aligned pointer
190 (32- or 64-bit platform dependent) that could exist. Further, it segments
191 that bit vector and only allocates each chunk when an address is seen within
192 that chunk. Since version 0.73, chunks are allocated in blocks of 2**16 bits
193 (ie 8K), accessed via a 256-way tree. The tree is 2 levels deep on a 32 bit
194 system, 6 levels deep on a 64 bit system. This avoids having make any
195 assumptions about address layout on 64 bit systems or trade offs about sizes
196 to allocate. It assumes that the addresses of allocated pointers are reasonably
197 contiguous, so that relevant parts of the tree stay in the CPU cache.
199 Besides saving a lot of memory, this change means that Devel::Size
200 runs significantly faster than previous versions.
202 =head1 Messages: texts originating from this module.
208 =item "Devel::Size: Unknown variable type"
210 The thing (or something contained within it) that you gave to
211 total_size() was unrecognisable as a Perl entity.
217 These messages warn you that for some types, the sizes calculated may not include
218 everything that could be associated with those types. The differences are usually
219 insignificant for most uses of this module.
221 These may be disabled by setting
223 $Devel::Size::warn = 0
227 =item "Devel::Size: Calculated sizes for CVs are incomplete"
229 =item "Devel::Size: Calculated sizes for FMs are incomplete"
231 =item "Devel::Size: Calculated sizes for compiled regexes are incompatible, and probably always will be"
235 =head2 New warnings since 0.72
237 Devel::Size has always been vulnerable to trapping when traversing Perl's
238 internal data structures, if it encounters uninitialised (dangling) pointers.
240 MSVC provides exception handling able to deal with this possibility, and when
241 built with MSVC Devel::Size will now attempt to ignore (or log) them and
242 continue. These messages are mainly of interest to Devel::Size and core
243 developers, and so are disabled by default.
245 They may be enabled by setting
247 $Devel::Size::dangle = 0
251 =item "Devel::Size: Can't determine class of operator OPx_XXXX, assuming BASEOP\n"
253 =item "Devel::Size: Encountered bad magic at: 0xXXXXXXXX"
255 =item "Devel::Size: Encountered dangling pointer in opcode at: 0xXXXXXXXX"
257 =item "Devel::Size: Encountered invalid pointer: 0xXXXXXXXX"
263 Doesn't currently walk all the bits for code refs, formats, and
264 IO. Those throw a warning, but a minimum size for them is returned.
266 Devel::Size only counts the memory that perl actually allocates. It
267 doesn't count 'dark' memory--memory that is lost due to fragmented free lists,
268 allocation alignments, or C library overhead.
272 Dan Sugalski dan@sidhe.org
274 Small portion taken from the B module as shipped with perl 5.6.2.
276 Previously maintained by Tels <http://bloodgate.com>
278 New pointer tracking & exception handling for 0.72 by BrowserUK
280 Currently maintained by Nicholas Clark
284 Copyright (C) 2005 Dan Sugalski, Copyright (C) 2007-2008 Tels
286 This module is free software; you can redistribute it and/or modify it
287 under the same terms as Perl v5.8.8.
291 perl(1), L<Devel::Size::Report>.