Commit | Line | Data |
a0d0e21e |
1 | =head1 NAME |
2 | |
954c1994 |
3 | perlguts - Introduction to the Perl API |
a0d0e21e |
4 | |
5 | =head1 DESCRIPTION |
6 | |
b3b6085d |
7 | This document attempts to describe how to use the Perl API, as well as |
06f6df17 |
8 | to provide some info on the basic workings of the Perl core. It is far |
b3b6085d |
9 | from complete and probably contains many errors. Please refer any |
10 | questions or comments to the author below. |
a0d0e21e |
11 | |
0a753a76 |
12 | =head1 Variables |
13 | |
5f05dabc |
14 | =head2 Datatypes |
a0d0e21e |
15 | |
16 | Perl has three typedefs that handle Perl's three main data types: |
17 | |
18 | SV Scalar Value |
19 | AV Array Value |
20 | HV Hash Value |
21 | |
d1b91892 |
22 | Each typedef has specific routines that manipulate the various data types. |
a0d0e21e |
23 | |
24 | =head2 What is an "IV"? |
25 | |
954c1994 |
26 | Perl uses a special typedef IV which is a simple signed integer type that is |
5f05dabc |
27 | guaranteed to be large enough to hold a pointer (as well as an integer). |
954c1994 |
28 | Additionally, there is the UV, which is simply an unsigned IV. |
a0d0e21e |
29 | |
d1b91892 |
30 | Perl also uses two special typedefs, I32 and I16, which will always be at |
954c1994 |
31 | least 32-bits and 16-bits long, respectively. (Again, there are U32 and U16, |
20dbd849 |
32 | as well.) They will usually be exactly 32 and 16 bits long, but on Crays |
33 | they will both be 64 bits. |
a0d0e21e |
34 | |
54310121 |
35 | =head2 Working with SVs |
a0d0e21e |
36 | |
20dbd849 |
37 | An SV can be created and loaded with one command. There are five types of |
38 | values that can be loaded: an integer value (IV), an unsigned integer |
39 | value (UV), a double (NV), a string (PV), and another scalar (SV). |
a0d0e21e |
40 | |
20dbd849 |
41 | The seven routines are: |
a0d0e21e |
42 | |
43 | SV* newSViv(IV); |
20dbd849 |
44 | SV* newSVuv(UV); |
a0d0e21e |
45 | SV* newSVnv(double); |
06f6df17 |
46 | SV* newSVpv(const char*, STRLEN); |
47 | SV* newSVpvn(const char*, STRLEN); |
46fc3d4c |
48 | SV* newSVpvf(const char*, ...); |
a0d0e21e |
49 | SV* newSVsv(SV*); |
50 | |
06f6df17 |
51 | C<STRLEN> is an integer type (Size_t, usually defined as size_t in |
52 | F<config.h>) guaranteed to be large enough to represent the size of |
53 | any string that perl can handle. |
54 | |
55 | In the unlikely case of a SV requiring more complex initialisation, you |
56 | can create an empty SV with newSV(len). If C<len> is 0 an empty SV of |
57 | type NULL is returned, else an SV of type PV is returned with len + 1 (for |
58 | the NUL) bytes of storage allocated, accessible via SvPVX. In both cases |
59 | the SV has value undef. |
20dbd849 |
60 | |
06f6df17 |
61 | SV *sv = newSV(0); /* no storage allocated */ |
62 | SV *sv = newSV(10); /* 10 (+1) bytes of uninitialised storage allocated */ |
20dbd849 |
63 | |
06f6df17 |
64 | To change the value of an I<already-existing> SV, there are eight routines: |
a0d0e21e |
65 | |
66 | void sv_setiv(SV*, IV); |
deb3007b |
67 | void sv_setuv(SV*, UV); |
a0d0e21e |
68 | void sv_setnv(SV*, double); |
08105a92 |
69 | void sv_setpv(SV*, const char*); |
06f6df17 |
70 | void sv_setpvn(SV*, const char*, STRLEN) |
46fc3d4c |
71 | void sv_setpvf(SV*, const char*, ...); |
5b3218b9 |
72 | void sv_vsetpvfn(SV*, const char*, STRLEN, va_list *, SV **, I32, bool *); |
a0d0e21e |
73 | void sv_setsv(SV*, SV*); |
74 | |
75 | Notice that you can choose to specify the length of the string to be |
9da1e3b5 |
76 | assigned by using C<sv_setpvn>, C<newSVpvn>, or C<newSVpv>, or you may |
77 | allow Perl to calculate the length by using C<sv_setpv> or by specifying |
78 | 0 as the second argument to C<newSVpv>. Be warned, though, that Perl will |
79 | determine the string's length by using C<strlen>, which depends on the |
9abd00ed |
80 | string terminating with a NUL character. |
81 | |
82 | The arguments of C<sv_setpvf> are processed like C<sprintf>, and the |
83 | formatted output becomes the value. |
84 | |
328bf373 |
85 | C<sv_vsetpvfn> is an analogue of C<vsprintf>, but it allows you to specify |
9abd00ed |
86 | either a pointer to a variable argument list or the address and length of |
87 | an array of SVs. The last argument points to a boolean; on return, if that |
88 | boolean is true, then locale-specific information has been used to format |
c2611fb3 |
89 | the string, and the string's contents are therefore untrustworthy (see |
9abd00ed |
90 | L<perlsec>). This pointer may be NULL if that information is not |
91 | important. Note that this function requires you to specify the length of |
92 | the format. |
93 | |
9da1e3b5 |
94 | The C<sv_set*()> functions are not generic enough to operate on values |
95 | that have "magic". See L<Magic Virtual Tables> later in this document. |
a0d0e21e |
96 | |
a3cb178b |
97 | All SVs that contain strings should be terminated with a NUL character. |
98 | If it is not NUL-terminated there is a risk of |
5f05dabc |
99 | core dumps and corruptions from code which passes the string to C |
100 | functions or system calls which expect a NUL-terminated string. |
101 | Perl's own functions typically add a trailing NUL for this reason. |
102 | Nevertheless, you should be very careful when you pass a string stored |
103 | in an SV to a C function or system call. |
104 | |
a0d0e21e |
105 | To access the actual value that an SV points to, you can use the macros: |
106 | |
107 | SvIV(SV*) |
954c1994 |
108 | SvUV(SV*) |
a0d0e21e |
109 | SvNV(SV*) |
110 | SvPV(SV*, STRLEN len) |
1fa8b10d |
111 | SvPV_nolen(SV*) |
a0d0e21e |
112 | |
954c1994 |
113 | which will automatically coerce the actual scalar type into an IV, UV, double, |
a0d0e21e |
114 | or string. |
115 | |
116 | In the C<SvPV> macro, the length of the string returned is placed into the |
1fa8b10d |
117 | variable C<len> (this is a macro, so you do I<not> use C<&len>). If you do |
118 | not care what the length of the data is, use the C<SvPV_nolen> macro. |
119 | Historically the C<SvPV> macro with the global variable C<PL_na> has been |
120 | used in this case. But that can be quite inefficient because C<PL_na> must |
121 | be accessed in thread-local storage in threaded Perl. In any case, remember |
122 | that Perl allows arbitrary strings of data that may both contain NULs and |
123 | might not be terminated by a NUL. |
a0d0e21e |
124 | |
ce2f5d8f |
125 | Also remember that C doesn't allow you to safely say C<foo(SvPV(s, len), |
126 | len);>. It might work with your compiler, but it won't work for everyone. |
127 | Break this sort of statement up into separate assignments: |
128 | |
b2f5ed49 |
129 | SV *s; |
ce2f5d8f |
130 | STRLEN len; |
131 | char * ptr; |
b2f5ed49 |
132 | ptr = SvPV(s, len); |
ce2f5d8f |
133 | foo(ptr, len); |
134 | |
07fa94a1 |
135 | If you want to know if the scalar value is TRUE, you can use: |
a0d0e21e |
136 | |
137 | SvTRUE(SV*) |
138 | |
139 | Although Perl will automatically grow strings for you, if you need to force |
140 | Perl to allocate more memory for your SV, you can use the macro |
141 | |
142 | SvGROW(SV*, STRLEN newlen) |
143 | |
144 | which will determine if more memory needs to be allocated. If so, it will |
145 | call the function C<sv_grow>. Note that C<SvGROW> can only increase, not |
5f05dabc |
146 | decrease, the allocated memory of an SV and that it does not automatically |
147 | add a byte for the a trailing NUL (perl's own string functions typically do |
8ebc5c01 |
148 | C<SvGROW(sv, len + 1)>). |
a0d0e21e |
149 | |
150 | If you have an SV and want to know what kind of data Perl thinks is stored |
151 | in it, you can use the following macros to check the type of SV you have. |
152 | |
153 | SvIOK(SV*) |
154 | SvNOK(SV*) |
155 | SvPOK(SV*) |
156 | |
157 | You can get and set the current length of the string stored in an SV with |
158 | the following macros: |
159 | |
160 | SvCUR(SV*) |
161 | SvCUR_set(SV*, I32 val) |
162 | |
cb1a09d0 |
163 | You can also get a pointer to the end of the string stored in the SV |
164 | with the macro: |
165 | |
166 | SvEND(SV*) |
167 | |
168 | But note that these last three macros are valid only if C<SvPOK()> is true. |
a0d0e21e |
169 | |
d1b91892 |
170 | If you want to append something to the end of string stored in an C<SV*>, |
171 | you can use the following functions: |
172 | |
08105a92 |
173 | void sv_catpv(SV*, const char*); |
e65f3abd |
174 | void sv_catpvn(SV*, const char*, STRLEN); |
46fc3d4c |
175 | void sv_catpvf(SV*, const char*, ...); |
328bf373 |
176 | void sv_vcatpvfn(SV*, const char*, STRLEN, va_list *, SV **, I32, bool); |
d1b91892 |
177 | void sv_catsv(SV*, SV*); |
178 | |
179 | The first function calculates the length of the string to be appended by |
180 | using C<strlen>. In the second, you specify the length of the string |
46fc3d4c |
181 | yourself. The third function processes its arguments like C<sprintf> and |
9abd00ed |
182 | appends the formatted output. The fourth function works like C<vsprintf>. |
183 | You can specify the address and length of an array of SVs instead of the |
184 | va_list argument. The fifth function extends the string stored in the first |
185 | SV with the string stored in the second SV. It also forces the second SV |
186 | to be interpreted as a string. |
187 | |
188 | The C<sv_cat*()> functions are not generic enough to operate on values that |
189 | have "magic". See L<Magic Virtual Tables> later in this document. |
d1b91892 |
190 | |
a0d0e21e |
191 | If you know the name of a scalar variable, you can get a pointer to its SV |
192 | by using the following: |
193 | |
4929bf7b |
194 | SV* get_sv("package::varname", FALSE); |
a0d0e21e |
195 | |
196 | This returns NULL if the variable does not exist. |
197 | |
d1b91892 |
198 | If you want to know if this variable (or any other SV) is actually C<defined>, |
a0d0e21e |
199 | you can call: |
200 | |
201 | SvOK(SV*) |
202 | |
06f6df17 |
203 | The scalar C<undef> value is stored in an SV instance called C<PL_sv_undef>. |
9adebda4 |
204 | |
205 | Its address can be used whenever an C<SV*> is needed. Make sure that |
206 | you don't try to compare a random sv with C<&PL_sv_undef>. For example |
207 | when interfacing Perl code, it'll work correctly for: |
208 | |
209 | foo(undef); |
210 | |
211 | But won't work when called as: |
212 | |
213 | $x = undef; |
214 | foo($x); |
215 | |
216 | So to repeat always use SvOK() to check whether an sv is defined. |
217 | |
218 | Also you have to be careful when using C<&PL_sv_undef> as a value in |
219 | AVs or HVs (see L<AVs, HVs and undefined values>). |
a0d0e21e |
220 | |
06f6df17 |
221 | There are also the two values C<PL_sv_yes> and C<PL_sv_no>, which contain |
222 | boolean TRUE and FALSE values, respectively. Like C<PL_sv_undef>, their |
223 | addresses can be used whenever an C<SV*> is needed. |
a0d0e21e |
224 | |
9cde0e7f |
225 | Do not be fooled into thinking that C<(SV *) 0> is the same as C<&PL_sv_undef>. |
a0d0e21e |
226 | Take this code: |
227 | |
228 | SV* sv = (SV*) 0; |
229 | if (I-am-to-return-a-real-value) { |
230 | sv = sv_2mortal(newSViv(42)); |
231 | } |
232 | sv_setsv(ST(0), sv); |
233 | |
234 | This code tries to return a new SV (which contains the value 42) if it should |
04343c6d |
235 | return a real value, or undef otherwise. Instead it has returned a NULL |
a0d0e21e |
236 | pointer which, somewhere down the line, will cause a segmentation violation, |
06f6df17 |
237 | bus error, or just weird results. Change the zero to C<&PL_sv_undef> in the |
238 | first line and all will be well. |
a0d0e21e |
239 | |
240 | To free an SV that you've created, call C<SvREFCNT_dec(SV*)>. Normally this |
3fe9a6f1 |
241 | call is not necessary (see L<Reference Counts and Mortality>). |
a0d0e21e |
242 | |
94dde4fb |
243 | =head2 Offsets |
244 | |
245 | Perl provides the function C<sv_chop> to efficiently remove characters |
246 | from the beginning of a string; you give it an SV and a pointer to |
da75cd15 |
247 | somewhere inside the PV, and it discards everything before the |
94dde4fb |
248 | pointer. The efficiency comes by means of a little hack: instead of |
249 | actually removing the characters, C<sv_chop> sets the flag C<OOK> |
250 | (offset OK) to signal to other functions that the offset hack is in |
251 | effect, and it puts the number of bytes chopped off into the IV field |
252 | of the SV. It then moves the PV pointer (called C<SvPVX>) forward that |
00aadd71 |
253 | many bytes, and adjusts C<SvCUR> and C<SvLEN>. |
94dde4fb |
254 | |
255 | Hence, at this point, the start of the buffer that we allocated lives |
256 | at C<SvPVX(sv) - SvIV(sv)> in memory and the PV pointer is pointing |
257 | into the middle of this allocated storage. |
258 | |
259 | This is best demonstrated by example: |
260 | |
261 | % ./perl -Ilib -MDevel::Peek -le '$a="12345"; $a=~s/.//; Dump($a)' |
262 | SV = PVIV(0x8128450) at 0x81340f0 |
263 | REFCNT = 1 |
264 | FLAGS = (POK,OOK,pPOK) |
265 | IV = 1 (OFFSET) |
266 | PV = 0x8135781 ( "1" . ) "2345"\0 |
267 | CUR = 4 |
268 | LEN = 5 |
269 | |
270 | Here the number of bytes chopped off (1) is put into IV, and |
271 | C<Devel::Peek::Dump> helpfully reminds us that this is an offset. The |
272 | portion of the string between the "real" and the "fake" beginnings is |
273 | shown in parentheses, and the values of C<SvCUR> and C<SvLEN> reflect |
274 | the fake beginning, not the real one. |
275 | |
fe854a6f |
276 | Something similar to the offset hack is performed on AVs to enable |
319cef53 |
277 | efficient shifting and splicing off the beginning of the array; while |
278 | C<AvARRAY> points to the first element in the array that is visible from |
279 | Perl, C<AvALLOC> points to the real start of the C array. These are |
280 | usually the same, but a C<shift> operation can be carried out by |
281 | increasing C<AvARRAY> by one and decreasing C<AvFILL> and C<AvLEN>. |
282 | Again, the location of the real start of the C array only comes into |
283 | play when freeing the array. See C<av_shift> in F<av.c>. |
284 | |
d1b91892 |
285 | =head2 What's Really Stored in an SV? |
a0d0e21e |
286 | |
287 | Recall that the usual method of determining the type of scalar you have is |
5f05dabc |
288 | to use C<Sv*OK> macros. Because a scalar can be both a number and a string, |
d1b91892 |
289 | usually these macros will always return TRUE and calling the C<Sv*V> |
a0d0e21e |
290 | macros will do the appropriate conversion of string to integer/double or |
291 | integer/double to string. |
292 | |
293 | If you I<really> need to know if you have an integer, double, or string |
294 | pointer in an SV, you can use the following three macros instead: |
295 | |
296 | SvIOKp(SV*) |
297 | SvNOKp(SV*) |
298 | SvPOKp(SV*) |
299 | |
300 | These will tell you if you truly have an integer, double, or string pointer |
d1b91892 |
301 | stored in your SV. The "p" stands for private. |
a0d0e21e |
302 | |
9e9796d6 |
303 | The are various ways in which the private and public flags may differ. |
304 | For example, a tied SV may have a valid underlying value in the IV slot |
305 | (so SvIOKp is true), but the data should be accessed via the FETCH |
306 | routine rather than directly, so SvIOK is false. Another is when |
307 | numeric conversion has occured and precision has been lost: only the |
308 | private flag is set on 'lossy' values. So when an NV is converted to an |
309 | IV with loss, SvIOKp, SvNOKp and SvNOK will be set, while SvIOK wont be. |
310 | |
07fa94a1 |
311 | In general, though, it's best to use the C<Sv*V> macros. |
a0d0e21e |
312 | |
54310121 |
313 | =head2 Working with AVs |
a0d0e21e |
314 | |
07fa94a1 |
315 | There are two ways to create and load an AV. The first method creates an |
316 | empty AV: |
a0d0e21e |
317 | |
318 | AV* newAV(); |
319 | |
54310121 |
320 | The second method both creates the AV and initially populates it with SVs: |
a0d0e21e |
321 | |
322 | AV* av_make(I32 num, SV **ptr); |
323 | |
5f05dabc |
324 | The second argument points to an array containing C<num> C<SV*>'s. Once the |
54310121 |
325 | AV has been created, the SVs can be destroyed, if so desired. |
a0d0e21e |
326 | |
54310121 |
327 | Once the AV has been created, the following operations are possible on AVs: |
a0d0e21e |
328 | |
329 | void av_push(AV*, SV*); |
330 | SV* av_pop(AV*); |
331 | SV* av_shift(AV*); |
332 | void av_unshift(AV*, I32 num); |
333 | |
334 | These should be familiar operations, with the exception of C<av_unshift>. |
335 | This routine adds C<num> elements at the front of the array with the C<undef> |
336 | value. You must then use C<av_store> (described below) to assign values |
337 | to these new elements. |
338 | |
339 | Here are some other functions: |
340 | |
5f05dabc |
341 | I32 av_len(AV*); |
a0d0e21e |
342 | SV** av_fetch(AV*, I32 key, I32 lval); |
a0d0e21e |
343 | SV** av_store(AV*, I32 key, SV* val); |
a0d0e21e |
344 | |
5f05dabc |
345 | The C<av_len> function returns the highest index value in array (just |
346 | like $#array in Perl). If the array is empty, -1 is returned. The |
347 | C<av_fetch> function returns the value at index C<key>, but if C<lval> |
348 | is non-zero, then C<av_fetch> will store an undef value at that index. |
04343c6d |
349 | The C<av_store> function stores the value C<val> at index C<key>, and does |
350 | not increment the reference count of C<val>. Thus the caller is responsible |
351 | for taking care of that, and if C<av_store> returns NULL, the caller will |
352 | have to decrement the reference count to avoid a memory leak. Note that |
353 | C<av_fetch> and C<av_store> both return C<SV**>'s, not C<SV*>'s as their |
354 | return value. |
d1b91892 |
355 | |
a0d0e21e |
356 | void av_clear(AV*); |
a0d0e21e |
357 | void av_undef(AV*); |
cb1a09d0 |
358 | void av_extend(AV*, I32 key); |
5f05dabc |
359 | |
360 | The C<av_clear> function deletes all the elements in the AV* array, but |
361 | does not actually delete the array itself. The C<av_undef> function will |
362 | delete all the elements in the array plus the array itself. The |
adc882cf |
363 | C<av_extend> function extends the array so that it contains at least C<key+1> |
364 | elements. If C<key+1> is less than the currently allocated length of the array, |
365 | then nothing is done. |
a0d0e21e |
366 | |
367 | If you know the name of an array variable, you can get a pointer to its AV |
368 | by using the following: |
369 | |
4929bf7b |
370 | AV* get_av("package::varname", FALSE); |
a0d0e21e |
371 | |
372 | This returns NULL if the variable does not exist. |
373 | |
04343c6d |
374 | See L<Understanding the Magic of Tied Hashes and Arrays> for more |
375 | information on how to use the array access functions on tied arrays. |
376 | |
54310121 |
377 | =head2 Working with HVs |
a0d0e21e |
378 | |
379 | To create an HV, you use the following routine: |
380 | |
381 | HV* newHV(); |
382 | |
54310121 |
383 | Once the HV has been created, the following operations are possible on HVs: |
a0d0e21e |
384 | |
08105a92 |
385 | SV** hv_store(HV*, const char* key, U32 klen, SV* val, U32 hash); |
386 | SV** hv_fetch(HV*, const char* key, U32 klen, I32 lval); |
a0d0e21e |
387 | |
5f05dabc |
388 | The C<klen> parameter is the length of the key being passed in (Note that |
389 | you cannot pass 0 in as a value of C<klen> to tell Perl to measure the |
390 | length of the key). The C<val> argument contains the SV pointer to the |
54310121 |
391 | scalar being stored, and C<hash> is the precomputed hash value (zero if |
5f05dabc |
392 | you want C<hv_store> to calculate it for you). The C<lval> parameter |
393 | indicates whether this fetch is actually a part of a store operation, in |
394 | which case a new undefined value will be added to the HV with the supplied |
395 | key and C<hv_fetch> will return as if the value had already existed. |
a0d0e21e |
396 | |
5f05dabc |
397 | Remember that C<hv_store> and C<hv_fetch> return C<SV**>'s and not just |
398 | C<SV*>. To access the scalar value, you must first dereference the return |
399 | value. However, you should check to make sure that the return value is |
400 | not NULL before dereferencing it. |
a0d0e21e |
401 | |
402 | These two functions check if a hash table entry exists, and deletes it. |
403 | |
08105a92 |
404 | bool hv_exists(HV*, const char* key, U32 klen); |
405 | SV* hv_delete(HV*, const char* key, U32 klen, I32 flags); |
a0d0e21e |
406 | |
5f05dabc |
407 | If C<flags> does not include the C<G_DISCARD> flag then C<hv_delete> will |
408 | create and return a mortal copy of the deleted value. |
409 | |
a0d0e21e |
410 | And more miscellaneous functions: |
411 | |
412 | void hv_clear(HV*); |
a0d0e21e |
413 | void hv_undef(HV*); |
5f05dabc |
414 | |
415 | Like their AV counterparts, C<hv_clear> deletes all the entries in the hash |
416 | table but does not actually delete the hash table. The C<hv_undef> deletes |
417 | both the entries and the hash table itself. |
a0d0e21e |
418 | |
d1b91892 |
419 | Perl keeps the actual data in linked list of structures with a typedef of HE. |
420 | These contain the actual key and value pointers (plus extra administrative |
421 | overhead). The key is a string pointer; the value is an C<SV*>. However, |
422 | once you have an C<HE*>, to get the actual key and value, use the routines |
423 | specified below. |
424 | |
a0d0e21e |
425 | I32 hv_iterinit(HV*); |
426 | /* Prepares starting point to traverse hash table */ |
427 | HE* hv_iternext(HV*); |
428 | /* Get the next entry, and return a pointer to a |
429 | structure that has both the key and value */ |
430 | char* hv_iterkey(HE* entry, I32* retlen); |
431 | /* Get the key from an HE structure and also return |
432 | the length of the key string */ |
cb1a09d0 |
433 | SV* hv_iterval(HV*, HE* entry); |
d1be9408 |
434 | /* Return an SV pointer to the value of the HE |
a0d0e21e |
435 | structure */ |
cb1a09d0 |
436 | SV* hv_iternextsv(HV*, char** key, I32* retlen); |
d1b91892 |
437 | /* This convenience routine combines hv_iternext, |
438 | hv_iterkey, and hv_iterval. The key and retlen |
439 | arguments are return values for the key and its |
440 | length. The value is returned in the SV* argument */ |
a0d0e21e |
441 | |
442 | If you know the name of a hash variable, you can get a pointer to its HV |
443 | by using the following: |
444 | |
4929bf7b |
445 | HV* get_hv("package::varname", FALSE); |
a0d0e21e |
446 | |
447 | This returns NULL if the variable does not exist. |
448 | |
8ebc5c01 |
449 | The hash algorithm is defined in the C<PERL_HASH(hash, key, klen)> macro: |
a0d0e21e |
450 | |
a0d0e21e |
451 | hash = 0; |
ab192400 |
452 | while (klen--) |
453 | hash = (hash * 33) + *key++; |
87275199 |
454 | hash = hash + (hash >> 5); /* after 5.6 */ |
ab192400 |
455 | |
87275199 |
456 | The last step was added in version 5.6 to improve distribution of |
ab192400 |
457 | lower bits in the resulting hash value. |
a0d0e21e |
458 | |
04343c6d |
459 | See L<Understanding the Magic of Tied Hashes and Arrays> for more |
460 | information on how to use the hash access functions on tied hashes. |
461 | |
1e422769 |
462 | =head2 Hash API Extensions |
463 | |
464 | Beginning with version 5.004, the following functions are also supported: |
465 | |
466 | HE* hv_fetch_ent (HV* tb, SV* key, I32 lval, U32 hash); |
467 | HE* hv_store_ent (HV* tb, SV* key, SV* val, U32 hash); |
c47ff5f1 |
468 | |
1e422769 |
469 | bool hv_exists_ent (HV* tb, SV* key, U32 hash); |
470 | SV* hv_delete_ent (HV* tb, SV* key, I32 flags, U32 hash); |
c47ff5f1 |
471 | |
1e422769 |
472 | SV* hv_iterkeysv (HE* entry); |
473 | |
474 | Note that these functions take C<SV*> keys, which simplifies writing |
475 | of extension code that deals with hash structures. These functions |
476 | also allow passing of C<SV*> keys to C<tie> functions without forcing |
477 | you to stringify the keys (unlike the previous set of functions). |
478 | |
479 | They also return and accept whole hash entries (C<HE*>), making their |
480 | use more efficient (since the hash number for a particular string |
4a4eefd0 |
481 | doesn't have to be recomputed every time). See L<perlapi> for detailed |
482 | descriptions. |
1e422769 |
483 | |
484 | The following macros must always be used to access the contents of hash |
485 | entries. Note that the arguments to these macros must be simple |
486 | variables, since they may get evaluated more than once. See |
4a4eefd0 |
487 | L<perlapi> for detailed descriptions of these macros. |
1e422769 |
488 | |
489 | HePV(HE* he, STRLEN len) |
490 | HeVAL(HE* he) |
491 | HeHASH(HE* he) |
492 | HeSVKEY(HE* he) |
493 | HeSVKEY_force(HE* he) |
494 | HeSVKEY_set(HE* he, SV* sv) |
495 | |
496 | These two lower level macros are defined, but must only be used when |
497 | dealing with keys that are not C<SV*>s: |
498 | |
499 | HeKEY(HE* he) |
500 | HeKLEN(HE* he) |
501 | |
04343c6d |
502 | Note that both C<hv_store> and C<hv_store_ent> do not increment the |
503 | reference count of the stored C<val>, which is the caller's responsibility. |
504 | If these functions return a NULL value, the caller will usually have to |
505 | decrement the reference count of C<val> to avoid a memory leak. |
1e422769 |
506 | |
a9381218 |
507 | =head2 AVs, HVs and undefined values |
508 | |
509 | Sometimes you have to store undefined values in AVs or HVs. Although |
510 | this may be a rare case, it can be tricky. That's because you're |
511 | used to using C<&PL_sv_undef> if you need an undefined SV. |
512 | |
513 | For example, intuition tells you that this XS code: |
514 | |
515 | AV *av = newAV(); |
516 | av_store( av, 0, &PL_sv_undef ); |
517 | |
518 | is equivalent to this Perl code: |
519 | |
520 | my @av; |
521 | $av[0] = undef; |
522 | |
523 | Unfortunately, this isn't true. AVs use C<&PL_sv_undef> as a marker |
524 | for indicating that an array element has not yet been initialized. |
525 | Thus, C<exists $av[0]> would be true for the above Perl code, but |
526 | false for the array generated by the XS code. |
527 | |
528 | Other problems can occur when storing C<&PL_sv_undef> in HVs: |
529 | |
530 | hv_store( hv, "key", 3, &PL_sv_undef, 0 ); |
531 | |
532 | This will indeed make the value C<undef>, but if you try to modify |
533 | the value of C<key>, you'll get the following error: |
534 | |
535 | Modification of non-creatable hash value attempted |
536 | |
537 | In perl 5.8.0, C<&PL_sv_undef> was also used to mark placeholders |
538 | in restricted hashes. This caused such hash entries not to appear |
539 | when iterating over the hash or when checking for the keys |
540 | with the C<hv_exists> function. |
541 | |
542 | You can run into similar problems when you store C<&PL_sv_true> or |
543 | C<&PL_sv_false> into AVs or HVs. Trying to modify such elements |
544 | will give you the following error: |
545 | |
546 | Modification of a read-only value attempted |
547 | |
548 | To make a long story short, you can use the special variables |
549 | C<&PL_sv_undef>, C<&PL_sv_true> and C<&PL_sv_false> with AVs and |
550 | HVs, but you have to make sure you know what you're doing. |
551 | |
552 | Generally, if you want to store an undefined value in an AV |
553 | or HV, you should not use C<&PL_sv_undef>, but rather create a |
554 | new undefined value using the C<newSV> function, for example: |
555 | |
556 | av_store( av, 42, newSV(0) ); |
557 | hv_store( hv, "foo", 3, newSV(0), 0 ); |
558 | |
a0d0e21e |
559 | =head2 References |
560 | |
d1b91892 |
561 | References are a special type of scalar that point to other data types |
562 | (including references). |
a0d0e21e |
563 | |
07fa94a1 |
564 | To create a reference, use either of the following functions: |
a0d0e21e |
565 | |
5f05dabc |
566 | SV* newRV_inc((SV*) thing); |
567 | SV* newRV_noinc((SV*) thing); |
a0d0e21e |
568 | |
5f05dabc |
569 | The C<thing> argument can be any of an C<SV*>, C<AV*>, or C<HV*>. The |
07fa94a1 |
570 | functions are identical except that C<newRV_inc> increments the reference |
571 | count of the C<thing>, while C<newRV_noinc> does not. For historical |
572 | reasons, C<newRV> is a synonym for C<newRV_inc>. |
573 | |
574 | Once you have a reference, you can use the following macro to dereference |
575 | the reference: |
a0d0e21e |
576 | |
577 | SvRV(SV*) |
578 | |
579 | then call the appropriate routines, casting the returned C<SV*> to either an |
d1b91892 |
580 | C<AV*> or C<HV*>, if required. |
a0d0e21e |
581 | |
d1b91892 |
582 | To determine if an SV is a reference, you can use the following macro: |
a0d0e21e |
583 | |
584 | SvROK(SV*) |
585 | |
07fa94a1 |
586 | To discover what type of value the reference refers to, use the following |
587 | macro and then check the return value. |
d1b91892 |
588 | |
589 | SvTYPE(SvRV(SV*)) |
590 | |
591 | The most useful types that will be returned are: |
592 | |
593 | SVt_IV Scalar |
594 | SVt_NV Scalar |
595 | SVt_PV Scalar |
5f05dabc |
596 | SVt_RV Scalar |
d1b91892 |
597 | SVt_PVAV Array |
598 | SVt_PVHV Hash |
599 | SVt_PVCV Code |
5f05dabc |
600 | SVt_PVGV Glob (possible a file handle) |
601 | SVt_PVMG Blessed or Magical Scalar |
602 | |
603 | See the sv.h header file for more details. |
d1b91892 |
604 | |
cb1a09d0 |
605 | =head2 Blessed References and Class Objects |
606 | |
06f6df17 |
607 | References are also used to support object-oriented programming. In perl's |
cb1a09d0 |
608 | OO lexicon, an object is simply a reference that has been blessed into a |
609 | package (or class). Once blessed, the programmer may now use the reference |
610 | to access the various methods in the class. |
611 | |
612 | A reference can be blessed into a package with the following function: |
613 | |
614 | SV* sv_bless(SV* sv, HV* stash); |
615 | |
06f6df17 |
616 | The C<sv> argument must be a reference value. The C<stash> argument |
617 | specifies which class the reference will belong to. See |
2ae324a7 |
618 | L<Stashes and Globs> for information on converting class names into stashes. |
cb1a09d0 |
619 | |
620 | /* Still under construction */ |
621 | |
622 | Upgrades rv to reference if not already one. Creates new SV for rv to |
8ebc5c01 |
623 | point to. If C<classname> is non-null, the SV is blessed into the specified |
624 | class. SV is returned. |
cb1a09d0 |
625 | |
08105a92 |
626 | SV* newSVrv(SV* rv, const char* classname); |
cb1a09d0 |
627 | |
e1c57cef |
628 | Copies integer, unsigned integer or double into an SV whose reference is C<rv>. SV is blessed |
8ebc5c01 |
629 | if C<classname> is non-null. |
cb1a09d0 |
630 | |
08105a92 |
631 | SV* sv_setref_iv(SV* rv, const char* classname, IV iv); |
e1c57cef |
632 | SV* sv_setref_uv(SV* rv, const char* classname, UV uv); |
08105a92 |
633 | SV* sv_setref_nv(SV* rv, const char* classname, NV iv); |
cb1a09d0 |
634 | |
5f05dabc |
635 | Copies the pointer value (I<the address, not the string!>) into an SV whose |
8ebc5c01 |
636 | reference is rv. SV is blessed if C<classname> is non-null. |
cb1a09d0 |
637 | |
08105a92 |
638 | SV* sv_setref_pv(SV* rv, const char* classname, PV iv); |
cb1a09d0 |
639 | |
8ebc5c01 |
640 | Copies string into an SV whose reference is C<rv>. Set length to 0 to let |
641 | Perl calculate the string length. SV is blessed if C<classname> is non-null. |
cb1a09d0 |
642 | |
e65f3abd |
643 | SV* sv_setref_pvn(SV* rv, const char* classname, PV iv, STRLEN length); |
cb1a09d0 |
644 | |
9abd00ed |
645 | Tests whether the SV is blessed into the specified class. It does not |
646 | check inheritance relationships. |
647 | |
08105a92 |
648 | int sv_isa(SV* sv, const char* name); |
9abd00ed |
649 | |
650 | Tests whether the SV is a reference to a blessed object. |
651 | |
652 | int sv_isobject(SV* sv); |
653 | |
654 | Tests whether the SV is derived from the specified class. SV can be either |
655 | a reference to a blessed object or a string containing a class name. This |
656 | is the function implementing the C<UNIVERSAL::isa> functionality. |
657 | |
08105a92 |
658 | bool sv_derived_from(SV* sv, const char* name); |
9abd00ed |
659 | |
00aadd71 |
660 | To check if you've got an object derived from a specific class you have |
9abd00ed |
661 | to write: |
662 | |
663 | if (sv_isobject(sv) && sv_derived_from(sv, class)) { ... } |
cb1a09d0 |
664 | |
5f05dabc |
665 | =head2 Creating New Variables |
cb1a09d0 |
666 | |
5f05dabc |
667 | To create a new Perl variable with an undef value which can be accessed from |
668 | your Perl script, use the following routines, depending on the variable type. |
cb1a09d0 |
669 | |
4929bf7b |
670 | SV* get_sv("package::varname", TRUE); |
671 | AV* get_av("package::varname", TRUE); |
672 | HV* get_hv("package::varname", TRUE); |
cb1a09d0 |
673 | |
674 | Notice the use of TRUE as the second parameter. The new variable can now |
675 | be set, using the routines appropriate to the data type. |
676 | |
5f05dabc |
677 | There are additional macros whose values may be bitwise OR'ed with the |
678 | C<TRUE> argument to enable certain extra features. Those bits are: |
cb1a09d0 |
679 | |
9a68f1db |
680 | =over |
681 | |
682 | =item GV_ADDMULTI |
683 | |
684 | Marks the variable as multiply defined, thus preventing the: |
685 | |
686 | Name <varname> used only once: possible typo |
687 | |
688 | warning. |
689 | |
9a68f1db |
690 | =item GV_ADDWARN |
691 | |
692 | Issues the warning: |
693 | |
694 | Had to create <varname> unexpectedly |
695 | |
696 | if the variable did not exist before the function was called. |
697 | |
698 | =back |
cb1a09d0 |
699 | |
07fa94a1 |
700 | If you do not specify a package name, the variable is created in the current |
701 | package. |
cb1a09d0 |
702 | |
5f05dabc |
703 | =head2 Reference Counts and Mortality |
a0d0e21e |
704 | |
d1be9408 |
705 | Perl uses a reference count-driven garbage collection mechanism. SVs, |
54310121 |
706 | AVs, or HVs (xV for short in the following) start their life with a |
55497cff |
707 | reference count of 1. If the reference count of an xV ever drops to 0, |
07fa94a1 |
708 | then it will be destroyed and its memory made available for reuse. |
55497cff |
709 | |
710 | This normally doesn't happen at the Perl level unless a variable is |
5f05dabc |
711 | undef'ed or the last variable holding a reference to it is changed or |
712 | overwritten. At the internal level, however, reference counts can be |
55497cff |
713 | manipulated with the following macros: |
714 | |
715 | int SvREFCNT(SV* sv); |
5f05dabc |
716 | SV* SvREFCNT_inc(SV* sv); |
55497cff |
717 | void SvREFCNT_dec(SV* sv); |
718 | |
719 | However, there is one other function which manipulates the reference |
07fa94a1 |
720 | count of its argument. The C<newRV_inc> function, you will recall, |
721 | creates a reference to the specified argument. As a side effect, |
722 | it increments the argument's reference count. If this is not what |
723 | you want, use C<newRV_noinc> instead. |
724 | |
725 | For example, imagine you want to return a reference from an XSUB function. |
726 | Inside the XSUB routine, you create an SV which initially has a reference |
727 | count of one. Then you call C<newRV_inc>, passing it the just-created SV. |
5f05dabc |
728 | This returns the reference as a new SV, but the reference count of the |
729 | SV you passed to C<newRV_inc> has been incremented to two. Now you |
07fa94a1 |
730 | return the reference from the XSUB routine and forget about the SV. |
731 | But Perl hasn't! Whenever the returned reference is destroyed, the |
732 | reference count of the original SV is decreased to one and nothing happens. |
733 | The SV will hang around without any way to access it until Perl itself |
734 | terminates. This is a memory leak. |
5f05dabc |
735 | |
736 | The correct procedure, then, is to use C<newRV_noinc> instead of |
faed5253 |
737 | C<newRV_inc>. Then, if and when the last reference is destroyed, |
738 | the reference count of the SV will go to zero and it will be destroyed, |
07fa94a1 |
739 | stopping any memory leak. |
55497cff |
740 | |
5f05dabc |
741 | There are some convenience functions available that can help with the |
54310121 |
742 | destruction of xVs. These functions introduce the concept of "mortality". |
07fa94a1 |
743 | An xV that is mortal has had its reference count marked to be decremented, |
744 | but not actually decremented, until "a short time later". Generally the |
745 | term "short time later" means a single Perl statement, such as a call to |
54310121 |
746 | an XSUB function. The actual determinant for when mortal xVs have their |
07fa94a1 |
747 | reference count decremented depends on two macros, SAVETMPS and FREETMPS. |
748 | See L<perlcall> and L<perlxs> for more details on these macros. |
55497cff |
749 | |
750 | "Mortalization" then is at its simplest a deferred C<SvREFCNT_dec>. |
751 | However, if you mortalize a variable twice, the reference count will |
752 | later be decremented twice. |
753 | |
00aadd71 |
754 | "Mortal" SVs are mainly used for SVs that are placed on perl's stack. |
755 | For example an SV which is created just to pass a number to a called sub |
06f6df17 |
756 | is made mortal to have it cleaned up automatically when it's popped off |
757 | the stack. Similarly, results returned by XSUBs (which are pushed on the |
758 | stack) are often made mortal. |
a0d0e21e |
759 | |
760 | To create a mortal variable, use the functions: |
761 | |
762 | SV* sv_newmortal() |
763 | SV* sv_2mortal(SV*) |
764 | SV* sv_mortalcopy(SV*) |
765 | |
00aadd71 |
766 | The first call creates a mortal SV (with no value), the second converts an existing |
5f05dabc |
767 | SV to a mortal SV (and thus defers a call to C<SvREFCNT_dec>), and the |
768 | third creates a mortal copy of an existing SV. |
00aadd71 |
769 | Because C<sv_newmortal> gives the new SV no value,it must normally be given one |
9a68f1db |
770 | via C<sv_setpv>, C<sv_setiv>, etc. : |
00aadd71 |
771 | |
772 | SV *tmp = sv_newmortal(); |
773 | sv_setiv(tmp, an_integer); |
774 | |
775 | As that is multiple C statements it is quite common so see this idiom instead: |
776 | |
777 | SV *tmp = sv_2mortal(newSViv(an_integer)); |
778 | |
779 | |
780 | You should be careful about creating mortal variables. Strange things |
781 | can happen if you make the same value mortal within multiple contexts, |
782 | or if you make a variable mortal multiple times. Thinking of "Mortalization" |
783 | as deferred C<SvREFCNT_dec> should help to minimize such problems. |
784 | For example if you are passing an SV which you I<know> has high enough REFCNT |
785 | to survive its use on the stack you need not do any mortalization. |
786 | If you are not sure then doing an C<SvREFCNT_inc> and C<sv_2mortal>, or |
787 | making a C<sv_mortalcopy> is safer. |
a0d0e21e |
788 | |
54310121 |
789 | The mortal routines are not just for SVs -- AVs and HVs can be |
faed5253 |
790 | made mortal by passing their address (type-casted to C<SV*>) to the |
07fa94a1 |
791 | C<sv_2mortal> or C<sv_mortalcopy> routines. |
a0d0e21e |
792 | |
5f05dabc |
793 | =head2 Stashes and Globs |
a0d0e21e |
794 | |
06f6df17 |
795 | A B<stash> is a hash that contains all variables that are defined |
796 | within a package. Each key of the stash is a symbol |
aa689395 |
797 | name (shared by all the different types of objects that have the same |
798 | name), and each value in the hash table is a GV (Glob Value). This GV |
799 | in turn contains references to the various objects of that name, |
800 | including (but not limited to) the following: |
cb1a09d0 |
801 | |
a0d0e21e |
802 | Scalar Value |
803 | Array Value |
804 | Hash Value |
a3cb178b |
805 | I/O Handle |
a0d0e21e |
806 | Format |
807 | Subroutine |
808 | |
06f6df17 |
809 | There is a single stash called C<PL_defstash> that holds the items that exist |
810 | in the C<main> package. To get at the items in other packages, append the |
811 | string "::" to the package name. The items in the C<Foo> package are in |
812 | the stash C<Foo::> in PL_defstash. The items in the C<Bar::Baz> package are |
813 | in the stash C<Baz::> in C<Bar::>'s stash. |
a0d0e21e |
814 | |
d1b91892 |
815 | To get the stash pointer for a particular package, use the function: |
a0d0e21e |
816 | |
08105a92 |
817 | HV* gv_stashpv(const char* name, I32 create) |
a0d0e21e |
818 | HV* gv_stashsv(SV*, I32 create) |
819 | |
820 | The first function takes a literal string, the second uses the string stored |
d1b91892 |
821 | in the SV. Remember that a stash is just a hash table, so you get back an |
cb1a09d0 |
822 | C<HV*>. The C<create> flag will create a new package if it is set. |
a0d0e21e |
823 | |
824 | The name that C<gv_stash*v> wants is the name of the package whose symbol table |
825 | you want. The default package is called C<main>. If you have multiply nested |
d1b91892 |
826 | packages, pass their names to C<gv_stash*v>, separated by C<::> as in the Perl |
827 | language itself. |
a0d0e21e |
828 | |
829 | Alternately, if you have an SV that is a blessed reference, you can find |
830 | out the stash pointer by using: |
831 | |
832 | HV* SvSTASH(SvRV(SV*)); |
833 | |
834 | then use the following to get the package name itself: |
835 | |
836 | char* HvNAME(HV* stash); |
837 | |
5f05dabc |
838 | If you need to bless or re-bless an object you can use the following |
839 | function: |
a0d0e21e |
840 | |
841 | SV* sv_bless(SV*, HV* stash) |
842 | |
843 | where the first argument, an C<SV*>, must be a reference, and the second |
844 | argument is a stash. The returned C<SV*> can now be used in the same way |
845 | as any other SV. |
846 | |
d1b91892 |
847 | For more information on references and blessings, consult L<perlref>. |
848 | |
54310121 |
849 | =head2 Double-Typed SVs |
0a753a76 |
850 | |
851 | Scalar variables normally contain only one type of value, an integer, |
852 | double, pointer, or reference. Perl will automatically convert the |
853 | actual scalar data from the stored type into the requested type. |
854 | |
855 | Some scalar variables contain more than one type of scalar data. For |
856 | example, the variable C<$!> contains either the numeric value of C<errno> |
857 | or its string equivalent from either C<strerror> or C<sys_errlist[]>. |
858 | |
859 | To force multiple data values into an SV, you must do two things: use the |
860 | C<sv_set*v> routines to add the additional scalar type, then set a flag |
861 | so that Perl will believe it contains more than one type of data. The |
862 | four macros to set the flags are: |
863 | |
864 | SvIOK_on |
865 | SvNOK_on |
866 | SvPOK_on |
867 | SvROK_on |
868 | |
869 | The particular macro you must use depends on which C<sv_set*v> routine |
870 | you called first. This is because every C<sv_set*v> routine turns on |
871 | only the bit for the particular type of data being set, and turns off |
872 | all the rest. |
873 | |
874 | For example, to create a new Perl variable called "dberror" that contains |
875 | both the numeric and descriptive string error values, you could use the |
876 | following code: |
877 | |
878 | extern int dberror; |
879 | extern char *dberror_list; |
880 | |
4929bf7b |
881 | SV* sv = get_sv("dberror", TRUE); |
0a753a76 |
882 | sv_setiv(sv, (IV) dberror); |
883 | sv_setpv(sv, dberror_list[dberror]); |
884 | SvIOK_on(sv); |
885 | |
886 | If the order of C<sv_setiv> and C<sv_setpv> had been reversed, then the |
887 | macro C<SvPOK_on> would need to be called instead of C<SvIOK_on>. |
888 | |
889 | =head2 Magic Variables |
a0d0e21e |
890 | |
d1b91892 |
891 | [This section still under construction. Ignore everything here. Post no |
892 | bills. Everything not permitted is forbidden.] |
893 | |
d1b91892 |
894 | Any SV may be magical, that is, it has special features that a normal |
895 | SV does not have. These features are stored in the SV structure in a |
5f05dabc |
896 | linked list of C<struct magic>'s, typedef'ed to C<MAGIC>. |
d1b91892 |
897 | |
898 | struct magic { |
899 | MAGIC* mg_moremagic; |
900 | MGVTBL* mg_virtual; |
901 | U16 mg_private; |
902 | char mg_type; |
903 | U8 mg_flags; |
904 | SV* mg_obj; |
905 | char* mg_ptr; |
906 | I32 mg_len; |
907 | }; |
908 | |
909 | Note this is current as of patchlevel 0, and could change at any time. |
910 | |
911 | =head2 Assigning Magic |
912 | |
913 | Perl adds magic to an SV using the sv_magic function: |
914 | |
08105a92 |
915 | void sv_magic(SV* sv, SV* obj, int how, const char* name, I32 namlen); |
d1b91892 |
916 | |
917 | The C<sv> argument is a pointer to the SV that is to acquire a new magical |
918 | feature. |
919 | |
920 | If C<sv> is not already magical, Perl uses the C<SvUPGRADE> macro to |
645c22ef |
921 | convert C<sv> to type C<SVt_PVMG>. Perl then continues by adding new magic |
922 | to the beginning of the linked list of magical features. Any prior entry |
923 | of the same type of magic is deleted. Note that this can be overridden, |
924 | and multiple instances of the same type of magic can be associated with an |
925 | SV. |
d1b91892 |
926 | |
54310121 |
927 | The C<name> and C<namlen> arguments are used to associate a string with |
928 | the magic, typically the name of a variable. C<namlen> is stored in the |
2d8d5d5a |
929 | C<mg_len> field and if C<name> is non-null then either a C<savepvn> copy of |
930 | C<name> or C<name> itself is stored in the C<mg_ptr> field, depending on |
931 | whether C<namlen> is greater than zero or equal to zero respectively. As a |
932 | special case, if C<(name && namlen == HEf_SVKEY)> then C<name> is assumed |
933 | to contain an C<SV*> and is stored as-is with its REFCNT incremented. |
d1b91892 |
934 | |
935 | The sv_magic function uses C<how> to determine which, if any, predefined |
936 | "Magic Virtual Table" should be assigned to the C<mg_virtual> field. |
06f6df17 |
937 | See the L<Magic Virtual Tables> section below. The C<how> argument is also |
14befaf4 |
938 | stored in the C<mg_type> field. The value of C<how> should be chosen |
06f6df17 |
939 | from the set of macros C<PERL_MAGIC_foo> found in F<perl.h>. Note that before |
645c22ef |
940 | these macros were added, Perl internals used to directly use character |
14befaf4 |
941 | literals, so you may occasionally come across old code or documentation |
75d0f26d |
942 | referring to 'U' magic rather than C<PERL_MAGIC_uvar> for example. |
d1b91892 |
943 | |
944 | The C<obj> argument is stored in the C<mg_obj> field of the C<MAGIC> |
945 | structure. If it is not the same as the C<sv> argument, the reference |
946 | count of the C<obj> object is incremented. If it is the same, or if |
645c22ef |
947 | the C<how> argument is C<PERL_MAGIC_arylen>, or if it is a NULL pointer, |
14befaf4 |
948 | then C<obj> is merely stored, without the reference count being incremented. |
d1b91892 |
949 | |
2d8d5d5a |
950 | See also C<sv_magicext> in L<perlapi> for a more flexible way to add magic |
951 | to an SV. |
952 | |
cb1a09d0 |
953 | There is also a function to add magic to an C<HV>: |
954 | |
955 | void hv_magic(HV *hv, GV *gv, int how); |
956 | |
957 | This simply calls C<sv_magic> and coerces the C<gv> argument into an C<SV>. |
958 | |
959 | To remove the magic from an SV, call the function sv_unmagic: |
960 | |
961 | void sv_unmagic(SV *sv, int type); |
962 | |
963 | The C<type> argument should be equal to the C<how> value when the C<SV> |
964 | was initially made magical. |
965 | |
d1b91892 |
966 | =head2 Magic Virtual Tables |
967 | |
d1be9408 |
968 | The C<mg_virtual> field in the C<MAGIC> structure is a pointer to an |
d1b91892 |
969 | C<MGVTBL>, which is a structure of function pointers and stands for |
970 | "Magic Virtual Table" to handle the various operations that might be |
971 | applied to that variable. |
972 | |
973 | The C<MGVTBL> has five pointers to the following routine types: |
974 | |
975 | int (*svt_get)(SV* sv, MAGIC* mg); |
976 | int (*svt_set)(SV* sv, MAGIC* mg); |
977 | U32 (*svt_len)(SV* sv, MAGIC* mg); |
978 | int (*svt_clear)(SV* sv, MAGIC* mg); |
979 | int (*svt_free)(SV* sv, MAGIC* mg); |
980 | |
06f6df17 |
981 | This MGVTBL structure is set at compile-time in F<perl.h> and there are |
d1b91892 |
982 | currently 19 types (or 21 with overloading turned on). These different |
983 | structures contain pointers to various routines that perform additional |
984 | actions depending on which function is being called. |
985 | |
986 | Function pointer Action taken |
987 | ---------------- ------------ |
8b0711c3 |
988 | svt_get Do something before the value of the SV is retrieved. |
d1b91892 |
989 | svt_set Do something after the SV is assigned a value. |
990 | svt_len Report on the SV's length. |
991 | svt_clear Clear something the SV represents. |
992 | svt_free Free any extra storage associated with the SV. |
993 | |
994 | For instance, the MGVTBL structure called C<vtbl_sv> (which corresponds |
14befaf4 |
995 | to an C<mg_type> of C<PERL_MAGIC_sv>) contains: |
d1b91892 |
996 | |
997 | { magic_get, magic_set, magic_len, 0, 0 } |
998 | |
14befaf4 |
999 | Thus, when an SV is determined to be magical and of type C<PERL_MAGIC_sv>, |
1000 | if a get operation is being performed, the routine C<magic_get> is |
1001 | called. All the various routines for the various magical types begin |
1002 | with C<magic_>. NOTE: the magic routines are not considered part of |
1003 | the Perl API, and may not be exported by the Perl library. |
d1b91892 |
1004 | |
1005 | The current kinds of Magic Virtual Tables are: |
1006 | |
14befaf4 |
1007 | mg_type |
1008 | (old-style char and macro) MGVTBL Type of magic |
1009 | -------------------------- ------ ---------------------------- |
1010 | \0 PERL_MAGIC_sv vtbl_sv Special scalar variable |
1011 | A PERL_MAGIC_overload vtbl_amagic %OVERLOAD hash |
1012 | a PERL_MAGIC_overload_elem vtbl_amagicelem %OVERLOAD hash element |
1013 | c PERL_MAGIC_overload_table (none) Holds overload table (AMT) |
1014 | on stash |
1015 | B PERL_MAGIC_bm vtbl_bm Boyer-Moore (fast string search) |
1016 | D PERL_MAGIC_regdata vtbl_regdata Regex match position data |
1017 | (@+ and @- vars) |
1018 | d PERL_MAGIC_regdatum vtbl_regdatum Regex match position data |
1019 | element |
1020 | E PERL_MAGIC_env vtbl_env %ENV hash |
1021 | e PERL_MAGIC_envelem vtbl_envelem %ENV hash element |
1022 | f PERL_MAGIC_fm vtbl_fm Formline ('compiled' format) |
1023 | g PERL_MAGIC_regex_global vtbl_mglob m//g target / study()ed string |
1024 | I PERL_MAGIC_isa vtbl_isa @ISA array |
1025 | i PERL_MAGIC_isaelem vtbl_isaelem @ISA array element |
1026 | k PERL_MAGIC_nkeys vtbl_nkeys scalar(keys()) lvalue |
1027 | L PERL_MAGIC_dbfile (none) Debugger %_<filename |
1028 | l PERL_MAGIC_dbline vtbl_dbline Debugger %_<filename element |
1029 | m PERL_MAGIC_mutex vtbl_mutex ??? |
645c22ef |
1030 | o PERL_MAGIC_collxfrm vtbl_collxfrm Locale collate transformation |
14befaf4 |
1031 | P PERL_MAGIC_tied vtbl_pack Tied array or hash |
1032 | p PERL_MAGIC_tiedelem vtbl_packelem Tied array or hash element |
1033 | q PERL_MAGIC_tiedscalar vtbl_packelem Tied scalar or handle |
1034 | r PERL_MAGIC_qr vtbl_qr precompiled qr// regex |
1035 | S PERL_MAGIC_sig vtbl_sig %SIG hash |
1036 | s PERL_MAGIC_sigelem vtbl_sigelem %SIG hash element |
1037 | t PERL_MAGIC_taint vtbl_taint Taintedness |
1038 | U PERL_MAGIC_uvar vtbl_uvar Available for use by extensions |
1039 | v PERL_MAGIC_vec vtbl_vec vec() lvalue |
92f0c265 |
1040 | V PERL_MAGIC_vstring (none) v-string scalars |
836995da |
1041 | w PERL_MAGIC_utf8 vtbl_utf8 UTF-8 length+offset cache |
14befaf4 |
1042 | x PERL_MAGIC_substr vtbl_substr substr() lvalue |
1043 | y PERL_MAGIC_defelem vtbl_defelem Shadow "foreach" iterator |
1044 | variable / smart parameter |
1045 | vivification |
1046 | * PERL_MAGIC_glob vtbl_glob GV (typeglob) |
1047 | # PERL_MAGIC_arylen vtbl_arylen Array length ($#ary) |
1048 | . PERL_MAGIC_pos vtbl_pos pos() lvalue |
1049 | < PERL_MAGIC_backref vtbl_backref ??? |
1050 | ~ PERL_MAGIC_ext (none) Available for use by extensions |
d1b91892 |
1051 | |
68dc0745 |
1052 | When an uppercase and lowercase letter both exist in the table, then the |
92f0c265 |
1053 | uppercase letter is typically used to represent some kind of composite type |
1054 | (a list or a hash), and the lowercase letter is used to represent an element |
1055 | of that composite type. Some internals code makes use of this case |
1056 | relationship. However, 'v' and 'V' (vec and v-string) are in no way related. |
14befaf4 |
1057 | |
1058 | The C<PERL_MAGIC_ext> and C<PERL_MAGIC_uvar> magic types are defined |
1059 | specifically for use by extensions and will not be used by perl itself. |
1060 | Extensions can use C<PERL_MAGIC_ext> magic to 'attach' private information |
1061 | to variables (typically objects). This is especially useful because |
1062 | there is no way for normal perl code to corrupt this private information |
1063 | (unlike using extra elements of a hash object). |
1064 | |
1065 | Similarly, C<PERL_MAGIC_uvar> magic can be used much like tie() to call a |
1066 | C function any time a scalar's value is used or changed. The C<MAGIC>'s |
bdbeb323 |
1067 | C<mg_ptr> field points to a C<ufuncs> structure: |
1068 | |
1069 | struct ufuncs { |
a9402793 |
1070 | I32 (*uf_val)(pTHX_ IV, SV*); |
1071 | I32 (*uf_set)(pTHX_ IV, SV*); |
bdbeb323 |
1072 | IV uf_index; |
1073 | }; |
1074 | |
1075 | When the SV is read from or written to, the C<uf_val> or C<uf_set> |
14befaf4 |
1076 | function will be called with C<uf_index> as the first arg and a pointer to |
1077 | the SV as the second. A simple example of how to add C<PERL_MAGIC_uvar> |
1526ead6 |
1078 | magic is shown below. Note that the ufuncs structure is copied by |
1079 | sv_magic, so you can safely allocate it on the stack. |
1080 | |
1081 | void |
1082 | Umagic(sv) |
1083 | SV *sv; |
1084 | PREINIT: |
1085 | struct ufuncs uf; |
1086 | CODE: |
1087 | uf.uf_val = &my_get_fn; |
1088 | uf.uf_set = &my_set_fn; |
1089 | uf.uf_index = 0; |
14befaf4 |
1090 | sv_magic(sv, 0, PERL_MAGIC_uvar, (char*)&uf, sizeof(uf)); |
5f05dabc |
1091 | |
14befaf4 |
1092 | Note that because multiple extensions may be using C<PERL_MAGIC_ext> |
1093 | or C<PERL_MAGIC_uvar> magic, it is important for extensions to take |
1094 | extra care to avoid conflict. Typically only using the magic on |
1095 | objects blessed into the same class as the extension is sufficient. |
1096 | For C<PERL_MAGIC_ext> magic, it may also be appropriate to add an I32 |
1097 | 'signature' at the top of the private data area and check that. |
5f05dabc |
1098 | |
ef50df4b |
1099 | Also note that the C<sv_set*()> and C<sv_cat*()> functions described |
1100 | earlier do B<not> invoke 'set' magic on their targets. This must |
1101 | be done by the user either by calling the C<SvSETMAGIC()> macro after |
1102 | calling these functions, or by using one of the C<sv_set*_mg()> or |
1103 | C<sv_cat*_mg()> functions. Similarly, generic C code must call the |
1104 | C<SvGETMAGIC()> macro to invoke any 'get' magic if they use an SV |
1105 | obtained from external sources in functions that don't handle magic. |
4a4eefd0 |
1106 | See L<perlapi> for a description of these functions. |
189b2af5 |
1107 | For example, calls to the C<sv_cat*()> functions typically need to be |
1108 | followed by C<SvSETMAGIC()>, but they don't need a prior C<SvGETMAGIC()> |
1109 | since their implementation handles 'get' magic. |
1110 | |
d1b91892 |
1111 | =head2 Finding Magic |
1112 | |
1113 | MAGIC* mg_find(SV*, int type); /* Finds the magic pointer of that type */ |
1114 | |
1115 | This routine returns a pointer to the C<MAGIC> structure stored in the SV. |
1116 | If the SV does not have that magical feature, C<NULL> is returned. Also, |
54310121 |
1117 | if the SV is not of type SVt_PVMG, Perl may core dump. |
d1b91892 |
1118 | |
08105a92 |
1119 | int mg_copy(SV* sv, SV* nsv, const char* key, STRLEN klen); |
d1b91892 |
1120 | |
1121 | This routine checks to see what types of magic C<sv> has. If the mg_type |
68dc0745 |
1122 | field is an uppercase letter, then the mg_obj is copied to C<nsv>, but |
1123 | the mg_type field is changed to be the lowercase letter. |
a0d0e21e |
1124 | |
04343c6d |
1125 | =head2 Understanding the Magic of Tied Hashes and Arrays |
1126 | |
14befaf4 |
1127 | Tied hashes and arrays are magical beasts of the C<PERL_MAGIC_tied> |
1128 | magic type. |
9edb2b46 |
1129 | |
1130 | WARNING: As of the 5.004 release, proper usage of the array and hash |
1131 | access functions requires understanding a few caveats. Some |
1132 | of these caveats are actually considered bugs in the API, to be fixed |
1133 | in later releases, and are bracketed with [MAYCHANGE] below. If |
1134 | you find yourself actually applying such information in this section, be |
1135 | aware that the behavior may change in the future, umm, without warning. |
04343c6d |
1136 | |
1526ead6 |
1137 | The perl tie function associates a variable with an object that implements |
9a68f1db |
1138 | the various GET, SET, etc methods. To perform the equivalent of the perl |
1526ead6 |
1139 | tie function from an XSUB, you must mimic this behaviour. The code below |
1140 | carries out the necessary steps - firstly it creates a new hash, and then |
1141 | creates a second hash which it blesses into the class which will implement |
1142 | the tie methods. Lastly it ties the two hashes together, and returns a |
1143 | reference to the new tied hash. Note that the code below does NOT call the |
1144 | TIEHASH method in the MyTie class - |
1145 | see L<Calling Perl Routines from within C Programs> for details on how |
1146 | to do this. |
1147 | |
1148 | SV* |
1149 | mytie() |
1150 | PREINIT: |
1151 | HV *hash; |
1152 | HV *stash; |
1153 | SV *tie; |
1154 | CODE: |
1155 | hash = newHV(); |
1156 | tie = newRV_noinc((SV*)newHV()); |
1157 | stash = gv_stashpv("MyTie", TRUE); |
1158 | sv_bless(tie, stash); |
899e16d0 |
1159 | hv_magic(hash, (GV*)tie, PERL_MAGIC_tied); |
1526ead6 |
1160 | RETVAL = newRV_noinc(hash); |
1161 | OUTPUT: |
1162 | RETVAL |
1163 | |
04343c6d |
1164 | The C<av_store> function, when given a tied array argument, merely |
1165 | copies the magic of the array onto the value to be "stored", using |
1166 | C<mg_copy>. It may also return NULL, indicating that the value did not |
9edb2b46 |
1167 | actually need to be stored in the array. [MAYCHANGE] After a call to |
1168 | C<av_store> on a tied array, the caller will usually need to call |
1169 | C<mg_set(val)> to actually invoke the perl level "STORE" method on the |
1170 | TIEARRAY object. If C<av_store> did return NULL, a call to |
1171 | C<SvREFCNT_dec(val)> will also be usually necessary to avoid a memory |
1172 | leak. [/MAYCHANGE] |
04343c6d |
1173 | |
1174 | The previous paragraph is applicable verbatim to tied hash access using the |
1175 | C<hv_store> and C<hv_store_ent> functions as well. |
1176 | |
1177 | C<av_fetch> and the corresponding hash functions C<hv_fetch> and |
1178 | C<hv_fetch_ent> actually return an undefined mortal value whose magic |
1179 | has been initialized using C<mg_copy>. Note the value so returned does not |
9edb2b46 |
1180 | need to be deallocated, as it is already mortal. [MAYCHANGE] But you will |
1181 | need to call C<mg_get()> on the returned value in order to actually invoke |
1182 | the perl level "FETCH" method on the underlying TIE object. Similarly, |
04343c6d |
1183 | you may also call C<mg_set()> on the return value after possibly assigning |
1184 | a suitable value to it using C<sv_setsv>, which will invoke the "STORE" |
9edb2b46 |
1185 | method on the TIE object. [/MAYCHANGE] |
04343c6d |
1186 | |
9edb2b46 |
1187 | [MAYCHANGE] |
04343c6d |
1188 | In other words, the array or hash fetch/store functions don't really |
1189 | fetch and store actual values in the case of tied arrays and hashes. They |
1190 | merely call C<mg_copy> to attach magic to the values that were meant to be |
1191 | "stored" or "fetched". Later calls to C<mg_get> and C<mg_set> actually |
1192 | do the job of invoking the TIE methods on the underlying objects. Thus |
9edb2b46 |
1193 | the magic mechanism currently implements a kind of lazy access to arrays |
04343c6d |
1194 | and hashes. |
1195 | |
1196 | Currently (as of perl version 5.004), use of the hash and array access |
1197 | functions requires the user to be aware of whether they are operating on |
9edb2b46 |
1198 | "normal" hashes and arrays, or on their tied variants. The API may be |
1199 | changed to provide more transparent access to both tied and normal data |
1200 | types in future versions. |
1201 | [/MAYCHANGE] |
04343c6d |
1202 | |
1203 | You would do well to understand that the TIEARRAY and TIEHASH interfaces |
1204 | are mere sugar to invoke some perl method calls while using the uniform hash |
1205 | and array syntax. The use of this sugar imposes some overhead (typically |
1206 | about two to four extra opcodes per FETCH/STORE operation, in addition to |
1207 | the creation of all the mortal variables required to invoke the methods). |
1208 | This overhead will be comparatively small if the TIE methods are themselves |
1209 | substantial, but if they are only a few statements long, the overhead |
1210 | will not be insignificant. |
1211 | |
d1c897a1 |
1212 | =head2 Localizing changes |
1213 | |
1214 | Perl has a very handy construction |
1215 | |
1216 | { |
1217 | local $var = 2; |
1218 | ... |
1219 | } |
1220 | |
1221 | This construction is I<approximately> equivalent to |
1222 | |
1223 | { |
1224 | my $oldvar = $var; |
1225 | $var = 2; |
1226 | ... |
1227 | $var = $oldvar; |
1228 | } |
1229 | |
1230 | The biggest difference is that the first construction would |
1231 | reinstate the initial value of $var, irrespective of how control exits |
9a68f1db |
1232 | the block: C<goto>, C<return>, C<die>/C<eval>, etc. It is a little bit |
d1c897a1 |
1233 | more efficient as well. |
1234 | |
1235 | There is a way to achieve a similar task from C via Perl API: create a |
1236 | I<pseudo-block>, and arrange for some changes to be automatically |
1237 | undone at the end of it, either explicit, or via a non-local exit (via |
1238 | die()). A I<block>-like construct is created by a pair of |
b687b08b |
1239 | C<ENTER>/C<LEAVE> macros (see L<perlcall/"Returning a Scalar">). |
1240 | Such a construct may be created specially for some important localized |
1241 | task, or an existing one (like boundaries of enclosing Perl |
1242 | subroutine/block, or an existing pair for freeing TMPs) may be |
1243 | used. (In the second case the overhead of additional localization must |
1244 | be almost negligible.) Note that any XSUB is automatically enclosed in |
1245 | an C<ENTER>/C<LEAVE> pair. |
d1c897a1 |
1246 | |
1247 | Inside such a I<pseudo-block> the following service is available: |
1248 | |
13a2d996 |
1249 | =over 4 |
d1c897a1 |
1250 | |
1251 | =item C<SAVEINT(int i)> |
1252 | |
1253 | =item C<SAVEIV(IV i)> |
1254 | |
1255 | =item C<SAVEI32(I32 i)> |
1256 | |
1257 | =item C<SAVELONG(long i)> |
1258 | |
1259 | These macros arrange things to restore the value of integer variable |
1260 | C<i> at the end of enclosing I<pseudo-block>. |
1261 | |
1262 | =item C<SAVESPTR(s)> |
1263 | |
1264 | =item C<SAVEPPTR(p)> |
1265 | |
1266 | These macros arrange things to restore the value of pointers C<s> and |
1267 | C<p>. C<s> must be a pointer of a type which survives conversion to |
1268 | C<SV*> and back, C<p> should be able to survive conversion to C<char*> |
1269 | and back. |
1270 | |
1271 | =item C<SAVEFREESV(SV *sv)> |
1272 | |
1273 | The refcount of C<sv> would be decremented at the end of |
26d9b02f |
1274 | I<pseudo-block>. This is similar to C<sv_2mortal> in that it is also a |
1275 | mechanism for doing a delayed C<SvREFCNT_dec>. However, while C<sv_2mortal> |
1276 | extends the lifetime of C<sv> until the beginning of the next statement, |
1277 | C<SAVEFREESV> extends it until the end of the enclosing scope. These |
1278 | lifetimes can be wildly different. |
1279 | |
1280 | Also compare C<SAVEMORTALIZESV>. |
1281 | |
1282 | =item C<SAVEMORTALIZESV(SV *sv)> |
1283 | |
1284 | Just like C<SAVEFREESV>, but mortalizes C<sv> at the end of the current |
1285 | scope instead of decrementing its reference count. This usually has the |
1286 | effect of keeping C<sv> alive until the statement that called the currently |
1287 | live scope has finished executing. |
d1c897a1 |
1288 | |
1289 | =item C<SAVEFREEOP(OP *op)> |
1290 | |
1291 | The C<OP *> is op_free()ed at the end of I<pseudo-block>. |
1292 | |
1293 | =item C<SAVEFREEPV(p)> |
1294 | |
1295 | The chunk of memory which is pointed to by C<p> is Safefree()ed at the |
1296 | end of I<pseudo-block>. |
1297 | |
1298 | =item C<SAVECLEARSV(SV *sv)> |
1299 | |
1300 | Clears a slot in the current scratchpad which corresponds to C<sv> at |
1301 | the end of I<pseudo-block>. |
1302 | |
1303 | =item C<SAVEDELETE(HV *hv, char *key, I32 length)> |
1304 | |
1305 | The key C<key> of C<hv> is deleted at the end of I<pseudo-block>. The |
1306 | string pointed to by C<key> is Safefree()ed. If one has a I<key> in |
1307 | short-lived storage, the corresponding string may be reallocated like |
1308 | this: |
1309 | |
9cde0e7f |
1310 | SAVEDELETE(PL_defstash, savepv(tmpbuf), strlen(tmpbuf)); |
d1c897a1 |
1311 | |
c76ac1ee |
1312 | =item C<SAVEDESTRUCTOR(DESTRUCTORFUNC_NOCONTEXT_t f, void *p)> |
d1c897a1 |
1313 | |
1314 | At the end of I<pseudo-block> the function C<f> is called with the |
c76ac1ee |
1315 | only argument C<p>. |
1316 | |
1317 | =item C<SAVEDESTRUCTOR_X(DESTRUCTORFUNC_t f, void *p)> |
1318 | |
1319 | At the end of I<pseudo-block> the function C<f> is called with the |
1320 | implicit context argument (if any), and C<p>. |
d1c897a1 |
1321 | |
1322 | =item C<SAVESTACK_POS()> |
1323 | |
1324 | The current offset on the Perl internal stack (cf. C<SP>) is restored |
1325 | at the end of I<pseudo-block>. |
1326 | |
1327 | =back |
1328 | |
1329 | The following API list contains functions, thus one needs to |
1330 | provide pointers to the modifiable data explicitly (either C pointers, |
00aadd71 |
1331 | or Perlish C<GV *>s). Where the above macros take C<int>, a similar |
d1c897a1 |
1332 | function takes C<int *>. |
1333 | |
13a2d996 |
1334 | =over 4 |
d1c897a1 |
1335 | |
1336 | =item C<SV* save_scalar(GV *gv)> |
1337 | |
1338 | Equivalent to Perl code C<local $gv>. |
1339 | |
1340 | =item C<AV* save_ary(GV *gv)> |
1341 | |
1342 | =item C<HV* save_hash(GV *gv)> |
1343 | |
1344 | Similar to C<save_scalar>, but localize C<@gv> and C<%gv>. |
1345 | |
1346 | =item C<void save_item(SV *item)> |
1347 | |
1348 | Duplicates the current value of C<SV>, on the exit from the current |
1349 | C<ENTER>/C<LEAVE> I<pseudo-block> will restore the value of C<SV> |
038fcae3 |
1350 | using the stored value. It doesn't handle magic. Use C<save_scalar> if |
1351 | magic is affected. |
d1c897a1 |
1352 | |
1353 | =item C<void save_list(SV **sarg, I32 maxsarg)> |
1354 | |
1355 | A variant of C<save_item> which takes multiple arguments via an array |
1356 | C<sarg> of C<SV*> of length C<maxsarg>. |
1357 | |
1358 | =item C<SV* save_svref(SV **sptr)> |
1359 | |
d1be9408 |
1360 | Similar to C<save_scalar>, but will reinstate an C<SV *>. |
d1c897a1 |
1361 | |
1362 | =item C<void save_aptr(AV **aptr)> |
1363 | |
1364 | =item C<void save_hptr(HV **hptr)> |
1365 | |
1366 | Similar to C<save_svref>, but localize C<AV *> and C<HV *>. |
1367 | |
1368 | =back |
1369 | |
1370 | The C<Alias> module implements localization of the basic types within the |
1371 | I<caller's scope>. People who are interested in how to localize things in |
1372 | the containing scope should take a look there too. |
1373 | |
0a753a76 |
1374 | =head1 Subroutines |
a0d0e21e |
1375 | |
68dc0745 |
1376 | =head2 XSUBs and the Argument Stack |
5f05dabc |
1377 | |
1378 | The XSUB mechanism is a simple way for Perl programs to access C subroutines. |
1379 | An XSUB routine will have a stack that contains the arguments from the Perl |
1380 | program, and a way to map from the Perl data structures to a C equivalent. |
1381 | |
1382 | The stack arguments are accessible through the C<ST(n)> macro, which returns |
1383 | the C<n>'th stack argument. Argument 0 is the first argument passed in the |
1384 | Perl subroutine call. These arguments are C<SV*>, and can be used anywhere |
1385 | an C<SV*> is used. |
1386 | |
1387 | Most of the time, output from the C routine can be handled through use of |
1388 | the RETVAL and OUTPUT directives. However, there are some cases where the |
1389 | argument stack is not already long enough to handle all the return values. |
1390 | An example is the POSIX tzname() call, which takes no arguments, but returns |
1391 | two, the local time zone's standard and summer time abbreviations. |
1392 | |
1393 | To handle this situation, the PPCODE directive is used and the stack is |
1394 | extended using the macro: |
1395 | |
924508f0 |
1396 | EXTEND(SP, num); |
5f05dabc |
1397 | |
924508f0 |
1398 | where C<SP> is the macro that represents the local copy of the stack pointer, |
1399 | and C<num> is the number of elements the stack should be extended by. |
5f05dabc |
1400 | |
00aadd71 |
1401 | Now that there is room on the stack, values can be pushed on it using C<PUSHs> |
06f6df17 |
1402 | macro. The pushed values will often need to be "mortal" (See |
d82b684c |
1403 | L</Reference Counts and Mortality>): |
5f05dabc |
1404 | |
00aadd71 |
1405 | PUSHs(sv_2mortal(newSViv(an_integer))) |
d82b684c |
1406 | PUSHs(sv_2mortal(newSVuv(an_unsigned_integer))) |
1407 | PUSHs(sv_2mortal(newSVnv(a_double))) |
00aadd71 |
1408 | PUSHs(sv_2mortal(newSVpv("Some String",0))) |
5f05dabc |
1409 | |
1410 | And now the Perl program calling C<tzname>, the two values will be assigned |
1411 | as in: |
1412 | |
1413 | ($standard_abbrev, $summer_abbrev) = POSIX::tzname; |
1414 | |
1415 | An alternate (and possibly simpler) method to pushing values on the stack is |
00aadd71 |
1416 | to use the macro: |
5f05dabc |
1417 | |
5f05dabc |
1418 | XPUSHs(SV*) |
1419 | |
00aadd71 |
1420 | This macro automatically adjust the stack for you, if needed. Thus, you |
5f05dabc |
1421 | do not need to call C<EXTEND> to extend the stack. |
00aadd71 |
1422 | |
1423 | Despite their suggestions in earlier versions of this document the macros |
d82b684c |
1424 | C<(X)PUSH[iunp]> are I<not> suited to XSUBs which return multiple results. |
1425 | For that, either stick to the C<(X)PUSHs> macros shown above, or use the new |
1426 | C<m(X)PUSH[iunp]> macros instead; see L</Putting a C value on Perl stack>. |
5f05dabc |
1427 | |
1428 | For more information, consult L<perlxs> and L<perlxstut>. |
1429 | |
1430 | =head2 Calling Perl Routines from within C Programs |
a0d0e21e |
1431 | |
1432 | There are four routines that can be used to call a Perl subroutine from |
1433 | within a C program. These four are: |
1434 | |
954c1994 |
1435 | I32 call_sv(SV*, I32); |
1436 | I32 call_pv(const char*, I32); |
1437 | I32 call_method(const char*, I32); |
1438 | I32 call_argv(const char*, I32, register char**); |
a0d0e21e |
1439 | |
954c1994 |
1440 | The routine most often used is C<call_sv>. The C<SV*> argument |
d1b91892 |
1441 | contains either the name of the Perl subroutine to be called, or a |
1442 | reference to the subroutine. The second argument consists of flags |
1443 | that control the context in which the subroutine is called, whether |
1444 | or not the subroutine is being passed arguments, how errors should be |
1445 | trapped, and how to treat return values. |
a0d0e21e |
1446 | |
1447 | All four routines return the number of arguments that the subroutine returned |
1448 | on the Perl stack. |
1449 | |
9a68f1db |
1450 | These routines used to be called C<perl_call_sv>, etc., before Perl v5.6.0, |
954c1994 |
1451 | but those names are now deprecated; macros of the same name are provided for |
1452 | compatibility. |
1453 | |
1454 | When using any of these routines (except C<call_argv>), the programmer |
d1b91892 |
1455 | must manipulate the Perl stack. These include the following macros and |
1456 | functions: |
a0d0e21e |
1457 | |
1458 | dSP |
924508f0 |
1459 | SP |
a0d0e21e |
1460 | PUSHMARK() |
1461 | PUTBACK |
1462 | SPAGAIN |
1463 | ENTER |
1464 | SAVETMPS |
1465 | FREETMPS |
1466 | LEAVE |
1467 | XPUSH*() |
cb1a09d0 |
1468 | POP*() |
a0d0e21e |
1469 | |
5f05dabc |
1470 | For a detailed description of calling conventions from C to Perl, |
1471 | consult L<perlcall>. |
a0d0e21e |
1472 | |
5f05dabc |
1473 | =head2 Memory Allocation |
a0d0e21e |
1474 | |
06f6df17 |
1475 | =head3 Allocation |
1476 | |
86058a2d |
1477 | All memory meant to be used with the Perl API functions should be manipulated |
1478 | using the macros described in this section. The macros provide the necessary |
1479 | transparency between differences in the actual malloc implementation that is |
1480 | used within perl. |
1481 | |
1482 | It is suggested that you enable the version of malloc that is distributed |
5f05dabc |
1483 | with Perl. It keeps pools of various sizes of unallocated memory in |
07fa94a1 |
1484 | order to satisfy allocation requests more quickly. However, on some |
1485 | platforms, it may cause spurious malloc or free errors. |
d1b91892 |
1486 | |
06f6df17 |
1487 | The following three macros are used to initially allocate memory : |
1488 | |
d1b91892 |
1489 | New(x, pointer, number, type); |
1490 | Newc(x, pointer, number, type, cast); |
1491 | Newz(x, pointer, number, type); |
1492 | |
5f05dabc |
1493 | The first argument C<x> was a "magic cookie" that was used to keep track |
1494 | of who called the macro, to help when debugging memory problems. However, |
07fa94a1 |
1495 | the current code makes no use of this feature (most Perl developers now |
1496 | use run-time memory checkers), so this argument can be any number. |
5f05dabc |
1497 | |
1498 | The second argument C<pointer> should be the name of a variable that will |
1499 | point to the newly allocated memory. |
d1b91892 |
1500 | |
d1b91892 |
1501 | The third and fourth arguments C<number> and C<type> specify how many of |
1502 | the specified type of data structure should be allocated. The argument |
1503 | C<type> is passed to C<sizeof>. The final argument to C<Newc>, C<cast>, |
1504 | should be used if the C<pointer> argument is different from the C<type> |
1505 | argument. |
1506 | |
1507 | Unlike the C<New> and C<Newc> macros, the C<Newz> macro calls C<memzero> |
1508 | to zero out all the newly allocated memory. |
1509 | |
06f6df17 |
1510 | =head3 Reallocation |
1511 | |
d1b91892 |
1512 | Renew(pointer, number, type); |
1513 | Renewc(pointer, number, type, cast); |
1514 | Safefree(pointer) |
1515 | |
1516 | These three macros are used to change a memory buffer size or to free a |
1517 | piece of memory no longer needed. The arguments to C<Renew> and C<Renewc> |
1518 | match those of C<New> and C<Newc> with the exception of not needing the |
1519 | "magic cookie" argument. |
1520 | |
06f6df17 |
1521 | =head3 Moving |
1522 | |
d1b91892 |
1523 | Move(source, dest, number, type); |
1524 | Copy(source, dest, number, type); |
1525 | Zero(dest, number, type); |
1526 | |
1527 | These three macros are used to move, copy, or zero out previously allocated |
1528 | memory. The C<source> and C<dest> arguments point to the source and |
1529 | destination starting points. Perl will move, copy, or zero out C<number> |
1530 | instances of the size of the C<type> data structure (using the C<sizeof> |
1531 | function). |
a0d0e21e |
1532 | |
5f05dabc |
1533 | =head2 PerlIO |
ce3d39e2 |
1534 | |
5f05dabc |
1535 | The most recent development releases of Perl has been experimenting with |
1536 | removing Perl's dependency on the "normal" standard I/O suite and allowing |
1537 | other stdio implementations to be used. This involves creating a new |
1538 | abstraction layer that then calls whichever implementation of stdio Perl |
68dc0745 |
1539 | was compiled with. All XSUBs should now use the functions in the PerlIO |
5f05dabc |
1540 | abstraction layer and not make any assumptions about what kind of stdio |
1541 | is being used. |
1542 | |
1543 | For a complete description of the PerlIO abstraction, consult L<perlapio>. |
1544 | |
8ebc5c01 |
1545 | =head2 Putting a C value on Perl stack |
ce3d39e2 |
1546 | |
1547 | A lot of opcodes (this is an elementary operation in the internal perl |
1548 | stack machine) put an SV* on the stack. However, as an optimization |
1549 | the corresponding SV is (usually) not recreated each time. The opcodes |
1550 | reuse specially assigned SVs (I<target>s) which are (as a corollary) |
1551 | not constantly freed/created. |
1552 | |
0a753a76 |
1553 | Each of the targets is created only once (but see |
ce3d39e2 |
1554 | L<Scratchpads and recursion> below), and when an opcode needs to put |
1555 | an integer, a double, or a string on stack, it just sets the |
1556 | corresponding parts of its I<target> and puts the I<target> on stack. |
1557 | |
1558 | The macro to put this target on stack is C<PUSHTARG>, and it is |
1559 | directly used in some opcodes, as well as indirectly in zillions of |
d82b684c |
1560 | others, which use it via C<(X)PUSH[iunp]>. |
ce3d39e2 |
1561 | |
1bd1c0d5 |
1562 | Because the target is reused, you must be careful when pushing multiple |
1563 | values on the stack. The following code will not do what you think: |
1564 | |
1565 | XPUSHi(10); |
1566 | XPUSHi(20); |
1567 | |
1568 | This translates as "set C<TARG> to 10, push a pointer to C<TARG> onto |
1569 | the stack; set C<TARG> to 20, push a pointer to C<TARG> onto the stack". |
1570 | At the end of the operation, the stack does not contain the values 10 |
1571 | and 20, but actually contains two pointers to C<TARG>, which we have set |
d82b684c |
1572 | to 20. |
1bd1c0d5 |
1573 | |
d82b684c |
1574 | If you need to push multiple different values then you should either use |
1575 | the C<(X)PUSHs> macros, or else use the new C<m(X)PUSH[iunp]> macros, |
1576 | none of which make use of C<TARG>. The C<(X)PUSHs> macros simply push an |
1577 | SV* on the stack, which, as noted under L</XSUBs and the Argument Stack>, |
1578 | will often need to be "mortal". The new C<m(X)PUSH[iunp]> macros make |
1579 | this a little easier to achieve by creating a new mortal for you (via |
1580 | C<(X)PUSHmortal>), pushing that onto the stack (extending it if necessary |
1581 | in the case of the C<mXPUSH[iunp]> macros), and then setting its value. |
1582 | Thus, instead of writing this to "fix" the example above: |
1583 | |
1584 | XPUSHs(sv_2mortal(newSViv(10))) |
1585 | XPUSHs(sv_2mortal(newSViv(20))) |
1586 | |
1587 | you can simply write: |
1588 | |
1589 | mXPUSHi(10) |
1590 | mXPUSHi(20) |
1591 | |
1592 | On a related note, if you do use C<(X)PUSH[iunp]>, then you're going to |
1bd1c0d5 |
1593 | need a C<dTARG> in your variable declarations so that the C<*PUSH*> |
d82b684c |
1594 | macros can make use of the local variable C<TARG>. See also C<dTARGET> |
1595 | and C<dXSTARG>. |
1bd1c0d5 |
1596 | |
8ebc5c01 |
1597 | =head2 Scratchpads |
ce3d39e2 |
1598 | |
54310121 |
1599 | The question remains on when the SVs which are I<target>s for opcodes |
5f05dabc |
1600 | are created. The answer is that they are created when the current unit -- |
1601 | a subroutine or a file (for opcodes for statements outside of |
1602 | subroutines) -- is compiled. During this time a special anonymous Perl |
ce3d39e2 |
1603 | array is created, which is called a scratchpad for the current |
1604 | unit. |
1605 | |
54310121 |
1606 | A scratchpad keeps SVs which are lexicals for the current unit and are |
ce3d39e2 |
1607 | targets for opcodes. One can deduce that an SV lives on a scratchpad |
1608 | by looking on its flags: lexicals have C<SVs_PADMY> set, and |
1609 | I<target>s have C<SVs_PADTMP> set. |
1610 | |
54310121 |
1611 | The correspondence between OPs and I<target>s is not 1-to-1. Different |
1612 | OPs in the compile tree of the unit can use the same target, if this |
ce3d39e2 |
1613 | would not conflict with the expected life of the temporary. |
1614 | |
2ae324a7 |
1615 | =head2 Scratchpads and recursion |
ce3d39e2 |
1616 | |
1617 | In fact it is not 100% true that a compiled unit contains a pointer to |
1618 | the scratchpad AV. In fact it contains a pointer to an AV of |
1619 | (initially) one element, and this element is the scratchpad AV. Why do |
1620 | we need an extra level of indirection? |
1621 | |
9a68f1db |
1622 | The answer is B<recursion>, and maybe B<threads>. Both |
ce3d39e2 |
1623 | these can create several execution pointers going into the same |
1624 | subroutine. For the subroutine-child not write over the temporaries |
1625 | for the subroutine-parent (lifespan of which covers the call to the |
1626 | child), the parent and the child should have different |
1627 | scratchpads. (I<And> the lexicals should be separate anyway!) |
1628 | |
5f05dabc |
1629 | So each subroutine is born with an array of scratchpads (of length 1). |
1630 | On each entry to the subroutine it is checked that the current |
ce3d39e2 |
1631 | depth of the recursion is not more than the length of this array, and |
1632 | if it is, new scratchpad is created and pushed into the array. |
1633 | |
1634 | The I<target>s on this scratchpad are C<undef>s, but they are already |
1635 | marked with correct flags. |
1636 | |
0a753a76 |
1637 | =head1 Compiled code |
1638 | |
1639 | =head2 Code tree |
1640 | |
1641 | Here we describe the internal form your code is converted to by |
1642 | Perl. Start with a simple example: |
1643 | |
1644 | $a = $b + $c; |
1645 | |
1646 | This is converted to a tree similar to this one: |
1647 | |
1648 | assign-to |
1649 | / \ |
1650 | + $a |
1651 | / \ |
1652 | $b $c |
1653 | |
7b8d334a |
1654 | (but slightly more complicated). This tree reflects the way Perl |
0a753a76 |
1655 | parsed your code, but has nothing to do with the execution order. |
1656 | There is an additional "thread" going through the nodes of the tree |
1657 | which shows the order of execution of the nodes. In our simplified |
1658 | example above it looks like: |
1659 | |
1660 | $b ---> $c ---> + ---> $a ---> assign-to |
1661 | |
1662 | But with the actual compile tree for C<$a = $b + $c> it is different: |
1663 | some nodes I<optimized away>. As a corollary, though the actual tree |
1664 | contains more nodes than our simplified example, the execution order |
1665 | is the same as in our example. |
1666 | |
1667 | =head2 Examining the tree |
1668 | |
06f6df17 |
1669 | If you have your perl compiled for debugging (usually done with |
1670 | C<-DDEBUGGING> on the C<Configure> command line), you may examine the |
0a753a76 |
1671 | compiled tree by specifying C<-Dx> on the Perl command line. The |
1672 | output takes several lines per node, and for C<$b+$c> it looks like |
1673 | this: |
1674 | |
1675 | 5 TYPE = add ===> 6 |
1676 | TARG = 1 |
1677 | FLAGS = (SCALAR,KIDS) |
1678 | { |
1679 | TYPE = null ===> (4) |
1680 | (was rv2sv) |
1681 | FLAGS = (SCALAR,KIDS) |
1682 | { |
1683 | 3 TYPE = gvsv ===> 4 |
1684 | FLAGS = (SCALAR) |
1685 | GV = main::b |
1686 | } |
1687 | } |
1688 | { |
1689 | TYPE = null ===> (5) |
1690 | (was rv2sv) |
1691 | FLAGS = (SCALAR,KIDS) |
1692 | { |
1693 | 4 TYPE = gvsv ===> 5 |
1694 | FLAGS = (SCALAR) |
1695 | GV = main::c |
1696 | } |
1697 | } |
1698 | |
1699 | This tree has 5 nodes (one per C<TYPE> specifier), only 3 of them are |
1700 | not optimized away (one per number in the left column). The immediate |
1701 | children of the given node correspond to C<{}> pairs on the same level |
1702 | of indentation, thus this listing corresponds to the tree: |
1703 | |
1704 | add |
1705 | / \ |
1706 | null null |
1707 | | | |
1708 | gvsv gvsv |
1709 | |
1710 | The execution order is indicated by C<===E<gt>> marks, thus it is C<3 |
1711 | 4 5 6> (node C<6> is not included into above listing), i.e., |
1712 | C<gvsv gvsv add whatever>. |
1713 | |
9afa14e3 |
1714 | Each of these nodes represents an op, a fundamental operation inside the |
1715 | Perl core. The code which implements each operation can be found in the |
1716 | F<pp*.c> files; the function which implements the op with type C<gvsv> |
1717 | is C<pp_gvsv>, and so on. As the tree above shows, different ops have |
1718 | different numbers of children: C<add> is a binary operator, as one would |
1719 | expect, and so has two children. To accommodate the various different |
1720 | numbers of children, there are various types of op data structure, and |
1721 | they link together in different ways. |
1722 | |
1723 | The simplest type of op structure is C<OP>: this has no children. Unary |
1724 | operators, C<UNOP>s, have one child, and this is pointed to by the |
1725 | C<op_first> field. Binary operators (C<BINOP>s) have not only an |
1726 | C<op_first> field but also an C<op_last> field. The most complex type of |
1727 | op is a C<LISTOP>, which has any number of children. In this case, the |
1728 | first child is pointed to by C<op_first> and the last child by |
1729 | C<op_last>. The children in between can be found by iteratively |
1730 | following the C<op_sibling> pointer from the first child to the last. |
1731 | |
1732 | There are also two other op types: a C<PMOP> holds a regular expression, |
1733 | and has no children, and a C<LOOP> may or may not have children. If the |
1734 | C<op_children> field is non-zero, it behaves like a C<LISTOP>. To |
1735 | complicate matters, if a C<UNOP> is actually a C<null> op after |
1736 | optimization (see L</Compile pass 2: context propagation>) it will still |
1737 | have children in accordance with its former type. |
1738 | |
06f6df17 |
1739 | Another way to examine the tree is to use a compiler back-end module, such |
1740 | as L<B::Concise>. |
1741 | |
0a753a76 |
1742 | =head2 Compile pass 1: check routines |
1743 | |
8870b5c7 |
1744 | The tree is created by the compiler while I<yacc> code feeds it |
1745 | the constructions it recognizes. Since I<yacc> works bottom-up, so does |
0a753a76 |
1746 | the first pass of perl compilation. |
1747 | |
1748 | What makes this pass interesting for perl developers is that some |
1749 | optimization may be performed on this pass. This is optimization by |
8870b5c7 |
1750 | so-called "check routines". The correspondence between node names |
0a753a76 |
1751 | and corresponding check routines is described in F<opcode.pl> (do not |
1752 | forget to run C<make regen_headers> if you modify this file). |
1753 | |
1754 | A check routine is called when the node is fully constructed except |
7b8d334a |
1755 | for the execution-order thread. Since at this time there are no |
0a753a76 |
1756 | back-links to the currently constructed node, one can do most any |
1757 | operation to the top-level node, including freeing it and/or creating |
1758 | new nodes above/below it. |
1759 | |
1760 | The check routine returns the node which should be inserted into the |
1761 | tree (if the top-level node was not modified, check routine returns |
1762 | its argument). |
1763 | |
1764 | By convention, check routines have names C<ck_*>. They are usually |
1765 | called from C<new*OP> subroutines (or C<convert>) (which in turn are |
1766 | called from F<perly.y>). |
1767 | |
1768 | =head2 Compile pass 1a: constant folding |
1769 | |
1770 | Immediately after the check routine is called the returned node is |
1771 | checked for being compile-time executable. If it is (the value is |
1772 | judged to be constant) it is immediately executed, and a I<constant> |
1773 | node with the "return value" of the corresponding subtree is |
1774 | substituted instead. The subtree is deleted. |
1775 | |
1776 | If constant folding was not performed, the execution-order thread is |
1777 | created. |
1778 | |
1779 | =head2 Compile pass 2: context propagation |
1780 | |
1781 | When a context for a part of compile tree is known, it is propagated |
a3cb178b |
1782 | down through the tree. At this time the context can have 5 values |
0a753a76 |
1783 | (instead of 2 for runtime context): void, boolean, scalar, list, and |
1784 | lvalue. In contrast with the pass 1 this pass is processed from top |
1785 | to bottom: a node's context determines the context for its children. |
1786 | |
1787 | Additional context-dependent optimizations are performed at this time. |
1788 | Since at this moment the compile tree contains back-references (via |
1789 | "thread" pointers), nodes cannot be free()d now. To allow |
1790 | optimized-away nodes at this stage, such nodes are null()ified instead |
1791 | of free()ing (i.e. their type is changed to OP_NULL). |
1792 | |
1793 | =head2 Compile pass 3: peephole optimization |
1794 | |
1795 | After the compile tree for a subroutine (or for an C<eval> or a file) |
1796 | is created, an additional pass over the code is performed. This pass |
1797 | is neither top-down or bottom-up, but in the execution order (with |
7b8d334a |
1798 | additional complications for conditionals). These optimizations are |
0a753a76 |
1799 | done in the subroutine peep(). Optimizations performed at this stage |
1800 | are subject to the same restrictions as in the pass 2. |
1801 | |
1ba7f851 |
1802 | =head2 Pluggable runops |
1803 | |
1804 | The compile tree is executed in a runops function. There are two runops |
1388f78e |
1805 | functions, in F<run.c> and in F<dump.c>. C<Perl_runops_debug> is used |
1806 | with DEBUGGING and C<Perl_runops_standard> is used otherwise. For fine |
1807 | control over the execution of the compile tree it is possible to provide |
1808 | your own runops function. |
1ba7f851 |
1809 | |
1810 | It's probably best to copy one of the existing runops functions and |
1811 | change it to suit your needs. Then, in the BOOT section of your XS |
1812 | file, add the line: |
1813 | |
1814 | PL_runops = my_runops; |
1815 | |
1816 | This function should be as efficient as possible to keep your programs |
1817 | running as fast as possible. |
1818 | |
9afa14e3 |
1819 | =head1 Examining internal data structures with the C<dump> functions |
1820 | |
1821 | To aid debugging, the source file F<dump.c> contains a number of |
1822 | functions which produce formatted output of internal data structures. |
1823 | |
1824 | The most commonly used of these functions is C<Perl_sv_dump>; it's used |
1825 | for dumping SVs, AVs, HVs, and CVs. The C<Devel::Peek> module calls |
1826 | C<sv_dump> to produce debugging output from Perl-space, so users of that |
00aadd71 |
1827 | module should already be familiar with its format. |
9afa14e3 |
1828 | |
1829 | C<Perl_op_dump> can be used to dump an C<OP> structure or any of its |
210b36aa |
1830 | derivatives, and produces output similar to C<perl -Dx>; in fact, |
9afa14e3 |
1831 | C<Perl_dump_eval> will dump the main root of the code being evaluated, |
1832 | exactly like C<-Dx>. |
1833 | |
1834 | Other useful functions are C<Perl_dump_sub>, which turns a C<GV> into an |
1835 | op tree, C<Perl_dump_packsubs> which calls C<Perl_dump_sub> on all the |
1836 | subroutines in a package like so: (Thankfully, these are all xsubs, so |
1837 | there is no op tree) |
1838 | |
1839 | (gdb) print Perl_dump_packsubs(PL_defstash) |
1840 | |
1841 | SUB attributes::bootstrap = (xsub 0x811fedc 0) |
1842 | |
1843 | SUB UNIVERSAL::can = (xsub 0x811f50c 0) |
1844 | |
1845 | SUB UNIVERSAL::isa = (xsub 0x811f304 0) |
1846 | |
1847 | SUB UNIVERSAL::VERSION = (xsub 0x811f7ac 0) |
1848 | |
1849 | SUB DynaLoader::boot_DynaLoader = (xsub 0x805b188 0) |
1850 | |
1851 | and C<Perl_dump_all>, which dumps all the subroutines in the stash and |
1852 | the op tree of the main root. |
1853 | |
954c1994 |
1854 | =head1 How multiple interpreters and concurrency are supported |
ee072b34 |
1855 | |
ee072b34 |
1856 | =head2 Background and PERL_IMPLICIT_CONTEXT |
1857 | |
1858 | The Perl interpreter can be regarded as a closed box: it has an API |
1859 | for feeding it code or otherwise making it do things, but it also has |
1860 | functions for its own use. This smells a lot like an object, and |
1861 | there are ways for you to build Perl so that you can have multiple |
acfe0abc |
1862 | interpreters, with one interpreter represented either as a C structure, |
1863 | or inside a thread-specific structure. These structures contain all |
1864 | the context, the state of that interpreter. |
1865 | |
9a68f1db |
1866 | Two macros control the major Perl build flavors: MULTIPLICITY and |
acfe0abc |
1867 | USE_5005THREADS. The MULTIPLICITY build has a C structure |
1868 | that packages all the interpreter state, and there is a similar thread-specific |
1869 | data structure under USE_5005THREADS. In both cases, |
54aff467 |
1870 | PERL_IMPLICIT_CONTEXT is also normally defined, and enables the |
1871 | support for passing in a "hidden" first argument that represents all three |
651a3225 |
1872 | data structures. |
54aff467 |
1873 | |
27da23d5 |
1874 | Two other "encapsulation" macros are the PERL_GLOBAL_STRUCT and |
1875 | PERL_GLOBAL_STRUCT_PRIVATE (the latter turns on the former, and the |
1876 | former turns on MULTIPLICITY.) The PERL_GLOBAL_STRUCT causes all the |
1877 | internal variables of Perl to be wrapped inside a single global struct, |
1878 | struct perl_vars, accessible as (globals) &PL_Vars or PL_VarsPtr or |
1879 | the function Perl_GetVars(). The PERL_GLOBAL_STRUCT_PRIVATE goes |
1880 | one step further, there is still a single struct (allocated in main() |
1881 | either from heap or from stack) but there are no global data symbols |
1882 | pointing to it. In either case the global struct should be initialised |
1883 | as the very first thing in main() using Perl_init_global_struct() and |
1884 | correspondingly tear it down after perl_free() using Perl_free_global_struct(), |
1885 | please see F<miniperlmain.c> for usage details. You may also need |
1886 | to use C<dVAR> in your coding to "declare the global variables" |
1887 | when you are using them. dTHX does this for you automatically. |
1888 | |
1889 | For backward compatibility reasons defining just PERL_GLOBAL_STRUCT |
1890 | doesn't actually hide all symbols inside a big global struct: some |
1891 | PerlIO_xxx vtables are left visible. The PERL_GLOBAL_STRUCT_PRIVATE |
1892 | then hides everything (see how the PERLIO_FUNCS_DECL is used). |
1893 | |
54aff467 |
1894 | All this obviously requires a way for the Perl internal functions to be |
acfe0abc |
1895 | either subroutines taking some kind of structure as the first |
ee072b34 |
1896 | argument, or subroutines taking nothing as the first argument. To |
acfe0abc |
1897 | enable these two very different ways of building the interpreter, |
ee072b34 |
1898 | the Perl source (as it does in so many other situations) makes heavy |
1899 | use of macros and subroutine naming conventions. |
1900 | |
54aff467 |
1901 | First problem: deciding which functions will be public API functions and |
00aadd71 |
1902 | which will be private. All functions whose names begin C<S_> are private |
954c1994 |
1903 | (think "S" for "secret" or "static"). All other functions begin with |
1904 | "Perl_", but just because a function begins with "Perl_" does not mean it is |
00aadd71 |
1905 | part of the API. (See L</Internal Functions>.) The easiest way to be B<sure> a |
1906 | function is part of the API is to find its entry in L<perlapi>. |
1907 | If it exists in L<perlapi>, it's part of the API. If it doesn't, and you |
1908 | think it should be (i.e., you need it for your extension), send mail via |
a422fd2d |
1909 | L<perlbug> explaining why you think it should be. |
ee072b34 |
1910 | |
1911 | Second problem: there must be a syntax so that the same subroutine |
1912 | declarations and calls can pass a structure as their first argument, |
1913 | or pass nothing. To solve this, the subroutines are named and |
1914 | declared in a particular way. Here's a typical start of a static |
1915 | function used within the Perl guts: |
1916 | |
1917 | STATIC void |
1918 | S_incline(pTHX_ char *s) |
1919 | |
acfe0abc |
1920 | STATIC becomes "static" in C, and may be #define'd to nothing in some |
1921 | configurations in future. |
ee072b34 |
1922 | |
651a3225 |
1923 | A public function (i.e. part of the internal API, but not necessarily |
1924 | sanctioned for use in extensions) begins like this: |
ee072b34 |
1925 | |
1926 | void |
2307c6d0 |
1927 | Perl_sv_setiv(pTHX_ SV* dsv, IV num) |
ee072b34 |
1928 | |
1929 | C<pTHX_> is one of a number of macros (in perl.h) that hide the |
1930 | details of the interpreter's context. THX stands for "thread", "this", |
1931 | or "thingy", as the case may be. (And no, George Lucas is not involved. :-) |
1932 | The first character could be 'p' for a B<p>rototype, 'a' for B<a>rgument, |
a7486cbb |
1933 | or 'd' for B<d>eclaration, so we have C<pTHX>, C<aTHX> and C<dTHX>, and |
1934 | their variants. |
ee072b34 |
1935 | |
a7486cbb |
1936 | When Perl is built without options that set PERL_IMPLICIT_CONTEXT, there is no |
1937 | first argument containing the interpreter's context. The trailing underscore |
ee072b34 |
1938 | in the pTHX_ macro indicates that the macro expansion needs a comma |
1939 | after the context argument because other arguments follow it. If |
1940 | PERL_IMPLICIT_CONTEXT is not defined, pTHX_ will be ignored, and the |
54aff467 |
1941 | subroutine is not prototyped to take the extra argument. The form of the |
1942 | macro without the trailing underscore is used when there are no additional |
ee072b34 |
1943 | explicit arguments. |
1944 | |
54aff467 |
1945 | When a core function calls another, it must pass the context. This |
2307c6d0 |
1946 | is normally hidden via macros. Consider C<sv_setiv>. It expands into |
ee072b34 |
1947 | something like this: |
1948 | |
2307c6d0 |
1949 | #ifdef PERL_IMPLICIT_CONTEXT |
1950 | #define sv_setiv(a,b) Perl_sv_setiv(aTHX_ a, b) |
ee072b34 |
1951 | /* can't do this for vararg functions, see below */ |
2307c6d0 |
1952 | #else |
1953 | #define sv_setiv Perl_sv_setiv |
1954 | #endif |
ee072b34 |
1955 | |
1956 | This works well, and means that XS authors can gleefully write: |
1957 | |
2307c6d0 |
1958 | sv_setiv(foo, bar); |
ee072b34 |
1959 | |
1960 | and still have it work under all the modes Perl could have been |
1961 | compiled with. |
1962 | |
ee072b34 |
1963 | This doesn't work so cleanly for varargs functions, though, as macros |
1964 | imply that the number of arguments is known in advance. Instead we |
1965 | either need to spell them out fully, passing C<aTHX_> as the first |
1966 | argument (the Perl core tends to do this with functions like |
1967 | Perl_warner), or use a context-free version. |
1968 | |
1969 | The context-free version of Perl_warner is called |
1970 | Perl_warner_nocontext, and does not take the extra argument. Instead |
1971 | it does dTHX; to get the context from thread-local storage. We |
1972 | C<#define warner Perl_warner_nocontext> so that extensions get source |
1973 | compatibility at the expense of performance. (Passing an arg is |
1974 | cheaper than grabbing it from thread-local storage.) |
1975 | |
acfe0abc |
1976 | You can ignore [pad]THXx when browsing the Perl headers/sources. |
ee072b34 |
1977 | Those are strictly for use within the core. Extensions and embedders |
1978 | need only be aware of [pad]THX. |
1979 | |
a7486cbb |
1980 | =head2 So what happened to dTHR? |
1981 | |
1982 | C<dTHR> was introduced in perl 5.005 to support the older thread model. |
1983 | The older thread model now uses the C<THX> mechanism to pass context |
1984 | pointers around, so C<dTHR> is not useful any more. Perl 5.6.0 and |
1985 | later still have it for backward source compatibility, but it is defined |
1986 | to be a no-op. |
1987 | |
ee072b34 |
1988 | =head2 How do I use all this in extensions? |
1989 | |
1990 | When Perl is built with PERL_IMPLICIT_CONTEXT, extensions that call |
1991 | any functions in the Perl API will need to pass the initial context |
1992 | argument somehow. The kicker is that you will need to write it in |
1993 | such a way that the extension still compiles when Perl hasn't been |
1994 | built with PERL_IMPLICIT_CONTEXT enabled. |
1995 | |
1996 | There are three ways to do this. First, the easy but inefficient way, |
1997 | which is also the default, in order to maintain source compatibility |
1998 | with extensions: whenever XSUB.h is #included, it redefines the aTHX |
1999 | and aTHX_ macros to call a function that will return the context. |
2000 | Thus, something like: |
2001 | |
2307c6d0 |
2002 | sv_setiv(sv, num); |
ee072b34 |
2003 | |
4375e838 |
2004 | in your extension will translate to this when PERL_IMPLICIT_CONTEXT is |
54aff467 |
2005 | in effect: |
ee072b34 |
2006 | |
2307c6d0 |
2007 | Perl_sv_setiv(Perl_get_context(), sv, num); |
ee072b34 |
2008 | |
54aff467 |
2009 | or to this otherwise: |
ee072b34 |
2010 | |
2307c6d0 |
2011 | Perl_sv_setiv(sv, num); |
ee072b34 |
2012 | |
2013 | You have to do nothing new in your extension to get this; since |
2fa86c13 |
2014 | the Perl library provides Perl_get_context(), it will all just |
ee072b34 |
2015 | work. |
2016 | |
2017 | The second, more efficient way is to use the following template for |
2018 | your Foo.xs: |
2019 | |
c52f9dcd |
2020 | #define PERL_NO_GET_CONTEXT /* we want efficiency */ |
2021 | #include "EXTERN.h" |
2022 | #include "perl.h" |
2023 | #include "XSUB.h" |
ee072b34 |
2024 | |
2025 | static my_private_function(int arg1, int arg2); |
2026 | |
c52f9dcd |
2027 | static SV * |
2028 | my_private_function(int arg1, int arg2) |
2029 | { |
2030 | dTHX; /* fetch context */ |
2031 | ... call many Perl API functions ... |
2032 | } |
ee072b34 |
2033 | |
2034 | [... etc ...] |
2035 | |
c52f9dcd |
2036 | MODULE = Foo PACKAGE = Foo |
ee072b34 |
2037 | |
c52f9dcd |
2038 | /* typical XSUB */ |
ee072b34 |
2039 | |
c52f9dcd |
2040 | void |
2041 | my_xsub(arg) |
2042 | int arg |
2043 | CODE: |
2044 | my_private_function(arg, 10); |
ee072b34 |
2045 | |
2046 | Note that the only two changes from the normal way of writing an |
2047 | extension is the addition of a C<#define PERL_NO_GET_CONTEXT> before |
2048 | including the Perl headers, followed by a C<dTHX;> declaration at |
2049 | the start of every function that will call the Perl API. (You'll |
2050 | know which functions need this, because the C compiler will complain |
2051 | that there's an undeclared identifier in those functions.) No changes |
2052 | are needed for the XSUBs themselves, because the XS() macro is |
2053 | correctly defined to pass in the implicit context if needed. |
2054 | |
2055 | The third, even more efficient way is to ape how it is done within |
2056 | the Perl guts: |
2057 | |
2058 | |
c52f9dcd |
2059 | #define PERL_NO_GET_CONTEXT /* we want efficiency */ |
2060 | #include "EXTERN.h" |
2061 | #include "perl.h" |
2062 | #include "XSUB.h" |
ee072b34 |
2063 | |
2064 | /* pTHX_ only needed for functions that call Perl API */ |
2065 | static my_private_function(pTHX_ int arg1, int arg2); |
2066 | |
c52f9dcd |
2067 | static SV * |
2068 | my_private_function(pTHX_ int arg1, int arg2) |
2069 | { |
2070 | /* dTHX; not needed here, because THX is an argument */ |
2071 | ... call Perl API functions ... |
2072 | } |
ee072b34 |
2073 | |
2074 | [... etc ...] |
2075 | |
c52f9dcd |
2076 | MODULE = Foo PACKAGE = Foo |
ee072b34 |
2077 | |
c52f9dcd |
2078 | /* typical XSUB */ |
ee072b34 |
2079 | |
c52f9dcd |
2080 | void |
2081 | my_xsub(arg) |
2082 | int arg |
2083 | CODE: |
2084 | my_private_function(aTHX_ arg, 10); |
ee072b34 |
2085 | |
2086 | This implementation never has to fetch the context using a function |
2087 | call, since it is always passed as an extra argument. Depending on |
2088 | your needs for simplicity or efficiency, you may mix the previous |
2089 | two approaches freely. |
2090 | |
651a3225 |
2091 | Never add a comma after C<pTHX> yourself--always use the form of the |
2092 | macro with the underscore for functions that take explicit arguments, |
2093 | or the form without the argument for functions with no explicit arguments. |
ee072b34 |
2094 | |
27da23d5 |
2095 | If one is compiling Perl with the C<-DPERL_GLOBAL_STRUCT> the C<dVAR> |
2096 | definition is needed if the Perl global variables (see F<perlvars.h> |
2097 | or F<globvar.sym>) are accessed in the function and C<dTHX> is not |
2098 | used (the C<dTHX> includes the C<dVAR> if necessary). One notices |
2099 | the need for C<dVAR> only with the said compile-time define, because |
2100 | otherwise the Perl global variables are visible as-is. |
2101 | |
a7486cbb |
2102 | =head2 Should I do anything special if I call perl from multiple threads? |
2103 | |
2104 | If you create interpreters in one thread and then proceed to call them in |
2105 | another, you need to make sure perl's own Thread Local Storage (TLS) slot is |
2106 | initialized correctly in each of those threads. |
2107 | |
2108 | The C<perl_alloc> and C<perl_clone> API functions will automatically set |
2109 | the TLS slot to the interpreter they created, so that there is no need to do |
2110 | anything special if the interpreter is always accessed in the same thread that |
2111 | created it, and that thread did not create or call any other interpreters |
2112 | afterwards. If that is not the case, you have to set the TLS slot of the |
2113 | thread before calling any functions in the Perl API on that particular |
2114 | interpreter. This is done by calling the C<PERL_SET_CONTEXT> macro in that |
2115 | thread as the first thing you do: |
2116 | |
2117 | /* do this before doing anything else with some_perl */ |
2118 | PERL_SET_CONTEXT(some_perl); |
2119 | |
2120 | ... other Perl API calls on some_perl go here ... |
2121 | |
ee072b34 |
2122 | =head2 Future Plans and PERL_IMPLICIT_SYS |
2123 | |
2124 | Just as PERL_IMPLICIT_CONTEXT provides a way to bundle up everything |
2125 | that the interpreter knows about itself and pass it around, so too are |
2126 | there plans to allow the interpreter to bundle up everything it knows |
2127 | about the environment it's running on. This is enabled with the |
acfe0abc |
2128 | PERL_IMPLICIT_SYS macro. Currently it only works with USE_ITHREADS |
4d1ff10f |
2129 | and USE_5005THREADS on Windows (see inside iperlsys.h). |
ee072b34 |
2130 | |
2131 | This allows the ability to provide an extra pointer (called the "host" |
2132 | environment) for all the system calls. This makes it possible for |
2133 | all the system stuff to maintain their own state, broken down into |
2134 | seven C structures. These are thin wrappers around the usual system |
2135 | calls (see win32/perllib.c) for the default perl executable, but for a |
2136 | more ambitious host (like the one that would do fork() emulation) all |
2137 | the extra work needed to pretend that different interpreters are |
2138 | actually different "processes", would be done here. |
2139 | |
2140 | The Perl engine/interpreter and the host are orthogonal entities. |
2141 | There could be one or more interpreters in a process, and one or |
2142 | more "hosts", with free association between them. |
2143 | |
a422fd2d |
2144 | =head1 Internal Functions |
2145 | |
2146 | All of Perl's internal functions which will be exposed to the outside |
06f6df17 |
2147 | world are prefixed by C<Perl_> so that they will not conflict with XS |
a422fd2d |
2148 | functions or functions used in a program in which Perl is embedded. |
2149 | Similarly, all global variables begin with C<PL_>. (By convention, |
06f6df17 |
2150 | static functions start with C<S_>.) |
a422fd2d |
2151 | |
2152 | Inside the Perl core, you can get at the functions either with or |
2153 | without the C<Perl_> prefix, thanks to a bunch of defines that live in |
2154 | F<embed.h>. This header file is generated automatically from |
dc9b1d22 |
2155 | F<embed.pl> and F<embed.fnc>. F<embed.pl> also creates the prototyping |
2156 | header files for the internal functions, generates the documentation |
2157 | and a lot of other bits and pieces. It's important that when you add |
2158 | a new function to the core or change an existing one, you change the |
2159 | data in the table in F<embed.fnc> as well. Here's a sample entry from |
2160 | that table: |
a422fd2d |
2161 | |
2162 | Apd |SV** |av_fetch |AV* ar|I32 key|I32 lval |
2163 | |
2164 | The second column is the return type, the third column the name. Columns |
2165 | after that are the arguments. The first column is a set of flags: |
2166 | |
2167 | =over 3 |
2168 | |
2169 | =item A |
2170 | |
2171 | This function is a part of the public API. |
2172 | |
2173 | =item p |
2174 | |
2175 | This function has a C<Perl_> prefix; ie, it is defined as C<Perl_av_fetch> |
2176 | |
2177 | =item d |
2178 | |
2179 | This function has documentation using the C<apidoc> feature which we'll |
2180 | look at in a second. |
2181 | |
2182 | =back |
2183 | |
2184 | Other available flags are: |
2185 | |
2186 | =over 3 |
2187 | |
2188 | =item s |
2189 | |
a7486cbb |
2190 | This is a static function and is defined as C<S_whatever>, and usually |
2191 | called within the sources as C<whatever(...)>. |
a422fd2d |
2192 | |
2193 | =item n |
2194 | |
2195 | This does not use C<aTHX_> and C<pTHX> to pass interpreter context. (See |
2196 | L<perlguts/Background and PERL_IMPLICIT_CONTEXT>.) |
2197 | |
2198 | =item r |
2199 | |
2200 | This function never returns; C<croak>, C<exit> and friends. |
2201 | |
2202 | =item f |
2203 | |
2204 | This function takes a variable number of arguments, C<printf> style. |
2205 | The argument list should end with C<...>, like this: |
2206 | |
2207 | Afprd |void |croak |const char* pat|... |
2208 | |
a7486cbb |
2209 | =item M |
a422fd2d |
2210 | |
00aadd71 |
2211 | This function is part of the experimental development API, and may change |
a422fd2d |
2212 | or disappear without notice. |
2213 | |
2214 | =item o |
2215 | |
2216 | This function should not have a compatibility macro to define, say, |
2217 | C<Perl_parse> to C<parse>. It must be called as C<Perl_parse>. |
2218 | |
a422fd2d |
2219 | =item x |
2220 | |
2221 | This function isn't exported out of the Perl core. |
2222 | |
dc9b1d22 |
2223 | =item m |
2224 | |
2225 | This is implemented as a macro. |
2226 | |
2227 | =item X |
2228 | |
2229 | This function is explicitly exported. |
2230 | |
2231 | =item E |
2232 | |
2233 | This function is visible to extensions included in the Perl core. |
2234 | |
2235 | =item b |
2236 | |
2237 | Binary backward compatibility; this function is a macro but also has |
2238 | a C<Perl_> implementation (which is exported). |
2239 | |
a422fd2d |
2240 | =back |
2241 | |
dc9b1d22 |
2242 | If you edit F<embed.pl> or F<embed.fnc>, you will need to run |
2243 | C<make regen_headers> to force a rebuild of F<embed.h> and other |
2244 | auto-generated files. |
a422fd2d |
2245 | |
6b4667fc |
2246 | =head2 Formatted Printing of IVs, UVs, and NVs |
9dd9db0b |
2247 | |
6b4667fc |
2248 | If you are printing IVs, UVs, or NVS instead of the stdio(3) style |
2249 | formatting codes like C<%d>, C<%ld>, C<%f>, you should use the |
2250 | following macros for portability |
9dd9db0b |
2251 | |
c52f9dcd |
2252 | IVdf IV in decimal |
2253 | UVuf UV in decimal |
2254 | UVof UV in octal |
2255 | UVxf UV in hexadecimal |
2256 | NVef NV %e-like |
2257 | NVff NV %f-like |
2258 | NVgf NV %g-like |
9dd9db0b |
2259 | |
6b4667fc |
2260 | These will take care of 64-bit integers and long doubles. |
2261 | For example: |
2262 | |
c52f9dcd |
2263 | printf("IV is %"IVdf"\n", iv); |
6b4667fc |
2264 | |
2265 | The IVdf will expand to whatever is the correct format for the IVs. |
9dd9db0b |
2266 | |
8908e76d |
2267 | If you are printing addresses of pointers, use UVxf combined |
2268 | with PTR2UV(), do not use %lx or %p. |
2269 | |
2270 | =head2 Pointer-To-Integer and Integer-To-Pointer |
2271 | |
2272 | Because pointer size does not necessarily equal integer size, |
2273 | use the follow macros to do it right. |
2274 | |
c52f9dcd |
2275 | PTR2UV(pointer) |
2276 | PTR2IV(pointer) |
2277 | PTR2NV(pointer) |
2278 | INT2PTR(pointertotype, integer) |
8908e76d |
2279 | |
2280 | For example: |
2281 | |
c52f9dcd |
2282 | IV iv = ...; |
2283 | SV *sv = INT2PTR(SV*, iv); |
8908e76d |
2284 | |
2285 | and |
2286 | |
c52f9dcd |
2287 | AV *av = ...; |
2288 | UV uv = PTR2UV(av); |
8908e76d |
2289 | |
0ca3a874 |
2290 | =head2 Exception Handling |
2291 | |
9b5c3821 |
2292 | There are a couple of macros to do very basic exception handling in XS |
2293 | modules. You have to define C<NO_XSLOCKS> before including F<XSUB.h> to |
2294 | be able to use these macros: |
2295 | |
2296 | #define NO_XSLOCKS |
2297 | #include "XSUB.h" |
2298 | |
2299 | You can use these macros if you call code that may croak, but you need |
2300 | to do some cleanup before giving control back to Perl. For example: |
0ca3a874 |
2301 | |
2302 | dXCPT; /* set up neccessary variables */ |
2303 | |
2304 | XCPT_TRY_START { |
2305 | code_that_may_croak(); |
2306 | } XCPT_TRY_END |
2307 | |
2308 | XCPT_CATCH |
2309 | { |
2310 | /* do cleanup here */ |
2311 | XCPT_RETHROW; |
2312 | } |
2313 | |
2314 | Note that you always have to rethrow an exception that has been |
2315 | caught. Using these macros, it is not possible to just catch the |
2316 | exception and ignore it. If you have to ignore the exception, you |
2317 | have to use the C<call_*> function. |
2318 | |
2319 | The advantage of using the above macros is that you don't have |
2320 | to setup an extra function for C<call_*>, and that using these |
2321 | macros is faster than using C<call_*>. |
2322 | |
a422fd2d |
2323 | =head2 Source Documentation |
2324 | |
2325 | There's an effort going on to document the internal functions and |
2326 | automatically produce reference manuals from them - L<perlapi> is one |
2327 | such manual which details all the functions which are available to XS |
2328 | writers. L<perlintern> is the autogenerated manual for the functions |
2329 | which are not part of the API and are supposedly for internal use only. |
2330 | |
2331 | Source documentation is created by putting POD comments into the C |
2332 | source, like this: |
2333 | |
2334 | /* |
2335 | =for apidoc sv_setiv |
2336 | |
2337 | Copies an integer into the given SV. Does not handle 'set' magic. See |
2338 | C<sv_setiv_mg>. |
2339 | |
2340 | =cut |
2341 | */ |
2342 | |
2343 | Please try and supply some documentation if you add functions to the |
2344 | Perl core. |
2345 | |
0d098d33 |
2346 | =head2 Backwards compatibility |
2347 | |
2348 | The Perl API changes over time. New functions are added or the interfaces |
2349 | of existing functions are changed. The C<Devel::PPPort> module tries to |
2350 | provide compatibility code for some of these changes, so XS writers don't |
2351 | have to code it themselves when supporting multiple versions of Perl. |
2352 | |
2353 | C<Devel::PPPort> generates a C header file F<ppport.h> that can also |
2354 | be run as a Perl script. To generate F<ppport.h>, run: |
2355 | |
2356 | perl -MDevel::PPPort -eDevel::PPPort::WriteFile |
2357 | |
2358 | Besides checking existing XS code, the script can also be used to retrieve |
2359 | compatibility information for various API calls using the C<--api-info> |
2360 | command line switch. For example: |
2361 | |
2362 | % perl ppport.h --api-info=sv_magicext |
2363 | |
2364 | For details, see C<perldoc ppport.h>. |
2365 | |
a422fd2d |
2366 | =head1 Unicode Support |
2367 | |
2368 | Perl 5.6.0 introduced Unicode support. It's important for porters and XS |
2369 | writers to understand this support and make sure that the code they |
2370 | write does not corrupt Unicode data. |
2371 | |
2372 | =head2 What B<is> Unicode, anyway? |
2373 | |
2374 | In the olden, less enlightened times, we all used to use ASCII. Most of |
2375 | us did, anyway. The big problem with ASCII is that it's American. Well, |
2376 | no, that's not actually the problem; the problem is that it's not |
2377 | particularly useful for people who don't use the Roman alphabet. What |
2378 | used to happen was that particular languages would stick their own |
2379 | alphabet in the upper range of the sequence, between 128 and 255. Of |
2380 | course, we then ended up with plenty of variants that weren't quite |
2381 | ASCII, and the whole point of it being a standard was lost. |
2382 | |
2383 | Worse still, if you've got a language like Chinese or |
2384 | Japanese that has hundreds or thousands of characters, then you really |
2385 | can't fit them into a mere 256, so they had to forget about ASCII |
2386 | altogether, and build their own systems using pairs of numbers to refer |
2387 | to one character. |
2388 | |
2389 | To fix this, some people formed Unicode, Inc. and |
2390 | produced a new character set containing all the characters you can |
2391 | possibly think of and more. There are several ways of representing these |
1e54db1a |
2392 | characters, and the one Perl uses is called UTF-8. UTF-8 uses |
a422fd2d |
2393 | a variable number of bytes to represent a character, instead of just |
b3b6085d |
2394 | one. You can learn more about Unicode at http://www.unicode.org/ |
a422fd2d |
2395 | |
1e54db1a |
2396 | =head2 How can I recognise a UTF-8 string? |
a422fd2d |
2397 | |
1e54db1a |
2398 | You can't. This is because UTF-8 data is stored in bytes just like |
2399 | non-UTF-8 data. The Unicode character 200, (C<0xC8> for you hex types) |
a422fd2d |
2400 | capital E with a grave accent, is represented by the two bytes |
2401 | C<v196.172>. Unfortunately, the non-Unicode string C<chr(196).chr(172)> |
2402 | has that byte sequence as well. So you can't tell just by looking - this |
2403 | is what makes Unicode input an interesting problem. |
2404 | |
2405 | The API function C<is_utf8_string> can help; it'll tell you if a string |
1e54db1a |
2406 | contains only valid UTF-8 characters. However, it can't do the work for |
a422fd2d |
2407 | you. On a character-by-character basis, C<is_utf8_char> will tell you |
1e54db1a |
2408 | whether the current character in a string is valid UTF-8. |
a422fd2d |
2409 | |
1e54db1a |
2410 | =head2 How does UTF-8 represent Unicode characters? |
a422fd2d |
2411 | |
1e54db1a |
2412 | As mentioned above, UTF-8 uses a variable number of bytes to store a |
a422fd2d |
2413 | character. Characters with values 1...128 are stored in one byte, just |
2414 | like good ol' ASCII. Character 129 is stored as C<v194.129>; this |
a31a806a |
2415 | continues up to character 191, which is C<v194.191>. Now we've run out of |
a422fd2d |
2416 | bits (191 is binary C<10111111>) so we move on; 192 is C<v195.128>. And |
2417 | so it goes on, moving to three bytes at character 2048. |
2418 | |
1e54db1a |
2419 | Assuming you know you're dealing with a UTF-8 string, you can find out |
a422fd2d |
2420 | how long the first character in it is with the C<UTF8SKIP> macro: |
2421 | |
2422 | char *utf = "\305\233\340\240\201"; |
2423 | I32 len; |
2424 | |
2425 | len = UTF8SKIP(utf); /* len is 2 here */ |
2426 | utf += len; |
2427 | len = UTF8SKIP(utf); /* len is 3 here */ |
2428 | |
1e54db1a |
2429 | Another way to skip over characters in a UTF-8 string is to use |
a422fd2d |
2430 | C<utf8_hop>, which takes a string and a number of characters to skip |
2431 | over. You're on your own about bounds checking, though, so don't use it |
2432 | lightly. |
2433 | |
1e54db1a |
2434 | All bytes in a multi-byte UTF-8 character will have the high bit set, |
3a2263fe |
2435 | so you can test if you need to do something special with this |
2436 | character like this (the UTF8_IS_INVARIANT() is a macro that tests |
2437 | whether the byte can be encoded as a single byte even in UTF-8): |
a422fd2d |
2438 | |
3a2263fe |
2439 | U8 *utf; |
2440 | UV uv; /* Note: a UV, not a U8, not a char */ |
a422fd2d |
2441 | |
3a2263fe |
2442 | if (!UTF8_IS_INVARIANT(*utf)) |
1e54db1a |
2443 | /* Must treat this as UTF-8 */ |
a422fd2d |
2444 | uv = utf8_to_uv(utf); |
2445 | else |
2446 | /* OK to treat this character as a byte */ |
2447 | uv = *utf; |
2448 | |
2449 | You can also see in that example that we use C<utf8_to_uv> to get the |
2450 | value of the character; the inverse function C<uv_to_utf8> is available |
1e54db1a |
2451 | for putting a UV into UTF-8: |
a422fd2d |
2452 | |
3a2263fe |
2453 | if (!UTF8_IS_INVARIANT(uv)) |
a422fd2d |
2454 | /* Must treat this as UTF8 */ |
2455 | utf8 = uv_to_utf8(utf8, uv); |
2456 | else |
2457 | /* OK to treat this character as a byte */ |
2458 | *utf8++ = uv; |
2459 | |
2460 | You B<must> convert characters to UVs using the above functions if |
1e54db1a |
2461 | you're ever in a situation where you have to match UTF-8 and non-UTF-8 |
2462 | characters. You may not skip over UTF-8 characters in this case. If you |
2463 | do this, you'll lose the ability to match hi-bit non-UTF-8 characters; |
2464 | for instance, if your UTF-8 string contains C<v196.172>, and you skip |
2465 | that character, you can never match a C<chr(200)> in a non-UTF-8 string. |
a422fd2d |
2466 | So don't do that! |
2467 | |
1e54db1a |
2468 | =head2 How does Perl store UTF-8 strings? |
a422fd2d |
2469 | |
2470 | Currently, Perl deals with Unicode strings and non-Unicode strings |
2471 | slightly differently. If a string has been identified as being UTF-8 |
2472 | encoded, Perl will set a flag in the SV, C<SVf_UTF8>. You can check and |
2473 | manipulate this flag with the following macros: |
2474 | |
2475 | SvUTF8(sv) |
2476 | SvUTF8_on(sv) |
2477 | SvUTF8_off(sv) |
2478 | |
2479 | This flag has an important effect on Perl's treatment of the string: if |
2480 | Unicode data is not properly distinguished, regular expressions, |
2481 | C<length>, C<substr> and other string handling operations will have |
2482 | undesirable results. |
2483 | |
2484 | The problem comes when you have, for instance, a string that isn't |
1e54db1a |
2485 | flagged is UTF-8, and contains a byte sequence that could be UTF-8 - |
2486 | especially when combining non-UTF-8 and UTF-8 strings. |
a422fd2d |
2487 | |
2488 | Never forget that the C<SVf_UTF8> flag is separate to the PV value; you |
2489 | need be sure you don't accidentally knock it off while you're |
2490 | manipulating SVs. More specifically, you cannot expect to do this: |
2491 | |
2492 | SV *sv; |
2493 | SV *nsv; |
2494 | STRLEN len; |
2495 | char *p; |
2496 | |
2497 | p = SvPV(sv, len); |
2498 | frobnicate(p); |
2499 | nsv = newSVpvn(p, len); |
2500 | |
2501 | The C<char*> string does not tell you the whole story, and you can't |
2502 | copy or reconstruct an SV just by copying the string value. Check if the |
1e54db1a |
2503 | old SV has the UTF-8 flag set, and act accordingly: |
a422fd2d |
2504 | |
2505 | p = SvPV(sv, len); |
2506 | frobnicate(p); |
2507 | nsv = newSVpvn(p, len); |
2508 | if (SvUTF8(sv)) |
2509 | SvUTF8_on(nsv); |
2510 | |
2511 | In fact, your C<frobnicate> function should be made aware of whether or |
1e54db1a |
2512 | not it's dealing with UTF-8 data, so that it can handle the string |
a422fd2d |
2513 | appropriately. |
2514 | |
3a2263fe |
2515 | Since just passing an SV to an XS function and copying the data of |
1e54db1a |
2516 | the SV is not enough to copy the UTF-8 flags, even less right is just |
3a2263fe |
2517 | passing a C<char *> to an XS function. |
2518 | |
1e54db1a |
2519 | =head2 How do I convert a string to UTF-8? |
a422fd2d |
2520 | |
1e54db1a |
2521 | If you're mixing UTF-8 and non-UTF-8 strings, you might find it necessary |
2522 | to upgrade one of the strings to UTF-8. If you've got an SV, the easiest |
a422fd2d |
2523 | way to do this is: |
2524 | |
2525 | sv_utf8_upgrade(sv); |
2526 | |
2527 | However, you must not do this, for example: |
2528 | |
2529 | if (!SvUTF8(left)) |
2530 | sv_utf8_upgrade(left); |
2531 | |
2532 | If you do this in a binary operator, you will actually change one of the |
b1866b2d |
2533 | strings that came into the operator, and, while it shouldn't be noticeable |
a422fd2d |
2534 | by the end user, it can cause problems. |
2535 | |
1e54db1a |
2536 | Instead, C<bytes_to_utf8> will give you a UTF-8-encoded B<copy> of its |
a422fd2d |
2537 | string argument. This is useful for having the data available for |
b1866b2d |
2538 | comparisons and so on, without harming the original SV. There's also |
a422fd2d |
2539 | C<utf8_to_bytes> to go the other way, but naturally, this will fail if |
2540 | the string contains any characters above 255 that can't be represented |
2541 | in a single byte. |
2542 | |
2543 | =head2 Is there anything else I need to know? |
2544 | |
2545 | Not really. Just remember these things: |
2546 | |
2547 | =over 3 |
2548 | |
2549 | =item * |
2550 | |
1e54db1a |
2551 | There's no way to tell if a string is UTF-8 or not. You can tell if an SV |
2552 | is UTF-8 by looking at is C<SvUTF8> flag. Don't forget to set the flag if |
2553 | something should be UTF-8. Treat the flag as part of the PV, even though |
a422fd2d |
2554 | it's not - if you pass on the PV to somewhere, pass on the flag too. |
2555 | |
2556 | =item * |
2557 | |
1e54db1a |
2558 | If a string is UTF-8, B<always> use C<utf8_to_uv> to get at the value, |
3a2263fe |
2559 | unless C<UTF8_IS_INVARIANT(*s)> in which case you can use C<*s>. |
a422fd2d |
2560 | |
2561 | =item * |
2562 | |
1e54db1a |
2563 | When writing a character C<uv> to a UTF-8 string, B<always> use |
3a2263fe |
2564 | C<uv_to_utf8>, unless C<UTF8_IS_INVARIANT(uv))> in which case |
2565 | you can use C<*s = uv>. |
a422fd2d |
2566 | |
2567 | =item * |
2568 | |
1e54db1a |
2569 | Mixing UTF-8 and non-UTF-8 strings is tricky. Use C<bytes_to_utf8> to get |
2570 | a new string which is UTF-8 encoded. There are tricks you can use to |
2571 | delay deciding whether you need to use a UTF-8 string until you get to a |
a422fd2d |
2572 | high character - C<HALF_UPGRADE> is one of those. |
2573 | |
2574 | =back |
2575 | |
53e06cf0 |
2576 | =head1 Custom Operators |
2577 | |
9a68f1db |
2578 | Custom operator support is a new experimental feature that allows you to |
53e06cf0 |
2579 | define your own ops. This is primarily to allow the building of |
2580 | interpreters for other languages in the Perl core, but it also allows |
2581 | optimizations through the creation of "macro-ops" (ops which perform the |
2582 | functions of multiple ops which are usually executed together, such as |
b7cb320d |
2583 | C<gvsv, gvsv, add>.) |
53e06cf0 |
2584 | |
b455bf3f |
2585 | This feature is implemented as a new op type, C<OP_CUSTOM>. The Perl |
53e06cf0 |
2586 | core does not "know" anything special about this op type, and so it will |
2587 | not be involved in any optimizations. This also means that you can |
2588 | define your custom ops to be any op structure - unary, binary, list and |
2589 | so on - you like. |
2590 | |
2591 | It's important to know what custom operators won't do for you. They |
2592 | won't let you add new syntax to Perl, directly. They won't even let you |
2593 | add new keywords, directly. In fact, they won't change the way Perl |
2594 | compiles a program at all. You have to do those changes yourself, after |
2595 | Perl has compiled the program. You do this either by manipulating the op |
2596 | tree using a C<CHECK> block and the C<B::Generate> module, or by adding |
2597 | a custom peephole optimizer with the C<optimize> module. |
2598 | |
2599 | When you do this, you replace ordinary Perl ops with custom ops by |
2600 | creating ops with the type C<OP_CUSTOM> and the C<pp_addr> of your own |
2601 | PP function. This should be defined in XS code, and should look like |
2602 | the PP ops in C<pp_*.c>. You are responsible for ensuring that your op |
2603 | takes the appropriate number of values from the stack, and you are |
2604 | responsible for adding stack marks if necessary. |
2605 | |
2606 | You should also "register" your op with the Perl interpreter so that it |
2607 | can produce sensible error and warning messages. Since it is possible to |
2608 | have multiple custom ops within the one "logical" op type C<OP_CUSTOM>, |
2609 | Perl uses the value of C<< o->op_ppaddr >> as a key into the |
2610 | C<PL_custom_op_descs> and C<PL_custom_op_names> hashes. This means you |
2611 | need to enter a name and description for your op at the appropriate |
2612 | place in the C<PL_custom_op_names> and C<PL_custom_op_descs> hashes. |
2613 | |
2614 | Forthcoming versions of C<B::Generate> (version 1.0 and above) should |
2615 | directly support the creation of custom ops by name; C<Opcodes::Custom> |
2616 | will provide functions which make it trivial to "register" custom ops to |
2617 | the Perl interpreter. |
2618 | |
954c1994 |
2619 | =head1 AUTHORS |
e89caa19 |
2620 | |
954c1994 |
2621 | Until May 1997, this document was maintained by Jeff Okamoto |
9b5bb84f |
2622 | E<lt>okamoto@corp.hp.comE<gt>. It is now maintained as part of Perl |
2623 | itself by the Perl 5 Porters E<lt>perl5-porters@perl.orgE<gt>. |
cb1a09d0 |
2624 | |
954c1994 |
2625 | With lots of help and suggestions from Dean Roehrich, Malcolm Beattie, |
2626 | Andreas Koenig, Paul Hudson, Ilya Zakharevich, Paul Marquess, Neil |
2627 | Bowers, Matthew Green, Tim Bunce, Spider Boardman, Ulrich Pfeifer, |
2628 | Stephen McCamant, and Gurusamy Sarathy. |
cb1a09d0 |
2629 | |
954c1994 |
2630 | =head1 SEE ALSO |
cb1a09d0 |
2631 | |
954c1994 |
2632 | perlapi(1), perlintern(1), perlxs(1), perlembed(1) |