3 perlxs - XS language reference manual
9 XS is a language used to create an extension interface
10 between Perl and some C library which one wishes to use with
11 Perl. The XS interface is combined with the library to
12 create a new library which can be linked to Perl. An B<XSUB>
13 is a function in the XS language and is the core component
14 of the Perl application interface.
16 The XS compiler is called B<xsubpp>. This compiler will embed
17 the constructs necessary to let an XSUB, which is really a C
18 function in disguise, manipulate Perl values and creates the
19 glue necessary to let Perl access the XSUB. The compiler
20 uses B<typemaps> to determine how to map C function parameters
21 and variables to Perl values. The default typemap handles
22 many common C types. A supplement typemap must be created
23 to handle special structures and types for the library being
26 See L<perlxstut> for a tutorial on the whole extension creation process.
30 Many of the examples which follow will concentrate on creating an
31 interface between Perl and the ONC+ RPC bind library functions.
32 Specifically, the rpcb_gettime() function will be used to demonstrate many
33 features of the XS language. This function has two parameters; the first
34 is an input parameter and the second is an output parameter. The function
35 also returns a status value.
37 bool_t rpcb_gettime(const char *host, time_t *timep);
39 From C this function will be called with the following
45 status = rpcb_gettime( "localhost", &timep );
47 If an XSUB is created to offer a direct translation between this function
48 and Perl, then this XSUB will be used from Perl with the following code.
49 The $status and $timep variables will contain the output of the function.
52 $status = rpcb_gettime( "localhost", $timep );
54 The following XS file shows an XS subroutine, or XSUB, which
55 demonstrates one possible interface to the rpcb_gettime()
56 function. This XSUB represents a direct translation between
57 C and Perl and so preserves the interface even from Perl.
58 This XSUB will be invoked from Perl with the usage shown
59 above. Note that the first three #include statements, for
60 C<EXTERN.h>, C<perl.h>, and C<XSUB.h>, will always be present at the
61 beginning of an XS file. This approach and others will be
62 expanded later in this document.
69 MODULE = RPC PACKAGE = RPC
72 rpcb_gettime(host,timep)
78 Any extension to Perl, including those containing XSUBs,
79 should have a Perl module to serve as the bootstrap which
80 pulls the extension into Perl. This module will export the
81 extension's functions and variables to the Perl program and
82 will cause the extension's XSUBs to be linked into Perl.
83 The following module will be used for most of the examples
84 in this document and should be used from Perl with the C<use>
85 command as shown earlier. Perl modules are explained in
86 more detail later in this document.
92 @ISA = qw(Exporter DynaLoader);
93 @EXPORT = qw( rpcb_gettime );
98 Throughout this document a variety of interfaces to the rpcb_gettime()
99 XSUB will be explored. The XSUBs will take their parameters in different
100 orders or will take different numbers of parameters. In each case the
101 XSUB is an abstraction between Perl and the real C rpcb_gettime()
102 function, and the XSUB must always ensure that the real rpcb_gettime()
103 function is called with the correct parameters. This abstraction will
104 allow the programmer to create a more Perl-like interface to the C
107 =head2 The Anatomy of an XSUB
109 The following XSUB allows a Perl program to access a C library function
110 called sin(). The XSUB will imitate the C function which takes a single
111 argument and returns a single value.
117 When using C pointers the indirection operator C<*> should be considered
118 part of the type and the address operator C<&> should be considered part of
119 the variable, as is demonstrated in the rpcb_gettime() function above. See
120 the section on typemaps for more about handling qualifiers and unary
121 operators in C types.
123 The function name and the return type must be placed on
132 =head2 The Argument Stack
134 The argument stack is used to store the values which are
135 sent as parameters to the XSUB and to store the XSUB's
136 return value. In reality all Perl functions keep their
137 values on this stack at the same time, each limited to its
138 own range of positions on the stack. In this document the
139 first position on that stack which belongs to the active
140 function will be referred to as position 0 for that function.
142 XSUBs refer to their stack arguments with the macro B<ST(x)>, where I<x>
143 refers to a position in this XSUB's part of the stack. Position 0 for that
144 function would be known to the XSUB as ST(0). The XSUB's incoming
145 parameters and outgoing return values always begin at ST(0). For many
146 simple cases the B<xsubpp> compiler will generate the code necessary to
147 handle the argument stack by embedding code fragments found in the
148 typemaps. In more complex cases the programmer must supply the code.
150 =head2 The RETVAL Variable
152 The RETVAL variable is a magic variable which always matches
153 the return type of the C library function. The B<xsubpp> compiler will
154 supply this variable in each XSUB and by default will use it to hold the
155 return value of the C library function being called. In simple cases the
156 value of RETVAL will be placed in ST(0) of the argument stack where it can
157 be received by Perl as the return value of the XSUB.
159 If the XSUB has a return type of C<void> then the compiler will
160 not supply a RETVAL variable for that function. When using
161 the PPCODE: directive the RETVAL variable may not be needed.
163 =head2 The MODULE Keyword
165 The MODULE keyword is used to start the XS code and to
166 specify the package of the functions which are being
167 defined. All text preceding the first MODULE keyword is
168 considered C code and is passed through to the output
169 untouched. Every XS module will have a bootstrap function
170 which is used to hook the XSUBs into Perl. The package name
171 of this bootstrap function will match the value of the last
172 MODULE statement in the XS source files. The value of
173 MODULE should always remain constant within the same XS
174 file, though this is not required.
176 The following example will start the XS code and will place
177 all functions in a package named RPC.
181 =head2 The PACKAGE Keyword
183 When functions within an XS source file must be separated into packages
184 the PACKAGE keyword should be used. This keyword is used with the MODULE
185 keyword and must follow immediately after it when used.
187 MODULE = RPC PACKAGE = RPC
189 [ XS code in package RPC ]
191 MODULE = RPC PACKAGE = RPCB
193 [ XS code in package RPCB ]
195 MODULE = RPC PACKAGE = RPC
197 [ XS code in package RPC ]
199 Although this keyword is optional and in some cases provides redundant
200 information it should always be used. This keyword will ensure that the
201 XSUBs appear in the desired package.
203 =head2 The PREFIX Keyword
205 The PREFIX keyword designates prefixes which should be
206 removed from the Perl function names. If the C function is
207 C<rpcb_gettime()> and the PREFIX value is C<rpcb_> then Perl will
208 see this function as C<gettime()>.
210 This keyword should follow the PACKAGE keyword when used.
211 If PACKAGE is not used then PREFIX should follow the MODULE
214 MODULE = RPC PREFIX = rpc_
216 MODULE = RPC PACKAGE = RPCB PREFIX = rpcb_
218 =head2 The OUTPUT: Keyword
220 The OUTPUT: keyword indicates that certain function parameters should be
221 updated (new values made visible to Perl) when the XSUB terminates or that
222 certain values should be returned to the calling Perl function. For
223 simple functions, such as the sin() function above, the RETVAL variable is
224 automatically designated as an output value. In more complex functions
225 the B<xsubpp> compiler will need help to determine which variables are output
228 This keyword will normally be used to complement the CODE: keyword.
229 The RETVAL variable is not recognized as an output variable when the
230 CODE: keyword is present. The OUTPUT: keyword is used in this
231 situation to tell the compiler that RETVAL really is an output
234 The OUTPUT: keyword can also be used to indicate that function parameters
235 are output variables. This may be necessary when a parameter has been
236 modified within the function and the programmer would like the update to
240 rpcb_gettime(host,timep)
246 The OUTPUT: keyword will also allow an output parameter to
247 be mapped to a matching piece of code rather than to a
251 rpcb_gettime(host,timep)
255 timep sv_setnv(ST(1), (double)timep);
257 =head2 The CODE: Keyword
259 This keyword is used in more complicated XSUBs which require
260 special handling for the C function. The RETVAL variable is
261 available but will not be returned unless it is specified
262 under the OUTPUT: keyword.
264 The following XSUB is for a C function which requires special handling of
265 its parameters. The Perl usage is given first.
267 $status = rpcb_gettime( "localhost", $timep );
272 rpcb_gettime(host,timep)
276 RETVAL = rpcb_gettime( host, &timep );
281 In many of the examples shown here the CODE: block (and
282 other blocks) will often be contained within braces ( C<{> and
283 C<}> ). This protects the CODE: block from complex INPUT
284 typemaps and ensures the resulting C code is legal.
286 =head2 The NO_INIT Keyword
288 The NO_INIT keyword is used to indicate that a function
289 parameter is being used as only an output value. The B<xsubpp>
290 compiler will normally generate code to read the values of
291 all function parameters from the argument stack and assign
292 them to C variables upon entry to the function. NO_INIT
293 will tell the compiler that some parameters will be used for
294 output rather than for input and that they will be handled
295 before the function terminates.
297 The following example shows a variation of the rpcb_gettime() function.
298 This function uses the timep variable as only an output variable and does
299 not care about its initial contents.
302 rpcb_gettime(host,timep)
304 time_t &timep = NO_INIT
308 =head2 Initializing Function Parameters
310 Function parameters are normally initialized with their
311 values from the argument stack. The typemaps contain the
312 code segments which are used to transfer the Perl values to
313 the C parameters. The programmer, however, is allowed to
314 override the typemaps and supply alternate initialization
317 The following code demonstrates how to supply initialization code for
318 function parameters. The initialization code is eval'd by the compiler
319 before it is added to the output so anything which should be interpreted
320 literally, such as double quotes, must be protected with backslashes.
323 rpcb_gettime(host,timep)
324 char *host = (char *)SvPV(ST(0),na);
329 This should not be used to supply default values for parameters. One
330 would normally use this when a function parameter must be processed by
331 another library function before it can be used. Default parameters are
332 covered in the next section.
334 =head2 Default Parameter Values
336 Default values can be specified for function parameters by
337 placing an assignment statement in the parameter list. The
338 default value may be a number or a string. Defaults should
339 always be used on the right-most parameters only.
341 To allow the XSUB for rpcb_gettime() to have a default host
342 value the parameters to the XSUB could be rearranged. The
343 XSUB will then call the real rpcb_gettime() function with
344 the parameters in the correct order. Perl will call this
345 XSUB with either of the following statements.
347 $status = rpcb_gettime( $timep, $host );
349 $status = rpcb_gettime( $timep );
351 The XSUB will look like the code which follows. A CODE:
352 block is used to call the real rpcb_gettime() function with
353 the parameters in the correct order for that function.
356 rpcb_gettime(timep,host="localhost")
358 time_t timep = NO_INIT
360 RETVAL = rpcb_gettime( host, &timep );
365 =head2 Variable-length Parameter Lists
367 XSUBs can have variable-length parameter lists by specifying an ellipsis
368 C<(...)> in the parameter list. This use of the ellipsis is similar to that
369 found in ANSI C. The programmer is able to determine the number of
370 arguments passed to the XSUB by examining the C<items> variable which the
371 B<xsubpp> compiler supplies for all XSUBs. By using this mechanism one can
372 create an XSUB which accepts a list of parameters of unknown length.
374 The I<host> parameter for the rpcb_gettime() XSUB can be
375 optional so the ellipsis can be used to indicate that the
376 XSUB will take a variable number of parameters. Perl should
377 be able to call this XSUB with either of the following statements.
379 $status = rpcb_gettime( $timep, $host );
381 $status = rpcb_gettime( $timep );
383 The XS code, with ellipsis, follows.
386 rpcb_gettime(timep, ...)
387 time_t timep = NO_INIT
390 char *host = "localhost";
393 host = (char *)SvPV(ST(1), na);
394 RETVAL = rpcb_gettime( host, &timep );
400 =head2 The PPCODE: Keyword
402 The PPCODE: keyword is an alternate form of the CODE: keyword and is used
403 to tell the B<xsubpp> compiler that the programmer is supplying the code to
404 control the argument stack for the XSUBs return values. Occasionally one
405 will want an XSUB to return a list of values rather than a single value.
406 In these cases one must use PPCODE: and then explicitly push the list of
407 values on the stack. The PPCODE: and CODE: keywords are not used
408 together within the same XSUB.
410 The following XSUB will call the C rpcb_gettime() function
411 and will return its two output values, timep and status, to
412 Perl as a single list.
421 status = rpcb_gettime( host, &timep );
423 PUSHs(sv_2mortal(newSViv(status)));
424 PUSHs(sv_2mortal(newSViv(timep)));
427 Notice that the programmer must supply the C code necessary
428 to have the real rpcb_gettime() function called and to have
429 the return values properly placed on the argument stack.
431 The C<void> return type for this function tells the B<xsubpp> compiler that
432 the RETVAL variable is not needed or used and that it should not be created.
433 In most scenarios the void return type should be used with the PPCODE:
436 The EXTEND() macro is used to make room on the argument
437 stack for 2 return values. The PPCODE: directive causes the
438 B<xsubpp> compiler to create a stack pointer called C<sp>, and it
439 is this pointer which is being used in the EXTEND() macro.
440 The values are then pushed onto the stack with the PUSHs()
443 Now the rpcb_gettime() function can be used from Perl with
444 the following statement.
446 ($status, $timep) = rpcb_gettime("localhost");
448 =head2 Returning Undef And Empty Lists
450 Occasionally the programmer will want to simply return
451 C<undef> or an empty list if a function fails rather than a
452 separate status value. The rpcb_gettime() function offers
453 just this situation. If the function succeeds we would like
454 to have it return the time and if it fails we would like to
455 have undef returned. In the following Perl code the value
456 of $timep will either be undef or it will be a valid time.
458 $timep = rpcb_gettime( "localhost" );
460 The following XSUB uses the C<void> return type to disable the generation of
461 the RETVAL variable and uses a CODE: block to indicate to the compiler
462 that the programmer has supplied all the necessary code. The
463 sv_newmortal() call will initialize the return value to undef, making that
464 the default return value.
473 ST(0) = sv_newmortal();
474 if( rpcb_gettime( host, &timep ) )
475 sv_setnv( ST(0), (double)timep);
478 The next example demonstrates how one would place an explicit undef in the
479 return value, should the need arise.
488 ST(0) = sv_newmortal();
489 if( rpcb_gettime( host, &timep ) ){
490 sv_setnv( ST(0), (double)timep);
497 To return an empty list one must use a PPCODE: block and
498 then not push return values on the stack.
506 if( rpcb_gettime( host, &timep ) )
507 PUSHs(sv_2mortal(newSViv(timep)));
509 /* Nothing pushed on stack, so an empty */
510 /* list is implicitly returned. */
514 =head2 The REQUIRE: Keyword
516 The REQUIRE: keyword is used to indicate the minimum version of the
517 B<xsubpp> compiler needed to compile the XS module. An XS module which
518 contains the following statement will only compile with B<xsubpp> version
523 =head2 The CLEANUP: Keyword
525 This keyword can be used when an XSUB requires special cleanup procedures
526 before it terminates. When the CLEANUP: keyword is used it must follow
527 any CODE:, PPCODE:, or OUTPUT: blocks which are present in the XSUB. The
528 code specified for the cleanup block will be added as the last statements
531 =head2 The BOOT: Keyword
533 The BOOT: keyword is used to add code to the extension's bootstrap
534 function. The bootstrap function is generated by the B<xsubpp> compiler and
535 normally holds the statements necessary to register any XSUBs with Perl.
536 With the BOOT: keyword the programmer can tell the compiler to add extra
537 statements to the bootstrap function.
539 This keyword may be used any time after the first MODULE keyword and should
540 appear on a line by itself. The first blank line after the keyword will
541 terminate the code block.
544 # The following message will be printed when the
545 # bootstrap function executes.
546 printf("Hello from the bootstrap!\n");
548 =head2 Inserting Comments and C Preprocessor Directives
550 Comments and C preprocessor directives are allowed within
551 CODE:, PPCODE:, BOOT:, and CLEANUP: blocks. The compiler
552 will pass the preprocessor directives through untouched and
553 will remove the commented lines. Comments can be added to
554 XSUBs by placing a C<#> at the beginning of the line. Care
555 should be taken to avoid making the comment look like a C
556 preprocessor directive, lest it be interpreted as such.
558 =head2 Using XS With C++
560 If a function is defined as a C++ method then it will assume
561 its first argument is an object pointer. The object pointer
562 will be stored in a variable called THIS. The object should
563 have been created by C++ with the new() function and should
564 be blessed by Perl with the sv_setref_pv() macro. The
565 blessing of the object by Perl can be handled by a typemap. An example
566 typemap is shown at the end of this section.
568 If the method is defined as static it will call the C++
569 function using the class::method() syntax. If the method is not static
570 the function will be called using the THIS->method() syntax.
572 The next examples will use the following C++ class.
579 void set_blue( int );
585 The XSUBs for the blue() and set_blue() methods are defined with the class
586 name but the parameter for the object (THIS, or "self") is implicit and is
593 color::set_blue( val )
596 Both functions will expect an object as the first parameter. The xsubpp
597 compiler will call that object C<THIS> and will use it to call the specified
598 method. So in the C++ code the blue() and set_blue() methods will be called
599 in the following manner.
601 RETVAL = THIS->blue();
603 THIS->set_blue( val );
605 If the function's name is B<DESTROY> then the C++ C<delete> function will be
606 called and C<THIS> will be given as its parameter.
611 The C++ code will call C<delete>.
615 If the function's name is B<new> then the C++ C<new> function will be called
616 to create a dynamic C++ object. The XSUB will expect the class name, which
617 will be kept in a variable called C<CLASS>, to be given as the first
623 The C++ code will call C<new>.
625 RETVAL = new color();
627 The following is an example of a typemap that could be used for this C++
634 # The Perl object is blessed into 'CLASS', which should be a
635 # char* having the name of the package for the blessing.
637 sv_setref_pv( $arg, CLASS, (void*)$var );
641 if( sv_isobject($arg) && (SvTYPE(SvRV($arg)) == SVt_PVMG) )
642 $var = ($type)SvIV((SV*)SvRV( $arg ));
644 warn( \"${Package}::$func_name() -- $var is not a blessed SV reference\" );
648 =head2 Interface Strategy
650 When designing an interface between Perl and a C library a straight
651 translation from C to XS is often sufficient. The interface will often be
652 very C-like and occasionally nonintuitive, especially when the C function
653 modifies one of its parameters. In cases where the programmer wishes to
654 create a more Perl-like interface the following strategy may help to
655 identify the more critical parts of the interface.
657 Identify the C functions which modify their parameters. The XSUBs for
658 these functions may be able to return lists to Perl, or may be
659 candidates to return undef or an empty list in case of failure.
661 Identify which values are used by only the C and XSUB functions
662 themselves. If Perl does not need to access the contents of the value
663 then it may not be necessary to provide a translation for that value
666 Identify the pointers in the C function parameter lists and return
667 values. Some pointers can be handled in XS with the & unary operator on
668 the variable name while others will require the use of the * operator on
669 the type name. In general it is easier to work with the & operator.
671 Identify the structures used by the C functions. In many
672 cases it may be helpful to use the T_PTROBJ typemap for
673 these structures so they can be manipulated by Perl as
676 =head2 Perl Objects And C Structures
678 When dealing with C structures one should select either
679 B<T_PTROBJ> or B<T_PTRREF> for the XS type. Both types are
680 designed to handle pointers to complex objects. The
681 T_PTRREF type will allow the Perl object to be unblessed
682 while the T_PTROBJ type requires that the object be blessed.
683 By using T_PTROBJ one can achieve a form of type-checking
684 because the XSUB will attempt to verify that the Perl object
685 is of the expected type.
687 The following XS code shows the getnetconfigent() function which is used
688 with ONC+ TIRPC. The getnetconfigent() function will return a pointer to a
689 C structure and has the C prototype shown below. The example will
690 demonstrate how the C pointer will become a Perl reference. Perl will
691 consider this reference to be a pointer to a blessed object and will
692 attempt to call a destructor for the object. A destructor will be
693 provided in the XS source to free the memory used by getnetconfigent().
694 Destructors in XS can be created by specifying an XSUB function whose name
695 ends with the word B<DESTROY>. XS destructors can be used to free memory
696 which may have been malloc'd by another XSUB.
698 struct netconfig *getnetconfigent(const char *netid);
700 A C<typedef> will be created for C<struct netconfig>. The Perl
701 object will be blessed in a class matching the name of the C
702 type, with the tag C<Ptr> appended, and the name should not
703 have embedded spaces if it will be a Perl package name. The
704 destructor will be placed in a class corresponding to the
705 class of the object and the PREFIX keyword will be used to
706 trim the name to the word DESTROY as Perl will expect.
708 typedef struct netconfig Netconfig;
710 MODULE = RPC PACKAGE = RPC
713 getnetconfigent(netid)
716 MODULE = RPC PACKAGE = NetconfigPtr PREFIX = rpcb_
719 rpcb_DESTROY(netconf)
722 printf("Now in NetconfigPtr::DESTROY\n");
725 This example requires the following typemap entry. Consult the typemap
726 section for more information about adding new typemaps for an extension.
731 This example will be used with the following Perl statements.
734 $netconf = getnetconfigent("udp");
736 When Perl destroys the object referenced by $netconf it will send the
737 object to the supplied XSUB DESTROY function. Perl cannot determine, and
738 does not care, that this object is a C struct and not a Perl object. In
739 this sense, there is no difference between the object created by the
740 getnetconfigent() XSUB and an object created by a normal Perl subroutine.
744 The typemap is a collection of code fragments which are used by the B<xsubpp>
745 compiler to map C function parameters and values to Perl values. The
746 typemap file may consist of three sections labeled C<TYPEMAP>, C<INPUT>, and
747 C<OUTPUT>. The INPUT section tells the compiler how to translate Perl values
748 into variables of certain C types. The OUTPUT section tells the compiler
749 how to translate the values from certain C types into values Perl can
750 understand. The TYPEMAP section tells the compiler which of the INPUT and
751 OUTPUT code fragments should be used to map a given C type to a Perl value.
752 Each of the sections of the typemap must be preceded by one of the TYPEMAP,
753 INPUT, or OUTPUT keywords.
755 The default typemap in the C<ext> directory of the Perl source contains many
756 useful types which can be used by Perl extensions. Some extensions define
757 additional typemaps which they keep in their own directory. These
758 additional typemaps may reference INPUT and OUTPUT maps in the main
759 typemap. The B<xsubpp> compiler will allow the extension's own typemap to
760 override any mappings which are in the default typemap.
762 Most extensions which require a custom typemap will need only the TYPEMAP
763 section of the typemap file. The custom typemap used in the
764 getnetconfigent() example shown earlier demonstrates what may be the typical
765 use of extension typemaps. That typemap is used to equate a C structure
766 with the T_PTROBJ typemap. The typemap used by getnetconfigent() is shown
767 here. Note that the C type is separated from the XS type with a tab and
768 that the C unary operator C<*> is considered to be a part of the C type name.
771 Netconfig *<tab>T_PTROBJ
775 File C<RPC.xs>: Interface to some ONC+ RPC bind library functions.
783 typedef struct netconfig Netconfig;
785 MODULE = RPC PACKAGE = RPC
788 rpcb_gettime(host="localhost")
793 ST(0) = sv_newmortal();
794 if( rpcb_gettime( host, &timep ) )
795 sv_setnv( ST(0), (double)timep );
799 getnetconfigent(netid="udp")
802 MODULE = RPC PACKAGE = NetconfigPtr PREFIX = rpcb_
805 rpcb_DESTROY(netconf)
808 printf("NetconfigPtr::DESTROY\n");
811 File C<typemap>: Custom typemap for RPC.xs.
816 File C<RPC.pm>: Perl module for the RPC extension.
822 @ISA = qw(Exporter DynaLoader);
823 @EXPORT = qw(rpcb_gettime getnetconfigent);
828 File C<rpctest.pl>: Perl test program for the RPC extension.
832 $netconf = getnetconfigent();
835 print "netconf = $netconf\n";
837 $netconf = getnetconfigent("tcp");
838 $a = rpcb_gettime("poplar");
840 print "netconf = $netconf\n";
845 Dean Roehrich F<E<lt>roehrich@cray.comE<gt>>