3 Pumpkin - Notes on handling the Perl Patch Pumpkin
7 There is no simple synopsis, yet.
11 This document attempts to begin to describe some of the considerations
12 involved in patching, porting, and maintaining perl.
14 This document is still under construction, and still subject to
15 significant changes. Still, I hope parts of it will be useful,
16 so I'm releasing it even though it's not done.
18 For the most part, it's a collection of anecdotal information that
19 already assumes some familiarity with the Perl sources. I really need
20 an introductory section that describes the organization of the sources
21 and all the various auxiliary files that are part of the distribution.
23 =head1 Where Do I Get Perl Sources and Related Material?
25 The Comprehensive Perl Archive Network (or CPAN) is the place to go.
26 There are many mirrors, but the easiest thing to use is probably
27 http://www.perl.com/CPAN/README.html , which automatically points you to a
28 mirror site "close" to you.
30 =head2 Perl5-porters mailing list
32 The mailing list perl5-porters@perl.org
33 is the main group working with the development of perl. If you're
34 interested in all the latest developments, you should definitely
35 subscribe. The list is high volume, but generally has a
36 fairly low noise level.
38 Subscribe by sending the message (in the body of your letter)
40 subscribe perl5-porters
42 to perl5-porters-request@perl.org .
44 Archives of the list are held at:
46 http://www.rosat.mpe-garching.mpg.de/mailing-lists/perl-porters/
48 =head1 How are Perl Releases Numbered?
50 Perl version numbers are floating point numbers, such as 5.004.
51 (Observations about the imprecision of floating point numbers for
52 representing reality probably have more relevance than you might
53 imagine :-) The major version number is 5 and the '004' is the
54 patchlevel. (Questions such as whether or not '004' is really a minor
55 version number can safely be ignored.:)
57 The version number is available as the magic variable $],
58 and can be used in comparisons, e.g.
60 print "You've got an old perl\n" if $] < 5.002;
62 You can also require particular version (or later) with
66 At some point in the future, we may need to decide what to call the
67 next big revision. In the .package file used by metaconfig to
68 generate Configure, there are two variables that might be relevant:
69 $baserev=5.0 and $package=perl5. At various times, I have suggested
70 we might change them to $baserev=5.1 and $package=perl5.1 if want
71 to signify a fairly major update. Or, we might want to jump to perl6.
72 Let's worry about that problem when we get there.
76 In addition, there may be "developer" sub-versions available. These
77 are not official releases. They may contain unstable experimental
78 features, and are subject to rapid change. Such developer
79 sub-versions are numbered with sub-version numbers. For example,
80 version 5.003_04 is the 4'th developer version built on top of
81 5.003. It might include the _01, _02, and _03 changes, but it
82 also might not. Sub-versions are allowed to be subversive. (But see
83 the next section for recent changes.)
85 These sub-versions can also be used as floating point numbers, so
86 you can do things such as
88 print "You've got an unstable perl\n" if $] == 5.00303;
90 You can also require particular version (or later) with
92 use 5.003_03; # the "_" is optional
94 Sub-versions produced by the members of perl5-porters are usually
95 available on CPAN in the F<src/5.0/unsupported> directory.
97 =head2 Maintenance and Development Subversions
99 As an experiment, starting with version 5.004, subversions _01 through
100 _49 will be reserved for bug-fix maintenance releases, and subversions
101 _50 through _99 will be available for unstable development versions.
103 The separate bug-fix track is being established to allow us an easy
104 way to distribute important bug fixes without waiting for the
105 developers to untangle all the other problems in the current
108 Trial releases of bug-fix maintenance releases are announced on
109 perl5-porters. Trial releases use the new subversion number (to avoid
110 testers installing it over the previous release) and include a 'local
111 patch' entry in patchlevel.h.
113 Watch for announcements of maintenance subversions in
114 comp.lang.perl.announce.
116 The first rule of maintenance work is "First, do no harm."
118 =head2 Why such a complicated scheme?
120 Two reasons, really. At least.
122 First, we need some way to identify and release collections of patches
123 that are known to have new features that need testing and exploration. The
124 subversion scheme does that nicely while fitting into the
127 Second, since most of the folks who help maintain perl do so on a
128 free-time voluntary basis, perl development does not proceed at a
129 precise pace, though it always seems to be moving ahead quickly.
130 We needed some way to pass around the "patch pumpkin" to allow
131 different people chances to work on different aspects of the
132 distribution without getting in each other's way. It wouldn't be
133 constructive to have multiple people working on incompatible
134 implementations of the same idea. Instead what was needed was
135 some kind of "baton" or "token" to pass around so everyone knew
138 =head2 Why is it called the patch pumpkin?
140 Chip Salzenberg gets credit for that, with a nod to his cow orker,
141 David Croy. We had passed around various names (baton, token, hot
142 potato) but none caught on. Then, Chip asked:
146 Who has the patch pumpkin?
148 To explain: David Croy once told me once that at a previous job,
149 there was one tape drive and multiple systems that used it for backups.
150 But instead of some high-tech exclusion software, they used a low-tech
151 method to prevent multiple simultaneous backups: a stuffed pumpkin.
152 No one was allowed to make backups unless they had the "backup pumpkin".
158 =head1 Philosophical Issues in Patching and Porting Perl
160 There are no absolute rules, but there are some general guidelines I
161 have tried to follow as I apply patches to the perl sources.
162 (This section is still under construction.)
164 =head2 Solve problems as generally as possible
166 Never implement a specific restricted solution to a problem when you
167 can solve the same problem in a more general, flexible way.
169 For example, for dynamic loading to work on some SVR4 systems, we had
170 to build a shared libperl.so library. In order to build "FAT" binaries
171 on NeXT 4.0 systems, we had to build a special libperl library. Rather
172 than continuing to build a contorted nest of special cases, I
173 generalized the process of building libperl so that NeXT and SVR4 users
174 could still get their work done, but others could build a shared
175 libperl if they wanted to as well.
177 Contain your changes carefully. Assume nothing about other operating
178 systems, not even closely related ones. Your changes must not affect
181 Spy shamelessly on how similar patching or porting issues have been
184 If feasible, try to keep filenames 8.3-compliant to humor those poor
185 souls that get joy from running Perl under such dire limitations.
187 =head2 Seek consensus on major changes
189 If you are making big changes, don't do it in secret. Discuss the
190 ideas in advance on perl5-porters.
192 =head2 Keep the documentation up-to-date
194 If your changes may affect how users use perl, then check to be sure
195 that the documentation is in sync with your changes. Be sure to
196 check all the files F<pod/*.pod> and also the F<INSTALL> document.
198 Consider writing the appropriate documentation first and then
199 implementing your change to correspond to the documentation.
201 =head2 Avoid machine-specific #ifdef's
203 To the extent reasonable, try to avoid machine-specific #ifdef's in
204 the sources. Instead, use feature-specific #ifdef's. The reason is
205 that the machine-specific #ifdef's may not be valid across major
206 releases of the operating system. Further, the feature-specific tests
207 may help out folks on another platform who have the same problem.
209 =head2 Machine-specific files
215 If you have many machine-specific #defines or #includes, consider
216 creating an "osish.h" (os2ish.h, vmsish.h, and so on) and including
217 that in perl.h. If you have several machine-specific files (function
218 emulations, function stubs, build utility wrappers) you may create a
219 separate subdirectory (djgpp, win32) and put the files in there.
220 Remember to update C<MANIFEST> when you add files.
222 If your system support dynamic loading but none of the existing
223 methods at F<ext/DynaLoader/dl_*.xs> work for you, you must write
224 a new one. Study the existing ones to see what kind of interface
229 There are two kinds of hints: hints for building Perl and hints for
230 extensions. The former live in the C<hints> subdirectory, the latter
231 in C<ext/*/hints> subdirectories.
233 The top level hints are Bourne-shell scripts that set, modify and
234 unset appropriate Configure variables, based on the Configure command
235 line options and possibly existing config.sh and Policy.sh files from
236 previous Configure runs.
238 The extension hints are written Perl (by the time they are used
239 miniperl has been built) and control the building of their respective
240 extensions. They can be used to for example manipulate compilation
243 =item build and installation Makefiles, scripts, and so forth
245 Sometimes you will also need to tweak the Perl build and installation
246 procedure itself, like for example F<Makefile.SH> and F<installperl>.
247 Tread very carefully, even more than usual. Contain your changes
252 Many of the tests in C<t> subdirectory assume machine-specific things
253 like existence of certain functions, something about filesystem
254 semantics, certain external utilities and their error messages. Use
255 the C<$^O> and the C<Config> module (which contains the results of the
256 Configure run, in effect the C<config.sh> converted to Perl) to either
257 skip (preferably not) or customize (preferable) the tests for your
262 Certain standard modules may need updating if your operating system
263 sports for example a native filesystem naming. You may want to update
264 some or all of the modules File::Basename, File::Spec, File::Path, and
265 File::Copy to become aware of your native filesystem syntax and
270 If your operating system comes from outside UNIX you almost certainly
271 will have differences in the available operating system functionality
272 (missing system calls, different semantics, whatever). Please
273 document these at F<pod/perlport.pod>. If your operating system is
274 the first B<not> to have a system call also update the list of
275 "portability-bewares" at the beginning of F<pod/perlfunc.pod>.
277 A file called F<README.youros> at the top level that explains things
278 like how to install perl at this platform, where to get any possibly
279 required additional software, and for example what test suite errors
280 to expect, is nice too.
282 You may also want to write a separate F<.pod> file for your operating
283 system to tell about existing mailing lists, os-specific modules,
284 documentation, whatever. Please name these along the lines of
285 F<perl>I<youros>.pod. [unfinished: where to put this file (the pod/
286 subdirectory, of course: but more importantly, which/what index files
291 =head2 Allow for lots of testing
293 We should never release a main version without testing it as a
296 =head2 Test popular applications and modules.
298 We should never release a main version without testing whether or not
299 it breaks various popular modules and applications. A partial list of
300 such things would include majordomo, metaconfig, apache, Tk, CGI,
301 libnet, and libwww, to name just a few. Of course it's quite possible
302 that some of those things will be just plain broken and need to be fixed,
303 but, in general, we ought to try to avoid breaking widely-installed
306 =head2 Automated generation of derivative files
308 The F<embed.h>, F<keywords.h>, F<opcode.h>, and F<perltoc.pod> files
309 are all automatically generated by perl scripts. In general, don't
310 patch these directly; patch the data files instead.
312 F<Configure> and F<config_h.SH> are also automatically generated by
313 B<metaconfig>. In general, you should patch the metaconfig units
314 instead of patching these files directly. However, very minor changes
315 to F<Configure> may be made in between major sync-ups with the
316 metaconfig units, which tends to be complicated operations. But be
317 careful, this can quickly spiral out of control. Running metaconfig
320 Also F<Makefile> is automatically produced from F<Makefile.SH>.
321 In general, look out for all F<*.SH> files.
323 Finally, the sample files in the F<Porting/> subdirectory are
324 generated automatically by the script F<U/mksample> included
325 with the metaconfig units. See L<"run metaconfig"> below for
326 information on obtaining the metaconfig units.
328 =head1 How to Make a Distribution
330 There really ought to be a 'make dist' target, but there isn't.
331 The 'dist' suite of tools also contains a number of tools that I haven't
332 learned how to use yet. Some of them may make this all a bit easier.
334 Here are the steps I go through to prepare a patch & distribution.
336 Lots of it could doubtless be automated but isn't. The Porting/makerel
337 (make release) perl script does now help automate some parts of it.
339 =head2 Announce your intentions
341 First, you should volunteer out loud to take the patch pumpkin. It's
342 generally counter-productive to have multiple people working in secret
345 At the same time, announce what you plan to do with the patch pumpkin,
346 to allow folks a chance to object or suggest alternatives, or do it for
347 you. Naturally, the patch pumpkin holder ought to incorporate various
348 bug fixes and documentation improvements that are posted while he or
349 she has the pumpkin, but there might also be larger issues at stake.
351 One of the precepts of the subversion idea is that we shouldn't give
352 the patch pumpkin to anyone unless we have some idea what he or she
353 is going to do with it.
355 =head2 refresh pod/perltoc.pod
357 Presumably, you have done a full C<make> in your working source
358 directory. Before you C<make spotless> (if you do), and if you have
359 changed any documentation in any module or pod file, change to the
360 F<pod> directory and run C<make toc>.
362 =head2 run installhtml to check the validity of the pod files
364 =head2 update patchlevel.h
366 Don't be shy about using the subversion number, even for a relatively
367 modest patch. We've never even come close to using all 99 subversions,
368 and it's better to have a distinctive number for your patch. If you
369 need feedback on your patch, go ahead and issue it and promise to
370 incorporate that feedback quickly (e.g. within 1 week) and send out a
373 =head2 run metaconfig
375 If you need to make changes to Configure or config_h.SH, it may be best to
376 change the appropriate metaconfig units instead, and regenerate Configure.
380 will regenerate Configure and config_h.SH. Much more information
381 on obtaining and running metaconfig is in the F<U/README> file
382 that comes with Perl's metaconfig units. Perl's metaconfig units
383 should be available on CPAN. A set of units that will work with
384 perl5.005 is in the file F<mc_units-5.005_00-01.tar.gz> under
385 http://www.perl.com/CPAN/authors/id/ANDYD/ . The mc_units tar file
386 should be unpacked in your main perl source directory. Note: those
387 units were for use with 5.005. There may have been changes since then.
388 Check for later versions or contact perl5-porters@perl.org to obtain a
389 pointer to the current version.
391 Alternatively, do consider if the F<*ish.h> files might be a better
392 place for your changes.
396 Make sure the MANIFEST is up-to-date. You can use dist's B<manicheck>
397 program for this. You can also use
399 perl -w -MExtUtils::Manifest=fullcheck -e fullcheck
401 Both commands will also list extra files in the directory that are not
404 The MANIFEST is normally sorted.
406 If you are using metaconfig to regenerate Configure, then you should note
407 that metaconfig actually uses MANIFEST.new, so you want to be sure
408 MANIFEST.new is up-to-date too. I haven't found the MANIFEST/MANIFEST.new
409 distinction particularly useful, but that's probably because I still haven't
410 learned how to use the full suite of tools in the dist distribution.
412 =head2 Check permissions
414 All the tests in the t/ directory ought to be executable. The
415 main makefile used to do a 'chmod t/*/*.t', but that resulted in
416 a self-modifying distribution--something some users would strongly
417 prefer to avoid. The F<t/TEST> script will check for this
418 and do the chmod if needed, but the tests still ought to be
421 In all, the following files should probably be executable:
436 vms/ext/Stdio/test.pl
440 Other things ought to be readable, at least :-).
442 Probably, the permissions for the files could be encoded in MANIFEST
443 somehow, but I'm reluctant to change MANIFEST itself because that
444 could break old scripts that use MANIFEST.
446 I seem to recall that some SVR3 systems kept some sort of file that listed
447 permissions for system files; something like that might be appropriate.
451 This will build a config.sh and config.h. You can skip this if you haven't
452 changed Configure or config_h.SH at all. I use the following command
454 sh Configure -Dprefix=/opt/perl -Doptimize=-O -Dusethreads \
456 -Dcf_email='yourname@yourhost.yourplace.com' \
457 -Dperladmin='yourname@yourhost.yourplace.com' \
458 -Dmydomain='.yourplace.com' \
459 -Dmyhostname='yourhost' \
462 =head2 Update Porting/config.sh and Porting/config_H
465 This section needs revision. We're currently working on easing
466 the task of keeping the vms, win32, and plan9 config.sh info
467 up-to-date. The plan is to use keep up-to-date 'canned' config.sh
468 files in the appropriate subdirectories and then generate 'canned'
469 config.h files for vms, win32, etc. from the generic config.sh file.
470 This is to ease maintenance. When Configure gets updated, the parts
471 sometimes get scrambled around, and the changes in config_H can
472 sometimes be very hard to follow. config.sh, on the other hand, can
473 safely be sorted, so it's easy to track (typically very small) changes
474 to config.sh and then propoagate them to a canned 'config.h' by any
475 number of means, including a perl script in win32/ or carrying
476 config.sh and config_h.SH to a Unix system and running sh
480 The Porting/config.sh and Porting/config_H files are provided to
481 help those folks who can't run Configure. It is important to keep
482 them up-to-date. If you have changed config_h.SH, those changes must
483 be reflected in config_H as well. (The name config_H was chosen to
484 distinguish the file from config.h even on case-insensitive file systems.)
485 Simply edit the existing config_H file; keep the first few explanatory
486 lines and then copy your new config.h below.
488 It may also be necessary to update win32/config.?c, vms/config.vms and
489 plan9/config.plan9, though you should be quite careful in doing so if
490 you are not familiar with those systems. You might want to issue your
491 patch with a promise to quickly issue a follow-up that handles those
494 =head2 make run_byacc
496 If you have byacc-1.8.2 (available from CPAN), and if there have been
497 changes to F<perly.y>, you can regenerate the F<perly.c> file. The
498 run_byacc makefile target does this by running byacc and then applying
499 some patches so that byacc dynamically allocates space, rather than
500 having fixed limits. This patch is handled by the F<perly.fixer>
501 script. Depending on the nature of the changes to F<perly.y>, you may
502 or may not have to hand-edit the patch to apply correctly. If you do,
503 you should include the edited patch in the new distribution. If you
504 have byacc-1.9, the patch won't apply cleanly. Changes to the printf
505 output statements mean the patch won't apply cleanly. Long ago I
506 started to fix F<perly.fixer> to detect this, but I never completed the
509 If C<perly.c> changes, make sure you run C<perl vms/vms_yfix.pl> to
510 update the corresponding VMS files. See L<VMS-specific updates>.
512 Some additional notes from Larry on this:
514 Don't forget to regenerate perly_c.diff.
518 patch perly.c <perly_c.diff
519 # manually apply any failed hunks
520 diff -c2 perly.c.orig perly.c >perly_c.diff
522 One chunk of lines that often fails begins with
526 and ends one line before
528 #define YYERRCODE 256
530 This only happens when you add or remove a token type. I suppose this
531 could be automated, but it doesn't happen very often nowadays.
535 =head2 make regen_headers
537 The F<embed.h>, F<keywords.h>, and F<opcode.h> files are all automatically
538 generated by perl scripts. Since the user isn't guaranteed to have a
539 working perl, we can't require the user to generate them. Hence you have
540 to, if you're making a distribution.
542 I used to include rules like the following in the makefile:
544 # The following three header files are generated automatically
545 # The correct versions should be already supplied with the perl kit,
546 # in case you don't have perl or 'sh' available.
547 # The - is to ignore error return codes in case you have the source
548 # installed read-only or you don't have perl yet.
549 keywords.h: keywords.pl
550 @echo "Don't worry if this fails."
554 However, I got B<lots> of mail consisting of people worrying because the
555 command failed. I eventually decided that I would save myself time
556 and effort by manually running C<make regen_headers> myself rather
557 than answering all the questions and complaints about the failing
560 =head2 global.sym, interp.sym and perlio.sym
562 Make sure these files are up-to-date. Read the comments in these
563 files and in perl_exp.SH to see what to do.
565 =head2 Binary compatibility
567 If you do change F<global.sym> or F<interp.sym>, think carefully about
568 what you are doing. To the extent reasonable, we'd like to maintain
569 souce and binary compatibility with older releases of perl. That way,
570 extensions built under one version of perl will continue to work with
571 new versions of perl.
573 Of course, some incompatible changes may well be necessary. I'm just
574 suggesting that we not make any such changes without thinking carefully
575 about them first. If possible, we should provide
576 backwards-compatibility stubs. There's a lot of XS code out there.
577 Let's not force people to keep changing it.
581 Be sure to update the F<Changes> file. Try to include both an overall
582 summary as well as detailed descriptions of the changes. Your
583 audience will include other developers and users, so describe
584 user-visible changes (if any) in terms they will understand, not in
585 code like "initialize foo variable in bar function".
587 There are differing opinions on whether the detailed descriptions
588 ought to go in the Changes file or whether they ought to be available
589 separately in the patch file (or both). There is no disagreement that
590 detailed descriptions ought to be easily available somewhere.
594 The F<Todo> file contains a roughly-catgorized unordered list of
595 aspects of Perl that could use enhancement, features that could be
596 added, areas that could be cleaned up, and so on. During your term as
597 pumpkin-holder, you will probably address some of these issues, and
598 perhaps identify others which, while you decide not to address them
599 this time around, may be tackled in the future. Update the file
600 reflect the situation as it stands when you hand over the pumpkin.
602 You might like, early in your pumpkin-holding career, to see if you
603 can find champions for partiticular issues on the to-do list: an issue
604 owned is an issue more likely to be resolved.
606 There are also some more porting-specific L<Todo> items later in this
609 =head2 OS/2-specific updates
611 In the os2 directory is F<diff.configure>, a set of OS/2-specific
612 diffs against B<Configure>. If you make changes to Configure, you may
613 want to consider regenerating this diff file to save trouble for the
616 You can also consider the OS/2 diffs as reminders of portability
617 things that need to be fixed in Configure.
619 =head2 VMS-specific updates
621 If you have changed F<perly.y> or F<perly.c>, then you most probably want
622 to update F<vms/perly_{h,c}.vms> by running C<perl vms/vms_yfix.pl>.
624 The Perl version number appears in several places under F<vms>.
625 It is courteous to update these versions. For example, if you are
626 making 5.004_42, replace "5.00441" with "5.00442".
628 =head2 Making the new distribution
630 Suppose, for example, that you want to make version 5.004_08. Then you can
631 do something like the following
633 mkdir ../perl5.004_08
634 awk '{print $1}' MANIFEST | cpio -pdm ../perl5.004_08
636 tar cf perl5.004_08.tar perl5.004_08
637 gzip --best perl5.004_08.tar
639 These steps, with extra checks, are automated by the Porting/makerel
642 =head2 Making a new patch
644 I find the F<makepatch> utility quite handy for making patches.
645 You can obtain it from any CPAN archive under
646 http://www.perl.com/CPAN/authors/Johan_Vromans/ . There are a couple
647 of differences between my version and the standard one. I have mine do
650 # Print a reassuring "End of Patch" note so people won't
651 # wonder if their mailer truncated patches.
652 print "\n\nEnd of Patch.\n";
654 at the end. That's because I used to get questions from people asking
655 if their mail was truncated.
657 It also writes Index: lines which include the new directory prefix
658 (change Index: print, approx line 294 or 310 depending on the version,
659 to read: print PATCH ("Index: $newdir$new\n");). That helps patches
660 work with more POSIX conformant patch programs.
662 Here's how I generate a new patch. I'll use the hypothetical
663 5.004_07 to 5.004_08 patch as an example.
665 # unpack perl5.004_07/
666 gzip -d -c perl5.004_07.tar.gz | tar -xof -
667 # unpack perl5.004_08/
668 gzip -d -c perl5.004_08.tar.gz | tar -xof -
669 makepatch perl5.004_07 perl5.004_08 > perl5.004_08.pat
671 Makepatch will automatically generate appropriate B<rm> commands to remove
672 deleted files. Unfortunately, it will not correctly set permissions
673 for newly created files, so you may have to do so manually. For example,
674 patch 5.003_04 created a new test F<t/op/gv.t> which needs to be executable,
675 so at the top of the patch, I inserted the following lines:
681 Now, of course, my patch is now wrong because makepatch didn't know I
682 was going to do that command, and it patched against /dev/null.
684 So, what I do is sort out all such shell commands that need to be in the
685 patch (including possible mv-ing of files, if needed) and put that in the
686 shell commands at the top of the patch. Next, I delete all the patch parts
687 of perl5.004_08.pat, leaving just the shell commands. Then, I do the
691 sh ../perl5.004_08.pat
693 makepatch perl5.004_07 perl5.004_08 >> perl5.004_08.pat
695 (Note the append to preserve my shell commands.)
696 Now, my patch will line up with what the end users are going to do.
698 =head2 Testing your patch
700 It seems obvious, but be sure to test your patch. That is, verify that
701 it produces exactly the same thing as your full distribution.
704 gzip -d -c perl5.004_07.tar.gz | tar -xf -
706 sh ../perl5.004_08.pat
707 patch -p1 -N < ../perl5.004_08.pat
709 gdiff -r perl5.004_07 perl5.004_08
711 where B<gdiff> is GNU diff. Other diff's may also do recursive checking.
715 Again, it's obvious, but you should test your new version as widely as you
716 can. You can be sure you'll hear about it quickly if your version doesn't
717 work on both ANSI and pre-ANSI compilers, and on common systems such as
718 SunOS 4.1.[34], Solaris, and Linux.
720 If your changes include conditional code, try to test the different
721 branches as thoroughly as you can. For example, if your system
722 supports dynamic loading, you can also test static loading with
726 You can also hand-tweak your config.h to try out different #ifdef
729 =head1 Common Gotcha's
735 The '#elif' preprocessor directive is not understood on all systems.
736 Specifically, I know that Pyramids don't understand it. Thus instead of the
747 You have to do the more Byzantine
759 Incidentally, whitespace between the leading '#' and the preprocessor
760 command is not guaranteed, but is very portable and you may use it freely.
761 I think it makes things a bit more readable, especially once things get
762 rather deeply nested. I also think that things should almost never get
763 too deeply nested, so it ought to be a moot point :-)
765 =item Probably Prefer POSIX
767 It's often the case that you'll need to choose whether to do
768 something the BSD-ish way or the POSIX-ish way. It's usually not
769 a big problem when the two systems use different names for similar
770 functions, such as memcmp() and bcmp(). The perl.h header file
771 handles these by appropriate #defines, selecting the POSIX mem*()
772 functions if available, but falling back on the b*() functions, if
775 More serious is the case where some brilliant person decided to
776 use the same function name but give it a different meaning or
777 calling sequence :-). getpgrp() and setpgrp() come to mind.
778 These are a real problem on systems that aim for conformance to
779 one standard (e.g. POSIX), but still try to support the other way
780 of doing things (e.g. BSD). My general advice (still not really
781 implemented in the source) is to do something like the following.
782 Suppose there are two alternative versions, fooPOSIX() and
786 /* use fooPOSIX(); */
789 /* try to emulate fooPOSIX() with fooBSD();
790 perhaps with the following: */
791 # define fooPOSIX fooBSD
793 # /* Uh, oh. We have to supply our own. */
794 # define fooPOSIX Perl_fooPOSIX
798 =item Think positively
800 If you need to add an #ifdef test, it is usually easier to follow if you
801 think positively, e.g.
803 #ifdef HAS_NEATO_FEATURE
804 /* use neato feature */
806 /* use some fallback mechanism */
809 rather than the more impenetrable
811 #ifndef MISSING_NEATO_FEATURE
812 /* Not missing it, so we must have it, so use it */
814 /* Are missing it, so fall back on something else. */
817 Of course for this toy example, there's not much difference. But when
818 the #ifdef's start spanning a couple of screen fulls, and the #else's
819 are marked something like
821 #else /* !MISSING_NEATO_FEATURE */
823 I find it easy to get lost.
825 =item Providing Missing Functions -- Problem
827 Not all systems have all the neat functions you might want or need, so
828 you might decide to be helpful and provide an emulation. This is
829 sound in theory and very kind of you, but please be careful about what
830 you name the function. Let me use the C<pause()> function as an
833 Perl5.003 has the following in F<perl.h>
836 #define pause() sleep((32767<<16)+32767)
839 Configure sets HAS_PAUSE if the system has the pause() function, so
840 this #define only kicks in if the pause() function is missing.
843 Unfortunately, some systems apparently have a prototype for pause()
844 in F<unistd.h>, but don't actually have the function in the library.
845 (Or maybe they do have it in a library we're not using.)
847 Thus, the compiler sees something like
849 extern int pause(void);
851 #define pause() sleep((32767<<16)+32767)
853 and dies with an error message. (Some compilers don't mind this;
854 others apparently do.)
856 To work around this, 5.003_03 and later have the following in perl.h:
858 /* Some unistd.h's give a prototype for pause() even though
859 HAS_PAUSE ends up undefined. This causes the #define
860 below to be rejected by the compiler. Sigh.
865 # define Pause() sleep((32767<<16)+32767)
870 The curious reader may wonder why I didn't do the following in
876 sleep((32767<<16)+32767);
880 That is, since the function is missing, just provide it.
881 Then things would probably be been alright, it would seem.
883 Well, almost. It could be made to work. The problem arises from the
884 conflicting needs of dynamic loading and namespace protection.
886 For dynamic loading to work on AIX (and VMS) we need to provide a list
887 of symbols to be exported. This is done by the script F<perl_exp.SH>,
888 which reads F<global.sym> and F<interp.sym>. Thus, the C<pause>
889 symbol would have to be added to F<global.sym> So far, so good.
891 On the other hand, one of the goals of Perl5 is to make it easy to
892 either extend or embed perl and link it with other libraries. This
893 means we have to be careful to keep the visible namespace "clean".
894 That is, we don't want perl's global variables to conflict with
895 those in the other application library. Although this work is still
896 in progress, the way it is currently done is via the F<embed.h> file.
897 This file is built from the F<global.sym> and F<interp.sym> files,
898 since those files already list the globally visible symbols. If we
899 had added C<pause> to global.sym, then F<embed.h> would contain the
902 #define pause Perl_pause
904 and calls to C<pause> in the perl sources would now point to
905 C<Perl_pause>. Now, when B<ld> is run to build the F<perl> executable,
906 it will go looking for C<perl_pause>, which probably won't exist in any
907 of the standard libraries. Thus the build of perl will fail.
909 Those systems where C<HAS_PAUSE> is not defined would be ok, however,
910 since they would get a C<Perl_pause> function in util.c. The rest of
911 the world would be in trouble.
913 And yes, this scenario has happened. On SCO, the function C<chsize>
914 is available. (I think it's in F<-lx>, the Xenix compatibility
915 library.) Since the perl4 days (and possibly before), Perl has
916 included a C<chsize> function that gets called something akin to
919 I32 chsize(fd, length)
925 #define chsize Perl_chsize
927 to F<embed.h>, the compile started failing on SCO systems.
929 The "fix" is to give the function a different name. The one
930 implemented in 5.003_05 isn't optimal, but here's what was done:
933 # ifdef my_chsize /* Probably #defined to Perl_my_chsize in embed.h */
936 # define my_chsize chsize
939 My explanatory comment in patch 5.003_05 said:
941 Undef and then re-define my_chsize from Perl_my_chsize to
942 just plain chsize if this system HAS_CHSIZE. This probably only
943 applies to SCO. This shows the perils of having internal
944 functions with the same name as external library functions :-).
946 Now, we can safely put C<my_chsize> in F<global.sym>, export it, and
947 hide it with F<embed.h>.
949 To be consistent with what I did for C<pause>, I probably should have
950 called the new function C<Chsize>, rather than C<my_chsize>.
951 However, the perl sources are quite inconsistent on this (Consider
952 New, Mymalloc, and Myremalloc, to name just a few.)
954 There is a problem with this fix, however, in that C<Perl_chsize>
955 was available as a F<libperl.a> library function in 5.003, but it
956 isn't available any more (as of 5.003_07). This means that we've
957 broken binary compatibility. This is not good.
959 =item Providing missing functions -- some ideas
961 We currently don't have a standard way of handling such missing
962 function names. Right now, I'm effectively thinking aloud about a
963 solution. Some day, I'll try to formally propose a solution.
965 Part of the problem is that we want to have some functions listed as
966 exported but not have their names mangled by embed.h or possibly
967 conflict with names in standard system headers. We actually already
968 have such a list at the end of F<perl_exp.SH> (though that list is
971 # extra globals not included above.
972 cat <<END >> perl.exp
996 This still needs much thought, but I'm inclined to think that one
997 possible solution is to prefix all such functions with C<perl_> in the
998 source and list them along with the other C<perl_*> functions in
1001 Thus, for C<chsize>, we'd do something like the following:
1005 # define perl_chsize chsize
1008 then in some file (e.g. F<util.c> or F<doio.c>) do
1011 I32 perl_chsize(fd, length)
1012 /* implement the function here . . . */
1015 Alternatively, we could just always use C<chsize> everywhere and move
1016 C<chsize> from F<global.sym> to the end of F<perl_exp.SH>. That would
1017 probably be fine as long as our C<chsize> function agreed with all the
1018 C<chsize> function prototypes in the various systems we'll be using.
1019 As long as the prototypes in actual use don't vary that much, this is
1020 probably a good alternative. (As a counter-example, note how Configure
1021 and perl have to go through hoops to find and use get Malloc_t and
1022 Free_t for C<malloc> and C<free>.)
1024 At the moment, this latter option is what I tend to prefer.
1026 =item All the world's a VAX
1028 Sorry, showing my age:-). Still, all the world is not BSD 4.[34],
1029 SVR4, or POSIX. Be aware that SVR3-derived systems are still quite
1030 common (do you have any idea how many systems run SCO?) If you don't
1031 have a bunch of v7 manuals handy, the metaconfig units (by default
1032 installed in F</usr/local/lib/dist/U>) are a good resource to look at
1037 =head1 Miscellaneous Topics
1041 Why does perl use a metaconfig-generated Configure script instead of an
1042 autoconf-generated configure script?
1044 Metaconfig and autoconf are two tools with very similar purposes.
1045 Metaconfig is actually the older of the two, and was originally written
1046 by Larry Wall, while autoconf is probably now used in a wider variety of
1047 packages. The autoconf info file discusses the history of autoconf and
1048 how it came to be. The curious reader is referred there for further
1051 Overall, both tools are quite good, I think, and the choice of which one
1052 to use could be argued either way. In March, 1994, when I was just
1053 starting to work on Configure support for Perl5, I considered both
1054 autoconf and metaconfig, and eventually decided to use metaconfig for the
1059 =item Compatibility with Perl4
1061 Perl4 used metaconfig, so many of the #ifdef's were already set up for
1062 metaconfig. Of course metaconfig had evolved some since Perl4's days,
1063 but not so much that it posed any serious problems.
1065 =item Metaconfig worked for me
1067 My system at the time was Interactive 2.2, a SVR3.2/386 derivative that
1068 also had some POSIX support. Metaconfig-generated Configure scripts
1069 worked fine for me on that system. On the other hand, autoconf-generated
1070 scripts usually didn't. (They did come quite close, though, in some
1071 cases.) At the time, I actually fetched a large number of GNU packages
1072 and checked. Not a single one configured and compiled correctly
1073 out-of-the-box with the system's cc compiler.
1075 =item Configure can be interactive
1077 With both autoconf and metaconfig, if the script works, everything is
1078 fine. However, one of my main problems with autoconf-generated scripts
1079 was that if it guessed wrong about something, it could be B<very> hard to
1080 go back and fix it. For example, autoconf always insisted on passing the
1081 -Xp flag to cc (to turn on POSIX behavior), even when that wasn't what I
1082 wanted or needed for that package. There was no way short of editing the
1083 configure script to turn this off. You couldn't just edit the resulting
1084 Makefile at the end because the -Xp flag influenced a number of other
1087 Metaconfig's Configure scripts, on the other hand, can be interactive.
1088 Thus if Configure is guessing things incorrectly, you can go back and fix
1089 them. This isn't as important now as it was when we were actively
1090 developing Configure support for new features such as dynamic loading,
1091 but it's still useful occasionally.
1095 At the time, autoconf-generated scripts were covered under the GNU Public
1096 License, and hence weren't suitable for inclusion with Perl, which has a
1097 different licensing policy. (Autoconf's licensing has since changed.)
1101 Metaconfig builds up Configure from a collection of discrete pieces
1102 called "units". You can override the standard behavior by supplying your
1103 own unit. With autoconf, you have to patch the standard files instead.
1104 I find the metaconfig "unit" method easier to work with. Others
1105 may find metaconfig's units clumsy to work with.
1109 =head2 @INC search order
1111 By default, the list of perl library directories in @INC is the
1119 Specifically, on my Solaris/x86 system, I run
1120 B<sh Configure -Dprefix=/opt/perl> and I have the following
1123 /opt/perl/lib/i86pc-solaris/5.00307
1125 /opt/perl/lib/site_perl/i86pc-solaris
1126 /opt/perl/lib/site_perl
1128 That is, perl's directories come first, followed by the site-specific
1131 The site libraries come second to support the usage of extensions
1132 across perl versions. Read the relevant section in F<INSTALL> for
1133 more information. If we ever make $sitearch version-specific, this
1134 topic could be revisited.
1136 =head2 Why isn't there a directory to override Perl's library?
1138 Mainly because no one's gotten around to making one. Note that
1139 "making one" involves changing perl.c, Configure, config_h.SH (and
1140 associated files, see above), and I<documenting> it all in the
1143 Apparently, most folks who want to override one of the standard library
1144 files simply do it by overwriting the standard library files.
1148 In the perl.c sources, you'll find an undocumented APPLLIB_EXP
1149 variable, sort of like PRIVLIB_EXP and ARCHLIB_EXP (which are
1150 documented in config_h.SH). Here's what APPLLIB_EXP is for, from
1151 a mail message from Larry:
1153 The main intent of APPLLIB_EXP is for folks who want to send out a
1154 version of Perl embedded in their product. They would set the symbol
1155 to be the name of the library containing the files needed to run or to
1156 support their particular application. This works at the "override"
1157 level to make sure they get their own versions of any library code that
1158 they absolutely must have configuration control over.
1160 As such, I don't see any conflict with a sysadmin using it for a
1161 override-ish sort of thing, when installing a generic Perl. It should
1162 probably have been named something to do with overriding though. Since
1163 it's undocumented we could still change it... :-)
1165 Given that it's already there, you can use it to override
1166 distribution modules. If you do
1168 sh Configure -Dccflags='-DAPPLLIB_EXP=/my/override'
1170 then perl.c will put /my/override ahead of ARCHLIB and PRIVLIB.
1172 =head2 Shared libperl.so location
1174 Why isn't the shared libperl.so installed in /usr/lib/ along
1175 with "all the other" shared libraries? Instead, it is installed
1176 in $archlib, which is typically something like
1178 /usr/local/lib/perl5/archname/5.00404
1180 and is architecture- and version-specific.
1182 The basic reason why a shared libperl.so gets put in $archlib is so that
1183 you can have more than one version of perl on the system at the same time,
1184 and have each refer to its own libperl.so.
1186 Three examples might help. All of these work now; none would work if you
1187 put libperl.so in /usr/lib.
1193 Suppose you want to have both threaded and non-threaded perl versions
1194 around. Configure will name both perl libraries "libperl.so" (so that
1195 you can link to them with -lperl). The perl binaries tell them apart
1196 by having looking in the appropriate $archlib directories.
1200 Suppose you have perl5.004_04 installed and you want to try to compile
1201 it again, perhaps with different options or after applying a patch.
1202 If you already have libperl.so installed in /usr/lib/, then it may be
1203 either difficult or impossible to get ld.so to find the new libperl.so
1204 that you're trying to build. If, instead, libperl.so is tucked away in
1205 $archlib, then you can always just change $archlib in the current perl
1206 you're trying to build so that ld.so won't find your old libperl.so.
1207 (The INSTALL file suggests you do this when building a debugging perl.)
1211 The shared perl library is not a "well-behaved" shared library with
1212 proper major and minor version numbers, so you can't necessarily
1213 have perl5.004_04 and perl5.004_05 installed simultaneously. Suppose
1214 perl5.004_04 were to install /usr/lib/libperl.so.4.4, and perl5.004_05
1215 were to install /usr/lib/libperl.so.4.5. Now, when you try to run
1216 perl5.004_04, ld.so might try to load libperl.so.4.5, since it has
1217 the right "major version" number. If this works at all, it almost
1218 certainly defeats the reason for keeping perl5.004_04 around. Worse,
1219 with development subversions, you certaily can't guarantee that
1220 libperl.so.4.4 and libperl.so.4.55 will be compatible.
1222 Anyway, all this leads to quite obscure failures that are sure to drive
1223 casual users crazy. Even experienced users will get confused :-). Upon
1224 reflection, I'd say leave libperl.so in $archlib.
1228 =head1 Upload Your Work to CPAN
1230 You can upload your work to CPAN if you have a CPAN id. Check out
1231 http://www.perl.com/CPAN/modules/04pause.html for information on
1232 _PAUSE_, the Perl Author's Upload Server.
1234 I typically upload both the patch file, e.g. F<perl5.004_08.pat.gz>
1235 and the full tar file, e.g. F<perl5.004_08.tar.gz>.
1237 If you want your patch to appear in the F<src/5.0/unsupported>
1238 directory on CPAN, send e-mail to the CPAN master librarian. (Check
1239 out http://www.perl.com/CPAN/CPAN.html ).
1241 =head1 Help Save the World
1243 You should definitely announce your patch on the perl5-porters list.
1244 You should also consider announcing your patch on
1245 comp.lang.perl.announce, though you should make it quite clear that a
1246 subversion is not a production release, and be prepared to deal with
1247 people who will not read your disclaimer.
1251 Here, in no particular order, are some Configure and build-related
1252 items that merit consideration. This list isn't exhaustive, it's just
1253 what I came up with off the top of my head.
1255 =head2 Good ideas waiting for round tuits
1261 I think we ought to support
1263 Configure -Dinstallprefix=/blah/blah
1265 Currently, we support B<-Dprefix=/blah/blah>, but the changing the install
1266 location has to be handled by something like the F<config.over> trick
1267 described in F<INSTALL>. AFS users also are treated specially.
1268 We should probably duplicate the metaconfig prefix stuff for an
1271 =item Configure -Dsrc=/blah/blah
1273 We should be able to emulate B<configure --srcdir>. Tom Tromey
1274 tromey@creche.cygnus.com has submitted some patches to
1275 the dist-users mailing list along these lines. They have been folded
1276 back into the main distribution, but various parts of the perl
1277 Configure/build/install process still assume src='.'.
1279 =item Hint file fixes
1281 Various hint files work around Configure problems. We ought to fix
1282 Configure so that most of them aren't needed.
1284 =item Hint file information
1286 Some of the hint file information (particularly dynamic loading stuff)
1287 ought to be fed back into the main metaconfig distribution.
1289 =item Catch GNU Libc "Stub" functions
1291 Some functions (such as lchown()) are present in libc, but are
1292 unimplmented. That is, they always fail and set errno=ENOSYS.
1294 Thomas Bushnell provided the following sample code and the explanation
1297 /* System header to define __stub macros and hopefully few prototypes,
1298 which can conflict with char FOO(); below. */
1300 /* Override any gcc2 internal prototype to avoid an error. */
1301 /* We use char because int might match the return type of a gcc2
1302 builtin and then its argument prototype would still apply. */
1307 /* The GNU C library defines this for functions which it implements
1308 to always fail with ENOSYS. Some functions are actually named
1309 something starting with __ and the normal name is an alias. */
1310 #if defined (__stub_FOO) || defined (__stub___FOO)
1318 The choice of <assert.h> is essentially arbitrary. The GNU libc
1319 macros are found in <gnu/stubs.h>. You can include that file instead
1320 of <assert.h> (which itself includes <gnu/stubs.h>) if you test for
1321 its existence first. <assert.h> is assumed to exist on every system,
1322 which is why it's used here. Any GNU libc header file will include
1323 the stubs macros. If either __stub_NAME or __stub___NAME is defined,
1324 then the function doesn't actually exist. Tests using <assert.h> work
1325 on every system around.
1327 The declaration of FOO is there to override builtin prototypes for
1332 =head2 Probably good ideas waiting for round tuits
1336 =item GNU configure --options
1338 I've received sensible suggestions for --exec_prefix and other
1339 GNU configure --options. It's not always obvious exactly what is
1340 intended, but this merits investigation.
1344 Currently, B<make clean> isn't all that useful, though
1345 B<make realclean> and B<make distclean> are. This needs a bit of
1346 thought and documentation before it gets cleaned up.
1348 =item Try gcc if cc fails
1350 Currently, we just give up.
1352 =item bypassing safe*alloc wrappers
1354 On some systems, it may be safe to call the system malloc directly
1355 without going through the util.c safe* layers. (Such systems would
1356 accept free(0), for example.) This might be a time-saver for systems
1357 that already have a good malloc. (Recent Linux libc's apparently have
1358 a nice malloc that is well-tuned for the system.)
1362 =head2 Vague possibilities
1368 Get some of the Macintosh stuff folded back into the main distribution.
1370 =item gconvert replacement
1372 Maybe include a replacement function that doesn't lose data in rare
1373 cases of coercion between string and numerical values.
1375 =item Improve makedepend
1377 The current makedepend process is clunky and annoyingly slow, but it
1378 works for most folks. Alas, it assumes that there is a filename
1379 $firstmakefile that the B<make> command will try to use before it uses
1380 F<Makefile>. Such may not be the case for all B<make> commands,
1381 particularly those on non-Unix systems.
1383 Probably some variant of the BSD F<.depend> file will be useful.
1384 We ought to check how other packages do this, if they do it at all.
1385 We could probably pre-generate the dependencies (with the exception of
1386 malloc.o, which could probably be determined at F<Makefile.SH>
1389 =item GNU Makefile standard targets
1391 GNU software generally has standardized Makefile targets. Unless we
1392 have good reason to do otherwise, I see no reason not to support them.
1396 Somehow, straighten out, document, and implement lockf(), flock(),
1397 and/or fcntl() file locking. It's a mess.
1403 Original author: Andy Dougherty doughera@lafcol.lafayette.edu .
1404 Additions by Chip Salzenberg chip@perl.com and
1405 Tim Bunce Tim.Bunce@ig.co.uk .
1407 All opinions expressed herein are those of the authorZ<>(s).
1409 =head1 LAST MODIFIED
1411 $Id: pumpkin.pod,v 1.22 1998/07/22 16:33:55 doughera Released $