Commit | Line | Data |
aa689395 |
1 | =head1 NAME |
2 | |
3 | Pumpkin - Notes on handling the Perl Patch Pumpkin |
4 | |
5 | =head1 SYNOPSIS |
6 | |
7 | There is no simple synopsis, yet. |
8 | |
9 | =head1 DESCRIPTION |
10 | |
11 | This document attempts to begin to describe some of the |
12 | considerations involved in patching and maintaining perl. |
13 | |
14 | This document is still under construction, and still subject to |
15 | significant changes. Still, I hope parts of it will be useful, |
16 | so I'm releasing it even though it's not done. |
17 | |
18 | For the most part, it's a collection of anecdotal information that |
19 | already assumes some familiarity with the Perl sources. I really need |
20 | an introductory section that describes the organization of the sources |
21 | and all the various auxiliary files that are part of the distribution. |
22 | |
23 | =head1 Where Do I Get Perl Sources and Related Material? |
24 | |
25 | The Comprehensive Perl Archive Network (or CPAN) is the place to go. |
26 | There are many mirrors, but the easiest thing to use is probably |
7b5757d1 |
27 | http://www.perl.com/CPAN/README.html , which automatically points you to a |
aa689395 |
28 | mirror site "close" to you. |
29 | |
30 | =head2 Perl5-porters mailing list |
31 | |
32 | The mailing list perl5-porters@perl.org |
33 | is the main group working with the development of perl. If you're |
34 | interested in all the latest developments, you should definitely |
35 | subscribe. The list is high volume, but generally has a |
36 | fairly low noise level. |
37 | |
38 | Subscribe by sending the message (in the body of your letter) |
39 | |
40 | subscribe perl5-porters |
41 | |
42 | to perl5-porters-request@perl.org . |
43 | |
44 | =head1 How are Perl Releases Numbered? |
45 | |
7b5757d1 |
46 | Perl version numbers are floating point numbers, such as 5.004. |
47 | (Observations about the imprecision of floating point numbers for |
48 | representing reality probably have more relevance than you might |
49 | imagine :-) The major version number is 5 and the '004' is the |
50 | patchlevel. (Questions such as whether or not '004' is really a minor |
51 | version number can safely be ignored.:) |
52 | |
53 | The version number is available as the magic variable $], |
aa689395 |
54 | and can be used in comparisons, e.g. |
55 | |
56 | print "You've got an old perl\n" if $] < 5.002; |
57 | |
aa689395 |
58 | You can also require particular version (or later) with |
59 | |
60 | use 5.002; |
61 | |
7b5757d1 |
62 | At some point in the future, we may need to decide what to call the |
63 | next big revision. In the .package file used by metaconfig to |
64 | generate Configure, there are two variables that might be relevant: |
65 | $baserev=5.0 and $package=perl5. At various times, I have suggested |
66 | we might change them to $baserev=5.1 and $package=perl5.1 if want |
67 | to signify a fairly major update. Or, we might want to jump to perl6. |
68 | Let's worry about that problem when we get there. |
69 | |
aa689395 |
70 | =head2 Subversions |
71 | |
72 | In addition, there may be "developer" sub-versions available. These |
73 | are not official releases. They may contain unstable experimental |
74 | features, and are subject to rapid change. Such developer |
75 | sub-versions are numbered with sub-version numbers. For example, |
76 | version 5.004_04 is the 4'th developer version built on top of |
77 | 5.004. It might include the _01, _02, and _03 changes, but it |
78 | also might not. Sub-versions are allowed to be subversive. |
79 | |
80 | These sub-versions can also be used as floating point numbers, so |
81 | you can do things such as |
82 | |
7b5757d1 |
83 | print "You've got an unstable perl\n" if $] == 5.00303; |
aa689395 |
84 | |
85 | You can also require particular version (or later) with |
86 | |
7b5757d1 |
87 | use 5.003_03; # the "_" is optional |
aa689395 |
88 | |
89 | Sub-versions produced by the members of perl5-porters are usually |
90 | available on CPAN in the F<src/5.0/unsupported> directory. |
91 | |
7b5757d1 |
92 | =head2 Maintenance and Development Subversions |
93 | |
94 | As an experiment, starting with version 5.004, subversions _01 through |
95 | _49 will be reserved for bug-fix maintenance releases, and subversions |
96 | _50 through _99 will be available for unstable development versions. |
97 | |
98 | The separate bug-fix track is being established to allow us an easy |
99 | way to distribute important bug fixes without waiting for the |
100 | developers to untangle all the other problems in the current |
101 | developer's release. |
102 | |
103 | Watch for announcements of maintenance subversions in |
104 | comp.lang.perl.announce. |
105 | |
aa689395 |
106 | =head2 Why such a complicated scheme? |
107 | |
108 | Two reasons, really. At least. |
109 | |
7b5757d1 |
110 | First, we need some way to identify and release collections of patches |
111 | that are known to have new features that need testing and exploration. The |
aa689395 |
112 | subversion scheme does that nicely while fitting into the |
113 | C<use 5.004;> mold. |
114 | |
115 | Second, since most of the folks who help maintain perl do so on a |
116 | free-time voluntary basis, perl development does not proceed at a |
117 | precise pace, though it always seems to be moving ahead quickly. |
118 | We needed some way to pass around the "patch pumpkin" to allow |
119 | different people chances to work on different aspects of the |
120 | distribution without getting in each other's way. It wouldn't be |
121 | constructive to have multiple people working on incompatible |
122 | implementations of the same idea. Instead what was needed was |
123 | some kind of "baton" or "token" to pass around so everyone knew |
124 | whose turn was next. |
125 | |
126 | =head2 Why is it called the patch pumpkin? |
127 | |
128 | Chip Salzenberg gets credit for that, with a nod to his cow orker, |
129 | David Croy. We had passed around various names (baton, token, hot |
130 | potato) but none caught on. Then, Chip asked: |
131 | |
132 | [begin quote] |
133 | |
134 | Who has the patch pumpkin? |
135 | |
136 | To explain: David Croy once told me once that at a previous job, |
137 | there was one tape drive and multiple systems that used it for backups. |
138 | But instead of some high-tech exclusion software, they used a low-tech |
139 | method to prevent multiple simultaneous backups: a stuffed pumpkin. |
140 | No one was allowed to make backups unless they had the "backup pumpkin". |
141 | |
142 | [end quote] |
143 | |
144 | The name has stuck. |
145 | |
146 | =head1 Philosophical Issues in Patching Perl |
147 | |
148 | There are no absolute rules, but there are some general guidelines I |
149 | have tried to follow as I apply patches to the perl sources. |
150 | (This section is still under construction.) |
151 | |
152 | =head2 Solve problems as generally as possible |
153 | |
7b5757d1 |
154 | Never implement a specific restricted solution to a problem when you |
155 | can solve the same problem in a more general, flexible way. |
156 | |
157 | For example, for dynamic loading to work on some SVR4 systems, we had |
158 | to build a shared libperl.so library. In order to build "FAT" binaries |
159 | on NeXT 4.0 systems, we had to build a special libperl library. Rather |
160 | than continuing to build a contorted nest of special cases, I |
161 | generalized the process of building libperl so that NeXT and SVR4 users |
162 | could still get their work done, but others could build a shared |
163 | libperl if they wanted to as well. |
aa689395 |
164 | |
165 | =head2 Seek consensus on major changes |
166 | |
167 | If you are making big changes, don't do it in secret. Discuss the |
168 | ideas in advance on perl5-porters. |
169 | |
170 | =head2 Keep the documentation up-to-date |
171 | |
172 | If your changes may affect how users use perl, then check to be sure |
173 | that the documentation is in sync with your changes. Be sure to |
174 | check all the files F<pod/*.pod> and also the F<INSTALL> document. |
175 | |
176 | Consider writing the appropriate documentation first and then |
7b5757d1 |
177 | implementing your change to correspond to the documentation. |
aa689395 |
178 | |
179 | =head2 Avoid machine-specific #ifdef's |
180 | |
181 | To the extent reasonable, try to avoid machine-specific #ifdef's in |
182 | the sources. Instead, use feature-specific #ifdef's. The reason is |
183 | that the machine-specific #ifdef's may not be valid across major |
184 | releases of the operating system. Further, the feature-specific tests |
185 | may help out folks on another platform who have the same problem. |
186 | |
187 | =head2 Allow for lots of testing |
188 | |
189 | We should never release a main version without testing it as a |
190 | subversion first. |
191 | |
7b5757d1 |
192 | =head2 Automate generation of derivative files |
aa689395 |
193 | |
194 | The F<embed.h>, F<keywords.h>, F<opcode.h>, and F<perltoc.pod> files |
195 | are all automatically generated by perl scripts. In general, don't |
196 | patch these directly; patch the data files instead. |
197 | |
198 | F<Configure> and F<config_h.SH> are also automatically generated by |
199 | B<metaconfig>. In general, you should patch the metaconfig units |
200 | instead of patching these files directly. However, minor changes to |
201 | F<Configure> may be made in between major sync-ups with the metaconfig |
202 | units, which tends to be complicated operations. |
203 | |
204 | =head1 How to Make a Distribution |
205 | |
206 | There really ought to be a 'make dist' target, but there isn't. |
207 | The 'dist' suite of tools also contains a number of tools that I haven't |
208 | learned how to use yet. Some of them may make this all a bit easier. |
209 | |
210 | Here are the steps I go through to prepare a patch & distribution. |
211 | |
212 | Lots of it could doubtless be automated but isn't. |
213 | |
214 | =head2 Announce your intentions |
215 | |
216 | First, you should volunteer out loud to take the patch pumpkin. It's |
217 | generally counter-productive to have multiple people working in secret |
218 | on the same thing. |
219 | |
220 | At the same time, announce what you plan to do with the patch pumpkin, |
221 | to allow folks a chance to object or suggest alternatives, or do it for |
222 | you. Naturally, the patch pumpkin holder ought to incorporate various |
223 | bug fixes and documentation improvements that are posted while he or |
224 | she has the pumpkin, but there might also be larger issues at stake. |
225 | |
226 | One of the precepts of the subversion idea is that we shouldn't give |
7b5757d1 |
227 | the patch pumpkin to anyone unless we have some idea what he or she |
228 | is going to do with it. |
aa689395 |
229 | |
230 | =head2 refresh pod/perltoc.pod |
231 | |
232 | Presumably, you have done a full C<make> in your working source |
233 | directory. Before you C<make spotless> (if you do), and if you have |
234 | changed any documentation in any module or pod file, change to the |
235 | F<pod> directory and run C<make toc>. |
236 | |
237 | =head2 update patchlevel.h |
238 | |
239 | Don't be shy about using the subversion number, even for a relatively |
240 | modest patch. We've never even come close to using all 99 subversions, |
241 | and it's better to have a distinctive number for your patch. If you |
242 | need feedback on your patch, go ahead and issue it and promise to |
243 | incorporate that feedback quickly (e.g. within 1 week) and send out a |
244 | second patch. |
245 | |
246 | =head2 run metaconfig |
247 | |
248 | If you need to make changes to Configure or config_h.SH, it may be best to |
249 | change the appropriate metaconfig units instead, and regenerate Configure. |
250 | |
251 | metaconfig -m |
252 | |
253 | will regenerate Configure and config_h.SH. More information on |
254 | obtaining and running metaconfig is in the F<U/README> file that comes |
255 | with Perl's metaconfig units. Perl's metaconfig units should be |
256 | available the same place you found this file. On CPAN, look under my |
257 | directory F<id/ANDYD/> for a file such as F<5.003_07-02.U.tar.gz>. |
258 | That file should be unpacked in your main perl source directory. It |
259 | contains the files needed to run B<metaconfig> to reproduce Perl's |
7b5757d1 |
260 | Configure script. (Those units are for 5.003_07. There have been |
261 | changes since then; please contact me if you want more recent |
262 | versions, and I will try to point you in the right direction.) |
aa689395 |
263 | |
264 | Alternatively, do consider if the F<*ish.h> files might be a better |
265 | place for your changes. |
266 | |
267 | =head2 MANIFEST |
268 | |
269 | Make sure the MANIFEST is up-to-date. You can use dist's B<manicheck> |
270 | program for this. You can also use |
271 | |
272 | perl -MExtUtils::Manifest -e fullcheck |
273 | |
274 | to do half the job. This will make sure everything listed in MANIFEST |
275 | is included in the distribution. dist's B<manicheck> command will |
276 | also list extra files in the directory that are not listed in |
277 | MANIFEST. |
278 | |
279 | The MANIFEST is normally sorted, with one exception. Perl includes |
280 | both a F<Configure> script and a F<configure> script. The |
281 | F<configure> script is a front-end to the main F<Configure>, but |
282 | is there to aid folks who use autoconf-generated F<configure> files |
283 | for other software. The problem is that F<Configure> and F<configure> |
284 | are the same on case-insensitive file systems, so I deliberately put |
285 | F<configure> first in the MANIFEST so that the extraction of |
286 | F<Configure> will overwrite F<configure> and leave you with the |
287 | correct script. (The F<configure> script must also have write |
288 | permission for this to work, so it's the only file in the distribution |
289 | I normally have with write permission.) |
290 | |
291 | If you are using metaconfig to regenerate Configure, then you should note |
292 | that metaconfig actually uses MANIFEST.new, so you want to be sure |
293 | MANIFEST.new is up-to-date too. I haven't found the MANIFEST/MANIFEST.new |
294 | distinction particularly useful, but that's probably because I still haven't |
295 | learned how to use the full suite of tools in the dist distribution. |
296 | |
297 | =head2 Check permissions |
298 | |
299 | All the tests in the t/ directory ought to be executable. The |
300 | main makefile used to do a 'chmod t/*/*.t', but that resulted in |
301 | a self-modifying distribution--something some users would strongly |
302 | prefer to avoid. Probably, the F<t/TEST> script should check for this |
303 | and do the chmod if needed, but it doesn't currently. |
304 | |
305 | In all, the following files should probably be executable: |
306 | |
307 | Configure |
308 | configpm |
309 | configure |
310 | embed.pl |
311 | installperl |
312 | installman |
313 | keywords.pl |
314 | lib/splain |
315 | myconfig |
316 | opcode.pl |
317 | perly.fixer |
318 | t/TEST |
319 | t/*/*.t |
320 | *.SH |
321 | vms/ext/Stdio/test.pl |
322 | vms/ext/filespec.t |
323 | vms/fndvers.com |
324 | x2p/*.SH |
325 | |
326 | Other things ought to be readable, at least :-). |
327 | |
328 | Probably, the permissions for the files could be encoded in MANIFEST |
329 | somehow, but I'm reluctant to change MANIFEST itself because that |
330 | could break old scripts that use MANIFEST. |
331 | |
332 | I seem to recall that some SVR3 systems kept some sort of file that listed |
333 | permissions for system files; something like that might be appropriate. |
334 | |
335 | =head2 Run Configure |
336 | |
337 | This will build a config.sh and config.h. You can skip this if you haven't |
338 | changed Configure or config_h.SH at all. |
339 | |
340 | =head2 Update config_H |
341 | |
342 | The config_H file is provided to help those folks who can't run Configure. |
343 | It is important to keep it up-to-date. If you have changed config_h.SH, |
344 | those changes must be reflected in config_H as well. (The name config_H was |
345 | chosen to distinguish the file from config.h even on case-insensitive file |
346 | systems.) Simply edit the existing config_H file; keep the first few |
347 | explanatory lines and then copy your new config.h below. |
348 | |
349 | It may also be necessary to update vms/config.vms and |
350 | plan9/config.plan9, though you should be quite careful in doing so if |
351 | you are not familiar with those systems. You might want to issue your |
352 | patch with a promise to quickly issue a follow-up that handles those |
353 | directories. |
354 | |
355 | =head2 make run_byacc |
356 | |
357 | If you have byacc-1.8.2 (available from CPAN), and if there have been |
358 | changes to F<perly.y>, you can regenerate the F<perly.c> file. The |
359 | run_byacc makefile target does this by running byacc and then applying |
360 | some patches so that byacc dynamically allocates space, rather than |
361 | having fixed limits. This patch is handled by the F<perly.fixer> |
362 | script. Depending on the nature of the changes to F<perly.y>, you may |
363 | or may not have to hand-edit the patch to apply correctly. If you do, |
364 | you should include the edited patch in the new distribution. If you |
365 | have byacc-1.9, the patch won't apply cleanly. Changes to the printf |
366 | output statements mean the patch won't apply cleanly. Long ago I |
367 | started to fix F<perly.fixer> to detect this, but I never completed the |
368 | task. |
369 | |
370 | Some additional notes from Larry on this: |
371 | |
372 | Don't forget to regenerate perly.c.diff. |
373 | |
7b5757d1 |
374 | byacc -d perly.y |
aa689395 |
375 | mv y.tab.c perly.c |
376 | patch perly.c <perly.c.diff |
377 | # manually apply any failed hunks |
378 | diff -c2 perly.c.orig perly.c >perly.c.diff |
379 | |
380 | One chunk of lines that often fails begins with |
381 | |
382 | #line 29 "perly.y" |
383 | |
384 | and ends one line before |
385 | |
386 | #define YYERRCODE 256 |
387 | |
388 | This only happens when you add or remove a token type. I suppose this |
389 | could be automated, but it doesn't happen very often nowadays. |
390 | |
391 | Larry |
392 | |
393 | =head2 make regen_headers |
394 | |
395 | The F<embed.h>, F<keywords.h>, and F<opcode.h> files are all automatically |
396 | generated by perl scripts. Since the user isn't guaranteed to have a |
397 | working perl, we can't require the user to generate them. Hence you have |
398 | to, if you're making a distribution. |
399 | |
400 | I used to include rules like the following in the makefile: |
401 | |
402 | # The following three header files are generated automatically |
403 | # The correct versions should be already supplied with the perl kit, |
404 | # in case you don't have perl or 'sh' available. |
405 | # The - is to ignore error return codes in case you have the source |
406 | # installed read-only or you don't have perl yet. |
407 | keywords.h: keywords.pl |
408 | @echo "Don't worry if this fails." |
409 | - perl keywords.pl |
410 | |
411 | |
7b5757d1 |
412 | However, I got B<lots> of mail consisting of people worrying because the |
aa689395 |
413 | command failed. I eventually decided that I would save myself time |
414 | and effort by manually running C<make regen_headers> myself rather |
415 | than answering all the questions and complaints about the failing |
416 | command. |
417 | |
418 | =head2 global.sym and interp.sym |
419 | |
420 | Make sure these files are up-to-date. Read the comments in these |
421 | files and in perl_exp.SH to see what to do. |
422 | |
423 | =head2 Binary compatibility |
424 | |
425 | If you do change F<global.sym> or F<interp.sym>, think carefully about |
426 | what you are doing. To the extent reasonable, we'd like to maintain |
427 | souce and binary compatibility with older releases of perl. That way, |
428 | extensions built under one version of perl will continue to work with |
429 | new versions of perl. |
430 | |
431 | Of course, some incompatible changes may well be necessary. I'm just |
432 | suggesting that we not make any such changes without thinking carefully |
433 | about them first. If possible, we should provide |
434 | backwards-compatibility stubs. There's a lot of XS code out there. |
435 | Let's not force people to keep changing it. |
436 | |
437 | =head2 Changes |
438 | |
439 | Be sure to update the F<Changes> file. Try to include both an overall |
440 | summary as well as detailed descriptions of the changes. Your |
441 | audience will include bother developers and users, so describe |
442 | user-visible changes (if any) in terms they will understand, not in |
443 | code like "initialize foo variable in bar function". |
444 | |
445 | There are differing opinions on whether the detailed descriptions |
446 | ought to go in the Changes file or whether they ought to be available |
447 | separately in the patch file (or both). There is no disagreement that |
448 | detailed descriptions ought to be easily available somewhere. |
449 | |
450 | =head2 OS/2-specific updates |
451 | |
452 | In the os2 directory is F<diff.configure>, a set of OS/2-specific |
453 | diffs against B<Configure>. If you make changes to Configure, you may |
454 | want to consider regenerating this diff file to save trouble for the |
455 | OS/2 maintainer. |
456 | |
7b5757d1 |
457 | You can also consider the OS/2 diffs as reminders of portability |
458 | things that need to be fixed in Configure. |
459 | |
aa689395 |
460 | =head2 VMS-specific updates |
461 | |
462 | If you have changed F<perly.y>, then you may want to update |
463 | F<vms/perly_{h,c}.vms> by running C<perl vms/vms_yfix.pl>. |
464 | |
465 | The Perl version number appears in several places under F<vms>. |
466 | It is courteous to update these versions. For example, if you are |
467 | making 5.004_42, replace "5.00441" with "5.00442". |
468 | |
469 | =head2 Making the new distribution |
470 | |
471 | Suppose, for example, that you want to make version 5.004_08. Then you can |
472 | do something like the following |
473 | |
474 | mkdir ../perl5.004_08 |
475 | awk '{print $1}' MANIFEST | cpio -pdm ../perl5.004_08 |
476 | cd ../ |
477 | tar cf perl5.004_08.tar perl5.004_08 |
478 | gzip --best perl5.004_08.tar |
479 | |
480 | =head2 Making a new patch |
481 | |
482 | I find the F<makepatch> utility quite handy for making patches. |
483 | You can obtain it from any CPAN archive under |
7b5757d1 |
484 | http://www.perl.com/CPAN/authors/Johan_Vromans/ . The only |
aa689395 |
485 | difference between my version and the standard one is that I have mine |
486 | do a |
487 | |
488 | # Print a reassuring "End of Patch" note so people won't |
489 | # wonder if their mailer truncated patches. |
490 | print "\n\nEnd of Patch.\n"; |
491 | |
492 | at the end. That's because I used to get questions from people asking if |
493 | their mail was truncated. |
494 | |
495 | Here's how I generate a new patch. I'll use the hypothetical |
496 | 5.004_07 to 5.004_08 patch as an example. |
497 | |
498 | # unpack perl5.004_07/ |
499 | gzip -d -c perl5.004_07.tar.gz | tar -xof - |
500 | # unpack perl5.004_08/ |
501 | gzip -d -c perl5.004_08.tar.gz | tar -xof - |
502 | makepatch perl5.004_07 perl5.004_08 > perl5.004_08.pat |
503 | |
504 | Makepatch will automatically generate appropriate B<rm> commands to remove |
505 | deleted files. Unfortunately, it will not correctly set permissions |
506 | for newly created files, so you may have to do so manually. For example, |
507 | patch 5.003_04 created a new test F<t/op/gv.t> which needs to be executable, |
508 | so at the top of the patch, I inserted the following lines: |
509 | |
510 | # Make a new test |
511 | touch t/op/gv.t |
512 | chmod +x t/opt/gv.t |
513 | |
514 | Now, of course, my patch is now wrong because makepatch didn't know I |
515 | was going to do that command, and it patched against /dev/null. |
516 | |
517 | So, what I do is sort out all such shell commands that need to be in the |
518 | patch (including possible mv-ing of files, if needed) and put that in the |
519 | shell commands at the top of the patch. Next, I delete all the patch parts |
520 | of perl5.004_08.pat, leaving just the shell commands. Then, I do the |
521 | following: |
522 | |
7b5757d1 |
523 | cd perl5.004_07 |
524 | sh ../perl5.004_08.pat |
aa689395 |
525 | cd .. |
7b5757d1 |
526 | makepatch perl5.004_07 perl5.004_08 >> perl5.004_08.pat |
aa689395 |
527 | |
528 | (Note the append to preserve my shell commands.) |
529 | Now, my patch will line up with what the end users are going to do. |
530 | |
531 | =head2 Testing your patch |
532 | |
533 | It seems obvious, but be sure to test your patch. That is, verify that |
534 | it produces exactly the same thing as your full distribution. |
535 | |
7b5757d1 |
536 | rm -rf perl5.004_07 |
537 | gzip -d -c perl5.004_07.tar.gz | tar -xf - |
538 | cd perl5.004_07 |
539 | sh ../perl5.004_08.pat |
540 | patch -p1 -N < ../perl5.004_08.pat |
aa689395 |
541 | cd .. |
7b5757d1 |
542 | gdiff -r perl5.004_07 perl5.004_08 |
aa689395 |
543 | |
544 | where B<gdiff> is GNU diff. Other diff's may also do recursive checking. |
545 | |
546 | =head2 More testing |
547 | |
548 | Again, it's obvious, but you should test your new version as widely as you |
549 | can. You can be sure you'll hear about it quickly if your version doesn't |
550 | work on both ANSI and pre-ANSI compilers, and on common systems such as |
551 | SunOS 4.1.[34], Solaris, and Linux. |
552 | |
553 | If your changes include conditional code, try to test the different |
554 | branches as thoroughly as you can. For example, if your system |
555 | supports dynamic loading, you can also test static loading with |
556 | |
557 | sh Configure -Uusedl |
558 | |
559 | You can also hand-tweak your config.h to try out different #ifdef |
560 | branches. |
561 | |
562 | =head1 Common Gotcha's |
563 | |
564 | =over 4 |
565 | |
566 | =item #elif |
567 | |
568 | The '#elif' preprocessor directive is not understood on all systems. |
569 | Specifically, I know that Pyramids don't understand it. Thus instead of the |
570 | simple |
571 | |
572 | #if defined(I_FOO) |
573 | # include <foo.h> |
574 | #elif defined(I_BAR) |
575 | # include <bar.h> |
576 | #else |
577 | # include <fubar.h> |
578 | #endif |
579 | |
580 | You have to do the more Byzantine |
581 | |
582 | #if defined(I_FOO) |
583 | # include <foo.h> |
584 | #else |
585 | # if defined(I_BAR) |
586 | # include <bar.h> |
587 | # else |
588 | # include <fubar.h> |
589 | # endif |
590 | #endif |
591 | |
592 | Incidentally, whitespace between the leading '#' and the preprocessor |
593 | command is not guaranteed, but is very portable and you may use it freely. |
594 | I think it makes things a bit more readable, especially once things get |
595 | rather deeply nested. I also think that things should almost never get |
596 | too deeply nested, so it ought to be a moot point :-) |
597 | |
598 | =item Probably Prefer POSIX |
599 | |
600 | It's often the case that you'll need to choose whether to do |
601 | something the BSD-ish way or the POSIX-ish way. It's usually not |
602 | a big problem when the two systems use different names for similar |
603 | functions, such as memcmp() and bcmp(). The perl.h header file |
604 | handles these by appropriate #defines, selecting the POSIX mem*() |
605 | functions if available, but falling back on the b*() functions, if |
606 | need be. |
607 | |
608 | More serious is the case where some brilliant person decided to |
609 | use the same function name but give it a different meaning or |
610 | calling sequence :-). getpgrp() and setpgrp() come to mind. |
611 | These are a real problem on systems that aim for conformance to |
612 | one standard (e.g. POSIX), but still try to support the other way |
613 | of doing things (e.g. BSD). My general advice (still not really |
614 | implemented in the source) is to do something like the following. |
615 | Suppose there are two alternative versions, fooPOSIX() and |
616 | fooBSD(). |
617 | |
618 | #ifdef HAS_FOOPOSIX |
619 | /* use fooPOSIX(); */ |
620 | #else |
621 | # ifdef HAS_FOOBSD |
622 | /* try to emulate fooPOSIX() with fooBSD(); |
623 | perhaps with the following: */ |
624 | # define fooPOSIX fooBSD |
625 | # else |
626 | # /* Uh, oh. We have to supply our own. */ |
627 | # define fooPOSIX Perl_fooPOSIX |
628 | # endif |
629 | #endif |
630 | |
631 | =item Think positively |
632 | |
633 | If you need to add an #ifdef test, it is usually easier to follow if you |
634 | think positively, e.g. |
635 | |
636 | #ifdef HAS_NEATO_FEATURE |
637 | /* use neato feature */ |
638 | #else |
639 | /* use some fallback mechanism */ |
640 | #endif |
641 | |
642 | rather than the more impenetrable |
643 | |
644 | #ifndef MISSING_NEATO_FEATURE |
645 | /* Not missing it, so we must have it, so use it */ |
646 | #else |
647 | /* Are missing it, so fall back on something else. */ |
648 | #endif |
649 | |
650 | Of course for this toy example, there's not much difference. But when |
651 | the #ifdef's start spanning a couple of screen fulls, and the #else's |
652 | are marked something like |
653 | |
654 | #else /* !MISSING_NEATO_FEATURE */ |
655 | |
656 | I find it easy to get lost. |
657 | |
658 | =item Providing Missing Functions -- Problem |
659 | |
660 | Not all systems have all the neat functions you might want or need, so |
661 | you might decide to be helpful and provide an emulation. This is |
662 | sound in theory and very kind of you, but please be careful about what |
663 | you name the function. Let me use the C<pause()> function as an |
664 | illustration. |
665 | |
666 | Perl5.003 has the following in F<perl.h> |
667 | |
668 | #ifndef HAS_PAUSE |
669 | #define pause() sleep((32767<<16)+32767) |
670 | #endif |
671 | |
672 | Configure sets HAS_PAUSE if the system has the pause() function, so |
673 | this #define only kicks in if the pause() function is missing. |
674 | Nice idea, right? |
675 | |
676 | Unfortunately, some systems apparently have a prototype for pause() |
677 | in F<unistd.h>, but don't actually have the function in the library. |
678 | (Or maybe they do have it in a library we're not using.) |
679 | |
680 | Thus, the compiler sees something like |
681 | |
682 | extern int pause(void); |
683 | /* . . . */ |
684 | #define pause() sleep((32767<<16)+32767) |
685 | |
686 | and dies with an error message. (Some compilers don't mind this; |
687 | others apparently do.) |
688 | |
689 | To work around this, 5.003_03 and later have the following in perl.h: |
690 | |
691 | /* Some unistd.h's give a prototype for pause() even though |
692 | HAS_PAUSE ends up undefined. This causes the #define |
693 | below to be rejected by the compiler. Sigh. |
694 | */ |
695 | #ifdef HAS_PAUSE |
696 | # define Pause pause |
697 | #else |
698 | # define Pause() sleep((32767<<16)+32767) |
699 | #endif |
700 | |
701 | This works. |
702 | |
703 | The curious reader may wonder why I didn't do the following in |
704 | F<util.c> instead: |
705 | |
706 | #ifndef HAS_PAUSE |
707 | void pause() |
708 | { |
709 | sleep((32767<<16)+32767); |
710 | } |
711 | #endif |
712 | |
713 | That is, since the function is missing, just provide it. |
714 | Then things would probably be been alright, it would seem. |
715 | |
716 | Well, almost. It could be made to work. The problem arises from the |
717 | conflicting needs of dynamic loading and namespace protection. |
718 | |
719 | For dynamic loading to work on AIX (and VMS) we need to provide a list |
720 | of symbols to be exported. This is done by the script F<perl_exp.SH>, |
721 | which reads F<global.sym> and F<interp.sym>. Thus, the C<pause> |
722 | symbol would have to be added to F<global.sym> So far, so good. |
723 | |
724 | On the other hand, one of the goals of Perl5 is to make it easy to |
725 | either extend or embed perl and link it with other libraries. This |
726 | means we have to be careful to keep the visible namespace "clean". |
727 | That is, we don't want perl's global variables to conflict with |
728 | those in the other application library. Although this work is still |
729 | in progress, the way it is currently done is via the F<embed.h> file. |
730 | This file is built from the F<global.sym> and F<interp.sym> files, |
731 | since those files already list the globally visible symbols. If we |
732 | had added C<pause> to global.sym, then F<embed.h> would contain the |
733 | line |
734 | |
735 | #define pause Perl_pause |
736 | |
737 | and calls to C<pause> in the perl sources would now point to |
738 | C<Perl_pause>. Now, when B<ld> is run to build the F<perl> executable, |
739 | it will go looking for C<perl_pause>, which probably won't exist in any |
740 | of the standard libraries. Thus the build of perl will fail. |
741 | |
742 | Those systems where C<HAS_PAUSE> is not defined would be ok, however, |
743 | since they would get a C<Perl_pause> function in util.c. The rest of |
744 | the world would be in trouble. |
745 | |
746 | And yes, this scenario has happened. On SCO, the function C<chsize> |
747 | is available. (I think it's in F<-lx>, the Xenix compatibility |
748 | library.) Since the perl4 days (and possibly before), Perl has |
749 | included a C<chsize> function that gets called something akin to |
750 | |
751 | #ifndef HAS_CHSIZE |
752 | I32 chsize(fd, length) |
753 | /* . . . */ |
754 | #endif |
755 | |
756 | When 5.003 added |
757 | |
758 | #define chsize Perl_chsize |
759 | |
760 | to F<embed.h>, the compile started failing on SCO systems. |
761 | |
762 | The "fix" is to give the function a different name. The one |
763 | implemented in 5.003_05 isn't optimal, but here's what was done: |
764 | |
765 | #ifdef HAS_CHSIZE |
766 | # ifdef my_chsize /* Probably #defined to Perl_my_chsize in embed.h */ |
767 | # undef my_chsize |
768 | # endif |
769 | # define my_chsize chsize |
770 | #endif |
771 | |
772 | My explanatory comment in patch 5.003_05 said: |
773 | |
774 | Undef and then re-define my_chsize from Perl_my_chsize to |
775 | just plain chsize if this system HAS_CHSIZE. This probably only |
776 | applies to SCO. This shows the perils of having internal |
777 | functions with the same name as external library functions :-). |
778 | |
779 | Now, we can safely put C<my_chsize> in F<global.sym>, export it, and |
780 | hide it with F<embed.h>. |
781 | |
782 | To be consistent with what I did for C<pause>, I probably should have |
783 | called the new function C<Chsize>, rather than C<my_chsize>. |
784 | However, the perl sources are quite inconsistent on this (Consider |
785 | New, Mymalloc, and Myremalloc, to name just a few.) |
786 | |
787 | There is a problem with this fix, however, in that C<Perl_chsize> |
788 | was available as a F<libperl.a> library function in 5.003, but it |
789 | isn't available any more (as of 5.003_07). This means that we've |
790 | broken binary compatibility. This is not good. |
791 | |
792 | =item Providing missing functions -- some ideas |
793 | |
794 | We currently don't have a standard way of handling such missing |
795 | function names. Right now, I'm effectively thinking aloud about a |
796 | solution. Some day, I'll try to formally propose a solution. |
797 | |
798 | Part of the problem is that we want to have some functions listed as |
799 | exported but not have their names mangled by embed.h or possibly |
800 | conflict with names in standard system headers. We actually already |
801 | have such a list at the end of F<perl_exp.SH> (though that list is |
802 | out-of-date): |
803 | |
804 | # extra globals not included above. |
805 | cat <<END >> perl.exp |
806 | perl_init_ext |
807 | perl_init_fold |
808 | perl_init_i18nl14n |
809 | perl_alloc |
810 | perl_construct |
811 | perl_destruct |
812 | perl_free |
813 | perl_parse |
814 | perl_run |
815 | perl_get_sv |
816 | perl_get_av |
817 | perl_get_hv |
818 | perl_get_cv |
819 | perl_call_argv |
820 | perl_call_pv |
821 | perl_call_method |
822 | perl_call_sv |
823 | perl_requirepv |
824 | safecalloc |
825 | safemalloc |
826 | saferealloc |
827 | safefree |
828 | |
829 | This still needs much thought, but I'm inclined to think that one |
830 | possible solution is to prefix all such functions with C<perl_> in the |
831 | source and list them along with the other C<perl_*> functions in |
832 | F<perl_exp.SH>. |
833 | |
834 | Thus, for C<chsize>, we'd do something like the following: |
835 | |
836 | /* in perl.h */ |
837 | #ifdef HAS_CHSIZE |
838 | # define perl_chsize chsize |
839 | #endif |
840 | |
841 | then in some file (e.g. F<util.c> or F<doio.c>) do |
842 | |
843 | #ifndef HAS_CHSIZE |
844 | I32 perl_chsize(fd, length) |
845 | /* implement the function here . . . */ |
846 | #endif |
847 | |
848 | Alternatively, we could just always use C<chsize> everywhere and move |
849 | C<chsize> from F<global.sym> to the end of F<perl_exp.SH>. That would |
850 | probably be fine as long as our C<chsize> function agreed with all the |
851 | C<chsize> function prototypes in the various systems we'll be using. |
852 | As long as the prototypes in actual use don't vary that much, this is |
853 | probably a good alternative. (As a counter-example, note how Configure |
854 | and perl have to go through hoops to find and use get Malloc_t and |
855 | Free_t for C<malloc> and C<free>.) |
856 | |
857 | At the moment, this latter option is what I tend to prefer. |
858 | |
859 | =item All the world's a VAX |
860 | |
861 | Sorry, showing my age:-). Still, all the world is not BSD 4.[34], |
862 | SVR4, or POSIX. Be aware that SVR3-derived systems are still quite |
863 | common (do you have any idea how many systems run SCO?) If you don't |
864 | have a bunch of v7 manuals handy, the metaconfig units (by default |
865 | installed in F</usr/local/lib/dist/U>) are a good resource to look at |
866 | for portability. |
867 | |
868 | =back |
869 | |
870 | =head1 Miscellaneous Topics |
871 | |
872 | =head2 Autoconf |
873 | |
874 | Why does perl use a metaconfig-generated Configure script instead of an |
875 | autoconf-generated configure script? |
876 | |
877 | Metaconfig and autoconf are two tools with very similar purposes. |
878 | Metaconfig is actually the older of the two, and was originally written |
879 | by Larry Wall, while autoconf is probably now used in a wider variety of |
880 | packages. The autoconf info file discusses the history of autoconf and |
881 | how it came to be. The curious reader is referred there for further |
882 | information. |
883 | |
884 | Overall, both tools are quite good, I think, and the choice of which one |
885 | to use could be argued either way. In March, 1994, when I was just |
886 | starting to work on Configure support for Perl5, I considered both |
887 | autoconf and metaconfig, and eventually decided to use metaconfig for the |
888 | following reasons: |
889 | |
890 | =over 4 |
891 | |
892 | =item Compatibility with Perl4 |
893 | |
894 | Perl4 used metaconfig, so many of the #ifdef's were already set up for |
895 | metaconfig. Of course metaconfig had evolved some since Perl4's days, |
896 | but not so much that it posed any serious problems. |
897 | |
898 | =item Metaconfig worked for me |
899 | |
900 | My system at the time was Interactive 2.2, a SVR3.2/386 derivative that |
901 | also had some POSIX support. Metaconfig-generated Configure scripts |
902 | worked fine for me on that system. On the other hand, autoconf-generated |
903 | scripts usually didn't. (They did come quite close, though, in some |
904 | cases.) At the time, I actually fetched a large number of GNU packages |
905 | and checked. Not a single one configured and compiled correctly |
906 | out-of-the-box with the system's cc compiler. |
907 | |
908 | =item Configure can be interactive |
909 | |
910 | With both autoconf and metaconfig, if the script works, everything is |
911 | fine. However, one of my main problems with autoconf-generated scripts |
912 | was that if it guessed wrong about something, it could be B<very> hard to |
913 | go back and fix it. For example, autoconf always insisted on passing the |
914 | -Xp flag to cc (to turn on POSIX behavior), even when that wasn't what I |
915 | wanted or needed for that package. There was no way short of editing the |
916 | configure script to turn this off. You couldn't just edit the resulting |
917 | Makefile at the end because the -Xp flag influenced a number of other |
918 | configure tests. |
919 | |
920 | Metaconfig's Configure scripts, on the other hand, can be interactive. |
921 | Thus if Configure is guessing things incorrectly, you can go back and fix |
922 | them. This isn't as important now as it was when we were actively |
923 | developing Configure support for new features such as dynamic loading, |
924 | but it's still useful occasionally. |
925 | |
926 | =item GPL |
927 | |
928 | At the time, autoconf-generated scripts were covered under the GNU Public |
929 | License, and hence weren't suitable for inclusion with Perl, which has a |
930 | different licensing policy. (Autoconf's licensing has since changed.) |
931 | |
932 | =item Modularity |
933 | |
934 | Metaconfig builds up Configure from a collection of discrete pieces |
935 | called "units". You can override the standard behavior by supplying your |
936 | own unit. With autoconf, you have to patch the standard files instead. |
937 | I find the metaconfig "unit" method easier to work with. Others |
938 | may find metaconfig's units clumsy to work with. |
939 | |
940 | =back |
941 | |
942 | =head2 @INC search order |
943 | |
944 | By default, the list of perl library directories in @INC is the |
945 | following: |
946 | |
947 | $archlib |
948 | $privlib |
949 | $sitearch |
950 | $sitelib |
951 | |
952 | Specifically, on my Solaris/x86 system, I run |
953 | B<sh Configure -Dprefix=/opt/perl> and I have the following |
954 | directories: |
955 | |
956 | /opt/perl/lib/i86pc-solaris/5.00307 |
957 | /opt/perl/lib |
958 | /opt/perl/lib/site_perl/i86pc-solaris |
959 | /opt/perl/lib/site_perl |
960 | |
961 | That is, perl's directories come first, followed by the site-specific |
962 | directories. |
963 | |
964 | The site libraries come second to support the usage of extensions |
965 | across perl versions. Read the relevant section in F<INSTALL> for |
966 | more information. If we ever make $sitearch version-specific, this |
967 | topic could be revisited. |
968 | |
969 | =head2 Why isn't there a directory to override Perl's library? |
970 | |
971 | Mainly because no one's gotten around to making one. Note that |
972 | "making one" involves changing perl.c, Configure, config_h.SH (and |
973 | associated files, see above), and I<documenting> it all in the |
974 | INSTALL file. |
975 | |
976 | Apparently, most folks who want to override one of the standard library |
977 | files simply do it by overwriting the standard library files. |
978 | |
979 | =head2 APPLLIB |
980 | |
981 | In the perl.c sources, you'll find an undocumented APPLLIB_EXP |
982 | variable, sort of like PRIVLIB_EXP and ARCHLIB_EXP (which are |
983 | documented in config_h.SH). Here's what APPLLIB_EXP is for, from |
984 | a mail message from Larry: |
985 | |
986 | The main intent of APPLLIB_EXP is for folks who want to send out a |
987 | version of Perl embedded in their product. They would set the symbol |
988 | to be the name of the library containing the files needed to run or to |
989 | support their particular application. This works at the "override" |
990 | level to make sure they get their own versions of any library code that |
991 | they absolutely must have configuration control over. |
992 | |
993 | As such, I don't see any conflict with a sysadmin using it for a |
994 | override-ish sort of thing, when installing a generic Perl. It should |
995 | probably have been named something to do with overriding though. Since |
996 | it's undocumented we could still change it... :-) |
997 | |
998 | Given that it's already there, you can use it to override |
999 | distribution modules. If you do |
1000 | |
1001 | sh Configure -Dccflags='-DAPPLLIB_EXP=/my/override' |
1002 | |
1003 | then perl.c will put /my/override ahead of ARCHLIB and PRIVLIB. |
1004 | |
1005 | =head1 Upload Your Work to CPAN |
1006 | |
1007 | You can upload your work to CPAN if you have a CPAN id. Check out |
1008 | http://www.perl.com/CPAN/modules/04pause.html for information on |
1009 | _PAUSE_, the Perl Author's Upload Server. |
1010 | |
1011 | I typically upload both the patch file, e.g. F<perl5.004_08.pat.gz> |
1012 | and the full tar file, e.g. F<perl5.004_08.tar.gz>. |
1013 | |
1014 | If you want your patch to appear in the F<src/5.0/unsupported> |
1015 | directory on CPAN, send e-mail to the CPAN master librarian. (Check |
7b5757d1 |
1016 | out http://www.perl.com/CPAN/CPAN.html ). |
aa689395 |
1017 | |
1018 | =head1 Help Save the World |
1019 | |
1020 | You should definitely announce your patch on the perl5-porters list. |
1021 | You should also consider announcing your patch on |
1022 | comp.lang.perl.announce, though you should make it quite clear that a |
1023 | subversion is not a production release, and be prepared to deal with |
1024 | people who will not read your disclaimer. |
1025 | |
1026 | =head1 Todo |
1027 | |
1028 | Here, in no particular order, are some Configure and build-related |
1029 | items that merit consideration. This list isn't exhaustive, it's just |
1030 | what I came up with off the top of my head. |
1031 | |
1032 | =head2 Good ideas waiting for round tuits |
1033 | |
1034 | =over 4 |
1035 | |
1036 | =item installprefix |
1037 | |
1038 | I think we ought to support |
1039 | |
1040 | Configure -Dinstallprefix=/blah/blah |
1041 | |
1042 | Currently, we support B<-Dprefix=/blah/blah>, but the changing the install |
1043 | location has to be handled by something like the F<config.over> trick |
1044 | described in F<INSTALL>. AFS users also are treated specially. |
1045 | We should probably duplicate the metaconfig prefix stuff for an |
1046 | install prefix. |
1047 | |
1048 | =item Configure -Dsrcdir=/blah/blah |
1049 | |
1050 | We should be able to emulate B<configure --srcdir>. Tom Tromey |
1051 | tromey@creche.cygnus.com has submitted some patches to |
1052 | the dist-users mailing list along these lines. Eventually, they ought |
1053 | to get folded back into the main distribution. |
1054 | |
1055 | =item Hint file fixes |
1056 | |
1057 | Various hint files work around Configure problems. We ought to fix |
1058 | Configure so that most of them aren't needed. |
1059 | |
1060 | =item Hint file information |
1061 | |
1062 | Some of the hint file information (particularly dynamic loading stuff) |
1063 | ought to be fed back into the main metaconfig distribution. |
1064 | |
1065 | =back |
1066 | |
1067 | =head2 Probably good ideas waiting for round tuits |
1068 | |
1069 | =over 4 |
1070 | |
1071 | =item GNU configure --options |
1072 | |
1073 | I've received sensible suggestions for --exec_prefix and other |
1074 | GNU configure --options. It's not always obvious exactly what is |
1075 | intended, but this merits investigation. |
1076 | |
1077 | =item make clean |
1078 | |
1079 | Currently, B<make clean> isn't all that useful, though |
1080 | B<make realclean> and B<make distclean> are. This needs a bit of |
1081 | thought and documentation before it gets cleaned up. |
1082 | |
1083 | =item Try gcc if cc fails |
1084 | |
1085 | Currently, we just give up. |
1086 | |
1087 | =item bypassing safe*alloc wrappers |
1088 | |
1089 | On some systems, it may be safe to call the system malloc directly |
1090 | without going through the util.c safe* layers. (Such systems would |
1091 | accept free(0), for example.) This might be a time-saver for systems |
1092 | that already have a good malloc. (Recent Linux libc's apparently have |
1093 | a nice malloc that is well-tuned for the system.) |
1094 | |
1095 | =back |
1096 | |
1097 | =head2 Vague possibilities |
1098 | |
1099 | =over 4 |
1100 | |
1101 | =item Win95, WinNT, and Win32 support |
1102 | |
1103 | We need to get something into the distribution for 32-bit Windows. |
1104 | I'm tired of all the private e-mail questions I get, and I'm saddened |
1105 | that so many folks keep trying to reinvent the same wheel. |
1106 | |
1107 | =item MacPerl |
1108 | |
1109 | Get some of the Macintosh stuff folded back into the main |
1110 | distribution. |
1111 | |
1112 | =item gconvert replacement |
1113 | |
1114 | Maybe include a replacement function that doesn't lose data in rare |
1115 | cases of coercion between string and numerical values. |
1116 | |
1117 | =item long long |
1118 | |
1119 | Can we support C<long long> on systems where C<long long> is larger |
1120 | than what we've been using for C<IV>? What if you can't C<sprintf> |
1121 | a C<long long>? |
1122 | |
1123 | =item Improve makedepend |
1124 | |
1125 | The current makedepend process is clunky and annoyingly slow, but it |
1126 | works for most folks. Alas, it assumes that there is a filename |
1127 | $firstmakefile that the B<make> command will try to use before it uses |
1128 | F<Makefile>. Such may not be the case for all B<make> commands, |
1129 | particularly those on non-Unix systems. |
1130 | |
1131 | Probably some variant of the BSD F<.depend> file will be useful. |
1132 | We ought to check how other packages do this, if they do it at all. |
1133 | We could probably pre-generate the dependencies (with the exception of |
1134 | malloc.o, which could probably be determined at F<Makefile.SH> |
1135 | extraction time. |
1136 | |
1137 | =item GNU Makefile standard targets |
1138 | |
1139 | GNU software generally has standardized Makefile targets. Unless we |
1140 | have good reason to do otherwise, I see no reason not to support them. |
1141 | |
1142 | =item File locking |
1143 | |
1144 | Somehow, straighten out, document, and implement lockf(), flock(), |
1145 | and/or fcntl() file locking. It's a mess. |
1146 | |
1147 | =back |
1148 | |
1149 | =head1 AUTHOR |
1150 | |
1151 | Andy Dougherty <doughera@lafcol.lafayette.edu>. |
1152 | |
1153 | Additions by Chip Salzenberg <chip@atlantic.net>. |
1154 | |
1155 | All opinions expressed herein are those of the authorZ<>(s). |
1156 | |
1157 | =head1 LAST MODIFIED |
1158 | |
7b5757d1 |
1159 | $Id: pumpkin.pod,v 1.9 1997/02/24 20:37:43 doughera Released $ |