=item Is the implementation generic enough to be portable?
The worst patches make use of a system-specific features. It's highly
-unlikely that nonportable additions to the Perl language will be
+unlikely that non-portable additions to the Perl language will be
accepted.
=item Is the implementation tested?
(Then creating the symlinks...)
The specifics may vary based on your operating system, of course.
-After you see this, you can abort the F<Configure> script, and you
+After it's all done, you
will see that the directory you are in has a tree of symlinks to the
F<perl-rsync> directories and files.
function edit {
if [ -L $1 ]; then
mv $1 $1.orig
- cp $1.orig $1
- vi $1
+ cp $1.orig $1
+ vi $1
else
- /bin/vi $1
- fi
+ vi $1
+ fi
}
function unedit {
if [ -L $1.orig ]; then
rm $1
- mv $1.orig $1
- fi
+ mv $1.orig $1
+ fi
}
Replace "vi" with your favorite flavor of editor.
mkpatchorig() {
local diffopts
- for f in `find . -name '*.orig' | sed s,^\./,,`
- do
- case `echo $f | sed 's,.orig$,,;s,.*\.,,'` in
- c) diffopts=-p ;;
+ for f in `find . -name '*.orig' | sed s,^\./,,`
+ do
+ case `echo $f | sed 's,.orig$,,;s,.*\.,,'` in
+ c) diffopts=-p ;;
pod) diffopts='-F^=' ;;
*) diffopts= ;;
- esac
- diff -du $diffopts $f `echo $f | sed 's,.orig$,,'`
- done
+ esac
+ diff -du $diffopts $f `echo $f | sed 's,.orig$,,'`
+ done
}
This function produces patches which include enough context to make
by Robert Spier. Become an administrator, and close any bugs you can get
your sticky mitts on:
- http://rt.perl.org/rt3/
+ http://bugs.perl.org/
To email the bug system administrators:
maint branch. Only patches that survive the heat of the development
branch get applied to maintenance versions.
-Your patch should also update the documentation and test suite. See
-L<Writing a test>.
+Your patch should update the documentation and test suite. See
+L<Writing a test>. If you have added or removed files in the distribution,
+edit the MANIFEST file accordingly, sort the MANIFEST file using
+C<make manisort>, and include those changes as part of your patch.
Patching documentation also follows the same order: if accepted, a patch
is first applied to B<development>, and if relevant then it's backported
The CPAN testers ( http://testers.cpan.org/ ) are a group of
volunteers who test CPAN modules on a variety of platforms. Perl
-Smokers ( http://www.nntp.perl.org/group/perl.daily-build )
+Smokers ( http://www.nntp.perl.org/group/perl.daily-build and
+http://www.nntp.perl.org/group/perl.daily-build.reports/ )
automatically test Perl source releases on platforms with various
-configurations. Both efforts welcome volunteers.
+configurations. Both efforts welcome volunteers. In order to get
+involved in smoke testing of the perl itself visit
+L<http://search.cpan.org/dist/Test-Smoke>. In order to start smoke
+testing CPAN modules visit L<http://search.cpan.org/dist/CPAN-YACSmoke/>
+or L<http://search.cpan.org/dist/POE-Component-CPAN-YACSmoke/> or
+L<http://search.cpan.org/dist/CPAN-Reporter/>.
It's a good idea to read and lurk for a while before chipping in.
That way you'll get to see the dynamic of the conversations, learn the
resembles the code found in L<perlembed>; most of the real action takes
place in F<perl.c>
+F<perlmain.c> is generated by L<writemain> from F<miniperlmain.c> at
+make time, so you should make perl to follow this along.
+
First, F<perlmain.c> allocates some memory and constructs a Perl
-interpreter:
+interpreter, along these lines:
1 PERL_SYS_INIT3(&argc,&argv,&env);
2
own C<malloc> as defined in F<malloc.c> if you selected that option at
configure time.
-Next, in line 7, we construct the interpreter; this sets up all the
-special variables that Perl needs, the stacks, and so on.
+Next, in line 7, we construct the interpreter using perl_construct,
+also in F<perl.c>; this sets up all the special variables that Perl
+needs, the stacks, and so on.
Now we pass Perl the command line options, and tell it to go:
exitstatus = perl_parse(my_perl, xs_init, argc, argv, (char **)NULL);
- if (!exitstatus) {
- exitstatus = perl_run(my_perl);
- }
+ if (!exitstatus)
+ perl_run(my_perl);
+
+ exitstatus = perl_destruct(my_perl);
+ perl_free(my_perl);
C<perl_parse> is actually a wrapper around C<S_parse_body>, as defined
in F<perl.c>, which processes the command line options, sets up any
=item Exception handing
-Perl's exception handing (i.e. C<die> etc) is built on top of the low-level
+Perl's exception handing (i.e. C<die> etc.) is built on top of the low-level
C<setjmp()>/C<longjmp()> C-library functions. These basically provide a
way to capture the current PC and SP registers and later restore them; i.e.
a C<longjmp()> continues at the point in code where a previous C<setjmp()>
=back
The following flags would be nice to have but they would first need
-their own Stygian stablemaster:
+their own Augean stablemaster:
=over 4
You may find it helpful to have a "macro dictionary", which you can
produce by saying C<cpp -dM perl.c | sort>. Even then, F<cpp> won't
-recursively apply those macros for you.
+recursively apply those macros for you.
=head2 gdb macro support
F<dump.c>; these work a little like an internal
L<Devel::Peek|Devel::Peek>, but they also cover OPs and other structures
that you can't get at from Perl. Let's take an example. We'll use the
-C<$a = $b + $c> we used before, but give it a bit of context:
+C<$a = $b + $c> we used before, but give it a bit of context:
C<$b = "6XXXX"; $c = 2.3;>. Where's a good place to stop and poke around?
What about C<pp_add>, the function we examined earlier to implement the
directly (if C<SvNOK> is set) or by calling the C<sv_2nv> function.
C<TOPs> takes the next SV from the top of the stack - yes, C<POPn> uses
C<TOPs> - but doesn't remove it. We then use C<SvNV> to get the NV from
-C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
+C<leftsv> in the same way as before - yes, C<POPn> uses C<SvNV>.
Since we don't have an NV for C<$b>, we'll have to use C<sv_2nv> to
convert it. If we step again, we'll find ourselves there:
All right, we've now had a look at how to navigate the Perl sources and
some things you'll need to know when fiddling with them. Let's now get
on and create a simple patch. Here's something Larry suggested: if a
-C<U> is the first active format during a C<pack>, (for example,
+C<U> is the first active format during a C<pack>, (for example,
C<pack "U3C8", @stuff>) then the resulting string should be treated as
UTF-8 encoded.
The regression tests for each operator live in F<t/op/>, and so we
make a copy of F<t/op/pack.t> to F<t/op/pack.t~>. Now we can add our
tests to the end. First, we'll test that the C<U> does indeed create
-Unicode strings.
+Unicode strings.
t/op/pack.t has a sensible ok() function, but if it didn't we could
use the one from t/test.pl.
we can write the more sensible (see L<Test::More> for a full
explanation of is() and other testing functions).
- is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
- "U* produces unicode" );
+ is( "1.20.300.4000", sprintf "%vd", pack("U*",1,20,300,4000),
+ "U* produces Unicode" );
Now we'll test that we got that space-at-the-beginning business right:
the first active format:
isnt( v1.20.300.4000, sprintf "%vd", pack("C0U*",1,20,300,4000),
- "U* not first isn't unicode" );
+ "U* not first isn't Unicode" );
Mustn't forget to change the number of tests which appears at the top,
or else the automated tester will get confused. This will either look
here. There are still some bits and pieces hanging around in here
that need to be moved. Perhaps you could move them? Thanks!
+=item F<t/mro/>
+
+Tests for perl's method resolution order implementations
+(see L<mro>).
+
=item F<t/op/>
Tests for perl's built in functions that don't fit into any of the
working on. This is a measure to prevent a high-level failure (such
as Config.pm breaking) from causing basic functionality tests to fail.
-=over 4
+=over 4
=item t/base t/comp
=back
When you say "make test" Perl uses the F<t/TEST> program to run the
-test suite (except under Win32 where it uses F<t/harness> instead.)
-All tests are run from the F<t/> directory, B<not> the directory
-which contains the test. This causes some problems with the tests
+test suite (except under Win32 where it uses F<t/harness> instead.)
+All tests are run from the F<t/> directory, B<not> the directory
+which contains the test. This causes some problems with the tests
in F<lib/>, so here's some opportunity for some patching.
You must be triply conscious of cross-platform concerns. This usually
=item minitest
Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
-F<t/op>, and F<t/uni> tests.
+F<t/op>, F<t/uni> and F<t/mro> tests.
=item test.valgrind check.valgrind utest.valgrind ucheck.valgrind
harness -v -torture -re LIST OF PATTERNS TO MATCH
If C<LIST OF FILES TO TEST> is omitted the file list is obtained from
-the manifest. The file list may include shell wildcards which will be
+the manifest. The file list may include shell wildcards which will be
expanded out.
=over 4
=item -v
-Run the tests under verbose mode so you can see what tests were run,
+Run the tests under verbose mode so you can see what tests were run,
and debug outbut.
=item -torture
=item -re LIST OF PATTERNS
-Filter the file list so that all the test files run match
+Filter the file list so that all the test files run match
/(LIST|OF|PATTERNS)/. Note that with this form the patterns
are joined by '|' and you cannot supply a list of files, instead
the test files are obtained from the MANIFEST.
except that the harnesses set up some environment variables that may
affect the execution of the test :
-=over 4
+=over 4
=item PERL_CORE=1
=back
+=head3 Other environment variables that may influence tests
+
+=over 4
+
+=item PERL_TEST_Net_Ping
+
+Setting this variable runs all the Net::Ping modules tests,
+otherwise some tests that interact with the outside world are skipped.
+See L<perl58delta>.
+
+=item PERL_TEST_NOVREXX
+
+Setting this variable skips the vrexx.t tests for OS2::REXX.
+
+=item PERL_TEST_NUMCONVERTS
+
+This sets a variable in op/numconvert.t.
+
+=back
+
+See also the documentation for the Test and Test::Harness modules,
+for more environment variables that affect testing.
+
=head2 Common problems when patching Perl source code
Perl source plays by ANSI C89 rules: no C99 (or C++) extensions. In
=item *
+Assuming the character set is ASCIIish
+
+Perl can compile and run under EBCDIC platforms. See L<perlebcdic>.
+This is transparent for the most part, but because the character sets
+differ, you shouldn't use numeric (decimal, octal, nor hex) constants
+to refer to characters. You can safely say 'A', but not 0x41.
+You can safely say '\n', but not \012.
+If a character doesn't have a trivial input form, you can
+create a #define for it in both C<utfebcdic.h> and C<utf8.h>, so that
+it resolves to different values depending on the character set being used.
+(There are three different EBCDIC character sets defined in C<utfebcdic.h>,
+so it might be best to insert the #define three times in that file.)
+
+Also, the range 'A' - 'Z' in ASCII is an unbroken sequence of 26 upper case
+alphabetic characters. That is not true in EBCDIC. Nor for 'a' to 'z'.
+But '0' - '9' is an unbroken range in both systems. Don't assume anything
+about other ranges.
+
+Many of the comments in the existing code ignore the possibility of EBCDIC,
+and may be wrong therefore, even if the code works.
+This is actually a tribute to the successful transparent insertion of being
+able to handle EBCDIC. without having to change pre-existing code.
+
+UTF-8 and UTF-EBCDIC are two different encodings used to represent Unicode
+code points as sequences of bytes. Macros
+with the same names (but different definitions)
+in C<utf8.h> and C<utfebcdic.h>
+are used to allow the calling code think that there is only one such encoding.
+This is almost always referred to as C<utf8>, but it means the EBCDIC
+version as well. Comments in the code may well be wrong even if the code
+itself is right.
+For example, the concept of C<invariant characters> differs between ASCII and
+EBCDIC.
+On ASCII platforms, only characters that do not have the high-order
+bit set (i.e. whose ordinals are strict ASCII, 0 - 127)
+are invariant, and the documentation and comments in the code
+may assume that,
+often referring to something like, say, C<hibit>.
+The situation differs and is not so simple on EBCDIC machines, but as long as
+the code itself uses the C<NATIVE_IS_INVARIANT()> macro appropriately, it
+works, even if the comments are wrong.
+
+=item *
+
+Assuming the character set is just ASCII
+
+ASCII is a 7 bit encoding, but bytes have 8 bits in them. The 128 extra
+characters have different meanings depending on the locale. Absent a locale,
+currently these extra characters are generally considered to be unassigned,
+and this has presented some problems.
+This is scheduled to be changed in 5.12 so that these characters will
+be considered to be Latin-1 (ISO-8859-1).
+
+=item *
+
Mixing #define and #ifdef
#define BURGLE(x) ... \
=item *
-Adding stuff after #endif or #else
+Adding non-comment stuff after #endif or #else
#ifdef SNOSH
...
=item *
-Binding together several statements
+Binding together several statements in a macro
Use the macros STMT_START and STMT_END.
#endif
How does the HAS_QUUX become defined where it needs to be? Well, if
-Foonix happens to be UNIXy enought to be able to run the Configure
+Foonix happens to be UNIXy enough to be able to run the Configure
script, and Configure has been taught about detecting and testing
quux(), the HAS_QUUX will be correctly defined. In other platforms,
the corresponding configuration step will hopefully do the same.
=back
+=head2 Problematic System Interfaces
+
+=over 4
+
+=item *
+
+malloc(0), realloc(0), calloc(0, 0) are non-portable. To be portable
+allocate at least one byte. (In general you should rarely need to
+work at this low level, but instead use the various malloc wrappers.)
+
+=item *
+
+snprintf() - the return type is unportable. Use my_snprintf() instead.
+
+=back
+
=head2 Security problems
Last but not least, here are various tips for safer coding.
is a good sign of these. Fixing these leaks is non-trivial,
unfortunately, but they must be fixed eventually.
+B<NOTE 4>: L<DynaLoader> will not clean up after itself completely
+unless Perl is built with the Configure option
+C<-Accflags=-DDL_UNLOAD_ALL_AT_EXIT>.
+
=head2 Rational Software's Purify
Purify is a commercial tool that is helpful in identifying
Once you've compiled a perl suitable for Purify'ing, then you
can just:
- make pureperl
+ make pureperl
which creates a binary named 'pureperl' that has been Purify'ed.
This binary is used in place of the standard 'perl' binary
make pureperl
cd t
- ../pureperl -I../lib harness
+ ../pureperl -I../lib harness
which would run Perl on test.pl and report any memory problems.
You should add -DPURIFY to the DEFINES line so the DEFINES
line looks something like:
- DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
+ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT $(CRYPT_FLAG) -DPURIFY=1
to disable Perl's arena memory allocation functions, as
well as to force use of memory allocation functions derived
cd win32
make
cd ../t
- purify ../perl -I../lib harness
+ purify ../perl -I../lib harness
which would instrument Perl in memory, run Perl on test.pl,
then finally report any memory problems.
=head2 valgrind
The excellent valgrind tool can be used to find out both memory leaks
-and illegal memory accesses. As of August 2003 it unfortunately works
-only on x86 (ELF) Linux. The special "test.valgrind" target can be used
-to run the tests under valgrind. Found errors and memory leaks are
-logged in files named F<test.valgrind>.
+and illegal memory accesses. As of version 3.3.0, Valgrind only
+supports Linux on x86, x86-64 and PowerPC. The special "test.valgrind"
+target can be used to run the tests under valgrind. Found errors
+and memory leaks are logged in files named F<testfile.valgrind>.
+
+Valgrind also provides a cachegrind tool, invoked on perl as:
+
+ VG_OPTS=--tool=cachegrind make test.valgrind
As system libraries (most notably glibc) are also triggering errors,
valgrind allows to suppress such errors using suppression files. The
converts C<new_SV()> from a macro into a real function, so you can use
your favourite debugger to discover where those pesky SVs were allocated.
+If you see that you're leaking memory at runtime, but neither valgrind
+nor C<-DDEBUG_LEAKING_SCALARS> will find anything, you're probably
+leaking SVs that are still reachable and will be properly cleaned up
+during destruction of the interpreter. In such cases, using the C<-Dm>
+switch can point you to the source of the leak. If the executable was
+built with C<-DDEBUG_LEAKING_SCALARS>, C<-Dm> will output SV allocations
+in addition to memory allocations. Each SV allocation has a distinct
+serial number that will be written on creation and destruction of the SV.
+So if you're executing the leaking code in a loop, you need to look for
+SVs that are created, but never destroyed between each cycle. If such an
+SV is found, set a conditional breakpoint within C<new_SV()> and make it
+break only when C<PL_sv_serial> is equal to the serial number of the
+leaking SV. Then you will catch the interpreter in exactly the state
+where the leaking SV is allocated, which is sufficient in many cases to
+find the source of the leak.
+
+As C<-Dm> is using the PerlIO layer for output, it will by itself
+allocate quite a bunch of SVs, which are hidden to avoid recursion.
+You can bypass the PerlIO layer if you use the SV logging provided
+by C<-DPERL_MEM_LOG> instead.
+
=head2 PERL_MEM_LOG
If compiled with C<-DPERL_MEM_LOG>, all Newx() and Renew() allocations
and at a higher level (the C<-Dm> is directly at the point of C<malloc()>,
while the C<PERL_MEM_LOG> is at the level of C<New()>).
+In addition to memory allocations, SV allocations will be logged, just as
+with C<-Dm>. However, since the logging doesn't use PerlIO, all SV allocations
+are logged and no extra SV allocations are introduced by enabling the logging.
+If compiled with C<-DDEBUG_LEAKING_SCALARS>, the serial number for each SV
+allocation is also logged.
+
+You can control the logging from your environment if you compile with
+C<-DPERL_MEM_LOG_ENV>. Then you need to explicitly set C<PERL_MEM_LOG> and/or
+C<PERL_SV_LOG> to a non-zero value to enable logging of memory and/or SV
+allocations.
+
=head2 Profiling
Depending on your platform there are various of profiling Perl.
For more detailed explanation of the available commands and output
formats, see your own local documentation of gprof.
+quick hint:
+
+ $ sh Configure -des -Dusedevel -Doptimize='-g' -Accflags='-pg' -Aldflags='-pg' && make
+ $ ./perl someprog # creates gmon.out in current directory
+ $ gprof perl > out
+ $ view out
+
=head2 GCC gcov Profiling
Starting from GCC 3.0 I<basic block profiling> is officially available
http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html#SEC132
+quick hint:
+
+ $ sh Configure -des -Doptimize='-g' -Accflags='-fprofile-arcs -ftest-coverage' \
+ -Aldflags='-fprofile-arcs -ftest-coverage' && make perl.gcov
+ $ rm -f regexec.c.gcov regexec.gcda
+ $ ./perl.gcov
+ $ gcov regexec.c
+ $ view regexec.c.gcov
+
=head2 Pixie Profiling
Pixie is a profiling tool available on IRIX and Tru64 (aka Digital
or 0xEFEFEFEF, you may be seeing the effect of the Poison() macros,
see L<perlclib>.
+=item *
+
+Under ithreads the optree is read only. If you want to enforce this, to check
+for write accesses from buggy code, compile with C<-DPL_OP_SLAB_ALLOC> to
+enable the OP slab allocator and C<-DPERL_DEBUG_READONLY_OPS> to enable code
+that allocates op memory via C<mmap>, and sets it read-only at run time.
+Any write access to an op results in a C<SIGBUS> and abort.
+
+This code is intended for development only, and may not be portable even to
+all Unix variants. Also, it is an 80% solution, in that it isn't able to make
+all ops read only. Specifically it
+
+=over
+
+=item 1
+
+Only sets read-only on all slabs of ops at C<CHECK> time, hence ops allocated
+later via C<require> or C<eval> will be re-write
+
+=item 2
+
+Turns an entire slab of ops read-write if the refcount of any op in the slab
+needs to be decreased.
+
+=item 3
+
+Turns an entire slab of ops read-write if any op from the slab is freed.
+
+=back
+
+It's not possible to turn the slabs to read-only after an action requiring
+read-write access, as either can happen during op tree building time, so
+there may still be legitimate write access.
+
+However, as an 80% solution it is still effective, as currently it catches
+a write access during the generation of F<Config.pm>, which means that we
+can't yet build F<perl> with this enabled.
+
=back
+
=head1 CONCLUSION
We've had a brief look around the Perl source, how to maintain quality
I'd now suggest you read over those references again, and then, as soon
as possible, get your hands dirty. The best way to learn is by doing,
-so:
+so:
=over 3
If you can do these things, you've started on the long road to Perl porting.
Thanks for wanting to help make Perl better - and happy hacking!
+=head2 Metaphoric Quotations
+
+If you recognized the quote about the Road above, you're in luck.
+
+Most software projects begin each file with a literal description of each
+file's purpose. Perl instead begins each with a literary allusion to that
+file's purpose.
+
+Like chapters in many books, all top-level Perl source files (along with a
+few others here and there) begin with an epigramic inscription that alludes,
+indirectly and metaphorically, to the material you're about to read.
+
+Quotations are taken from writings of J.R.R Tolkien pertaining to his
+Legendarium, almost always from I<The Lord of the Rings>. Chapters and
+page numbers are given using the following editions:
+
+=over 4
+
+=item *
+
+I<The Hobbit>, by J.R.R. Tolkien. The hardcover, 70th-anniversary
+edition of 2007 was used, published in the UK by Harper Collins Publishers
+and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lord of the Rings>, by J.R.R. Tolkien. The hardcover,
+50th-anniversary edition of 2004 was used, published in the UK by Harper
+Collins Publishers and in the US by the Houghton Mifflin Company.
+
+=item *
+
+I<The Lays of Beleriand>, by J.R.R. Tolkien and published posthumously by his
+son and literary executor, C.J.R. Tolkien, being the 3rd of the 12 volumes
+in Christopher's mammoth I<History of Middle Earth>. Page numbers derive
+from the hardcover edition, first published in 1983 by George Allen &
+Unwin; no page numbers changed for the special 3-volume omnibus edition of
+2002 or the various trade-paper editions, all again now by Harper Collins
+or Houghton Mifflin.
+
+=back
+
+Other JRRT books fair game for quotes would thus include I<The Adventures of
+Tom Bombadil>, I<The Silmarillion>, I<Unfinished Tales>, and I<The Tale of
+the Children of Hurin>, all but the first posthumously assembled by CJRT.
+But I<The Lord of the Rings> itself is perfectly fine and probably best to
+quote from, provided you can find a suitable quote there.
+
+So if you were to supply a new, complete, top-level source file to add to
+Perl, you should conform to this peculiar practice by yourself selecting an
+appropriate quotation from Tolkien, retaining the original spelling and
+punctuation and using the same format the rest of the quotes are in.
+Indirect and oblique is just fine; remember, it's a metaphor, so being meta
+is, after all, what it's for.
+
=head1 AUTHOR
This document was written by Nathan Torkington, and is maintained by
the perl5-porters mailing list.
+
projects / p5sagit/p5-mst-13.2.git / blobdiff