X-Git-Url: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?a=blobdiff_plain;f=pod%2Fperltodo.pod;h=2ab61f30521231de933ea77bc5c2cc7d104cd57c;hb=002b9267a385cf8ff0e7534241cdf3798da8636c;hp=0f4ea6353dfc906ea3b5f013dad86f5dc9da8448;hpb=3298bd4d0902a683adad1b8abafe8009c3a99cd9;p=p5sagit%2Fp5-mst-13.2.git

diff --git a/pod/perltodo.pod b/pod/perltodo.pod
index 0f4ea63..2ab61f3 100644
--- a/pod/perltodo.pod
+++ b/pod/perltodo.pod
@@ -4,810 +4,674 @@ perltodo - Perl TO-DO List
 
 =head1 DESCRIPTION
 
-This is a list of wishes for Perl.  Send updates to
-I<perl5-porters@perl.org>.  If you want to work on any of these
-projects, be sure to check the perl5-porters archives for past ideas,
-flames, and propaganda.  This will save you time and also prevent you
-from implementing something that Larry has already vetoed.  One set
-of archives may be found at:
+This is a list of wishes for Perl. The tasks we think are smaller or easier
+are listed first. Anyone is welcome to work on any of these, but it's a good
+idea to first contact I<perl5-porters@perl.org> to avoid duplication of
+effort. By all means contact a pumpking privately first if you prefer.
 
-    http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/
-
-=head1 To do during 5.6.x
-
-=head2 Support for I/O disciplines
-
-C<perlio> provides this, but the interface could be a lot more
-straightforward.
-
-=head2 Autoload bytes.pm
-
-When the lexer sees, for instance, C<bytes::length>, it should
-automatically load the C<bytes> pragma.
-
-=head2 Make "\u{XXXX}" et al work
-
-Danger, Will Robinson! Discussing the semantics of C<"\x{F00}">,
-C<"\xF00"> and C<"\U{F00}"> on P5P I<will> lead to a long and boring
-flamewar.
-
-=head2 Create a char *sv_pvprintify(sv, STRLEN *lenp, UV flags)
-
-For displaying PVs with control characters, embedded nulls, and Unicode.
-This would be useful for printing warnings, or data and regex dumping,
-not_a_number(), and so on.
-
-Requirements: should handle both byte and UTF8 strings.  isPRINT()
-characters printed as-is, character less than 256 as \xHH, Unicode
-characters as \x{HHH}.  Don't assume ASCII-like, either, get somebody
-on EBCDIC to test the output.
-
-Possible options, controlled by the flags:
-- whitespace (other than ' ' of isPRINT()) printed as-is
-- use isPRINT_LC() instead of isPRINT()
-- print control characters like this: "\cA"
-- print control characters like this: "^A"
-- non-PRINTables printed as '.' instead of \xHH
-- use \OOO instead of \xHH
-- use the C/Perl-metacharacters like \n, \t
-- have a maximum length for the produced string (read it from *lenp)
-- append a "..." to the produced string if the maximum length is exceeded
-- really fancy: print unicode characters as \N{...}
-
-=head2 Overloadable regex assertions
+Whilst patches to make the list shorter are most welcome, ideas to add to
+the list are also encouraged. Check the perl5-porters archives for past
+ideas, and any discussion about them. One set of archives may be found at:
 
-This may or may not be possible with the current regular expression
-engine. The idea is that, for instance, C<\b> needs to be
-algorithmically computed if you're dealing with Thai text. Hence, the
-B<\b> assertion wants to be overloaded by a function.
+    http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/
 
-=head2 Unicode case mappings 
+What can we offer you in return? Fame, fortune, and everlasting glory? Maybe
+not, but if your patch is incorporated, then we'll add your name to the
+F<AUTHORS> file, which ships in the official distribution. How many other
+programming languages offer you 1 line of immortality?
 
-    Case Mappings? http://www.unicode.org/unicode/reports/tr21/
+=head1 The roadmap to 5.10
 
-=head2 Unicode regular expression character classes
+The roadmap to 5.10 envisages feature based releases, as various items in this
+TODO are completed.
 
-They have some tricks Perl doesn't yet implement like character
-class subtraction.
+=head2 Needed for a 5.9.3 release
 
-	http://www.unicode.org/unicode/reports/tr18/
+=over
 
-=head2 use Thread for iThreads
+=item *
+Implement L</lexical pragmas>
 
-Artur Bergman's C<iThreads> module is a start on this, but needs to
-be more mature.
+=back
 
-=head2 make perl_clone optionally clone ops
+=head2 Needed for a 5.9.4 release
 
-So that pseudoforking, mod_perl, iThreads and nvi will work properly
-(but not as efficiently) until the regex engine is fixed to be threadsafe.
+=over
 
-=head2 Work out exit/die semantics for threads
+=item *
+Review assertions. Review syntax to combine assertions. Can assertions  take
+advantage of the lexical pragams work? L</What hooks would assertions need?>
 
-=head2 Typed lexicals for compiler
+=back
 
-=head2 Compiler workarounds for Win32
+=head2 Needed for a 5.9.5 release
 
-=head2 AUTOLOADing in the compiler
+=over
 
-=head2 Fixing comppadlist when compiling
+=item *
+Implement L</_ prototype character>
 
-=head2 Cleaning up exported namespace
+=item *
+Implement L</state variables>
 
-=head2 Complete signal handling
+=back
 
-Add C<PERL_ASYNC_CHECK> to opcodes which loop; replace C<sigsetjmp> with
-C<sigjmp>; check C<wait> for signal safety.
+=head2 Needed for a 5.9.6 release
 
-=head2 Out-of-source builds
+Stabilisation. If all goes well, this will be the equivalent of a 5.10-beta.
 
-This was done for 5.6.0, but needs reworking for 5.7.x
+=head1 Tasks that only need Perl knowledge
 
-=head2 POSIX realtime support
+=head2 common test code for timed bail out
 
-POSIX 1003.1 1996 Edition support--realtime stuff: POSIX semaphores,
-message queues, shared memory, realtime clocks, timers, signals (the
-metaconfig units mostly already exist for these)
+Write portable self destruct code for tests to stop them burning CPU in
+infinite loops. This needs to avoid using alarm, as some of the tests are
+testing alarm/sleep or timers.
 
-=head2 UNIX98 support
+=head2 POD -> HTML conversion in the core still sucks
 
-Reader-writer locks, realtime/asynchronous IO
+Which is crazy given just how simple POD purports to be, and how simple HTML
+can be. It's not actually I<as> simple as it sounds, particularly with the
+flexibility POD allows for C<=item>, but it would be good to improve the
+visual appeal of the HTML generated, and to avoid it having any validation
+errors. See also L</make HTML install work>, as the layout of installation tree
+is needed to improve the cross-linking.
 
-=head2 IPv6 Support
+The addition of C<Pod::Simple> and its related modules may make this task
+easier to complete.
 
-There are non-core modules, such as C<Net::IPv6>, but these will need
-integrating when IPv6 actually starts to really happen.  See RFC 2292
-and RFC 2553.
+=head2 Parallel testing
 
-=head2 Long double conversion
+The core regression test suite is getting ever more comprehensive, which has
+the side effect that it takes longer to run. This isn't so good. Investigate
+whether it would be feasible to give the harness script the B<option> of
+running sets of tests in parallel. This would be useful for tests in
+F<t/op/*.t> and F<t/uni/*.t> and maybe some sets of tests in F<lib/>.
 
-Floating point formatting is still causing some weird test failures.
+Questions to answer
 
-=head2 Locales
+=over 4
 
-Locales and Unicode interact with each other in unpleasant ways.
-One possible solution would be to adopt/support ICU:
+=item 1
 
-	http://oss.software.ibm.com/developerworks/opensource/icu/project/
+How does screen layout work when you're running more than one test?
 
-=head2 Thread-safe regexes
+=item 2
 
-The regular expression engine is currently non-threadsafe.
+How does the caller of test specify how many tests to run in parallel?
 
-=head2 Arithmetic on non-Arabic numerals
+=item 3
 
-C<[1234567890]> aren't the only numerals any more.
+How do setup/teardown tests identify themselves?
 
-=head2 POSIX Unicode character classes
+=back
 
-([=a=] for equivalance classes, [.ch.] for collation.)
-These are dependent on Unicode normalization and collation.
+Pugs already does parallel testing - can their approach be re-used?
 
-=head2 Factoring out common suffices/prefices in regexps (trie optimization)
+=head2 Make Schwern poorer
 
-Currently, the user has to optimize C<foo|far> and C<foo|goo> into
-C<f(?:oo|ar)> and C<[fg]oo> by hand; this could be done automatically.
+We should have for everything. When all the core's modules are tested,
+Schwern has promised to donate to $500 to TPF. We may need volunteers to
+hold him upside down and shake vigorously in order to actually extract the
+cash.
 
-=head2 Security audit shipped utilities
+See F<t/lib/1_compile.t> for the 3 remaining modules that need tests.
 
-All the code we ship with Perl needs to be sensible about temporary file
-handling, locking, input validation, and so on.
+=head2 Improve the coverage of the core tests
 
-=head2 Custom opcodes
+Use Devel::Cover to ascertain the core's test coverage, then add tests that
+are currently missing.
 
-Have a way to introduce user-defined opcodes without the subroutine call
-overhead of an XSUB; the user should be able to create PP code. Simon
-Cozens has some ideas on this.
+=head2 test B
 
-=head2 spawnvp() on Win32
+A full test suite for the B module would be nice.
 
-Win32 has problems spawning processes, particularly when the arguments
-to the child process contain spaces, quotes or tab characters.
+=head2 A decent benchmark
 
-=head2 DLL Versioning
+C<perlbench> seems impervious to any recent changes made to the perl core. It
+would be useful to have a reasonable general benchmarking suite that roughly
+represented what current perl programs do, and measurably reported whether
+tweaks to the core improve, degrade or don't really affect performance, to
+guide people attempting to optimise the guts of perl. Gisle would welcome
+new tests for perlbench.
 
-Windows needs a way to know what version of a XS or C<libperl> DLL it's
-loading.
+=head2 fix tainting bugs
 
-=head2 Introduce @( and @)
+Fix the bugs revealed by running the test suite with the C<-t> switch (via
+C<make test.taintwarn>).
 
-C<$(> may return "foo bar baz". Unfortunately, since groups can
-theoretically have spaces in their names, this could be one, two or
-three groups.
+=head2 Dual life everything
 
-=head2 Floating point handling
+As part of the "dists" plan, anything that doesn't belong in the smallest perl
+distribution needs to be dual lifed. Anything else can be too. Figure out what
+changes would be needed to package that module and its tests up for CPAN, and
+do so. Test it with older perl releases, and fix the problems you find.
 
-C<NaN> and C<inf> support is particularly troublesome.
-(fp_classify(), fp_class(), fp_class_d(), class(), isinf(),
-isfinite(), finite(), isnormal(), unordered(), <ieeefp.h>,
-<fp_class.h> (there are metaconfig units for all these) (I think),
-fp_setmask(), fp_getmask(), fp_setround(), fp_getround()
-(no metaconfig units yet for these).  Don't forget finitel(), fp_classl(),
-fp_class_l(), (yes, both do, unfortunately, exist), and unorderedl().)
+=head2 Improving C<threads::shared>
 
-As of Perl 5.6.1 is a Perl macro, Perl_isnan().
+Investigate whether C<threads::shared> could share aggregates properly with
+only Perl level changes to shared.pm
 
-=head2 IV/UV preservation
+=head2 POSIX memory footprint
 
-Nicholas Clark has done a lot of work on this, but work is continuing.
-C<+>, C<-> and C<*> work, but guards need to be in place for C<%>, C</>,
-C<&>, C<oct>, C<hex> and C<pack>.
+Ilya observed that use POSIX; eats memory like there's no tomorrow, and at
+various times worked to cut it down. There is probably still fat to cut out -
+for example POSIX passes Exporter some very memory hungry data structures.
 
-=head2 Replace pod2html with something using Pod::Parser
 
-The CPAN module C<Malik::Pod::Html> may be a more suitable basis for a
-C<pod2html> convertor; the current one duplicates the functionality
-abstracted in C<Pod::Parser>, which makes updating the POD language
-difficult.
 
-=head2 Automate module testing on CPAN
 
-When a new Perl is being beta tested, porters have to manually grab
-their favourite CPAN modules and test them - this should be done
-automatically.
 
-=head2 sendmsg and recvmsg
 
-We have all the other BSD socket functions but these. There are
-metaconfig units for these functions which can be added. To avoid these
-being new opcodes, a solution similar to the way C<sockatmark> was added
-would be preferable. (Autoload the C<IO::whatever> module.)
 
-=head2 Rewrite perlre documentation
+=head1 Tasks that need a little sysadmin-type knowledge
 
-The new-style patterns need full documentation, and the whole document
-needs to be a lot clearer.
+Or if you prefer, tasks that you would learn from, and broaden your skills
+base...
 
-=head2 Convert example code to IO::Handle filehandles
+=head2 Relocatable perl
 
-=head2 Document Win32 choices
+The C level patches needed to create a relocatable perl binary are done, as
+is the work on F<Config.pm>. All that's left to do is the C<Configure> tweaking
+to let people specify how they want to do the install.
 
-=head2 Check new modules
+=head2 make HTML install work
 
-=head2 Make roffitall find pods and libs itself
+There is an C<installhtml> target in the Makefile. It's marked as
+"experimental". It would be good to get this tested, make it work reliably, and
+remove the "experimental" tag. This would include
 
-Simon Cozens has done some work on this but it needs a rethink.
+=over 4
 
-=head1 To do at some point
+=item 1
 
-These are ideas that have been regularly tossed around, that most
-people believe should be done maybe during 5.8.x
+Checking that cross linking between various parts of the documentation works.
+In particular that links work between the modules (files with POD in F<lib/>)
+and the core documentation (files in F<pod/>)
 
-=head2 Remove regular expression recursion
+=item 2
 
-Because the regular expression engine is recursive, badly designed
-expressions can lead to lots of recursion filling up the stack. Ilya
-claims that it is easy to convert the engine to being iterative, but
-this has still not yet been done. There may be a regular expression
-engine hit squad meeting at TPC5.
+Work out how to split C<perlfunc> into chunks, preferably one per function
+group, preferably with general case code that could be used elsewhere.
+Challenges here are correctly identifying the groups of functions that go
+together, and making the right named external cross-links point to the right
+page. Things to be aware of are C<-X>, groups such as C<getpwnam> to
+C<endservent>, two or more C<=items> giving the different parameter lists, such
+as
 
-=head2 Memory leaks after failed eval
+    =item substr EXPR,OFFSET,LENGTH,REPLACEMENT
+    
+    =item substr EXPR,OFFSET,LENGTH
+    
+    =item substr EXPR,OFFSET
 
-Perl will leak memory if you C<eval "hlagh hlagh hlagh hlagh">. This is
-partially because it attempts to build up an op tree for that code and
-doesn't properly free it. The same goes for non-syntactically-correct
-regular expressions. Hugo looked into this, but decided it needed a
-mark-and-sweep GC implementation. 
+and different parameter lists having different meanings. (eg C<select>)
 
-Alan notes that: The basic idea was to extend the parser token stack
-(C<YYSTYPE>) to include a type field so we knew what sort of thing each
-element of the stack was.  The F<<perly.c> code would then have to be
-postprocessed to record the type of each entry on the stack as it was
-created, and the parser patched so that it could unroll the stack
-properly on error.
+=back
 
-This is possible to do, but would be pretty messy to implement, as it
-would rely on even more sed hackery in F<perly.fixer>.
+=head2 compressed man pages
 
-=head2 pack "(stuff)*"
+Be able to install them. This would probably need a configure test to see how
+the system does compressed man pages (same directory/different directory?
+same filename/different filename), as well as tweaking the F<installman> script
+to compress as necessary.
 
-That's to say, C<pack "(sI)40"> would be the same as C<pack "sI"x40>
+=head2 Add a code coverage target to the Makefile
 
-=head2 bitfields in pack
+Make it easy for anyone to run Devel::Cover on the core's tests. The steps
+to do this manually are roughly
 
-=head2 Cross compilation
+=over 4
 
-Make Perl buildable with a cross-compiler. This will play havoc with
-Configure, which needs to how how the target system will respond to
-its tests; maybe C<microperl> will be a good starting point here.
-(Indeed, Bart Schuller reports that he compiled up C<microperl> for
-the Agenda PDA and it works fine.)  A really big spanner in the works
-is the bootstrapping build process of Perl: if the filesystem the
-target systems sees is not the same what the build host sees, various
-input, output, and (Perl) library files need to be copied back and forth.
+=item *
 
-As of 5.8.0 Configure mostly works for cross-compilation
-(used successfully for iPAQ Linux), miniperl gets built,
-but then building DynaLoader (and other extensions) fails
-since MakeMaker knows nothing of cross-compilation.
-(See INSTALL/Cross-compilation for the state of things.)
+do a normal C<Configure>, but include Devel::Cover as a module to install
+(see F<INSTALL> for how to do this)
 
-=head2 Perl preprocessor / macros
+=item *
 
-Source filters help with this, but do not get us all the way. For
-instance, it should be possible to implement the C<??> operator somehow;
-source filters don't (quite) cut it.
+    make perl
 
-=head2 Perl lexer in Perl
+=item *
 
-Damian Conway is planning to work on this, but it hasn't happened yet.
+    cd t; HARNESS_PERL_SWITCHES=-MDevel::Cover ./perl -I../lib harness
 
-=head2 Using POSIX calls internally
+=item *
 
-When faced with a BSD vs. SySV -style interface to some library or
-system function, perl's roots show in that it typically prefers the BSD
-interface (but falls back to the SysV one).  One example is getpgrp().
-Other examples include C<memcpy> vs. C<bcopy>.  There are others, mostly in
-F<<pp_sys.c>.
+Process the resulting Devel::Cover database
 
-Mostly, this item is a suggestion for which way to start a journey into
-an C<#ifdef> forest.  It is not primarily a suggestion to eliminate any of
-the C<#ifdef> forests.
+=back
 
-POSIX calls are perhaps more likely to be portable to unexpected
-architectures. They are also perhaps more likely to be actively
-maintained by a current vendor.  They are also perhaps more likely to be
-available in thread-safe versions, if appropriate.
+This just give you the coverage of the F<.pm>s. To also get the C level
+coverage you need to
 
-=head2 -i rename file when changed
+=over 4
 
-It's only necessary to rename a file when inplace editing when the file
-has changed. Detecting a change is perhaps the difficult bit.
+=item *
 
-=head2 All ARGV input should act like E<lt>E<gt>
+Additionally tell C<Configure> to use the appropriate C compiler flags for
+C<gcov>
 
-=head2 Support for rerunning debugger
+=item *
 
-There should be a way of restarting the debugger on demand.
+    make perl.gcov
 
-=head2 Test Suite for the Debugger
+(instead of C<make perl>)
 
-The debugger is a complex piece of software and fixing something
-here may inadvertently break something else over there.  To tame
-this chaotic behaviour, a test suite is necessary. 
+=item *
 
-=head2 my sub foo { }
+After running the tests run C<gcov> to generate all the F<.gcov> files.
+(Including down in the subdirectories of F<ext/>
 
-The basic principle is sound, but there are problems with the semantics
-of self-referential and mutually referential lexical subs: how to
-declare the subs?
+=item *
 
-=head2 One-pass global destruction
+(From the top level perl directory) run C<gcov2perl> on all the C<.gcov> files
+to get their stats into the cover_db directory.
 
-Sweeping away all the allocated memory in one go is a laudable goal, but
-it's difficult and in most cases, it's easier to let the memory get
-freed by exiting.
+=item *
 
-=head2 Rewrite regexp parser
+Then process the Devel::Cover database
 
-There has been talk recently of rewriting the regular expression parser
-to produce an optree instead of a chain of opcodes; it's unclear whether
-or not this would be a win. 
+=back
 
-=head2 Cache recently used regexps
+It would be good to add a single switch to C<Configure> to specify that you
+wanted to perform perl level coverage, and another to specify C level
+coverage, and have C<Configure> and the F<Makefile> do all the right things
+automatically.
 
-This is to speed up 
+=head2 Make Config.pm cope with differences between build and installed perl
 
-    for my $re (@regexps) {
-        $matched++ if /$re/
-    }
+Quite often vendors ship a perl binary compiled with their (pay-for)
+compilers.  People install a free compiler, such as gcc. To work out how to
+build extensions, Perl interrogates C<%Config>, so in this situation
+C<%Config> describes compilers that aren't there, and extension building
+fails. This forces people into choosing between re-compiling perl themselves
+using the compiler they have, or only using modules that the vendor ships.
 
-C<qr//> already gives us a way of saving compiled regexps, but it should
-be done automatically.
+It would be good to find a way teach C<Config.pm> about the installation setup,
+possibly involving probing at install time or later, so that the C<%Config> in
+a binary distribution better describes the installed machine, when the
+installed machine differs from the build machine in some significant way.
 
-=head2 Re-entrant functions
+=head2 make parallel builds work
 
-Add configure probes for C<_r> forms of system calls and fit them to the
-core. Unfortunately, calling conventions for these functions and not
-standardised.
+Currently parallel builds (such as C<make -j3>) don't work reliably. We believe
+that this is due to incomplete dependency specification in the F<Makefile>.
+It would be good if someone were able to track down the causes of these
+problems, so that parallel builds worked properly.
 
-=head2 Cross-compilation support
+=head2 linker specification files
+
+Some platforms mandate that you provide a list of a shared library's external
+symbols to the linker, so the core already has the infrastructure in place to
+do this for generating shared perl libraries. My understanding is that the
+GNU toolchain can accept an optional linker specification file, and restrict
+visibility just to symbols declared in that file. It would be good to extend
+F<makedef.pl> to support this format, and to provide a means within
+C<Configure> to enable it. This would allow Unix users to test that the
+export list is correct, and to build a perl that does not pollute the global
+namespace with private symbols.
+
+
+
+
+=head1 Tasks that need a little C knowledge
+
+These tasks would need a little C knowledge, but don't need any specific
+background or experience with XS, or how the Perl interpreter works
 
-Bart Schuller reports that using C<microperl> and a cross-compiler, he
-got Perl working on the Agenda PDA. However, one cannot build a full
-Perl because Configure needs to get the results for the target platform,
-for the host.
+=head2 Make it clear from -v if this is the exact official release
+
+Currently perl from C<p4>/C<rsync> ships with a F<patchlevel.h> file that
+usually defines one local patch, of the form "MAINT12345" or "RC1". The output
+of perl -v doesn't report that a perl isn't an official release, and this
+information can get lost in bugs reports. Because of this, the minor version
+isn't bumped up until RC time, to minimise the possibility of versions of perl
+escaping that believe themselves to be newer than they actually are.
+
+It would be useful to find an elegant way to have the "this is an interim
+maintenance release" or "this is a release candidate" in the terse -v output,
+and have it so that it's easy for the pumpking to remove this just as the
+release tarball is rolled up. This way the version pulled out of rsync would
+always say "I'm a development release" and it would be safe to bump the
+reported minor version as soon as a release ships, which would aid perl
+developers.
 
-=head2 Bit-shifting bitvectors
+This task is really about thinking of an elegant way to arrange the C source
+such that it's trivial for the Pumpking to flag "this is an official release"
+when making a tarball, yet leave the default source saying "I'm not the
+official release".
 
-Given:
+=head2 Tidy up global variables
 
-    vec($v, 1000, 1) = 1;
+There's a note in F<intrpvar.h>
 
-One should be able to do
+  /* These two variables are needed to preserve 5.8.x bincompat because
+     we can't change function prototypes of two exported functions.
+     Probably should be taken out of blead soon, and relevant prototypes
+     changed.  */
 
-    $v <<= 1;
+So doing this, and removing any of the unused variables still present would
+be good.
 
-and have the 999'th bit set.
+=head2 Ordering of "global" variables.
 
-Currently if you try with shift bitvectors you shift the NV/UV, instead
-of the bits in the PV.  Not very logical.
+F<thrdvar.h> and F<intrpvarh> define the "global" variables that need to be
+per-thread under ithreads, where the variables are actually elements in a
+structure. As C dictates, the variables must be laid out in order of
+declaration. There is a comment
+C</* Important ones in the first cache line (if alignment is done right) */>
+which implies that at some point in the past the ordering was carefully chosen
+(at least in part). However, it's clear that the ordering is less than perfect,
+as currently there are things such as 7 C<bool>s in a row, then something
+typically requiring 4 byte alignment, and then an odd C<bool> later on.
+(C<bool>s are typically defined as C<char>s). So it would be good for someone
+to review the ordering of the variables, to see how much alignment padding can
+be removed.
 
-=head2 debugger pragma
+=head2 bincompat functions
 
-The debugger is implemented in Perl in F<perl5db.pl>; turning it into a
-pragma should be easy, but making it work lexically might be more
-difficult. Fiddling with C<$^P> would be necessary.
+There are lots of functions which are retained for binary compatibility.
+Clean these up. Move them to mathom.c, and don't compile for blead?
 
-=head2 use less pragma
+=head2 am I hot or not?
 
-Identify areas where speed/memory tradeoffs can be made and have a hint
-to switch between them.
+The idea of F<pp_hot.c> is that it contains the I<hot> ops, the ops that are
+most commonly used. The idea is that by grouping them, their object code will
+be adjacent in the executable, so they have a greater chance of already being
+in the CPU cache (or swapped in) due to being near another op already in use.
 
-=head2 switch structures
+Except that it's not clear if these really are the most commonly used ops. So
+anyone feeling like exercising their skill with coverage and profiling tools
+might want to determine what ops I<really> are the most commonly used. And in
+turn suggest evictions and promotions to achieve a better F<pp_hot.c>.
 
-Although we have C<Switch.pm> in core, Larry points to the dormant
-C<nswitch> and C<cswitch> ops in F<pp.c>; using these opcodes would be
-much faster.
+=head2 emulate the per-thread memory pool on Unix
 
-=head2 Cache eval tree
+For Windows, ithreads allocates memory for each thread from a separate pool,
+which it discards at thread exit. It also checks that memory is free()d to
+the correct pool. Neither check is done on Unix, so code developed there won't
+be subject to such strictures, so can harbour bugs that only show up when the
+code reaches Windows.
 
-=head2 rcatmaybe
+It would be good to be able to optionally emulate the Window pool system on
+Unix, to let developers who only have access to Unix, or want to use
+Unix-specific debugging tools, check for these problems. To do this would
+involve figuring out how the C<PerlMem_*> macros wrap C<malloc()> access, and
+providing a layer that records/checks the identity of the thread making the
+call, and recording all the memory allocated by each thread via this API so
+that it can be summarily free()d at thread exit. One implementation idea
+would be to increase the size of allocation, and store the C<my_perl> pointer
+(to identify the thread) at the start, along with pointers to make a linked
+list of blocks for this thread. To avoid alignment problems it would be
+necessary to do something like
 
-=head2 Shrink opcode tables
+  union memory_header_padded {
+    struct memory_header {
+      void *thread_id;   /* For my_perl */
+      void *next;        /* Pointer to next block for this thread */
+    } data;
+    long double padding; /* whatever type has maximal alignment constraint */
+  };
 
-=head2 Optimize away @_
 
-Look at the "reification" code in C<av.c>
+although C<long double> might not be the only type to add to the padding
+union.
 
-=head2 Prototypes versus indirect objects
+=head2 reduce duplication in sv_setsv_flags
 
-Currently, indirect object syntax bypasses prototype checks.
+C<Perl_sv_setsv_flags> has a comment
+C</* There's a lot of redundancy below but we're going for speed here */>
 
-=head2 Install HMTL
+Whilst this was true 10 years ago, the growing disparity between RAM and CPU
+speeds mean that the trade offs have changed. In addition, the duplicate code
+adds to the maintenance burden. It would be good to see how much of the
+redundancy can be pruned, particular in the less common paths. (Profiling
+tools at the ready...). For example, why does the test for
+"Can't redefine active sort subroutine" need to occur in two places?
 
-HTML versions of the documentation need to be installed by default; a
-call to C<installhtml> from C<installperl> may be all that's necessary.
 
-=head2 Prototype method calls
 
-=head2 Return context prototype declarations
 
-=head2 magic_setisa
+=head1 Tasks that need a knowledge of XS
 
-=head2 Garbage collection
+These tasks would need C knowledge, and roughly the level of knowledge of
+the perl API that comes from writing modules that use XS to interface to
+C.
 
-There have been persistent mumblings about putting a mark-and-sweep
-garbage detector into Perl; Alan Burlison has some ideas about this.
+=head2 IPv6
 
-=head2 IO tutorial
+Clean this up. Check everything in core works
 
-Mark-Jason Dominus has the beginnings of one of these.
+=head2 shrink C<GV>s, C<CV>s
 
-=head2 pack/unpack tutorial
+By removing unused elements and careful re-ordering, the structures for C<AV>s
+and C<HV>s have recently been shrunk considerably. It's probable that the same
+approach would find savings in C<GV>s and C<CV>s, if not all the other
+larger-than-C<PVMG> types.
 
-Simon Cozens has the beginnings of one of these.
+=head2 merge Perl_sv_2[inpu]v
 
-=head2 Rewrite perldoc
+There's a lot of code shared between C<Perl_sv_2iv_flags>,
+C<Perl_sv_2uv_flags>, C<Perl_sv_2nv>, and C<Perl_sv_2pv_flags>. It would be
+interesting to see if some of it can be merged into common shared static
+functions. In particular, C<Perl_sv_2uv_flags> started out as a cut&paste
+from C<Perl_sv_2iv_flags> around 5.005_50 time, and it may be possible to
+replace both with a single function that returns a value or union which is
+split out by the macros in F<sv.h>
 
-There are a few suggestions for what to do with C<perldoc>: maybe a
-full-text search, an index function, locating pages on a particular
-high-level subject, and so on.
+=head2 UTF8 caching code
 
-=head2 Install .3p manpages
+The string position/offset cache is not optional. It should be.
 
-This is a bone of contention; we can create C<.3p> manpages for each
-built-in function, but should we install them by default? Tcl does this,
-and it clutters up C<apropos>.
+=head2 Implicit Latin 1 => Unicode translation
 
-=head2 Unicode tutorial
+Conversions from byte strings to UTF-8 currently map high bit characters
+to Unicode without translation (or, depending on how you look at it, by
+implicitly assuming that the byte strings are in Latin-1). As perl assumes
+the C locale by default, upgrading a string to UTF-8 may change the
+meaning of its contents regarding character classes, case mapping, etc.
+This should probably emit a warning (at least).
 
-Simon Cozens promises to do this before he gets old.
+This task is incremental - even a little bit of work on it will help.
 
-=head2 Update POSIX.pm for 1003.1-2
+=head2 autovivification
 
-=head2 Retargetable installation
+Make all autovivification consistent w.r.t LVALUE/RVALUE and strict/no strict;
 
-Allow C<@INC> to be changed after Perl is built.
+This task is incremental - even a little bit of work on it will help.
 
-=head2 POSIX emulation on non-POSIX systems
+=head2 Unicode in Filenames
 
-Make C<POSIX.pm> behave as POSIXly as possible everywhere, meaning we
-have to implement POSIX equivalents for some functions if necessary.
+chdir, chmod, chown, chroot, exec, glob, link, lstat, mkdir, open,
+opendir, qx, readdir, readlink, rename, rmdir, stat, symlink, sysopen,
+system, truncate, unlink, utime, -X.  All these could potentially accept
+Unicode filenames either as input or output (and in the case of system
+and qx Unicode in general, as input or output to/from the shell).
+Whether a filesystem - an operating system pair understands Unicode in
+filenames varies.
 
-=head2 Rename Win32 headers
+Known combinations that have some level of understanding include
+Microsoft NTFS, Apple HFS+ (In Mac OS 9 and X) and Apple UFS (in Mac
+OS X), NFS v4 is rumored to be Unicode, and of course Plan 9.  How to
+create Unicode filenames, what forms of Unicode are accepted and used
+(UCS-2, UTF-16, UTF-8), what (if any) is the normalization form used,
+and so on, varies.  Finding the right level of interfacing to Perl
+requires some thought.  Remember that an OS does not implicate a
+filesystem.
 
-=head2 Finish off lvalue functions
+(The Windows -C command flag "wide API support" has been at least
+temporarily retired in 5.8.1, and the -C has been repurposed, see
+L<perlrun>.)
 
-They don't work in the debugger, and they don't work for list or hash
-slices.
+=head2 Unicode in %ENV
 
-=head2 Update sprintf documentation
+Currently the %ENV entries are always byte strings.
 
-Hugo van der Sanden plans to look at this.
+=head2 use less 'memory'
 
-=head2 Use fchown/fchmod internally
+Investigate trade offs to switch out perl's choices on memory usage.
+Particularly perl should be able to give memory back.
 
-This has been done in places, but needs a thorough code review.
-Also fchdir is available in some platforms.
+This task is incremental - even a little bit of work on it will help.
 
-=head1 Vague ideas
+=head2 Re-implement C<:unique> in a way that is actually thread-safe
 
-Ideas which have been discussed, and which may or may not happen.
+The old implementation made bad assumptions on several levels. A good 90%
+solution might be just to make C<:unique> work to share the string buffer
+of SvPVs. That way large constant strings can be shared between ithreads,
+such as the configuration information in F<Config>.
 
-=head2 ref() in list context
+=head2 Make tainting consistent
 
-It's unclear what this should do or how to do it without breaking old
-code.
+Tainting would be easier to use if it didn't take documented shortcuts and
+allow taint to "leak" everywhere within an expression.
 
-=head2 Make tr/// return histogram of characters in list context
+=head2 readpipe(LIST)
 
-There is a patch for this, but it may require Unicodification.
+system() accepts a LIST syntax (and a PROGRAM LIST syntax) to avoid
+running a shell. readpipe() (the function behind qx//) could be similarly
+extended.
 
-=head2 Compile to real threaded code
 
-=head2 Structured types
 
-=head2 Modifiable $1 et al.
 
-    ($x = "elephant") =~ /e(ph)/;
-    $1 = "g"; # $x = "elegant"
 
-What happens if there are multiple (nested?) brackets? What if the
-string changes between the match and the assignment?
+=head1 Tasks that need a knowledge of the interpreter
 
-=head2 Procedural interfaces for IO::*, etc.
+These tasks would need C knowledge, and knowledge of how the interpreter works,
+or a willingness to learn.
 
-Some core modules have been accused of being overly-OO. Adding
-procedural interfaces could demystify them.
+=head2 lexical pragmas
 
-=head2 RPC modules
+Reimplement the mechanism of lexical pragmas to be more extensible. Fix
+current pragmas that don't work well (or at all) with lexical scopes or in
+run-time eval(STRING) (C<sort>, C<re>, C<encoding> for example). MJD has a
+preliminary patch that implements this.
 
 =head2 Attach/detach debugger from running program
 
-With C<gdb>, you can attach the debugger to a running program if you
-pass the process ID. It would be good to do this with the Perl debugger
-on a running Perl program, although I'm not sure how it would be done.
-
-=head2 Alternative RE syntax module
-
-    use Regex::Newbie;
-    $re = Regex::Newbie->new
-                       ->start
-                       ->match("foo")
-                       ->repeat(Regex::Newbie->class("char"),3)
-                       ->end;
-    /$re/;
-
-=head2 GUI::Native
-
-A non-core module that would use "native" GUI to create graphical
-applications.
-
-=head2 foreach(reverse ...)
-
-Currently
-
-    foreach (reverse @_) { ... }
-
-puts C<@_> on the stack, reverses it putting the reversed version on the
-stack, then iterates forwards. Instead, it could be special-cased to put
-C<@_> on the stack then iterate backwards.
-
-=head2 Constant function cache
-
-=head2 Approximate regular expression matching
-
-=head1 Ongoing
-
-These items B<always> need doing:
-
-=head2 Update guts documentation
-
-Simon Cozens tries to do this when possible, and contributions to the
-C<perlapi> documentation is welcome.
-
-=head2 Add more tests
-
-Michael Schwern will donate $500 to Yet Another Society when all core
-modules have tests.
-
-=head2 Update auxiliary tools
-
-The code we ship with Perl should look like good Perl 5.
-
-=head1 Recently done things
-
-These are things which have been on the todo lists in previous releases
-but have recently been completed.
-
-=head2 Safe signal handling
-
-A new signal model went into 5.7.1 without much fanfare. Operations and
-C<malloc>s are no longer interrupted by signals, which are handled
-between opcodes. This means that C<PERL_ASYNC_CHECK> now actually does
-something. However, there are still a few things that need to be done.
-
-=head2 Tie Modules
-
-Modules which implement arrays in terms of strings, substrings or files
-can be found on the CPAN.
-
-=head2 gettimeofday
-
-C<Time::Hires> has been integrated into the core.
-
-=head2 setitimer and getimiter
-
-Adding C<Time::Hires> got us this too.
-
-=head2 Testing __DIE__ hook
-
-Tests have been added.
-
-=head2 CPP equivalent in Perl
-
-A C Yardley will probably have done this by the time you can read this.
-This allows for a generalization of the C constant detection used in
-building C<Errno.pm>.
-
-=head2 Explicit switch statements
+The old perltodo notes "With C<gdb>, you can attach the debugger to a running
+program if you pass the process ID. It would be good to do this with the Perl
+debugger on a running Perl program, although I'm not sure how it would be
+done." ssh and screen do this with named pipes in /tmp. Maybe we can too.
 
-C<Switch.pm> has been integrated into the core to give you all manner of
-C<switch...case> semantics.
+=head2 Constant folding
 
-=head2 autocroak
+The peephole optimiser should trap errors during constant folding, and give
+up on the folding, rather than bailing out at compile time.  It is quite
+possible that the unfoldable constant is in unreachable code, eg something
+akin to C<$a = 0/0 if 0;>
 
-This is C<Fatal.pm>.
+=head2 LVALUE functions for lists
 
-=head2 UTF/EBCDIC
+The old perltodo notes that lvalue functions don't work for list or hash
+slices. This would be good to fix.
 
-Nick Ing-Simmons has made UTF-EBCDIC (UTR13) work with Perl.
+=head2 LVALUE functions in the debugger
 
-    EBCDIC?        http://www.unicode.org/unicode/reports/tr16/
+The old perltodo notes that lvalue functions don't work in the debugger. This
+would be good to fix.
 
-=head2 UTF Regexes
+=head2 _ prototype character
 
-Although there are probably some small bugs to be rooted out, Jarkko
-Hietaniemi has made regular expressions polymorphic between bytes and
-characters.
+Study the possibility of adding a new prototype character, C<_>, meaning
+"this argument defaults to $_".
 
-=head2 perlcc to produce executable
+=head2 state variables
 
-C<perlcc> was recently rewritten, and can now produce standalone
-executables.
+C<my $foo if 0;> is deprecated, and should be replaced with
+C<state $x = "initial value\n";> the syntax from Perl 6.
 
-=head2 END blocks saved in compiled output
+=head2 @INC source filter to Filter::Simple
 
-=head2 Secure temporary file module
+The second return value from a sub in @INC can be a source filter. This isn't
+documented. It should be changed to use Filter::Simple, tested and documented.
 
-Tim Jenness' C<File::Temp> is now in core.
+=head2 regexp optimiser optional
 
-=head2 Integrate Time::HiRes
+The regexp optimiser is not optional. It should configurable to be, to allow
+its performance to be measured, and its bugs to be easily demonstrated.
 
-This module is now part of core.
+=head2 UNITCHECK
 
-=head2 Turn Cwd into XS
+Introduce a new special block, UNITCHECK, which is run at the end of a
+compilation unit (module, file, eval(STRING) block). This will correspond to
+the Perl 6 CHECK. Perl 5's CHECK cannot be changed or removed because the
+O.pm/B.pm backend framework depends on it.
 
-Benjamin Sugars has done this.
+=head2 optional optimizer
 
-=head2 Mmap for input
+Make the peephole optimizer optional. Currently it performs two tasks as
+it walks the optree - genuine peephole optimisations, and necessary fixups of
+ops. It would be good to find an efficient way to switch out the
+optimisations whilst keeping the fixups.
 
-Nick Ing-Simmons' C<perlio> supports an C<mmap> IO method.
+=head2 You WANT *how* many
 
-=head2 Byte to/from UTF8 and UTF8 to/from local conversion
+Currently contexts are void, scalar and list. split has a special mechanism in
+place to pass in the number of return values wanted. It would be useful to
+have a general mechanism for this, backwards compatible and little speed hit.
+This would allow proposals such as short circuiting sort to be implemented
+as a module on CPAN.
 
-C<Encode> provides this.
+=head2 lexical aliases
 
-=head2 Add sockatmark support
+Allow lexical aliases (maybe via the syntax C<my \$alias = \$foo>.
 
-Added in 5.7.1
+=head2 entersub XS vs Perl
 
-=head2 Mailing list archives
+At the moment pp_entersub is huge, and has code to deal with entering both
+perl and XS subroutines. Subroutine implementations rarely change between 
+perl and XS at run time, so investigate using 2 ops to enter subs (one for
+XS, one for perl) and swap between if a sub is redefined.
 
-http://lists.perl.org/, http://archive.develooper.com/
+=head2 Self ties
 
-=head2 Bug tracking
+self ties are currently illegal because they caused too many segfaults. Maybe
+the causes of these could be tracked down and self-ties on all types re-
+instated.
 
-Richard Foley has written the bug tracking system at http://bugs.perl.org/
-
-=head2 Integrate MacPerl
-
-Chris Nandor and Matthias Neeracher have integrated the MacPerl changes
-into 5.6.0.
-
-=head2 Web "nerve center" for Perl
-
-http://use.perl.org/ is what you're looking for.
-
-=head2 Regular expression tutorial
-
-C<perlretut>, provided by Mark Kvale.
-
-=head2 Debugging Tutorial
-
-C<perldebtut>, written by Richard Foley.
-
-=head2 Integrate new modules
-
-Jarkko has been integrating madly into 5.7.x
-
-=head2 Integrate profiler
-
-C<Devel::DProf> is now a core module.
-
-=head2 Y2K error detection
-
-There's a configure option to detect unsafe concatenation with "19", and
-a CPAN module. (C<D'oh::Year>)
-
-=head2 Regular expression debugger
-
-While not part of core, Mark-Jason Dominus has written C<Rx> and has
-also come up with a generalised strategy for regular expression
-debugging.
-
-=head2 POD checker
-
-That's, uh, F<podchecker>
-
-=head2 "Dynamic" lexicals
-
-=head2 Cache precompiled modules
-
-=head1 Deprecated Wishes
-
-These are items which used to be in the todo file, but have been
-deprecated for some reason.
-
-=head2 Loop control on do{}
-
-This would break old code; use C<do{{ }}> instead.
-
-=head2 Lexically scoped typeglobs
-
-Not needed now we have lexical IO handles.
-
-=head2 format BOTTOM
-
-=head2 report HANDLE
-
-Damian Conway's text formatting modules seem to be the Way To Go.
-
-=head2 Generalised want()/caller())
-
-=head2 Named prototypes
-
-These both seem to be delayed until Perl 6.
-
-=head2 Built-in globbing
-
-The C<File::Glob> module has been used to replace the C<glob> function.
-
-=head2 Regression tests for suidperl
-
-C<suidperl> is deprecated in favour of common sense.
-
-=head2 Cached hash values
-
-We have shared hash keys, which perform the same job.
-
-=head2 Add compression modules
-
-The compression modules are a little heavy; meanwhile, Nick Clark is
-working on experimental pragmata to do transparent decompression on
-input.
-
-=head2 Reorganise documentation into tutorials/references
-
-Could not get consensus on P5P about this.
-
-=head2 Remove distinction between functions and operators
-
-Caution: highly flammable.
-
-=head2 Make XS easier to use
-
-Use C<Inline> instead, or SWIG.
-
-=head2 Make embedding easier to use
-
-Use C<Inline::CPR>.
+=head2 Optimize away @_
 
-=head2 man for perl
+The old perltodo notes "Look at the "reification" code in C<av.c>".
 
-See the Perl Power Tools. (http://language.perl.com/ppt/)
+=head2 What hooks would assertions need?
 
-=head2 my $Package::variable
+Assertions are in the core, and work. However, assertions needed to be added
+as a core patch, rather than an XS module in ext, or a CPAN module, because
+the core has no hooks in the necessary places. It would be useful to
+investigate what hooks would need to be added to make it possible to provide
+the full assertion support from a CPAN module, so that we aren't constraining
+the imagination of future CPAN authors.
 
-Use C<our> instead.
 
-=head2 "or" tests defined, not truth
 
-Suggesting this on P5P B<will> cause a boring and interminable flamewar.
 
-=head2 "class"-based lexicals
 
-Use flyweight objects, secure hashes or, dare I say it, pseudo-hashes instead.
-(Or whatever will replace pseudohashes in 5.10.)
+=head1 Big projects
 
-=head2 byteperl
+Tasks that will get your name mentioned in the description of the "Highlights
+of 5.10"
 
-C<ByteLoader> covers this.
+=head2 make ithreads more robust
 
-=head2 Lazy evaluation / tail recursion removal
+Generally make ithreads more robust. See also L</iCOW>
 
-C<List::Util> gives first() (a short-circuiting grep); tail recursion
-removal is done manually, with C<goto &whoami;>. (However, MJD has
-found that C<goto &whoami> introduces a performance penalty, so maybe
-there should be a way to do this after all: C<sub foo {START: ... goto
-START;> is better.)
+This task is incremental - even a little bit of work on it will help, and
+will be greatly appreciated.
 
-=head2 Make "use utf8" the default
+=head2 iCOW
 
-Because of backward compatibility this is difficult: scripts could not
-contain B<any legacy eight-bit data> (like Latin-1) anymore, even in
-string literals or pod.  Also would introduce a measurable slowdown of
-at least few percentages since all regular expression operations would
-be done in full UTF-8.  But if you want to try this, add
--DUSE_UTF8_SCRIPTS to your compilation flags.
+Sarathy and Arthur have a proposal for an improved Copy On Write which
+specifically will be able to COW new ithreads. If this can be implemented
+it would be a good thing.
 
-=head2 Unicode collation and normalization
+=head2 (?{...}) closures in regexps
 
-The Unicode::Collate and Unicode::Normalize modules
-by SADAHIRO Tomoyuki have been included since 5.8.0.
+Fix (or rewrite) the implementation of the C</(?{...})/> closures.
 
-    Collation?     http://www.unicode.org/unicode/reports/tr10/
-    Normalization? http://www.unicode.org/unicode/reports/tr15/
+=head2 A re-entrant regexp engine
 
-=cut
+This will allow the use of a regex from inside (?{ }), (??{ }) and
+(?(?{ })|) constructs.