3 perltodo - Perl TO-DO List
7 This is a list of wishes for Perl. Send updates to
8 I<perl5-porters@perl.org>. If you want to work on any of these
9 projects, be sure to check the perl5-porters archives for past ideas,
10 flames, and propaganda. This will save you time and also prevent you
11 from implementing something that Larry has already vetoed. One set
12 of archives may be found at:
14 http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/
16 =head1 To do during 5.6.x
18 =head2 Support for I/O disciplines
20 C<perlio> provides this, but the interface could be a lot more
23 =head2 Autoload bytes.pm
25 When the lexer sees, for instance, C<bytes::length>, it should
26 automatically load the C<bytes> pragma.
28 =head2 Make "\u{XXXX}" et al work
30 Danger, Will Robinson! Discussing the semantics of C<"\x{F00}">,
31 C<"\xF00"> and C<"\U{F00}"> on P5P I<will> lead to a long and boring
34 =head2 Create a char *sv_pvprintify(sv, STRLEN *lenp, UV flags)
36 For displaying PVs with control characters, embedded nulls, and Unicode.
37 This would be useful for printing warnings, or data and regex dumping,
38 not_a_number(), and so on.
40 Requirements: should handle both byte and UTF8 strings. isPRINT()
41 characters printed as-is, character less than 256 as \xHH, Unicode
42 characters as \x{HHH}. Don't assume ASCII-like, either, get somebody
43 on EBCDIC to test the output.
45 Possible options, controlled by the flags:
46 - whitespace (other than ' ' of isPRINT()) printed as-is
47 - use isPRINT_LC() instead of isPRINT()
48 - print control characters like this: "\cA"
49 - print control characters like this: "^A"
50 - non-PRINTables printed as '.' instead of \xHH
51 - use \OOO instead of \xHH
52 - use the C/Perl-metacharacters like \n, \t
53 - have a maximum length for the produced string (read it from *lenp)
54 - append a "..." to the produced string if the maximum length is exceeded
55 - really fancy: print unicode characters as \N{...}
57 =head2 Overloadable regex assertions
59 This may or may not be possible with the current regular expression
60 engine. The idea is that, for instance, C<\b> needs to be
61 algorithmically computed if you're dealing with Thai text. Hence, the
62 B<\b> assertion wants to be overloaded by a function.
64 =head2 Unicode collation and normalization
66 Simon Cozens promises to work on this.
68 Collation? http://www.unicode.org/unicode/reports/tr10/
69 Normalization? http://www.unicode.org/unicode/reports/tr15/
71 =head2 Unicode case mappings
73 Case Mappings? http://www.unicode.org/unicode/reports/tr21/
75 =head2 Unicode regular expression character classes
77 They have some tricks Perl doesn't yet implement like character
80 http://www.unicode.org/unicode/reports/tr18/
82 =head2 use Thread for iThreads
84 Artur Bergman's C<iThreads> module is a start on this, but needs to
87 =head2 make perl_clone optionally clone ops
89 So that pseudoforking, mod_perl, iThreads and nvi will work properly
90 (but not as efficiently) until the regex engine is fixed to be threadsafe.
92 =head2 Work out exit/die semantics for threads
94 =head2 Typed lexicals for compiler
96 =head2 Compiler workarounds for Win32
98 =head2 AUTOLOADing in the compiler
100 =head2 Fixing comppadlist when compiling
102 =head2 Cleaning up exported namespace
104 =head2 Complete signal handling
106 Add C<PERL_ASYNC_CHECK> to opcodes which loop; replace C<sigsetjmp> with
107 C<sigjmp>; check C<wait> for signal safety.
109 =head2 Out-of-source builds
111 This was done for 5.6.0, but needs reworking for 5.7.x
113 =head2 POSIX realtime support
115 POSIX 1003.1 1996 Edition support--realtime stuff: POSIX semaphores,
116 message queues, shared memory, realtime clocks, timers, signals (the
117 metaconfig units mostly already exist for these)
119 =head2 UNIX98 support
121 Reader-writer locks, realtime/asynchronous IO
125 There are non-core modules, such as C<Net::IPv6>, but these will need
126 integrating when IPv6 actually starts to really happen. See RFC 2292
129 =head2 Long double conversion
131 Floating point formatting is still causing some weird test failures.
135 Locales and Unicode interact with each other in unpleasant ways.
136 One possible solution would be to adopt/support ICU:
138 http://oss.software.ibm.com/developerworks/opensource/icu/project/
140 =head2 Thread-safe regexes
142 The regular expression engine is currently non-threadsafe.
144 =head2 Arithmetic on non-Arabic numerals
146 C<[1234567890]> aren't the only numerals any more.
148 =head2 POSIX Unicode character classes
150 ([=a=] for equivalance classes, [.ch.] for collation.)
151 These are dependent on Unicode normalization and collation.
153 =head2 Factoring out common suffices/prefices in regexps (trie optimization)
155 Currently, the user has to optimize C<foo|far> and C<foo|goo> into
156 C<f(?:oo|ar)> and C<[fg]oo> by hand; this could be done automatically.
158 =head2 Security audit shipped utilities
160 All the code we ship with Perl needs to be sensible about temporary file
161 handling, locking, input validation, and so on.
163 =head2 Custom opcodes
165 Have a way to introduce user-defined opcodes without the subroutine call
166 overhead of an XSUB; the user should be able to create PP code. Simon
167 Cozens has some ideas on this.
169 =head2 spawnvp() on Win32
171 Win32 has problems spawning processes, particularly when the arguments
172 to the child process contain spaces, quotes or tab characters.
174 =head2 DLL Versioning
176 Windows needs a way to know what version of a XS or C<libperl> DLL it's
179 =head2 Introduce @( and @)
181 C<$(> may return "foo bar baz". Unfortunately, since groups can
182 theoretically have spaces in their names, this could be one, two or
185 =head2 Floating point handling
187 C<NaN> and C<inf> support is particularly troublesome.
188 (fp_classify(), fp_class(), fp_class_d(), class(), isinf(),
189 isfinite(), finite(), isnormal(), unordered(), <ieeefp.h>,
190 <fp_class.h> (there are metaconfig units for all these) (I think),
191 fp_setmask(), fp_getmask(), fp_setround(), fp_getround()
192 (no metaconfig units yet for these). Don't forget finitel(), fp_classl(),
193 fp_class_l(), (yes, both do, unfortunately, exist), and unorderedl().)
195 As of Perl 5.6.1 is a Perl macro, Perl_isnan().
197 =head2 IV/UV preservation
199 Nicholas Clark has done a lot of work on this, but work is continuing.
200 C<+>, C<-> and C<*> work, but guards need to be in place for C<%>, C</>,
201 C<&>, C<oct>, C<hex> and C<pack>.
203 =head2 Replace pod2html with something using Pod::Parser
205 The CPAN module C<Malik::Pod::Html> may be a more suitable basis for a
206 C<pod2html> convertor; the current one duplicates the functionality
207 abstracted in C<Pod::Parser>, which makes updating the POD language
210 =head2 Automate module testing on CPAN
212 When a new Perl is being beta tested, porters have to manually grab
213 their favourite CPAN modules and test them - this should be done
216 =head2 sendmsg and recvmsg
218 We have all the other BSD socket functions but these. There are
219 metaconfig units for these functions which can be added. To avoid these
220 being new opcodes, a solution similar to the way C<sockatmark> was added
221 would be preferable. (Autoload the C<IO::whatever> module.)
223 =head2 Rewrite perlre documentation
225 The new-style patterns need full documentation, and the whole document
226 needs to be a lot clearer.
228 =head2 Convert example code to IO::Handle filehandles
230 =head2 Document Win32 choices
232 =head2 Check new modules
234 =head2 Make roffitall find pods and libs itself
236 Simon Cozens has done some work on this but it needs a rethink.
238 =head1 To do at some point
240 These are ideas that have been regularly tossed around, that most
241 people believe should be done maybe during 5.8.x
243 =head2 Remove regular expression recursion
245 Because the regular expression engine is recursive, badly designed
246 expressions can lead to lots of recursion filling up the stack. Ilya
247 claims that it is easy to convert the engine to being iterative, but
248 this has still not yet been done. There may be a regular expression
249 engine hit squad meeting at TPC5.
251 =head2 Memory leaks after failed eval
253 Perl will leak memory if you C<eval "hlagh hlagh hlagh hlagh">. This is
254 partially because it attempts to build up an op tree for that code and
255 doesn't properly free it. The same goes for non-syntactically-correct
256 regular expressions. Hugo looked into this, but decided it needed a
257 mark-and-sweep GC implementation.
259 Alan notes that: The basic idea was to extend the parser token stack
260 (C<YYSTYPE>) to include a type field so we knew what sort of thing each
261 element of the stack was. The F<<perly.c> code would then have to be
262 postprocessed to record the type of each entry on the stack as it was
263 created, and the parser patched so that it could unroll the stack
266 This is possible to do, but would be pretty messy to implement, as it
267 would rely on even more sed hackery in F<perly.fixer>.
269 =head2 pack "(stuff)*"
271 That's to say, C<pack "(sI)40"> would be the same as C<pack "sI"x40>
273 =head2 bitfields in pack
275 =head2 Cross compilation
277 Make Perl buildable with a cross-compiler. This will play havoc with
278 Configure, which needs to how how the target system will respond to
279 its tests; maybe C<microperl> will be a good starting point here.
280 (Indeed, Bart Schuller reports that he compiled up C<microperl> for
281 the Agenda PDA and it works fine.) A really big spanner in the works
282 is the bootstrapping build process of Perl: if the filesystem the
283 target systems sees is not the same what the build host sees, various
284 input, output, and (Perl) library files need to be copied back and forth.
286 =head2 Perl preprocessor / macros
288 Source filters help with this, but do not get us all the way. For
289 instance, it should be possible to implement the C<??> operator somehow;
290 source filters don't (quite) cut it.
292 =head2 Perl lexer in Perl
294 Damian Conway is planning to work on this, but it hasn't happened yet.
296 =head2 Using POSIX calls internally
298 When faced with a BSD vs. SySV -style interface to some library or
299 system function, perl's roots show in that it typically prefers the BSD
300 interface (but falls back to the SysV one). One example is getpgrp().
301 Other examples include C<memcpy> vs. C<bcopy>. There are others, mostly in
304 Mostly, this item is a suggestion for which way to start a journey into
305 an C<#ifdef> forest. It is not primarily a suggestion to eliminate any of
306 the C<#ifdef> forests.
308 POSIX calls are perhaps more likely to be portable to unexpected
309 architectures. They are also perhaps more likely to be actively
310 maintained by a current vendor. They are also perhaps more likely to be
311 available in thread-safe versions, if appropriate.
313 =head2 -i rename file when changed
315 It's only necessary to rename a file when inplace editing when the file
316 has changed. Detecting a change is perhaps the difficult bit.
318 =head2 All ARGV input should act like E<lt>E<gt>
320 =head2 Support for rerunning debugger
322 There should be a way of restarting the debugger on demand.
324 =head2 Test Suite for the Debugger
326 The debugger is a complex piece of software and fixing something
327 here may inadvertently break something else over there. To tame
328 this chaotic behaviour, a test suite is necessary.
330 =head2 my sub foo { }
332 The basic principle is sound, but there are problems with the semantics
333 of self-referential and mutually referential lexical subs: how to
336 =head2 One-pass global destruction
338 Sweeping away all the allocated memory in one go is a laudable goal, but
339 it's difficult and in most cases, it's easier to let the memory get
342 =head2 Rewrite regexp parser
344 There has been talk recently of rewriting the regular expression parser
345 to produce an optree instead of a chain of opcodes; it's unclear whether
346 or not this would be a win.
348 =head2 Cache recently used regexps
352 for my $re (@regexps) {
356 C<qr//> already gives us a way of saving compiled regexps, but it should
357 be done automatically.
359 =head2 Re-entrant functions
361 Add configure probes for C<_r> forms of system calls and fit them to the
362 core. Unfortunately, calling conventions for these functions and not
365 =head2 Cross-compilation support
367 Bart Schuller reports that using C<microperl> and a cross-compiler, he
368 got Perl working on the Agenda PDA. However, one cannot build a full
369 Perl because Configure needs to get the results for the target platform,
372 =head2 Bit-shifting bitvectors
376 vec($v, 1000, 1) = 1;
378 One should be able to do
382 and have the 999'th bit set.
384 Currently if you try with shift bitvectors you shift the NV/UV, instead
385 of the bits in the PV. Not very logical.
387 =head2 debugger pragma
389 The debugger is implemented in Perl in F<perl5db.pl>; turning it into a
390 pragma should be easy, but making it work lexically might be more
391 difficult. Fiddling with C<$^P> would be necessary.
393 =head2 use less pragma
395 Identify areas where speed/memory tradeoffs can be made and have a hint
396 to switch between them.
398 =head2 switch structures
400 Although we have C<Switch.pm> in core, Larry points to the dormant
401 C<nswitch> and C<cswitch> ops in F<pp.c>; using these opcodes would be
404 =head2 Cache eval tree
408 =head2 Shrink opcode tables
410 =head2 Optimize away @_
412 Look at the "reification" code in C<av.c>
414 =head2 Prototypes versus indirect objects
416 Currently, indirect object syntax bypasses prototype checks.
420 HTML versions of the documentation need to be installed by default; a
421 call to C<installhtml> from C<installperl> may be all that's necessary.
423 =head2 Prototype method calls
425 =head2 Return context prototype declarations
429 =head2 Garbage collection
431 There have been persistent mumblings about putting a mark-and-sweep
432 garbage detector into Perl; Alan Burlison has some ideas about this.
436 Mark-Jason Dominus has the beginnings of one of these.
438 =head2 pack/unpack tutorial
440 Simon Cozens has the beginnings of one of these.
442 =head2 Rewrite perldoc
444 There are a few suggestions for what to do with C<perldoc>: maybe a
445 full-text search, an index function, locating pages on a particular
446 high-level subject, and so on.
448 =head2 Install .3p manpages
450 This is a bone of contention; we can create C<.3p> manpages for each
451 built-in function, but should we install them by default? Tcl does this,
452 and it clutters up C<apropos>.
454 =head2 Unicode tutorial
456 Simon Cozens promises to do this before he gets old.
458 =head2 Update POSIX.pm for 1003.1-2
460 =head2 Retargetable installation
462 Allow C<@INC> to be changed after Perl is built.
464 =head2 POSIX emulation on non-POSIX systems
466 Make C<POSIX.pm> behave as POSIXly as possible everywhere, meaning we
467 have to implement POSIX equivalents for some functions if necessary.
469 =head2 Rename Win32 headers
471 =head2 Finish off lvalue functions
473 They don't work in the debugger, and they don't work for list or hash
476 =head2 Update sprintf documentation
478 Hugo van der Sanden plans to look at this.
480 =head2 Use fchown/fchmod internally
482 This has been done in places, but needs a thorough code review.
483 Also fchdir is available in some platforms.
487 Ideas which have been discussed, and which may or may not happen.
489 =head2 ref() in list context
491 It's unclear what this should do or how to do it without breaking old
494 =head2 Make tr/// return histogram
496 There is a patch for this, but it may require Unicodification.
498 =head2 Compile to real threaded code
500 =head2 Structured types
502 =head2 Modifiable $1 et al.
504 ($x = "elephant") =~ /e(ph)/;
505 $1 = "g"; # $x = "elegant"
507 What happens if there are multiple (nested?) brackets? What if the
508 string changes between the match and the assignment?
510 =head2 Procedural interfaces for IO::*, etc.
512 Some core modules have been accused of being overly-OO. Adding
513 procedural interfaces could demystify them.
517 =head2 Attach/detach debugger from running program
519 With C<gdb>, you can attach the debugger to a running program if you
520 pass the process ID. It would be good to do this with the Perl debugger
521 on a running Perl program, although I'm not sure how it would be done.
523 =head2 Alternative RE syntax module
526 $re = Regex::Newbie->new
529 ->repeat(Regex::Newbie->class("char"),3)
535 A non-core module that would use "native" GUI to create graphical
538 =head2 foreach(reverse ...)
542 foreach (reverse @_) { ... }
544 puts C<@_> on the stack, reverses it putting the reversed version on the
545 stack, then iterates forwards. Instead, it could be special-cased to put
546 C<@_> on the stack then iterate backwards.
548 =head2 Constant function cache
550 =head2 Approximate regular expression matching
554 These items B<always> need doing:
556 =head2 Update guts documentation
558 Simon Cozens tries to do this when possible, and contributions to the
559 C<perlapi> documentation is welcome.
561 =head2 Add more tests
563 Michael Schwern will donate $500 to Yet Another Society when all core
566 =head2 Update auxiliary tools
568 The code we ship with Perl should look like good Perl 5.
570 =head1 Recently done things
572 These are things which have been on the todo lists in previous releases
573 but have recently been completed.
575 =head2 Safe signal handling
577 A new signal model went into 5.7.1 without much fanfare. Operations and
578 C<malloc>s are no longer interrupted by signals, which are handled
579 between opcodes. This means that C<PERL_ASYNC_CHECK> now actually does
580 something. However, there are still a few things that need to be done.
584 Modules which implement arrays in terms of strings, substrings or files
585 can be found on the CPAN.
589 C<Time::Hires> has been integrated into the core.
591 =head2 setitimer and getimiter
593 Adding C<Time::Hires> got us this too.
595 =head2 Testing __DIE__ hook
597 Tests have been added.
599 =head2 CPP equivalent in Perl
601 A C Yardley will probably have done this by the time you can read this.
602 This allows for a generalization of the C constant detection used in
603 building C<Errno.pm>.
605 =head2 Explicit switch statements
607 C<Switch.pm> has been integrated into the core to give you all manner of
608 C<switch...case> semantics.
616 Nick Ing-Simmons has made UTF-EBCDIC (UTR13) work with Perl.
618 EBCDIC? http://www.unicode.org/unicode/reports/tr16/
622 Although there are probably some small bugs to be rooted out, Jarkko
623 Hietaniemi has made regular expressions polymorphic between bytes and
626 =head2 perlcc to produce executable
628 C<perlcc> was recently rewritten, and can now produce standalone
631 =head2 END blocks saved in compiled output
633 =head2 Secure temporary file module
635 Tim Jenness' C<File::Temp> is now in core.
637 =head2 Integrate Time::HiRes
639 This module is now part of core.
641 =head2 Turn Cwd into XS
643 Benjamin Sugars has done this.
645 =head2 Mmap for input
647 Nick Ing-Simmons' C<perlio> supports an C<mmap> IO method.
649 =head2 Byte to/from UTF8 and UTF8 to/from local conversion
651 C<Encode> provides this.
653 =head2 Add sockatmark support
657 =head2 Mailing list archives
659 http://lists.perl.org/, http://archive.develooper.com/
663 Richard Foley has written the bug tracking system at http://bugs.perl.org/
665 =head2 Integrate MacPerl
667 Chris Nandor and Matthias Neeracher have integrated the MacPerl changes
670 =head2 Web "nerve center" for Perl
672 http://use.perl.org/ is what you're looking for.
674 =head2 Regular expression tutorial
676 C<perlretut>, provided by Mark Kvale.
678 =head2 Debugging Tutorial
680 C<perldebtut>, written by Richard Foley.
682 =head2 Integrate new modules
684 Jarkko has been integrating madly into 5.7.x
686 =head2 Integrate profiler
688 C<Devel::DProf> is now a core module.
690 =head2 Y2K error detection
692 There's a configure option to detect unsafe concatenation with "19", and
693 a CPAN module. (C<D'oh::Year>)
695 =head2 Regular expression debugger
697 While not part of core, Mark-Jason Dominus has written C<Rx> and has
698 also come up with a generalised strategy for regular expression
703 That's, uh, F<podchecker>
705 =head2 "Dynamic" lexicals
707 =head2 Cache precompiled modules
709 =head1 Deprecated Wishes
711 These are items which used to be in the todo file, but have been
712 deprecated for some reason.
714 =head2 Loop control on do{}
716 This would break old code; use C<do{{ }}> instead.
718 =head2 Lexically scoped typeglobs
720 Not needed now we have lexical IO handles.
726 Damian Conway's text formatting modules seem to be the Way To Go.
728 =head2 Generalised want()/caller())
730 =head2 Named prototypes
732 These both seem to be delayed until Perl 6.
734 =head2 Built-in globbing
736 The C<File::Glob> module has been used to replace the C<glob> function.
738 =head2 Regression tests for suidperl
740 C<suidperl> is deprecated in favour of common sense.
742 =head2 Cached hash values
744 We have shared hash keys, which perform the same job.
746 =head2 Add compression modules
748 The compression modules are a little heavy; meanwhile, Nick Clark is
749 working on experimental pragmata to do transparent decompression on
752 =head2 Reorganise documentation into tutorials/references
754 Could not get consensus on P5P about this.
756 =head2 Remove distinction between functions and operators
758 Caution: highly flammable.
760 =head2 Make XS easier to use
762 Use C<Inline> instead, or SWIG.
764 =head2 Make embedding easier to use
770 See the Perl Power Tools. (http://language.perl.com/ppt/)
772 =head2 my $Package::variable
776 =head2 "or" tests defined, not truth
778 Suggesting this on P5P B<will> cause a boring and interminable flamewar.
780 =head2 "class"-based lexicals
782 Use flyweight objects, secure hashes or, dare I say it, pseudo-hashes instead.
786 C<ByteLoader> covers this.
788 =head2 Lazy evaluation / tail recursion removal
790 C<List::Util> in core gives some of these; tail recursion removal is
791 done manually, with C<goto &whoami;>. (However, MJD has found that
792 C<goto &whoami> introduces a performance penalty, so maybe there should
793 be a way to do this after all: C<sub foo {START: ... goto START;> is
796 =head2 Make "use utf8" the default
798 There is a patch available for this, search p5p archives for
799 the Subject "[EXPERIMENTAL PATCH] make unicode (utf8) default"
800 but this would be unacceptable because of backward compatibility:
801 scripts could not contain B<any legacy eight-bit data>. Also would
802 introduce a measurable slowdown of at least few percentages since all
803 regular expression operations would be done in full UTF-8.