1 .\" Automatically generated by Pod::Man 2.22 (Pod::Simple 3.10)
4 .\" ========================================================================
5 .de Sp \" Vertical space (when we can't use .PP)
9 .de Vb \" Begin verbatim text
14 .de Ve \" End verbatim text
18 .\" Set up some character translations and predefined strings. \*(-- will
19 .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
20 .\" double quote, and \*(R" will give a right double quote. \*(C+ will
21 .\" give a nicer C++. Capital omega is used to do unbreakable dashes and
22 .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff,
23 .\" nothing in troff, for use with C<>.
25 .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
29 . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
30 . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch
43 .\" Escape single quotes in literal strings from groff's Unicode transform.
47 .\" If the F register is turned on, we'll generate index entries on stderr for
48 .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
49 .\" entries marked with X<> in POD. Of course, you'll have to process the
50 .\" output yourself in some meaningful fashion.
53 . tm Index:\\$1\t\\n%\t"\\$2"
63 .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
64 .\" Fear. Run. Save yourself. No user-serviceable parts.
65 . \" fudge factors for nroff and troff
74 . ds #H ((1u-(\\\\n(.fu%2u))*.13m)
80 . \" simple accents for nroff and troff
90 . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
91 . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
92 . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
93 . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
94 . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
95 . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
97 . \" troff and (daisy-wheel) nroff accents
98 .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
99 .ds 8 \h'\*(#H'\(*b\h'-\*(#H'
100 .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
101 .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
102 .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
103 .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
104 .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
105 .ds ae a\h'-(\w'a'u*4/10)'e
106 .ds Ae A\h'-(\w'A'u*4/10)'E
107 . \" corrections for vroff
108 .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
109 .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
110 . \" for low resolution devices (crt and lpr)
111 .if \n(.H>23 .if \n(.V>19 \
124 .\" ========================================================================
127 .TH PPI 3 "2009-08-08" "perl v5.8.7" "User Contributed Perl Documentation"
128 .\" For nroff, turn off justification. Always turn off hyphenation; it makes
129 .\" way too many mistakes in technical documents.
133 PPI \- Parse, Analyze and Manipulate Perl (without perl)
135 .IX Header "SYNOPSIS"
139 \& # Create a new empty document
140 \& my $Document = PPI::Document\->new;
142 \& # Create a document from source
143 \& $Document = PPI::Document\->new(\e\*(Aqprint "Hello World!\en"\*(Aq);
145 \& # Load a Document from a file
146 \& $Document = PPI::Document\->new(\*(AqModule.pm\*(Aq);
148 \& # Does it contain any POD?
149 \& if ( $Document\->find_any(\*(AqPPI::Token::Pod\*(Aq) ) {
150 \& print "Module contains POD\en";
153 \& # Get the name of the main package
154 \& $pkg = $Document\->find_first(\*(AqPPI::Statement::Package\*(Aq)\->namespace;
156 \& # Remove all that nasty documentation
157 \& $Document\->prune(\*(AqPPI::Token::Pod\*(Aq);
158 \& $Document\->prune(\*(AqPPI::Token::Comment\*(Aq);
161 \& $Document\->save(\*(AqModule.pm.stripped\*(Aq);
164 .IX Header "DESCRIPTION"
165 .SS "About this Document"
166 .IX Subsection "About this Document"
167 This is the \s-1PPI\s0 manual. It describes its reason for existing, its general
168 structure, its use, an overview of the \s-1API\s0, and provides a few
169 implementation samples.
171 .IX Subsection "Background"
172 The ability to read, and manipulate Perl (the language) programmatically
173 other than with perl (the application) was one that caused difficulty
176 The cause of this problem was Perl's complex and dynamic grammar.
177 Although there is typically not a huge diversity in the grammar of most
178 Perl code, certain issues cause large problems when it comes to parsing.
180 Indeed, quite early in Perl's history Tom Christenson introduced the Perl
181 community to the quote \fI\*(L"Nothing but perl can parse Perl\*(R"\fR, or as it is
182 more often stated now as a truism:
184 \&\fB\*(L"Only perl can parse Perl\*(R"\fR
186 One example of the sorts of things the prevent Perl being easily parsed are
187 function signatures, as demonstrated by the following.
190 \& @result = (dothis $foo, $bar);
192 \& # Which of the following is it equivalent to?
193 \& @result = (dothis($foo), $bar);
194 \& @result = dothis($foo, $bar);
197 The first line above can be interpreted in two different ways, depending
198 on whether the \f(CW&dothis\fR function is expecting one argument, or two,
201 A \*(L"code parser\*(R" (something that parses for the purpose of execution) such
202 as perl needs information that is not found in the immediate vicinity of
203 the statement being parsed.
205 The information might not just be elsewhere in the file, it might not even be
206 in the same file at all. It might also not be able to determine this
207 information without the prior execution of a \f(CW\*(C`BEGIN {}\*(C'\fR block, or the
208 loading and execution of one or more external modules. Or worse the &dothis
209 function may not even have been written yet.
211 \&\fBWhen parsing Perl as code, you must also execute it\fR
213 Even perl itself never really fully understands the structure of the source
214 code after and indeed \fBas\fR it processes it, and in that sense doesn't
215 \&\*(L"parse\*(R" Perl source into anything remotely like a structured document.
216 This makes it of no real use for any task that needs to treat the source
217 code as a document, and do so reliably and robustly.
219 For more information on why it is impossible to parse perl, see Randal
220 Schwartz's seminal response to the question of \*(L"Why can't you parse Perl\*(R".
222 <http://www.perlmonks.org/index.pl?node_id=44722>
224 The purpose of \s-1PPI\s0 is \fBnot\fR to parse Perl \fICode\fR, but to parse Perl
225 \&\fIDocuments\fR. By treating the problem this way, we are able to parse a
226 single file containing Perl source code \*(L"isolated\*(R" from any other
227 resources, such as libraries upon which the code may depend, and
228 without needing to run an instance of perl alongside or inside the parser.
230 Historically, using an embedded perl parser was widely considered to be
231 the most likely avenue for finding a solution to \f(CW\*(C`Parse::Perl\*(C'\fR. It was
232 investigated from time to time and attempts have generally failed or
233 suffered from sufficiently bad corner cases that they were abandoned.
234 .SS "What Does \s-1PPI\s0 Stand For?"
235 .IX Subsection "What Does PPI Stand For?"
236 \&\f(CW\*(C`PPI\*(C'\fR is an acronym for the longer original module name
237 \&\f(CW\*(C`Parse::Perl::Isolated\*(C'\fR. And in the spirit or the silly acronym games
238 played by certain unnamed Open Source projects you may have \fIhurd\fR of,
239 it also a reverse backronym of \*(L"I Parse Perl\*(R".
241 Of course, I could just be lying and have just made that second bit up
242 10 minutes before the release of \s-1PPI\s0 1.000. Besides, \fBall\fR the cool
243 Perl packages have TLAs (Three Letter Acronyms). It's a rule or something.
245 Why don't you just think of it as the \fBPerl Parsing Interface\fR for simplicity.
247 The original name was shortened to prevent the author (and you the users)
248 from contracting \s-1RSI\s0 by having to type crazy things like
249 \&\f(CW\*(C`Parse::Perl::Isolated::Token::QuoteLike::Backtick\*(C'\fR 100 times a day.
251 In acknowledgment that someone may some day come up with a valid solution
252 for the grammar problem it was decided at the commencement of the project
253 to leave the \f(CW\*(C`Parse::Perl\*(C'\fR namespace free for any such effort.
255 Since that time I've been able to prove to my own satisfaction that it
256 \&\fBis\fR truly impossible to accurately parse Perl as both code and document
257 at once. For the academics, parsing Perl suffers from the \*(L"Halting Problem\*(R".
259 With this in mind \f(CW\*(C`Parse::Perl\*(C'\fR has now been co-opted as the title for
260 the SourceForge project that publishes \s-1PPI\s0 and a large collection of other
261 applications and modules related to the (document) parsing of Perl source
264 You can find this project at <http://sf.net/projects/parseperl>,
265 however we no longer use the SourceForge \s-1CVS\s0 server. Instead, the
266 current development version of \s-1PPI\s0 is available via \s-1SVN\s0 at
267 <http://svn.ali.as/cpan/trunk/PPI/>.
268 .SS "Why Parse Perl?"
269 .IX Subsection "Why Parse Perl?"
270 Once you can accept that we will never be able to parse Perl well enough
271 to meet the standards of things that treat Perl as code, it is worth
272 re-examining \f(CW\*(C`why\*(C'\fR we want to \*(L"parse\*(R" Perl at all.
274 What are the things that people might want a \*(L"Perl parser\*(R" for.
275 .IP "Documentation" 4
276 .IX Item "Documentation"
277 Analyzing the contents of a Perl document to automatically generate
278 documentation, in parallel to, or as a replacement for, \s-1POD\s0 documentation.
280 Allow an indexer to to locate and process all the comments and
281 documentation from code for \*(L"full text search\*(R" applications.
282 .IP "Structural and Quality Analysis" 4
283 .IX Item "Structural and Quality Analysis"
284 Determine quality or other metrics across a body of code, and identify
285 situations relating to particular phrases, techniques or locations.
287 Index functions, variables and packages within Perl code, and doing search
288 and graph (in the node/edge sense) analysis of large code bases.
290 .IX Item "Refactoring"
291 Make structural, syntax, or other changes to code in an automated manner,
292 either independently or in assistance to an editor. This sort of task list
293 includes backporting, forward porting, partial evaluation, \*(L"improving\*(R" code,
294 or whatever. All the sort of things you'd want from a Perl::Editor.
297 Change the layout of code without changing its meaning. This includes
298 techniques such as tidying (like perltidy), obfuscation, compressing and
299 \&\*(L"squishing\*(R", or to implement formatting preferences or policies.
301 .IX Item "Presentation"
302 This includes methods of improving the presentation of code, without changing
303 the content of the code. Modify, improve, syntax colour etc the presentation
304 of a Perl document. Generating \*(L"IntelliText\*(R"\-like functions.
306 If we treat this as a baseline for the sort of things we are going to have
307 to build on top of Perl, then it becomes possible to identify a standard
308 for how good a Perl parser needs to be.
309 .SS "How good is Good Enough(\s-1TM\s0)"
310 .IX Subsection "How good is Good Enough(TM)"
311 \&\s-1PPI\s0 seeks to be good enough to achieve all of the above tasks, or to provide
312 a sufficiently good \s-1API\s0 on which to allow others to implement modules in
313 these and related areas.
315 However, there are going to be limits to this process. Because \s-1PPI\s0 cannot
316 adapt to changing grammars, any code written using source filters should not
317 be assumed to be parsable.
319 At one extreme, this includes anything munged by Acme::Bleach, as well
320 as (arguably) more common cases like Switch. We do not pretend to be
321 able to always parse code using these modules, although as long as it still
322 follows a format that looks like Perl syntax, it may be possible to extend
323 the lexer to handle them.
325 The ability to extend \s-1PPI\s0 to handle lexical additions to the language is on
326 the drawing board to be done some time post\-1.0
328 The goal for success was originally to be able to successfully parse 99% of
329 all Perl documents contained in \s-1CPAN\s0. This means the entire file in each
332 \&\s-1PPI\s0 has succeeded in this goal far beyond the expectations of even the
333 author. At time of writing there are only 28 non-Acme Perl modules in \s-1CPAN\s0
334 that \s-1PPI\s0 is incapable of parsing. Most of these are so badly broken they
335 do not compile as Perl code anyway.
337 So unless you are actively going out of your way to break \s-1PPI\s0, you should
338 expect that it will handle your code just fine.
339 .SS "Internationalisation"
340 .IX Subsection "Internationalisation"
341 \&\s-1PPI\s0 provides partial support for internationalisation and localisation.
343 Specifically, it allows the use characters from the Latin\-1 character
344 set to be used in quotes, comments, and \s-1POD\s0. Primarily, this covers
345 languages from Europe and South America.
347 \&\s-1PPI\s0 does \fBnot\fR currently provide support for Unicode, although there
348 is an initial implementation available in a development branch from
351 If you need Unicode support, and would like to help stress test the
352 Unicode support so we can move it to the main branch and enable it
353 in the main release should contact the author. (contact details below)
354 .SS "Round Trip Safe"
355 .IX Subsection "Round Trip Safe"
356 When \s-1PPI\s0 parses a file it builds \fBeverything\fR into the model, including
357 whitespace. This is needed in order to make the Document fully \*(L"Round Trip\*(R"
360 The general concept behind a \*(L"Round Trip\*(R" parser is that it knows what it
361 is parsing is somewhat uncertain, and so \fBexpects\fR to get things wrong
362 from time to time. In the cases where it parses code wrongly the tree
363 will serialize back out to the same string of code that was read in,
364 repairing the parser's mistake as it heads back out to the file.
366 The end result is that if you parse in a file and serialize it back out
367 without changing the tree, you are guaranteed to get the same file you
368 started with. \s-1PPI\s0 does this correctly and reliably for 100% of all known
371 \&\fBWhat goes in, will come out. Every time.\fR
373 The one minor exception at this time is that if the newlines for your file
374 are wrong (meaning not matching the platform newline format), \s-1PPI\s0 will
375 localise them for you. (It isn't to be convenient, supporting
376 arbitrary newlines would make some of the code more complicated)
378 Better control of the newline type is on the wish list though, and
379 anyone wanting to help out is encouraged to contact the author.
381 .IX Header "IMPLEMENTATION"
383 .IX Subsection "General Layout"
384 \&\s-1PPI\s0 is built upon two primary \*(L"parsing\*(R" components, PPI::Tokenizer
385 and PPI::Lexer, and a large tree of about 50 classes which implement
386 the various the \fIPerl Document Object Model\fR (\s-1PDOM\s0).
388 The \s-1PDOM\s0 is conceptually similar in style and intent to the regular \s-1DOM\s0 or
389 other code Abstract Syntax Trees (ASTs), but contains some differences
390 to handle perl-specific cases, and to assist in treating the code as a
391 document. Please note that it is \fBnot\fR an implementation of the official
392 Document Object Model specification, only somewhat similar to it.
394 On top of the Tokenizer, Lexer and the classes of the \s-1PDOM\s0, sit a number
395 of classes intended to make life a little easier when dealing with \s-1PDOM\s0
398 Both the major parsing components were hand-coded from scratch with only
399 plain Perl code and a few small utility modules. There are no grammar or
400 patterns mini-languages, no \s-1YACC\s0 or \s-1LEX\s0 style tools and only a small number
401 of regular expressions.
403 This is primarily because of the sheer volume of accumulated cruft that
404 exists in Perl. Not even perl itself is capable of parsing Perl documents
405 (remember, it just parses and executes it as code).
407 As a result, \s-1PPI\s0 needed to be cruftier than perl itself. Feel free to
408 shudder at this point, and hope you never have to understand the Tokenizer
409 codebase. Speaking of which...
411 .IX Subsection "The Tokenizer"
412 The Tokenizer takes source code and converts it into a series of tokens. It
413 does this using a slow but thorough character by character manual process,
414 rather than using a pattern system or complex regexes.
416 Or at least it does so conceptually. If you were to actually trace the code
417 you would find it's not truly character by character due to a number of
418 regexps and optimisations throughout the code. This lets the Tokenizer
419 \&\*(L"skip ahead\*(R" when it can find shortcuts, so it tends to jump around a line
420 a bit wildly at times.
422 In practice, the number of times the Tokenizer will \fBactually\fR move the
423 character cursor itself is only about 5% \- 10% higher than the number of
424 tokens contained in the file. This makes it about as optimal as it can be
425 made without implementing it in something other than Perl.
427 In 2001 when \s-1PPI\s0 was started, this structure made \s-1PPI\s0 quite slow, and not
428 really suitable for interactive tasks. This situation has improved greatly
429 with multi-gigahertz processors, but can still be painful when working with
432 The target parsing rate for \s-1PPI\s0 is about 5000 lines per gigacycle. It is
433 currently believed to be at about 1500, and main avenue for making it to
434 the target speed has now become \s-1PPI::XS\s0, a drop-in \s-1XS\s0 accelerator for
437 Since \s-1PPI::XS\s0 has only just gotten off the ground and is currently only
438 at proof-of-concept stage, this may take a little while. Anyone interested
439 in helping out with \s-1PPI::XS\s0 is \fBhighly\fR encouraged to contact the
440 author. In fact, the design of \s-1PPI::XS\s0 means it's possible to port
441 one function at a time safely and reliably. So every little bit will help.
443 .IX Subsection "The Lexer"
444 The Lexer takes a token stream, and converts it to a lexical tree. Because
445 we are parsing Perl \fBdocuments\fR this includes whitespace, comments, and
446 all number of weird things that have no relevance when code is actually
449 An instantiated PPI::Lexer consumes PPI::Tokenizer objects and
450 produces PPI::Document objects. However you should probably never be
451 working with the Lexer directly. You should just be able to create
452 PPI::Document objects and work with them directly.
453 .SS "The Perl Document Object Model"
454 .IX Subsection "The Perl Document Object Model"
455 The \s-1PDOM\s0 is a structured collection of data classes that together provide
456 a correct and scalable model for documents that follow the standard Perl
458 .SS "The \s-1PDOM\s0 Class Tree"
459 .IX Subsection "The PDOM Class Tree"
460 The following lists all of the 67 current \s-1PDOM\s0 classes, listing with indentation
461 based on inheritance.
467 \& PPI::Document::Fragment
469 \& PPI::Statement::Package
470 \& PPI::Statement::Include
471 \& PPI::Statement::Sub
472 \& PPI::Statement::Scheduled
473 \& PPI::Statement::Compound
474 \& PPI::Statement::Break
475 \& PPI::Statement::Given
476 \& PPI::Statement::When
477 \& PPI::Statement::Data
478 \& PPI::Statement::End
479 \& PPI::Statement::Expression
480 \& PPI::Statement::Variable
481 \& PPI::Statement::Null
482 \& PPI::Statement::UnmatchedBrace
483 \& PPI::Statement::Unknown
485 \& PPI::Structure::Block
486 \& PPI::Structure::Subscript
487 \& PPI::Structure::Constructor
488 \& PPI::Structure::Condition
489 \& PPI::Structure::List
490 \& PPI::Structure::For
491 \& PPI::Structure::Given
492 \& PPI::Structure::When
493 \& PPI::Structure::Unknown
495 \& PPI::Token::Whitespace
496 \& PPI::Token::Comment
498 \& PPI::Token::Number
499 \& PPI::Token::Number::Binary
500 \& PPI::Token::Number::Octal
501 \& PPI::Token::Number::Hex
502 \& PPI::Token::Number::Float
503 \& PPI::Token::Number::Exp
504 \& PPI::Token::Number::Version
506 \& PPI::Token::DashedWord
507 \& PPI::Token::Symbol
509 \& PPI::Token::ArrayIndex
510 \& PPI::Token::Operator
512 \& PPI::Token::Quote::Single
513 \& PPI::Token::Quote::Double
514 \& PPI::Token::Quote::Literal
515 \& PPI::Token::Quote::Interpolate
516 \& PPI::Token::QuoteLike
517 \& PPI::Token::QuoteLike::Backtick
518 \& PPI::Token::QuoteLike::Command
519 \& PPI::Token::QuoteLike::Regexp
520 \& PPI::Token::QuoteLike::Words
521 \& PPI::Token::QuoteLike::Readline
522 \& PPI::Token::Regexp
523 \& PPI::Token::Regexp::Match
524 \& PPI::Token::Regexp::Substitute
525 \& PPI::Token::Regexp::Transliterate
526 \& PPI::Token::HereDoc
528 \& PPI::Token::Structure
530 \& PPI::Token::Separator
533 \& PPI::Token::Prototype
534 \& PPI::Token::Attribute
535 \& PPI::Token::Unknown
538 To summarize the above layout, all \s-1PDOM\s0 objects inherit from the
541 Under this are PPI::Token, strings of content with a known type,
542 and PPI::Node, syntactically significant containers that hold other
545 The three most important of these are the PPI::Document, the
546 PPI::Statement and the PPI::Structure classes.
547 .SS "The Document, Statement and Structure"
548 .IX Subsection "The Document, Statement and Structure"
549 At the top of all complete \s-1PDOM\s0 trees is a PPI::Document object. It
550 represents a complete file of Perl source code as you might find it on
553 There are some specialised types of document, such as PPI::Document::File
554 and PPI::Document::Normalized but for the purposes of the \s-1PDOM\s0 they are
555 all just considered to be the same thing.
557 Each Document will contain a number of \fBStatements\fR, \fBStructures\fR and
560 A PPI::Statement is any series of Tokens and Structures that are treated
561 as a single contiguous statement by perl itself. You should note that a
562 Statement is as close as \s-1PPI\s0 can get to \*(L"parsing\*(R" the code in the sense that
563 perl-itself parses Perl code when it is building the op-tree.
565 Because of the isolation and Perl's syntax, it is provably impossible for
566 \&\s-1PPI\s0 to accurately determine precedence of operators or which tokens are
567 implicit arguments to a sub call.
569 So rather than lead you on with a bad guess that has a strong chance of
570 being wrong, \s-1PPI\s0 does not attempt to determine precedence or sub parameters
573 At a fundamental level, it only knows that this series of elements
574 represents a single Statement as perl sees it, but it can do so with
575 enough certainty that it can be trusted.
577 However, for specific Statement types the \s-1PDOM\s0 is able to derive additional
578 useful information about their meaning. For the best, most useful, and most
579 heavily used example, see PPI::Statement::Include.
581 A PPI::Structure is any series of tokens contained within matching braces.
582 This includes code blocks, conditions, function argument braces, anonymous
583 array and hash constructors, lists, scoping braces and all other syntactic
584 structures represented by a matching pair of braces, including (although it
585 may not seem obvious at first) \f(CW\*(C`<READLINE>\*(C'\fR braces.
587 Each Structure contains none, one, or many Tokens and Structures (the rules
588 for which vary for the different Structure subclasses)
590 Under the \s-1PDOM\s0 structure rules, a Statement can \fBnever\fR directly contain
591 another child Statement, a Structure can \fBnever\fR directly contain another
592 child Structure, and a Document can \fBnever\fR contain another Document
593 anywhere in the tree.
595 Aside from these three rules, the \s-1PDOM\s0 tree is extremely flexible.
596 .SS "The \s-1PDOM\s0 at Work"
597 .IX Subsection "The PDOM at Work"
598 To demonstrate the \s-1PDOM\s0 in use lets start with an example showing how the
599 tree might look for the following chunk of simple Perl code.
604 \& print( "Hello World!" );
609 Translated into a \s-1PDOM\s0 tree it would have the following structure (as shown
610 via the included PPI::Dumper).
614 \& PPI::Token::Comment \*(Aq#!/usr/bin/perl\en\*(Aq
615 \& PPI::Token::Whitespace \*(Aq\en\*(Aq
616 \& PPI::Statement::Expression
617 \& PPI::Token::Bareword \*(Aqprint\*(Aq
618 \& PPI::Structure::List ( ... )
619 \& PPI::Token::Whitespace \*(Aq \*(Aq
620 \& PPI::Statement::Expression
621 \& PPI::Token::Quote::Double \*(Aq"Hello World!"\*(Aq
622 \& PPI::Token::Whitespace \*(Aq \*(Aq
623 \& PPI::Token::Structure \*(Aq;\*(Aq
624 \& PPI::Token::Whitespace \*(Aq\en\*(Aq
625 \& PPI::Token::Whitespace \*(Aq\en\*(Aq
626 \& PPI::Statement::Expression
627 \& PPI::Token::Bareword \*(Aqexit\*(Aq
628 \& PPI::Structure::List ( ... )
629 \& PPI::Token::Structure \*(Aq;\*(Aq
630 \& PPI::Token::Whitespace \*(Aq\en\*(Aq
633 Please note that in this this example, strings are only listed for the
634 \&\fBactual\fR PPI::Token that contains that string. Structures are listed
635 with the type of brace characters it represents noted.
637 The PPI::Dumper module can be used to generate similar trees yourself.
639 We can make that \s-1PDOM\s0 dump a little easier to read if we strip out all the
640 whitespace. Here it is again, sans the distracting whitespace tokens.
644 \& PPI::Token::Comment \*(Aq#!/usr/bin/perl\en\*(Aq
645 \& PPI::Statement::Expression
646 \& PPI::Token::Bareword \*(Aqprint\*(Aq
647 \& PPI::Structure::List ( ... )
648 \& PPI::Statement::Expression
649 \& PPI::Token::Quote::Double \*(Aq"Hello World!"\*(Aq
650 \& PPI::Token::Structure \*(Aq;\*(Aq
651 \& PPI::Statement::Expression
652 \& PPI::Token::Bareword \*(Aqexit\*(Aq
653 \& PPI::Structure::List ( ... )
654 \& PPI::Token::Structure \*(Aq;\*(Aq
657 As you can see, the tree can get fairly deep at time, especially when every
658 isolated token in a bracket becomes its own statement. This is needed to
659 allow anything inside the tree the ability to grow. It also makes the
660 search and analysis algorithms much more flexible.
662 Because of the depth and complexity of \s-1PDOM\s0 trees, a vast number of very easy
663 to use methods have been added wherever possible to help people working with
664 \&\s-1PDOM\s0 trees do normal tasks relatively quickly and efficiently.
665 .SS "Overview of the Primary Classes"
666 .IX Subsection "Overview of the Primary Classes"
667 The main \s-1PPI\s0 classes, and links to their own documentation, are listed
668 here in alphabetical order.
669 .IP "PPI::Document" 4
670 .IX Item "PPI::Document"
671 The Document object, the root of the \s-1PDOM\s0.
672 .IP "PPI::Document::Fragment" 4
673 .IX Item "PPI::Document::Fragment"
674 A cohesive fragment of a larger Document. Although not of any real current
675 use, it is needed for use in certain internal tree manipulation
678 For example, doing things like cut/copy/paste etc. Very similar to a
679 PPI::Document, but has some additional methods and does not represent
680 a lexical scope boundary.
682 A document fragment is also non-serializable, and so cannot be written out
685 .IX Item "PPI::Dumper"
686 A simple class for dumping readable debugging versions of \s-1PDOM\s0 structures,
687 such as in the demonstration above.
689 .IX Item "PPI::Element"
690 The Element class is the abstract base class for all objects within the \s-1PDOM\s0
693 Implements an instantiable object form of a \s-1PDOM\s0 tree search.
695 .IX Item "PPI::Lexer"
696 The \s-1PPI\s0 Lexer. Converts Token streams into \s-1PDOM\s0 trees.
699 The Node object, the abstract base class for all \s-1PDOM\s0 objects that can
700 contain other Elements, such as the Document, Statement and Structure
702 .IP "PPI::Statement" 4
703 .IX Item "PPI::Statement"
704 The base class for all Perl statements. Generic \*(L"evaluate for side-effects\*(R"
705 statements are of this actual type. Other more interesting statement types
706 belong to one of its children.
708 See it's own documentation for a longer description and list of all of the
709 different statement types and sub-classes.
710 .IP "PPI::Structure" 4
711 .IX Item "PPI::Structure"
712 The abstract base class for all structures. A Structure is a language
713 construct consisting of matching braces containing a set of other elements.
715 See the PPI::Structure documentation for a description and
716 list of all of the different structure types and sub-classes.
718 .IX Item "PPI::Token"
719 A token is the basic unit of content. At its most basic, a Token is just
720 a string tagged with metadata (its class, and some additional flags in
722 .IP "PPI::Token::_QuoteEngine" 4
723 .IX Item "PPI::Token::_QuoteEngine"
724 The PPI::Token::Quote and PPI::Token::QuoteLike classes provide
725 abstract base classes for the many and varied types of quote and
726 quote-like things in Perl. However, much of the actual quote login is
727 implemented in a separate quote engine, based at
728 PPI::Token::_QuoteEngine.
730 Classes that inherit from PPI::Token::Quote, PPI::Token::QuoteLike
731 and PPI::Token::Regexp are generally parsed only by the Quote Engine.
732 .IP "PPI::Tokenizer" 4
733 .IX Item "PPI::Tokenizer"
734 The \s-1PPI\s0 Tokenizer. One Tokenizer consumes a chunk of text and provides
735 access to a stream of PPI::Token objects.
737 The Tokenizer is very very complicated, to the point where even the author
738 treads carefully when working with it.
740 Most of the complication is the result of optimizations which have tripled
741 the tokenization speed, at the expense of maintainability. We cope with the
742 spaghetti by heavily commenting everything.
743 .IP "PPI::Transform" 4
744 .IX Item "PPI::Transform"
745 The Perl Document Transformation \s-1API\s0. Provides a standard interface and
746 abstract base class for objects and classes that manipulate Documents.
748 .IX Header "INSTALLING"
749 The core \s-1PPI\s0 distribution is pure Perl and has been kept as tight as
750 possible and with as few dependencies as possible.
752 It should download and install normally on any platform from within
753 the \s-1CPAN\s0 and \s-1CPANPLUS\s0 applications, or directly using the distribution
754 tarball. If installing by hand, you may need to install a few small
755 utility modules first. The exact ones will depend on your version of
758 There are no special install instructions for \s-1PPI\s0, and the normal
759 \&\f(CW\*(C`Perl Makefile.PL\*(C'\fR, \f(CW\*(C`make\*(C'\fR, \f(CW\*(C`make test\*(C'\fR, \f(CW\*(C`make install\*(C'\fR instructions
762 .IX Header "EXTENDING"
763 The \s-1PPI\s0 namespace itself is reserved for the sole use of the modules under
764 the umbrella of the \f(CW\*(C`Parse::Perl\*(C'\fR SourceForge project.
766 <http://sf.net/projects/parseperl>
768 You are recommended to use the PPIx:: namespace for PPI-specific
769 modifications or prototypes thereof, or Perl:: for modules which provide
770 a general Perl language-related functions.
772 If what you wish to implement looks like it fits into PPIx:: namespace,
773 you should consider contacting the \f(CW\*(C`Parse::Perl\*(C'\fR mailing list (detailed on
774 the SourceForge site) first, as what you want may already be in progress,
775 or you may wish to consider joining the team and doing it within the
776 \&\f(CW\*(C`Parse::Perl\*(C'\fR project itself.
779 \&\- Many more analysis and utility methods for \s-1PDOM\s0 classes
781 \&\- Creation of a PPI::Tutorial document
783 \&\- Add many more key functions to \s-1PPI::XS\s0
785 \&\- We can \fBalways\fR write more and better unit tests
787 \&\- Complete the full implementation of \->literal (1.200)
789 \&\- Full understanding of scoping (due 1.300)
792 This module is stored in an Open Repository at the following address.
794 <http://svn.ali.as/cpan/trunk/PPI>
796 Write access to the repository is made available automatically to any
797 published \s-1CPAN\s0 author, and to most other volunteers on request.
799 If you are able to submit your bug report in the form of new (failing)
800 unit tests, or can apply your fix directly instead of submitting a patch,
801 you are \fBstrongly\fR encouraged to do so, as the author currently maintains
802 over 100 modules and it can take some time to deal with non\-\*(L"Critical\*(R" bug
805 This will also guarentee that your issue will be addressed in the next
806 release of the module.
808 For large changes though, please consider creating a branch so that they
809 can be properly reviewed and trialed before being applied to the trunk.
811 If you cannot provide a direct test or fix, or don't have time to do so,
812 then regular bug reports are still accepted and appreciated via the \s-1CPAN\s0
815 <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=PPI>
817 For other issues or questions, contact the \f(CW\*(C`Parse::Perl\*(C'\fR project mailing
820 For commercial or media-related enquiries, or to have your \s-1SVN\s0 commit bit
821 enabled, contact the author.
824 Adam Kennedy <adamk@cpan.org>
825 .SH "ACKNOWLEDGMENTS"
826 .IX Header "ACKNOWLEDGMENTS"
827 A huge thank you to Phase N Australia (<http://phase\-n.com/>) for
828 permitting the original open sourcing and release of this distribution
829 from what was originally several thousand hours of commercial work.
831 Another big thank you to The Perl Foundation
832 (<http://www.perlfoundation.org/>) for funding for the final big
833 refactoring and completion run.
835 Also, to the various co-maintainers that have contributed both large and
836 small with tests and patches and especially to those rare few who have
837 deep-dived into the guts to (gasp) add a feature.
840 \& \- Dan Brook : PPIx::XPath, Acme::PerlML
841 \& \- Audrey Tang : "Line Noise" Testing
842 \& \- Arjen Laarhoven : Three\-element \->location support
843 \& \- Elliot Shank : Perl 5.10 support, five\-element \->location
846 And finally, thanks to those brave ( and foolish :) ) souls willing to dive
847 in and use, test drive and provide feedback on \s-1PPI\s0 before version 1.000,
848 in some cases before it made it to beta quality, and still did extremely
849 distasteful things (like eating 50 meg of \s-1RAM\s0 a second).
851 I owe you all a beer. Corner me somewhere and collect at your convenience.
852 If I missed someone who wasn't in my email history, thank you too :)
855 \& # In approximate order of appearance
856 \& \- Claes Jacobsson
857 \& \- Michael Schwern
858 \& \- Jeff T. Parsons
859 \& \- CPAN Author "CHOCOLATEBOY"
860 \& \- Robert Rotherberg
861 \& \- CPAN Author "PODMASTER"
862 \& \- Richard Soderberg
863 \& \- Nadim ibn Hamouda el Khemir
864 \& \- Graciliano M. P.
869 \& \- Michael Schilli
872 \& \- Tony Stubblebine
873 \& \- Tatsuhiko Miyagawa
874 \& \- CPAN Author "CHROMATIC"
879 \& \- Johan Lindstrom
882 And to single one person out, thanks go to Randal Schwartz who
883 spent a great number of hours in \s-1IRC\s0 over a critical 6 month period
884 explaining why Perl is impossibly unparsable and constantly shoving evil
885 and ugly corner cases in my face. He remained a tireless devil's advocate,
886 and without his support this project genuinely could never have been
889 So for my schooling in the Deep Magiks, you have my deepest gratitude Randal.
891 .IX Header "COPYRIGHT"
892 Copyright 2001 \- 2009 Adam Kennedy.
894 This program is free software; you can redistribute
895 it and/or modify it under the same terms as Perl itself.
897 The full text of the license can be found in the
898 \&\s-1LICENSE\s0 file included with this module.
projects / catagits/Gitalist.git / blob