Commit | Line | Data |
adfe19db |
1 | =head1 NAME |
2 | |
3 | HACKERS - Devel::PPPort internals for hackers |
4 | |
5 | =head1 SYNOPSIS |
6 | |
7 | So you probably want to hack C<Devel::PPPort>? |
8 | |
9 | Well, here's some information to get you started with what's |
10 | lying around in this distribution. |
11 | |
12 | =head1 DESCRIPTION |
13 | |
c1a049cb |
14 | =head2 How to build 114 versions of Perl |
adfe19db |
15 | |
16 | C<Devel::PPPort> supports Perl versions between 5.003 and bleadperl. |
17 | To guarantee this support, I need some of these versions on my |
c1a049cb |
18 | machine. I currently have 114 different Perl version/configuration |
adfe19db |
19 | combinations installed on my laptop. |
20 | |
21 | As many of the old Perl distributions need patching to compile |
c1a049cb |
22 | cleanly on newer systems (and because building 114 Perls by hand |
adfe19db |
23 | just isn't fun), I wrote a tool to build all the different |
24 | versions and configurations. You can find it in F<devel/buildperl.pl>. |
25 | It can currently build the following Perl releases: |
26 | |
27 | 5.003 |
28 | 5.004 - 5.004_05 |
29 | 5.005 - 5.005_04 |
30 | 5.6.x |
31 | 5.7.x |
32 | 5.8.x |
33 | 5.9.x |
c1a049cb |
34 | 5.10.x |
adfe19db |
35 | |
36 | =head2 Fully automatic API checks |
37 | |
38 | Knowing which parts of the API are not backwards compatible and |
39 | probably need C<Devel::PPPort> support is another problem that's |
40 | not easy to deal with manually. If you run |
41 | |
42 | perl Makefile.PL --with-apicheck |
43 | |
44 | a C file is generated by F<parts/apicheck.pl> that is compiled |
45 | and linked with C<Devel::PPPort>. This C file has the purpose of |
46 | using each of the public API functions/macros once. |
47 | |
48 | The required information is derived from C<parts/embed.fnc> (just |
679ad62d |
49 | a copy of bleadperl's C<embed.fnc>), C<parts/apidoc.fnc> (which |
adfe19db |
50 | is generated by F<devel/mkapidoc.sh> and simply collects the rest |
679ad62d |
51 | of the apidoc entries spread over the Perl source code) and |
52 | C<parts/ppport.fnc> (which lists all API provided purely by |
53 | Devel::PPPort). |
adfe19db |
54 | The generated C file C<apicheck.c> is currently about 500k in size |
55 | and takes quite a while to compile. |
56 | |
57 | Usually, C<apicheck.c> won't compile with older perls. And even if |
58 | it compiles, there's still a good chance of the dynamic linker |
59 | failing at C<make test> time. But that's on purpose! |
60 | |
61 | We can use these failures to find changes in the API automatically. |
62 | The two Perl scripts F<devel/mktodo> and F<devel/mktodo.pl> |
63 | repeatedly run C<Devel::PPPort> with the apicheck code through |
64 | all different versions of perl. Scanning the output of the compiler |
65 | and the dynamic linker for errors, the files in F<parts/todo/> are |
66 | generated. These files list all parts of the public API that don't |
67 | work with less than a certain version of Perl. |
68 | |
69 | This information is in turn used by F<parts/apicheck.pl> to mask |
70 | API calls in the generated C file for these versions, so the |
71 | process can be stopped by the time F<apicheck.c> compiles cleanly |
ba120f6f |
72 | and the dynamic linker is happy. (Actually, this process may generate |
73 | false positives, so by default each API call is checked once more |
74 | afterwards.) |
adfe19db |
75 | |
ba120f6f |
76 | Running C<devel/mktodo> takes about an hour, depending of course |
77 | on the machine you're running it on. If you run it with |
78 | the C<--nocheck> option, it won't recheck the API calls that failed |
79 | in the compilation stage and it'll take significantly less time. |
80 | Running with C<--nocheck> should usually be safe. |
adfe19db |
81 | |
82 | When running C<devel/mktodo> with the C<--base> option, it will |
83 | generate the I<baseline> todo files by disabling all functionality |
84 | provided by C<Devel::PPPort>. These are required for implementing |
85 | the C<--compat-version> option of the C<ppport.h> script. The |
86 | baseline todo files hold the information about which version of |
87 | Perl lacks a certain part of the API. |
88 | |
89 | However, only the documented public API can be checked this way. |
90 | And since C<Devel::PPPort> provides more macros, these would not be |
91 | affected by C<--compat-version>. It's the job of F<devel/scanprov> |
92 | to figure out the baseline information for all remaining provided |
93 | macros by scanning the include files in the F<CORE> directory of |
94 | various Perl versions. |
95 | |
ba120f6f |
96 | The whole process isn't platform independent. It has currently been |
97 | tested only under Linux, and it definitely requires at least C<gcc> and |
98 | the C<nm> utility. |
99 | |
100 | It's not very often that one has to regenerate the baseline and todo |
101 | files. If you have to, you can either run F<devel/regenerate> or just |
102 | execute the following steps by hand: |
adfe19db |
103 | |
104 | =over 4 |
105 | |
106 | =item * |
107 | |
108 | You need a whole bunch of different Perls. The more, the better. |
109 | You can use F<devel/buildperl.pl> to build them. I keep my perls |
110 | in F</tmp/perl>, so most of the tools take this as a default. |
111 | |
112 | =item * |
113 | |
56093a11 |
114 | You also need a freshly built bleadperl that is in the path under |
115 | exactly this name. (The name of the executable is currently hardcoded |
116 | in F<devel/mktodo> and F<devel/scanprov>.) |
117 | |
118 | =item * |
119 | |
adfe19db |
120 | Remove all existing todo files in the F<parts/base> and |
121 | F<parts/todo> directories. |
122 | |
123 | =item * |
124 | |
125 | Update the API information. Copy the latest F<embed.fnc> file from |
126 | bleadperl to the F<parts> directory and run F<devel/mkapidoc.sh> to |
127 | collect the remaining information in F<parts/apidoc.fnc>. |
128 | |
129 | =item * |
130 | |
131 | Build the new baseline by running |
132 | |
133 | perl devel/mktodo --base |
134 | |
135 | in the root directory of the distribution. When it's finished, |
136 | move all files from the F<parts/todo> directory to F<parts/base>. |
137 | |
138 | =item * |
139 | |
140 | Build the new todo files by running |
141 | |
142 | perl devel/mktodo |
143 | |
144 | in the root directory of the distribution. |
145 | |
146 | =item * |
147 | |
148 | Finally, add the remaining baseline information by running |
149 | |
4a582685 |
150 | perl Makefile.PL && make |
151 | perl devel/scanprov write |
adfe19db |
152 | |
153 | =back |
154 | |
155 | =head2 Implementation |
156 | |
157 | Residing in F<parts/inc/> is the "heart" of C<Devel::PPPort>. Each |
158 | of the files implements a part of the supported API, along with |
159 | hints, dependency information, XS code and tests. |
160 | The files are in a POD-like format that is parsed using the |
161 | functions in F<parts/ppptools.pl>. |
162 | |
163 | The scripts F<PPPort_pm.PL>, F<PPPort_xs.PL> and F<mktests.PL> all |
164 | use the information in F<parts/inc/> to generate the main module |
1d088ed8 |
165 | F<PPPort.pm>, the XS code in F<RealPPPort.xs> and various test files |
adfe19db |
166 | in F<t/>. |
167 | |
168 | All of these files could be generated on the fly while building |
1d088ed8 |
169 | C<Devel::PPPort>, but not having the tests in C<t/> will confuse |
170 | TEST/harness in the core. Not having F<PPPort.pm> will be bad for |
171 | viewing the docs on C<search.cpan.org>. So unfortunately, it's |
172 | unavoidable to put some redundancy into the package. |
adfe19db |
173 | |
174 | =head2 Adding stuff to Devel::PPPort |
175 | |
176 | First, check if the code you plan to add fits into one of the |
177 | existing files in F<parts/inc/>. If not, just start a new one and |
178 | remember to include it from within F<PPPort_pm.PL>. |
179 | |
180 | Each file holds all relevant data for implementing a certain part |
181 | of the API: |
182 | |
183 | =over 2 |
184 | |
185 | =item * |
186 | |
187 | A list of the provided API in the C<=provides> section. |
188 | |
189 | =item * |
190 | |
191 | The implementation to add to F<ppport.h> in the C<=implementation> |
192 | section. |
193 | |
194 | =item * |
195 | |
196 | The code required to add to PPPort.xs for testing the implementation. |
197 | This code goes into the C<=xshead>, C<=xsinit>, C<=xsmisc>, C<=xsboot> |
0d0f8426 |
198 | and C<=xsubs> section. Have a look at the template at the bottom |
199 | of F<PPPort_xs.PL> to see where the code ends up. |
adfe19db |
200 | |
201 | =item * |
202 | |
203 | The tests in the C<=tests> section. Remember not to use any fancy |
204 | modules or syntax elements, as the test code should be able to run |
205 | with Perl 5.003, which, for example, doesn't support C<my> in |
206 | C<for>-loops: |
207 | |
0d0f8426 |
208 | for my $x (1, 2, 3) { } # won't work with 5.003 |
adfe19db |
209 | |
0d0f8426 |
210 | You can use C<ok()> to report success or failure: |
211 | |
212 | ok($got == 42); |
213 | ok($got, $expected); |
214 | |
215 | Regular expressions are not supported as the second argument to C<ok>, |
216 | because older perls do not support the C<qr> operator. |
adfe19db |
217 | |
218 | =back |
219 | |
220 | It's usually the best approach to just copy an existing file and |
221 | use it as a template. |
222 | |
0d0f8426 |
223 | =head2 Implementation Hints |
224 | |
225 | In the C<=implementation> section, you can use |
226 | |
227 | __UNDEFINED__ macro some definition |
228 | |
229 | instead of |
230 | |
231 | #ifndef macro |
232 | # define macro some definition |
233 | #endif |
234 | |
235 | The macro can have optional arguments and the definition can even |
236 | span multiple lines, like in |
237 | |
238 | __UNDEFINED__ SvMAGIC_set(sv, val) \ |
239 | STMT_START { assert(SvTYPE(sv) >= SVt_PVMG); \ |
240 | (((XPVMG*) SvANY(sv))->xmg_magic = (val)); } STMT_END |
241 | |
242 | This usually makes the code more compact and readable. And you |
243 | only have to add C<__UNDEFINED__> to the C<=provided> section. |
244 | |
245 | Version checking can be tricky if you want to do it correct. |
246 | You can use |
247 | |
248 | #if { VERSION < 5.9.3 } |
249 | |
250 | instead of |
251 | |
252 | #if ((PERL_VERSION < 9) || (PERL_VERSION == 9 && PERL_SUBVERSION < 3)) |
253 | |
254 | The version number can be either of the new form C<5.x.x> or of the older |
255 | form C<5.00x_yy>. Both are translated into the correct preprocessor |
256 | statements. It is also possible to combine this with other statements: |
257 | |
258 | #if { VERSION >= 5.004 } && !defined(sv_vcatpvf) |
259 | /* a */ |
260 | #elif { VERSION < 5.004_63 } && { VERSION != 5.004_05 } |
261 | /* b */ |
262 | #endif |
263 | |
264 | This not only works in the C<=implementation> section, but also in |
265 | the C<=xsubs>, C<=xsinit>, C<=xsmisc>, C<=xshead> and C<=xsboot> sections. |
266 | |
adfe19db |
267 | =head2 Testing |
268 | |
269 | To automatically test C<Devel::PPPort> with lots of different Perl |
270 | versions, you can use the F<soak> script. Just pass it a list of |
271 | all Perl binaries you want to test. |
272 | |
273 | =head2 Special Makefile targets |
274 | |
4a582685 |
275 | You can use |
adfe19db |
276 | |
277 | make regen |
278 | |
4a582685 |
279 | to regenerate all of the autogenerated files. To get rid of all |
280 | generated files (except for F<parts/todo/*> and F<parts/base/*>), |
281 | use |
adfe19db |
282 | |
283 | make purge_all |
284 | |
285 | That's it. |
286 | |
0d0f8426 |
287 | =head2 Submitting Patches |
288 | |
289 | If you've added some functionality to C<Devel::PPPort>, please |
290 | consider submitting a patch with your work to either the author |
291 | (E<lt>mhx@cpan.orgE<gt>) or to the CPAN Request Tracker at |
292 | L<http://rt.cpan.org>. |
293 | |
294 | When submitting patches, please only add the relevant changes |
295 | and don't include the differences of the generated files. You |
296 | can use the C<purge_all> target to delete all autogenerated |
297 | files. |
298 | |
56093a11 |
299 | =head2 Integrating into the Perl core |
300 | |
301 | When integrating this module into the Perl core, be sure to |
302 | remove the following files from the distribution. They are |
303 | either not needed or generated on the fly when building this |
304 | module in the core: |
305 | |
306 | MANIFEST |
307 | META.yml |
308 | PPPort.pm |
309 | |
adfe19db |
310 | =head1 COPYRIGHT |
311 | |
51d6c659 |
312 | Version 3.x, Copyright (C) 2004-2009, Marcus Holland-Moritz. |
adfe19db |
313 | |
314 | Version 2.x, Copyright (C) 2001, Paul Marquess. |
315 | |
316 | Version 1.x, Copyright (C) 1999, Kenneth Albanowski. |
317 | |
318 | This program is free software; you can redistribute it and/or |
319 | modify it under the same terms as Perl itself. |
320 | |
321 | =head1 SEE ALSO |
322 | |
ba120f6f |
323 | See L<ppport.h> and L<devel/regenerate>. |
adfe19db |
324 | |
325 | =cut |
326 | |