A slightly better layout for the diagram.
[p5sagit/p5-mst-13.2.git] / Porting / repository.pod
CommitLineData
0c97a5ed 1=head1 NAME
2
3repository - Using the Perl repository
4
0cfb3454 5=head1 Synopsis
6
7First, we assume here that you have already decided that you will
8need B<write> access to the repository. If all you need is B<read>
9access, there are much better ways to access the most current state of
10the perl repository, or explore individual files and patches therein.
11See L<perlhack> for details.
12
13This document describes what a Perl Porter needs to do to start using
14the Perl repository.
0c97a5ed 15
16=head1 Prerequisites
17
18You'll need to get hold of the following software.
19
20=over 4
21
22=item Perforce
23
24Download a perforce client from:
25
26 http://www.perforce.com/perforce/loadprog.html
27
28You'll probably also want to look at:
29
30 http://www.perforce.com/perforce/technical.html
31
32where you can look at or download its documentation.
33
34=item ssh
35
36If you don't already have access to an ssh client, then look at its
37home site C<http://www.cs.hut.fi/ssh> which mentions ftp sites from
38which it's available. You only need to build the client parts (ssh
39and ssh-keygen should suffice).
40
dbb93a7d 41If you're on Windows then you might like to obtain MSYS (Minimal System)
42from:
43
44 http://www.mingw.org/download.shtml
45
46which contains an ssh client. If you use this outside of the MSYS
47environment then you'll need to ensure the HOME environment variable
48is set to a suitable directory: ssh.exe will want to access files in
49a F<.ssh> sub-directory of %HOME%.
50
51Alternatively, the "plink" program, part of PuTTY:
52
53 http://www.chiark.greenend.org.uk/~sgtatham/putty/
54
55should also work fine for Windows users.
56
0c97a5ed 57=back
58
59=head1 Creating an SSH Key Pair
60
61If you already use ssh and want to use the same key pair for perl
62repository access then you can skip the rest of this section.
63Otherwise, generate an ssh key pair for use with the repository
64by typing the command
65
66 ssh-keygen
67
68After generating a key pair and testing it, ssh-keygen will ask you
69to enter a filename in which to save the key. The default it offers
70will be the file F<~/.ssh/identity> which is suitable unless you
71particularly want to keep separate ssh identities for some reason.
72If so, you could save the perl repository private key in the file
73F<~/.ssh/perl>, for example, but I will use the standard filename
74in the remainder of the examples of this document.
75
76After typing in the filename, it will prompt you to type in a
77passphrase. The private key will itself be encrypted so that it is
78usable only when that passphrase is typed. (When using ssh, you will
79be prompted when it requires a pass phrase to unlock a private key.)
80If you provide a blank passphrase then no passphrase will be needed
81to unlock the key and, as a consequence, anyone who gains access to
82the key file gains access to accounts protected with that key
83(barring additional configuration to restrict access by IP address).
84
85When you have typed the passphrase in twice, ssh-keygen will confirm
86where it has saved the private key (in the filename you gave and
87with permissions set to be only readable by you), what your public
88key is (don't worry: you don't need to memorise it) and where it
89has saved the corresponding public key. The public key is saved in
90a filename corresponding to your private key's filename but with
91".pub" appended, usually F<~/.ssh/identity.pub>. That public key
92can be (but need not be) world readable. It is not used by your
93own system at all.
94
dbb93a7d 95Note that the above process creates a key pair for ssh protocol 1.
96You can request ssh protocol 2 (RSA) instead if you prefer (if your
97particular ssh client supports it), via the command
98
99 ssh-keygen -t rsa
100
101This will create private/public identity files called F<~/.ssh/id_rsa>
102and F<~/.ssh/id_rsa.pub> respectively. Protocol 2 offers a higher
103level of security than protocol 1. This is not required for access to
104the Perl repository -- ssh is used for authentication rather than
105encryption (the Perl sources are open anyway) -- but either protocol
106is supported by the server.
107
0c97a5ed 108=head1 Notifying the Repository Keeper
109
110Mail the contents of that public key file to the keeper of the perl
111repository (see L</Contact Information> below).
112When the key is added to the repository host's configuration file,
113you will be able to connect to it with ssh by using the corresponding
114private key file (after unlocking it with your chosen passphrase).
115
dbb93a7d 116There is no harm in creating both protocol 1 and protocol 2 keys and
117mailing them both in. That way you'll be able to connect using either
118protocol, which may be useful if you later find yourself using a client
119that only supports one or the other protocol.
120
0c97a5ed 121=head1 Connecting to the Repository
122
123Connections to the repository are made by using ssh to provide a
124TCP "tunnel" rather than by using ssh to login to or invoke any
dbb93a7d 125ordinary commands on the repository.
126
127The ssh (secure shell) protocol runs over port number 22, so if you
128have a firewall installed at the client end then you must ensure that
129it is configured to allow you to make an outgoing connection to port 22
130on sickle.activestate.com.
131
132When you want to start a session using the repository, use the command:
0c97a5ed 133
4bc101a5 134 ssh -l perlrep -f -q -x -L 1666:127.0.0.1:1666 sickle.activestate.com foo
0c97a5ed 135
dbb93a7d 136If you are not using the default filename of F<~/.ssh/identity> or
137F<~/.ssh/id_rsa> to hold your perl repository private key then you'll
138need to add the option B<-i filename> to tell ssh where it is. Unless
139you chose a blank passphrase for that private key, ssh will prompt you
140for the passphrase to unlock that key. Then ssh will fork and put itself
0c97a5ed 141in the background, returning you (silently) to your shell prompt.
dbb93a7d 142
143Note that the first time you connect you may see a message like
144"The authenticity of host 'sickle.activestate.com' can't be established,"
145and asking you if you want to continue. Just answer yes and sickle's
146details will be cached in a F<known_hosts> or F<known_hosts2> file. You
147will not see that message again unless you delete the cache file.
148
0c97a5ed 149The tunnel for repository access is now ready for use.
150
151For the sake of completeness (and for the case where the chosen
152port of 1666 is already in use on your machine), I'll briefly
153describe what all those ssh arguments are for.
154
155=over 4
156
4bc101a5 157=item B<-l perlrep>
0c97a5ed 158
4bc101a5 159Use a remote username of perlrep. (The account on the repository which
160provides the end-point of the ssh tunnel is named "perlrep".)
0c97a5ed 161
162=item B<-f>
163
164Tells ssh to fork and remain running in the background. Since ssh
165is only being used for its tunnelling capabilities, the command
166that ssh runs never does any I/O and can sit silently in the
167background.
168
169=item B<-q>
170
171Tells ssh to be quiet. Without this option, ssh will output a
172message each time you use a p4 command (since each p4 command
173tunnels over the ssh connection to reach the repository).
174
175=item B<-x>
176
177Tells ssh not to bother to set up a tunnel for X11 connections.
178The repository doesn't allow this anyway.
179
180=item B<-L 1666:127.0.0.1:1666>
181
182This is the important option. It tells ssh to listen out for
183connections made to port 1666 on your local machine. When such
184a connection is made, the ssh client tells the remote side
185(the corresponding ssh daemon on the repository) to make a
186connection to IP address 127.0.0.1, port 1666. Data flowing
187along that connection is tunnelled over the ssh connection
188(encrypted). The perforce daemon running on the repository
189only accepts connections from localhost and that is exactly
190where ssh-tunnelled connections appear to come from.
191
192If port 1666 is already in use on your machine then you can
193choose any non-privileged port (a number between 1024 and 65535)
194which happens to be free on your machine. It's the first of the
195three colon separated values that you should change. Picking
196port 2345 would mean changing the option to
197B<-L 2345:127.0.0.1:1666>. Whatever port number you choose should
198be used for the value of the P4PORT environment variable (q.v.).
199
200=item sickle.activestate.com
201
4bc101a5 202This is the canonical name of the host on which the perl repository
1cb8ef30 203resides.
0c97a5ed 204
205=item foo
206
207This is a dummy place holder argument. Without an argument
208here, ssh will try to perform an interactive login to the
209repository which is not allowed. Ordinarily, this argument
210is for the one-off command which is to be executed on the
211remote host. However, the repository's ssh configuration
212file uses the "command=" option to force a particular
213command to run so the actual value of the argument is
214ignored. The command that's actually run merely pauses and
215waits for the ssh connection to drop, then exits.
216
217=back
218
219=head1 Problems
220
221You should normally get a prompt that asks for the passphrase
222for your RSA key when you connect with the ssh command shown
223above. If you see a prompt that looks like:
224
225 perlrep@sickle.activestate.com's password:
226
dbb93a7d 227Then you either don't have a F<~/.ssh/identity> or F<~/.ssh/id_rsa>
228file corresponding to your public key, or that file is not readable.
0c97a5ed 229Fix the problem and try again.
230
dbb93a7d 231If you only had the public key file for one protocol installed at the
232server end then make sure your client is using the corresponding
233protocol. An ssh client that supports protocol 2 will probably choose
234that by default, which will fail if the server end only has your public
235key file for protocol 1. Some ssh clients have "-1" and "-2" arguments
236to force which protocol to use.
237
238The "-v" (verbose) flag can be useful for seeing what protocol your
239client is actually trying to connect with, and for spotting any other
240problems. The flag can be specified multiple times to increase
241verbosity. Note that specifying the "-q" flag as well might override
242your request for verbose output, so drop the "-q" flag when trying this.
243
0c97a5ed 244=head1 Using the Perforce Client
245
246Remember to read the documentation for Perforce. You need
247to make sure that three environment variable are set
248correctly before using the p4 client with the perl repository.
249
250=over 4
251
252=item P4PORT
253
254Set this to localhost:1666 (the port for your ssh client to listen on)
255unless that port is already in use on your host. If it is, see
256the section above on the B<-L 1666:127.0.0.1:1666> option to ssh.
257
258=item P4CLIENT
259
260The value of this is the name by which Perforce knows your
2f6eead3 261host's workspace. You need to pick a name (normally, your
83d05790 262Perforce username, a dash, and your host's short name)
0c97a5ed 263when you first start using the perl repository and then
2f6eead3 264stick with it.
265
266Perforce keeps track of the files you have on your machine. It
267does this through your client. When you first sync a version of a
268file, the file comes from the server to your machine. If you sync
269the same file again the server does nothing because it
270knows you already have the file.
271
272You should NOT use the same client on different machines. If you do
273you probably won't get the files you expect, and may end up with
274nasty corruption. Perforce allows you to have as many clients as
275you want. For example, sally-home, sally-openbsd, sally-laptop.
276
277Also, never change the client's root and view at the same time.
278See C<http://www.perforce.com/perforce/doc.002/manuals/p4guide/04_details.html#1048341>
279
0c97a5ed 280If you have multiple hosts sharing the same directory structure
2f6eead3 281via NFS then you may be able to get away with only one client name,
282but be careful.
0c97a5ed 283
284The C<p4 clients> command lists all currently known clients.
285
286=item P4USER
287
288This is the username by which perforce knows you. Use your
289username if you have a well known or obvious one or else pick
290a new one which other perl5-porters will recognise. There is
2f6eead3 291a licence limit on the number of these usernames, so be sure not
292to use more than one.
293
294It is very important to set a password for your Perforce username,
295or else anyone can impersonate you. Use the C<p4 passwd> command
296to do this. Once a password is set for your account, you'll need
297to tell Perforce what it is. You can do this by setting the
298environment variable P4PASSWD, or you can use the C<-P> flag
299with the C<p4> command.
300
301There are a few techniques you can use to avoid having to either
302set an environment variable or type the password on every command.
303One is to create a shell alias, for example, in bash, add something like
304 alias p4='p4 -P secret'
305to your F<.bash_profile> file. Another way is to create a small shell
306script, for example
307 #!/bin/bash
308 p4 -P secret $@
309And use this instead of running C<p4> directly.
310
311With either of these, be sure the file containing your password
312(the F<.bash_profile> or shell script file) is only readable by you.
0c97a5ed 313
314The C<p4 users> command lists all currently known users.
315
316=back
317
dbb93a7d 318Note that on Windows P4PORT and P4USER are requested when installing
319Perforce. They are stored in the registry, so they do not need to be
320set in the environment.
321
0c97a5ed 322Once these three environment variables are set, you can use the
323perforce p4 client exactly as described in its documentation.
dbb93a7d 324
0c97a5ed 325After setting these variables and connecting to the repository
83d05790 326for the first time, you should use the C<p4 user> command to
dbb93a7d 327set a valid email address for yourself. Messages to the commit list
328are sent (faked) from whatever email address you set here.
329
330Also use the C<p4 client> command to specify your workspace
331specifications for each individual client from which you will interact
332with the repository. The P4CLIENT environment variable, of course,
333needs to be set to one of these client workspace names.
0c97a5ed 334
335=head1 Ending a Repository Session
336
337When you have finished a session using the repository, you
338should kill off the ssh client process to break the tunnel.
339Since ssh forked itself into the background, you'll need to use
340something like ps with the appropriate options to find the ssh
341process and then kill it manually. The default signal of
342SIGTERM is fine.
343
344=head1 Overview of the Repository
345
346Please read at least the introductory sections of the Perforce
347User Guide (and perhaps the Quick Start Guide as well) before
348reading this section.
349
350Every repository user typically "owns" a "branch" of the mainline
351code in the repository. They hold the "pumpkin" for things in this
352area, and are usually the only user who will modify files there.
353This is not strictly enforced in order to allow the flexibility
354of other users stealing the pumpkin for short periods with the
355owner's permission.
356
dbb93a7d 357Here is (part of) the current structure of the repository:
0c97a5ed 358
359 /----+-----perl - Mainline development (bleadperl)
f704d51e 360 +-----perlio - PerlIO Pumpkin's Perl
0c97a5ed 361 +-----vmsperl - VMS Pumpkin's Perl
362 +-----maint-5.004------perl - Maintainance branches
363 +-----maint-5.005------perl
dbb93a7d 364 +-----maint-5.6--------perl
365 +-----maint-5.8--------perl
366 +-----pureperl---------pureperl
0c97a5ed 367
368Perforce uses a branching model that simply tracks relationships
369between files. It does not care about directories at all, so
370any file can be a branch of any other file--the fully qualified
371depot path name (of the form //depot/foo/bar.c) uniquely determines
372a file for the purpose of establishing branching relationships.
373Since a branch usually involves hundreds of files, such relationships
374are typically specified en masse using a branch map (try `p4 help branch`).
375`p4 branches` lists the existing branches that have been set up.
376`p4 branch -o branchname` can be used to view the map for a particular
377branch, if you want to determine the ancestor for a particular set of
378files.
379
380The mainline (aka "trunk") code in the Perl repository is under
381"//depot/perl/...". Most branches typically map its entire
382contents under a directory that goes by the same name as the branch
f704d51e 383name. Thus the contents of the perlio branch are to be found
384in //depot/perlio.
0c97a5ed 385
386Run `p4 client` to specify how the repository contents should map to
387your local disk. Most users will typically have a client map that
388includes at least their entire branch and the contents of the mainline.
389
390Run `p4 changes -l -m10` to check on the activity in the repository.
391//depot/perl/Porting/genlog is useful to get an annotated changelog
392that shows files and branches. You can use this listing to determine
393if there are any changes in the mainline that you need to merge into
394your own branch. A typical merging session looks like this:
395
f704d51e 396 % cd ~/p4view/perlio
397 % p4 integrate -b perlio # to bring parent changes into perlio
398 % p4 resolve -am ./... # auto merge the changes
0c97a5ed 399 % p4 resolve ./... # manual merge conflicting changes
400 % p4 submit ./... # check in
401
f704d51e 402If the owner of the mainline wants to bring the changes in perlio
0c97a5ed 403back into the mainline, they do:
404
f704d51e 405 % p4 integrate -r -b perlio
0c97a5ed 406 ...
407
408Generating a patch for change#42 is done as follows:
409
a2c6387b 410 % p4genpatch 42 > change-42.patch
0c97a5ed 411
a2c6387b 412F<p4genpatch> is to be found in //depot/perl/Porting/.
f704d51e 413
414The usual routine to apply a patch is
415
416 % p4 edit file.c file.h
417 % patch < patch.txt
418
419(any necessary, re-Configure, make regen_headers, make clean, etc, here)
420
421 % make all test
422
423(preferably make all test in several platforms and under several
424different Configurations)
425
426 % while unhappy
427 do
428 $EDITOR
429 make all test
430 done
431 % p4 submit
432
433Other useful Perforce commands
434
435 % p4 describe -du 12345 # show change 12345
436
437Note: the output of "p4 describe" is not in proper diff format, use
a2c6387b 438the F<Porting/p4genpatch> to get a diff-compatible format.
44897049 439(Note that it may be easier to get one already prepared: grep
440L<perlhack> for APC, and append eg "/diffs/12345.gz" to one of the
441URLs to get a usable patch.)
f704d51e 442
443 % p4 diff -se ./... # have I modified something but forgotten
444 # to "p4 edit", easy faux pas with autogenerated
445 # files like proto.h, or if one forgets to
446 # look carefully which files a patch modifies
447 % p4 sync file.h # if someone else has modified file.h
448 % p4 opened # which files are opened (p4 edit) by me
449 % p4 opened -a # which files are opened by anybody
450 % p4 diff -du file.c # what changes have I done
451 % p4 revert file.h # never mind my changes
452 % p4 sync -f argh.c # forcibly synchronize your file
453 # from the repository
454 % p4 diff -sr | p4 -x - revert
455 # throw away (opened but) unchanged files
456 # (in Perforce it's a little bit too easy
457 # to checkin unchanged files)
458
459Integrate patch 12345 from the mainline to the maint-5.6 branch:
460(you have to in the directory that has both the mainline and
461the maint-5.6/perl as subdirectories)
462
463 % p4 integrate -d perl/...@12345,12345 maint-5.6/perl/...
464
465Integrate patches 12347-12350 from the perlio branch to the mainline:
466
467 % p4 integrate -d perlio/...@12347,12350 perl/...
0c97a5ed 468
469=head1 Contact Information
470
dbb93a7d 471The mail alias E<lt>perl-repository-keepers@perl.orgE<gt> can be used to reach
b09defb6 472all current users of the repository.
0c97a5ed 473
474The repository keeper is currently Gurusamy Sarathy
dbb93a7d 475E<lt>gsar@activestate.comE<gt>.
0c97a5ed 476
477=head1 AUTHORS
478
dbb93a7d 479Malcolm Beattie, E<lt>mbeattie@sable.ox.ac.ukE<gt>, 24 June 1997.
480
481Gurusamy Sarathy, E<lt>gsar@activestate.comE<gt>, 8 May 1999.
0c97a5ed 482
dbb93a7d 483Slightly updated by Simon Cozens, E<lt>simon@brecon.co.ukE<gt>, 3 July 2000.
0c97a5ed 484
dbb93a7d 485More updates by Jarkko Hietaniemi, E<lt>jhi@iki.fiE<gt>, 28 June 2001.
f704d51e 486
dbb93a7d 487Perforce clarifications by Randall Gellens, E<lt>rcg@users.sourceforge.netE<gt>, 12 July 2001.
0c97a5ed 488
dbb93a7d 489Windows-related updates by Steve Hay E<lt>shay@cpan.orgE<gt>, 23 July 2004.
2f6eead3 490
0c97a5ed 491=cut