=head2 Working with SVs
An SV can be created and loaded with one command. There are four types of
-values that can be loaded: an integer value (IV), a double (NV), a string,
-(PV), and another scalar (SV).
+values that can be loaded: an integer value (IV), a double (NV),
+a string (PV), and another scalar (SV).
The six routines are:
HE* hv_fetch_ent (HV* tb, SV* key, I32 lval, U32 hash);
HE* hv_store_ent (HV* tb, SV* key, SV* val, U32 hash);
-
+
bool hv_exists_ent (HV* tb, SV* key, U32 hash);
SV* hv_delete_ent (HV* tb, SV* key, I32 flags, U32 hash);
-
+
SV* hv_iterkeysv (HE* entry);
Note that these functions take C<SV*> keys, which simplifies writing
They also return and accept whole hash entries (C<HE*>), making their
use more efficient (since the hash number for a particular string
-doesn't have to be recomputed every time). See L<API LISTING> later in
-this document for detailed descriptions.
+doesn't have to be recomputed every time). See L<perlapi> for detailed
+descriptions.
The following macros must always be used to access the contents of hash
entries. Note that the arguments to these macros must be simple
variables, since they may get evaluated more than once. See
-L<API LISTING> later in this document for detailed descriptions of these
-macros.
+L<perlapi> for detailed descriptions of these macros.
HePV(HE* he, STRLEN len)
HeVAL(HE* he)
a vtbl_amagicelem %OVERLOAD hash element
c (none) Holds overload table (AMT) on stash
B vtbl_bm Boyer-Moore (fast string search)
+ D vtbl_regdata Regex match position data (@+ and @- vars)
+ d vtbl_regdatum Regex match position data element
E vtbl_env %ENV hash
e vtbl_envelem %ENV hash element
f vtbl_fm Formline ('compiled' format)
C<sv_cat*_mg()> functions. Similarly, generic C code must call the
C<SvGETMAGIC()> macro to invoke any 'get' magic if they use an SV
obtained from external sources in functions that don't handle magic.
-L<API LISTING> later in this document identifies such functions.
+See L<perlapi> for a description of these functions.
For example, calls to the C<sv_cat*()> functions typically need to be
followed by C<SvSETMAGIC()>, but they don't need a prior C<SvGETMAGIC()>
since their implementation handles 'get' magic.
instances of the size of the C<type> data structure (using the C<sizeof>
function).
+Here is a handy table of equivalents between ordinary C and Perl's
+memory abstraction layer:
+
+ Instead Of: Use:
+
+ malloc New
+ calloc Newz
+ realloc Renew
+ memcopy Copy
+ memmove Move
+ free Safefree
+ strdup savepv
+ strndup savepvn (Hey, strndup doesn't exist!)
+ memcpy/*(struct foo *) StructCopy
+
=head2 PerlIO
The most recent development releases of Perl has been experimenting with
=head2 Compile pass 1: check routines
-The tree is created by the I<pseudo-compiler> while yacc code feeds it
-the constructions it recognizes. Since yacc works bottom-up, so does
+The tree is created by the compiler while I<yacc> code feeds it
+the constructions it recognizes. Since I<yacc> works bottom-up, so does
the first pass of perl compilation.
What makes this pass interesting for perl developers is that some
optimization may be performed on this pass. This is optimization by
-so-called I<check routines>. The correspondence between node names
+so-called "check routines". The correspondence between node names
and corresponding check routines is described in F<opcode.pl> (do not
forget to run C<make regen_headers> if you modify this file).
=head1 How multiple interpreters and concurrency are supported
-WARNING: This information is subject to radical changes prior to
-the Perl 5.6 release. Use with caution.
-
=head2 Background and PERL_IMPLICIT_CONTEXT
The Perl interpreter can be regarded as a closed box: it has an API
which will be private. All functions whose names begin C<S_> are private
(think "S" for "secret" or "static"). All other functions begin with
"Perl_", but just because a function begins with "Perl_" does not mean it is
-part of the API. The easiest way to be B<sure> a function is part of the API
-is to find its entry in L<perlapi>. If it exists in L<perlapi>, it's part
-of the API. If it doesn't, and you think it should be (i.e., you need it fo
-r your extension), send mail via L<perlbug> explaining why you think it
-should be.
-
-(L<perlapi> itself is generated by embed.pl, a Perl script that generates
-significant portions of the Perl source code. It has a list of almost
-all the functions defined by the Perl interpreter along with their calling
-characteristics and some flags. Functions that are part of the public API
-are marked with an 'A' in its flags.)
+part of the API. (See L</Internal Functions>.) The easiest way to be B<sure> a
+function is part of the API is to find its entry in L<perlapi>.
+If it exists in L<perlapi>, it's part of the API. If it doesn't, and you
+think it should be (i.e., you need it for your extension), send mail via
+L<perlbug> explaining why you think it should be.
Second problem: there must be a syntax so that the same subroutine
declarations and calls can pass a structure as their first argument,
sv_setsv(asv, bsv);
-in your extesion will translate to this when PERL_IMPLICIT_CONTEXT is
+in your extension will translate to this when PERL_IMPLICIT_CONTEXT is
in effect:
- Perl_sv_setsv(GetPerlInterpreter(), asv, bsv);
+ Perl_sv_setsv(Perl_get_context(), asv, bsv);
or to this otherwise:
Perl_sv_setsv(asv, bsv);
You have to do nothing new in your extension to get this; since
-the Perl library provides GetPerlInterpreter(), it will all just
+the Perl library provides Perl_get_context(), it will all just
work.
The second, more efficient way is to use the following template for
There could be one or more interpreters in a process, and one or
more "hosts", with free association between them.
+=head1 Internal Functions
+
+All of Perl's internal functions which will be exposed to the outside
+world are be prefixed by C<Perl_> so that they will not conflict with XS
+functions or functions used in a program in which Perl is embedded.
+Similarly, all global variables begin with C<PL_>. (By convention,
+static functions start with C<S_>)
+
+Inside the Perl core, you can get at the functions either with or
+without the C<Perl_> prefix, thanks to a bunch of defines that live in
+F<embed.h>. This header file is generated automatically from
+F<embed.pl>. F<embed.pl> also creates the prototyping header files for
+the internal functions, generates the documentation and a lot of other
+bits and pieces. It's important that when you add a new function to the
+core or change an existing one, you change the data in the table at the
+end of F<embed.pl> as well. Here's a sample entry from that table:
+
+ Apd |SV** |av_fetch |AV* ar|I32 key|I32 lval
+
+The second column is the return type, the third column the name. Columns
+after that are the arguments. The first column is a set of flags:
+
+=over 3
+
+=item A
+
+This function is a part of the public API.
+
+=item p
+
+This function has a C<Perl_> prefix; ie, it is defined as C<Perl_av_fetch>
+
+=item d
+
+This function has documentation using the C<apidoc> feature which we'll
+look at in a second.
+
+=back
+
+Other available flags are:
+
+=over 3
+
+=item s
+
+This is a static function and is defined as C<S_whatever>.
+
+=item n
+
+This does not use C<aTHX_> and C<pTHX> to pass interpreter context. (See
+L<perlguts/Background and PERL_IMPLICIT_CONTEXT>.)
+
+=item r
+
+This function never returns; C<croak>, C<exit> and friends.
+
+=item f
+
+This function takes a variable number of arguments, C<printf> style.
+The argument list should end with C<...>, like this:
+
+ Afprd |void |croak |const char* pat|...
+
+=item m
+
+This function is part of the experimental development API, and may change
+or disappear without notice.
+
+=item o
+
+This function should not have a compatibility macro to define, say,
+C<Perl_parse> to C<parse>. It must be called as C<Perl_parse>.
+
+=item j
+
+This function is not a member of C<CPerlObj>. If you don't know
+what this means, don't use it.
+
+=item x
+
+This function isn't exported out of the Perl core.
+
+=back
+
+If you edit F<embed.pl>, you will need to run C<make regen_headers> to
+force a rebuild of F<embed.h> and other auto-generated files.
+
+=head2 Formatted Printing of IVs and UVs
+
+If you are printing IVs or UVs instead of the stdio(3) style formatting
+codes like C<%d> you should use the following macros for portability
+
+ IVdf IV in decimal
+ UVuf UV in decimal
+ UVof UV in octal
+ UVxf UV in hexadecimal
+
+For example: printf("IV is %"IVdf"\n", iv); That will expand
+to whatever is the correct format for the IVs.
+
+=head2 Source Documentation
+
+There's an effort going on to document the internal functions and
+automatically produce reference manuals from them - L<perlapi> is one
+such manual which details all the functions which are available to XS
+writers. L<perlintern> is the autogenerated manual for the functions
+which are not part of the API and are supposedly for internal use only.
+
+Source documentation is created by putting POD comments into the C
+source, like this:
+
+ /*
+ =for apidoc sv_setiv
+
+ Copies an integer into the given SV. Does not handle 'set' magic. See
+ C<sv_setiv_mg>.
+
+ =cut
+ */
+
+Please try and supply some documentation if you add functions to the
+Perl core.
+
+=head1 Unicode Support
+
+Perl 5.6.0 introduced Unicode support. It's important for porters and XS
+writers to understand this support and make sure that the code they
+write does not corrupt Unicode data.
+
+=head2 What B<is> Unicode, anyway?
+
+In the olden, less enlightened times, we all used to use ASCII. Most of
+us did, anyway. The big problem with ASCII is that it's American. Well,
+no, that's not actually the problem; the problem is that it's not
+particularly useful for people who don't use the Roman alphabet. What
+used to happen was that particular languages would stick their own
+alphabet in the upper range of the sequence, between 128 and 255. Of
+course, we then ended up with plenty of variants that weren't quite
+ASCII, and the whole point of it being a standard was lost.
+
+Worse still, if you've got a language like Chinese or
+Japanese that has hundreds or thousands of characters, then you really
+can't fit them into a mere 256, so they had to forget about ASCII
+altogether, and build their own systems using pairs of numbers to refer
+to one character.
+
+To fix this, some people formed Unicode, Inc. and
+produced a new character set containing all the characters you can
+possibly think of and more. There are several ways of representing these
+characters, and the one Perl uses is called UTF8. UTF8 uses
+a variable number of bytes to represent a character, instead of just
+one. You can learn more about Unicode at
+L<http://www.unicode.org/|http://www.unicode.org/>
+
+=head2 How can I recognise a UTF8 string?
+
+You can't. This is because UTF8 data is stored in bytes just like
+non-UTF8 data. The Unicode character 200, (C<0xC8> for you hex types)
+capital E with a grave accent, is represented by the two bytes
+C<v196.172>. Unfortunately, the non-Unicode string C<chr(196).chr(172)>
+has that byte sequence as well. So you can't tell just by looking - this
+is what makes Unicode input an interesting problem.
+
+The API function C<is_utf8_string> can help; it'll tell you if a string
+contains only valid UTF8 characters. However, it can't do the work for
+you. On a character-by-character basis, C<is_utf8_char> will tell you
+whether the current character in a string is valid UTF8.
+
+=head2 How does UTF8 represent Unicode characters?
+
+As mentioned above, UTF8 uses a variable number of bytes to store a
+character. Characters with values 1...128 are stored in one byte, just
+like good ol' ASCII. Character 129 is stored as C<v194.129>; this
+contines up to character 191, which is C<v194.191>. Now we've run out of
+bits (191 is binary C<10111111>) so we move on; 192 is C<v195.128>. And
+so it goes on, moving to three bytes at character 2048.
+
+Assuming you know you're dealing with a UTF8 string, you can find out
+how long the first character in it is with the C<UTF8SKIP> macro:
+
+ char *utf = "\305\233\340\240\201";
+ I32 len;
+
+ len = UTF8SKIP(utf); /* len is 2 here */
+ utf += len;
+ len = UTF8SKIP(utf); /* len is 3 here */
+
+Another way to skip over characters in a UTF8 string is to use
+C<utf8_hop>, which takes a string and a number of characters to skip
+over. You're on your own about bounds checking, though, so don't use it
+lightly.
+
+All bytes in a multi-byte UTF8 character will have the high bit set, so
+you can test if you need to do something special with this character
+like this:
+
+ UV uv;
+
+ if (utf & 0x80)
+ /* Must treat this as UTF8 */
+ uv = utf8_to_uv(utf);
+ else
+ /* OK to treat this character as a byte */
+ uv = *utf;
+
+You can also see in that example that we use C<utf8_to_uv> to get the
+value of the character; the inverse function C<uv_to_utf8> is available
+for putting a UV into UTF8:
+
+ if (uv > 0x80)
+ /* Must treat this as UTF8 */
+ utf8 = uv_to_utf8(utf8, uv);
+ else
+ /* OK to treat this character as a byte */
+ *utf8++ = uv;
+
+You B<must> convert characters to UVs using the above functions if
+you're ever in a situation where you have to match UTF8 and non-UTF8
+characters. You may not skip over UTF8 characters in this case. If you
+do this, you'll lose the ability to match hi-bit non-UTF8 characters;
+for instance, if your UTF8 string contains C<v196.172>, and you skip
+that character, you can never match a C<chr(200)> in a non-UTF8 string.
+So don't do that!
+
+=head2 How does Perl store UTF8 strings?
+
+Currently, Perl deals with Unicode strings and non-Unicode strings
+slightly differently. If a string has been identified as being UTF-8
+encoded, Perl will set a flag in the SV, C<SVf_UTF8>. You can check and
+manipulate this flag with the following macros:
+
+ SvUTF8(sv)
+ SvUTF8_on(sv)
+ SvUTF8_off(sv)
+
+This flag has an important effect on Perl's treatment of the string: if
+Unicode data is not properly distinguished, regular expressions,
+C<length>, C<substr> and other string handling operations will have
+undesirable results.
+
+The problem comes when you have, for instance, a string that isn't
+flagged is UTF8, and contains a byte sequence that could be UTF8 -
+especially when combining non-UTF8 and UTF8 strings.
+
+Never forget that the C<SVf_UTF8> flag is separate to the PV value; you
+need be sure you don't accidentally knock it off while you're
+manipulating SVs. More specifically, you cannot expect to do this:
+
+ SV *sv;
+ SV *nsv;
+ STRLEN len;
+ char *p;
+
+ p = SvPV(sv, len);
+ frobnicate(p);
+ nsv = newSVpvn(p, len);
+
+The C<char*> string does not tell you the whole story, and you can't
+copy or reconstruct an SV just by copying the string value. Check if the
+old SV has the UTF8 flag set, and act accordingly:
+
+ p = SvPV(sv, len);
+ frobnicate(p);
+ nsv = newSVpvn(p, len);
+ if (SvUTF8(sv))
+ SvUTF8_on(nsv);
+
+In fact, your C<frobnicate> function should be made aware of whether or
+not it's dealing with UTF8 data, so that it can handle the string
+appropriately.
+
+=head2 How do I convert a string to UTF8?
+
+If you're mixing UTF8 and non-UTF8 strings, you might find it necessary
+to upgrade one of the strings to UTF8. If you've got an SV, the easiest
+way to do this is:
+
+ sv_utf8_upgrade(sv);
+
+However, you must not do this, for example:
+
+ if (!SvUTF8(left))
+ sv_utf8_upgrade(left);
+
+If you do this in a binary operator, you will actually change one of the
+strings that came into the operator, and, while it shouldn't be noticable
+by the end user, it can cause problems.
+
+Instead, C<bytes_to_utf8> will give you a UTF8-encoded B<copy> of its
+string argument. This is useful for having the data available for
+comparisons and so on, without harming the orginal SV. There's also
+C<utf8_to_bytes> to go the other way, but naturally, this will fail if
+the string contains any characters above 255 that can't be represented
+in a single byte.
+
+=head2 Is there anything else I need to know?
+
+Not really. Just remember these things:
+
+=over 3
+
+=item *
+
+There's no way to tell if a string is UTF8 or not. You can tell if an SV
+is UTF8 by looking at is C<SvUTF8> flag. Don't forget to set the flag if
+something should be UTF8. Treat the flag as part of the PV, even though
+it's not - if you pass on the PV to somewhere, pass on the flag too.
+
+=item *
+
+If a string is UTF8, B<always> use C<utf8_to_uv> to get at the value,
+unless C<!(*s & 0x80)> in which case you can use C<*s>.
+
+=item *
+
+When writing to a UTF8 string, B<always> use C<uv_to_utf8>, unless
+C<uv < 0x80> in which case you can use C<*s = uv>.
+
+=item *
+
+Mixing UTF8 and non-UTF8 strings is tricky. Use C<bytes_to_utf8> to get
+a new string which is UTF8 encoded. There are tricks you can use to
+delay deciding whether you need to use a UTF8 string until you get to a
+high character - C<HALF_UPGRADE> is one of those.
+
+=back
+
=head1 AUTHORS
Until May 1997, this document was maintained by Jeff Okamoto
projects / p5sagit/p5-mst-13.2.git / blobdiff