[perl #18107] lc(), uc() and ucfirst() broken inside utf8 regex

Front page | perl.perl5.porters | Postings from October 2002

[perl #18107] lc(), uc() and ucfirst() broken inside utf8 regex

Thread Next

From:

Autrijus Tang

Date:

October 28, 2002 18:43

Subject:

[perl #18107] lc(), uc() and ucfirst() broken inside utf8 regex

Message ID:

rt-18107-40609.17.7804073596322@bugs6.perl.org

# New Ticket Created by  Autrijus Tang 
# Please include the string:  [perl #18107]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt2/Ticket/Display.html?id=18107 >


This is a bug report for perl from autrijus@autrijus.org,
generated with the help of perlbug 1.33 running under perl v5.8.0.


-----------------------------------------------------------------
[Please enter your report here]

The following snippet demonstrated corrupted utf8 strings for
lc(), uc() and ucfirst(), but not lcfirst():

utf8::upgrade($_="t.est"); s/([a-z]+)/lc($1)/ge;      print "lc: $_\n";
utf8::upgrade($_="t.est"); s/([a-z]+)/uc($1)/ge;      print "uc: $_\n";
utf8::upgrade($_="t.est"); s/([a-z]+)/lcfirst($1)/ge; print "lcfirst: $_\n";
utf8::upgrade($_="t.est"); s/([a-z]+)/ucfirst($1)/ge; print "ucfirst: $_\n";

It seem to only happen when the following combination is met:
- String with unicode flag on
- Regular expression with captured variables
- Variables manipulated by lc, uc, ucfirst or the equivalent \L escapes

Since it occurred in Mail::Header it effectively corrupted
utf8-enabled mail applications writing in perl.

Thanks,
/Autrijus/

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
This perlbug was built using Perl v5.8.0 - Mon Jun 10 19:48:03 CST 2002
It is being executed now by  Perl v5.8.0 - Sat Oct  5 11:17:02 GMT 2002.

Site configuration information for perl v5.8.0:

Configured by root at Sat Oct  5 11:17:02 GMT 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=freebsd, osvers=4.7-rc, archname=i386-freebsd
    uname='freebsd gohan17.freebsd.org 4.7-rc freebsd 4.7-rc #0: sun apr 1 02:34:56 pst 2002 asami@bento.freebsd.org:usrsrcsyscompilebento i386 '
    config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.8.0/mach -Dprivlib=/usr/local/lib/perl5/5.8.0 -Dman3dir=/usr/local/lib/perl5/5.8.0/man/man3 -Dsitearch=/usr/local/lib/perl5/site_perl/5.8.0/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.8.0 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.0/BSDPAN" -Ui_gdbm -Dusemymalloc=n'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.0/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include',
    optimize='-O -pipe ',
    cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.0/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.95.4 20020320 [FreeBSD]', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-Wl,-E  -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lm -lc -lcrypt -lutil
    perllibs=-lm -lc -lcrypt -lutil
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
    cccdlflags='-DPIC -fPIC', lddlflags='-shared  -L/usr/local/lib'

Locally applied patches:
    DEVEL17060

---
@INC for perl v5.8.0:
    /usr/local/lib/perl5/site_perl/5.8.0/mach
    /usr/local/lib/perl5/site_perl/5.8.0
    /usr/local/lib/perl5/site_perl
    /usr/local/lib/perl5/5.8.0/BSDPAN
    /usr/local/lib/perl5/5.8.0/mach
    /usr/local/lib/perl5/5.8.0
    .

---
Environment for perl v5.8.0:
    HOME=/home/autrijus
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=en_US.ISO_8859-1
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/autrijus/bin:/usr/X11R6/bin:/usr/games:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/home/autrijus/Komodo-2.0
    PERL_BADLANG (unset)
    SHELL=/bin/tcsh

Thread Next