Everything

php

Status of Multibyte support in PHP

php.internals

Al Baker

22 years ago

Hi,

I'm trying to find status on the multibyte support in PHP.  The manual
shows the functions in mbstring to be experimental and it's hard to find
evidence of how stable it really is.

Is this natively supported in PHP after 4.3.x?

Thanks,
Al

Moriyoshi Koizumi

22 years ago

Hi,

On 2004/08/28, at 5:15, Al Baker wrote:

> I'm trying to find status on the multibyte support in PHP.  The manual
> shows the functions in mbstring to be experimental and it's hard to
> find
> evidence of how stable it really is.

Which manual says that mbstring is still experimental? :) it was marked
"not experimental" two years ago.

No matter what that says, and no matter who says it's experimental,
it's been used for a fairly long time by lots of people and I've
never heard of a serious flaw / deficiency in its functionality.

There are also some alternatives that'd be worth paying attention to:

- PHP5 iconv extension
- php-i18n project led by l0t3k

> Is this natively supported in PHP after 4.3.x?

While the extension is there, PHP doesn't yet natively supports
multibyte
encodings in the sense that the language implementation always works
singlebyte-wise
and doesn't automagicallytake care of these fancy characters.

Moriyoshi

Al Baker

22 years ago

Thanks for the confirmation.  The experimental warning appears on any of
the individual help files for mbstring (e.g. mb_ereg).

Do you know if the regular expression support lets you match alpha and
numeric content like you would normally?

Al

On Sat, 2004-08-28 at 05:52 +0900, Moriyoshi Koizumi wrote:

Sean Coates

22 years ago

Are these functions in fact non-experimental? If so, I will remove the
warning from the mb_* functions in the manual.

S

Al Baker wrote:

Moriyoshi Koizumi

22 years ago

On 2004/08/28, at 8:17, Sean Coates wrote:

> Are these functions in fact non-experimental? If so, I will remove the
> warning from the mb_* functions in the manual.

Yep, definitely. Thanks in advance :)

Moriyoshi

Moriyoshi Koizumi

22 years ago

Hi,

On 2004/08/28, at 8:38, Al Baker wrote:

> Thanks for the confirmation.  The experimental warning appears on any
> of
> the individual help files for mbstring (e.g. mb_ereg).

Hmm, warnings seem to persist in manual pages of the multibyte regular
expression functions. I'll remove them later :)

> Do you know if the regular expression support lets you match alpha and
> numeric content like you would normally?

Maybe sorts of character classes don't work as you expect, because
they lack comprehensive collation support that fully conforms to the
Unicode
specification. It's stable I think, but actually not complete yet.

> Al

Moriyoshi

Rasmus Lerdorf

22 years ago

On Sat, 28 Aug 2004, Moriyoshi Koizumi wrote:

> Maybe sorts of character classes don't work as you expect, because
> they lack comprehensive collation support that fully conforms to the
> Unicode
> specification. It's stable I think, but actually not complete yet.

ICU has gotten a lot more modular these days, so it may be possible to
pick out smaller pieces of it to address things like this.  Last time we
looked at it some 4+ years ago, it was a huge all-or-nothing beast.  There
was also an icu extension proposed on pecl-dev recently.

-Rasmus

Derick Rethans

22 years ago

On Fri, 27 Aug 2004, Al Baker wrote:

> Thanks for the confirmation.  The experimental warning appears on any of
> the individual help files for mbstring (e.g. mb_ereg).

Can you file a "Documentation" problem for that on bugs.php.net please?

regards,
Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

Dave

22 years ago

Derick Rethans wrote:

> On Fri, 27 Aug 2004, Al Baker wrote:
>
>
>>Thanks for the confirmation.  The experimental warning appears on any of
>>the individual help files for mbstring (e.g. mb_ereg).
>
>
> Can you file a "Documentation" problem for that on bugs.php.net please?

Don't bother, I have fixed this in the documentation.

> regards,
> Derick

Dave

Al Baker

22 years ago

So, all the mb_* functions will have the experimental removed?  I see a
few that still have it, or is that just the mirrors not up to date yet?

Al

On Sat, 2004-08-28 at 23:35 +1000, Dave Barr wrote:

Mehdi Achour

22 years ago

The warning will be gone after the next build of the manual.

Mehdi

Al Baker wrote:

Derick Rethans

22 years ago

On Sat, 28 Aug 2004, Al Baker wrote:

> So, all the mb_* functions will have the experimental removed?  I see a
> few that still have it, or is that just the mirrors not up to date yet?

I'd go for the iconv() functions as you need to external extension for
this in PHP 5 (it's enabled by default).

Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

Adam Q

22 years ago

Actually I noticed that the Zend Engine 2 page [1] mentions
"Internationalization" as a _new_ feature of PHP 5...
Has there been big changes to the way PHP5 handles i18n?

[1] http://www.zend.com/zend/zend-engine-summary.php

Thanks,
Adam

On 30/08/2004, at 2:11 PM, Derick Rethans wrote:

Derick Rethans

22 years ago

On Mon, 30 Aug 2004, Adam Q wrote:

> Actually I noticed that the Zend Engine 2 page [1] mentions
> "Internationalization" as a _new_ feature of PHP 5...
> Has there been big changes to the way PHP5 handles i18n?
>
> [1] http://www.zend.com/zend/zend-engine-summary.php

That paragraph is utter bollocks if you ask me.

Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

Adam Maccabee Trachtenberg

22 years ago

On Mon, 30 Aug 2004, Derick Rethans wrote:

> On Mon, 30 Aug 2004, Adam Q wrote:
>
> > Actually I noticed that the Zend Engine 2 page [1] mentions
> > "Internationalization" as a _new_ feature of PHP 5...
> > Has there been big changes to the way PHP5 handles i18n?
> >
> > [1] http://www.zend.com/zend/zend-engine-summary.php
>
> That paragraph is utter bollocks if you ask me.

This looks like a *really* old document. It still lists the
possibility of MI in PHP 5.

Neither MI or i18n has been added to ZE 2, and there haven't been
talks about them in months, if not years. :)

-adam

--
adam@trachtenberg.com
author of o'reilly's "upgrading to php 5" and "php cookbook"
avoid the holiday rush, buy your copies today!

Andi Gutmans

22 years ago

Adam,

That link is in the History section of the Zend Engine II section. We left
it and our original RFC there so that people can take a look but PHP 5 has
changed very much since then.

Andi

At 07:08 PM 8/30/2004 +0800, Adam Q wrote:

Al Baker

22 years ago

What is the "--enable-zend-multibyte" configuration option used for
then?

I've noticed a few bug reports where this option was the culprit.

Al

On Mon, 2004-08-30 at 11:50 -0700, Andi Gutmans wrote:

Masaki Fujimoto

22 years ago

Hello,

the option is, to handle following issues (though I've been away
for months... :(

* parsing scripts written in flex-unfriendly encodings
(e.g. Shift_JIS, BIG-5, etc)
* parsing scripts containing BOM (Byte Order Mark)
* converting encoding of each script to one "internal" encoding
* parsing scripts written in wide characters (like UTF-16)
-- experimental

in short, this option enables Zend Engine II to parse scripts
written in various encodings. however, these features apparently
cause some performance loss comapring to simple parsing, so I think
this option will remains one of additional features.

anyway, would please tell me the id of the bugs you mentioned? as I
stated, I always take care not to do any harm to the other codes, so
of course I'll fix them ASAP if it's caused by my option.

regards,
Masaki Fujimoto

The Message From Al Baker on Mon, 30 Aug 2004 15:13:06 -0500:

> What is the "--enable-zend-multibyte" configuration option used for
> then?
>
> I've noticed a few bug reports where this option was the culprit.
>
> Al
>
> On Mon, 2004-08-30 at 11:50 -0700, Andi Gutmans wrote:
> > Adam,
> >
> > That link is in the History section of the Zend Engine II section. We left
> > it and our original RFC there so that people can take a look but PHP 5 has
> > changed very much since then.
> >
> > Andi
> >
> > At 07:08 PM 8/30/2004 +0800, Adam Q wrote:
> > >Actually I noticed that the Zend Engine 2 page [1] mentions
> > >"Internationalization" as a _new_ feature of PHP 5...
> > >Has there been big changes to the way PHP5 handles i18n?
> > >
> > >[1] http://www.zend.com/zend/zend-engine-summary.php
> > >
> > >Thanks,
> > >Adam
> > >
> > >On 30/08/2004, at 2:11 PM, Derick Rethans wrote:
> > >
> > >>On Sat, 28 Aug 2004, Al Baker wrote:
> > >>
> > >>>So, all the mb_* functions will have the experimental removed?  I see a
> > >>>few that still have it, or is that just the mirrors not up to date yet?
> > >>
> > >>I'd go for the iconv() functions as you need to external extension for
> > >>this in PHP 5 (it's enabled by default).
> > >>
> > >>Derick
> > >>
> > >>--
> > >>Derick Rethans
> > >>http://derickrethans.nl | http://ez.no | http://xdebug.org
> > >>
> > >>--
> > >>PHP Internals - PHP Runtime Development Mailing List
> > >>To unsubscribe, visit: http://www.php.net/unsub.php
> > >
> > >--
> > >PHP Internals - PHP Runtime Development Mailing List
> > >To unsubscribe, visit: http://www.php.net/unsub.php
> >
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php

--
Masaki Fujimoto
fujimoto@php.net

Derick Rethans

22 years ago

On Mon, 30 Aug 2004, Andi Gutmans wrote:

> Adam,
>
> That link is in the History section of the Zend Engine II section. We left
> it and our original RFC there so that people can take a look but PHP 5 has
> changed very much since then.

Perhaps it's a good idea to add a BIG disclaimer on that same page? As
it's not obvious when you find it through a search engine...

Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

Andi Gutmans

22 years ago

Yeah good idea. I'll try and see what I can do.

Andi

At 08:14 AM 8/31/2004 +0200, Derick Rethans wrote: