regex operators

php.internals

David Sklar

22 years ago
I was thinking about adding one or two regex-related features to the engine: 1. "preg_case": this would behave just like case but instead of doing an equality comparison, would match against a regular expression, e.g. switch($data) { preg_case '/^\d{5}(-\d{4})?$/': print "US Postal Code"; break; preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; print "Canadian Postal Code"; break; default: print "something else!"; } Where should any captured subpatterns go? 2. A regex match operator that returns an array containing subpatterns. If there is no match against the regex, then an empty array (or just false?) would be returned. if ($data =~ '/^(\d{5})(-\d{4})?$/') { print "The whole postal code is $data[0]."; print "The first five digits is $data[1]."; if ($data[2]) { print "The ZIP+4 is $data[2].";} } Some issues with adding these features: - It creates an engine dependency on the PCRE library. - There would have to be some new opcodes and parser tokens - Ideally the code that implements these operators could share as much as possible with what's already been done in the PCRE extension -- is that possible? Comments? Thanks, David

Andrey Hristov

22 years ago
Hi David, 2 weeks ago Hartmut Holzgraefe had similar idea : switch ($data, "preg_match") { case '/foo.*?bar/i' : /* computation here */ break; } The second argument is a callback. Andrey David Sklar wrote:

George Schlossnagle

22 years ago
Aesthetics aside, how is that different in effect from: switch(true) { case preg_match('/foo.*?bar/i', $data): /** stuff **/ break; case preg_match('/baz/', $data); /** other stuff **/ break; default: break; } ? On Tuesday, October 14, 2003, at 10:58 AM, Andrey Hristov wrote:
> Hi David, > 2 weeks ago Hartmut Holzgraefe had similar idea : > switch ($data, "preg_match") { > case '/foo.*?bar/i' : /* computation here */ > break; > } > > > The second argument is a callback. > > Andrey > > David Sklar wrote: > >> I was thinking about adding one or two regex-related features to the >> engine: >> >> 1. "preg_case": this would behave just like case but instead of doing >> an >> equality comparison, would match against a regular expression, e.g. >> >> switch($data) { >> preg_case '/^\d{5}(-\d{4})?$/': >> print "US Postal Code"; >> break; >> preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; >> print "Canadian Postal Code"; >> break; >> default: >> print "something else!"; >> } >> >> Where should any captured subpatterns go? >> >> 2. A regex match operator that returns an array containing >> subpatterns. If >> there is no match against the regex, then an empty array (or just >> false?) >> would be returned. >> >> if ($data =~ '/^(\d{5})(-\d{4})?$/') { >> print "The whole postal code is $data[0]."; >> print "The first five digits is $data[1]."; >> if ($data[2]) { print "The ZIP+4 is $data[2].";} >> } >> >> Some issues with adding these features: >> >> - It creates an engine dependency on the PCRE library. >> - There would have to be some new opcodes and parser tokens >> - Ideally the code that implements these operators could share as >> much as >> possible with what's already been done in the PCRE extension -- is >> that >> possible? >> >> Comments? >> >> Thanks, >> David >> >> > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
-- George Schlossnagle -- Principal Consultant -- OmniTI Computer Consulting, Inc. -- +1.410.872.4910 x202 -- 1024D/1100A5A0 1370 F70A 9365 96C9 2F5E 56C2 B2B9 262F 1100 A5A0

David Sklar

22 years ago
One thing that I suppose you get by using "case preg_match()" like George does below (as opposed to my preg_case operator or Hartmut's callback) is you can deal with arbitrary arguments to the preg_match() (or other) function -- storing captured subpatterns from the regex, for example. David On Tuesday, October 14, 2003 11:04 AM, mailto:george@omniti.com wrote:

Derick Rethans

22 years ago
On Tue, 14 Oct 2003, David Sklar wrote:
> One thing that I suppose you get by using "case preg_match()" like George > does below (as opposed to my preg_case operator or Hartmut's callback) is > you can deal with arbitrary arguments to the preg_match() (or other) > function -- storing captured subpatterns from the regex, for example.
Right, and as George's example already works I see no point in adding more 'magic' operators that look like Perl to me. Derick
-- "Interpreting what the GPL actually means is a job best left to those that read the future by examining animal entrails." ------------------------------------------------------------------------- Derick Rethans http://derickrethans.nl/ International PHP Magazine http://php-mag.net/ -------------------------------------------------------------------------

David Sklar

22 years ago
On Tuesday, October 14, 2003 11:18 AM, mailto:derick@php.net wrote:
> Right, and as George's example already works I see no point in adding > more 'magic' operators that look like Perl to me.
What about a match operator? I realize that similar functionality can be achieved with preg_match(), but we could duplicate all operators with functions: if (equals($foo,$bar)) {} instead of if ($foo == bar). Obviously, testing equality is a frequent enough operation that it would be clumsy to not have an == operator. In the same vein, matching against a regex seems like a more frequent operation in web programming than, say bitshifting with assignment ala the <<= and >>= operators. Which is why I think it could be a useful addition. David

Robert Cummings

22 years ago
On Tue, 2003-10-14 at 13:02, David Sklar wrote:
> On Tuesday, October 14, 2003 11:18 AM, mailto:derick@php.net wrote: > > > Right, and as George's example already works I see no point in adding > > more 'magic' operators that look like Perl to me. > > What about a match operator? I realize that similar functionality can be > achieved with preg_match(), but we could duplicate all operators with > functions: if (equals($foo,$bar)) {} instead of if ($foo == bar). Obviously, > testing equality is a frequent enough operation that it would be clumsy to > not have an == operator. In the same vein, matching against a regex seems > like a more frequent operation in web programming than, say bitshifting with > assignment ala the <<= and >>= operators. Which is why I think it could be a > useful addition.
Bitshifting being a commonly used C operator makes conversion of C programs to PHP simplistic. IMHO PHP isn't PERL and if I wanted cryptic non standard operators I'd be using PERL. Cheers, Rob.
-- .------------------------------------------------------------. | InterJinn Application Framework - http://www.interjinn.com | :------------------------------------------------------------: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `------------------------------------------------------------'

Rasmus Lerdorf

22 years ago
On Tue, 14 Oct 2003, David Sklar wrote:
> On Tuesday, October 14, 2003 11:18 AM, mailto:derick@php.net wrote: > > > Right, and as George's example already works I see no point in adding > > more 'magic' operators that look like Perl to me. > > What about a match operator? I realize that similar functionality can be > achieved with preg_match(), but we could duplicate all operators with > functions: if (equals($foo,$bar)) {} instead of if ($foo == bar). Obviously, > testing equality is a frequent enough operation that it would be clumsy to > not have an == operator. In the same vein, matching against a regex seems > like a more frequent operation in web programming than, say bitshifting with > assignment ala the <<= and >>= operators. Which is why I think it could be a > useful addition.
You are pushing towards $_~=/^\.*?\$$/; This is not human-readable code and one of the basic characteristics that sets PHP apart from Perl. Every non-trivial line of PHP code has a decypherable keyword that you can plug into the manual to figure out what that line is doing. We make sure of this by keeping the number of operators to a minimum. As for your bitshifting example. It has nothing to do with the frequency of use, it has to do with readability. -Rasmus

David Sklar

22 years ago
On Tuesday, October 14, 2003 1:10 PM, mailto:rasmus@lerdorf.com wrote:
> You are pushing towards > > $_~=/^\.*?\$$/; > > This is not human-readable code and one of the basic characteristics > that sets PHP apart from Perl.
Actually, I'm pushing towards if (! ($_REQUEST['email'] =~ '/^[^@\s]+@([-a-z0-9]+\.)+[a-z]{2,}$/i')) { $form->addError('Please enter a valid e-mail address.'); } There's not much we can practically do about the punctuation density of regular expressions, but we can make their use more widespread by changing the syntax of how they're invoked.
> Every non-trivial line of PHP code > has a decypherable keyword that you can plug into the manual to > figure out what that line is doing.
I think this is a great aspect of PHP.
> We make sure of this by keeping > the number of operators to a minimum. As for your bitshifting > example. It has nothing to do with the frequency of use, it has to > do with readability.
So why is the === operator an operator and not an is_identical() function? My motivation for this operator is to encourage regular expression use as part of the core toolbox of PHP programmers. I think, especially in a web context, where so much work has to do with data validation and manipulation, that this is a reasonable goal. The features that the preg_* functions provide are great -- I think we should explore ways to have an operator syntax for regular expressions. David

Robert Cummings

22 years ago
On Tue, 2003-10-14 at 14:14, David Sklar wrote:
> On Tuesday, October 14, 2003 1:10 PM, mailto:rasmus@lerdorf.com wrote: > > > You are pushing towards > > > > $_~=/^\.*?\$$/; > > > > This is not human-readable code and one of the basic characteristics > > that sets PHP apart from Perl. > > Actually, I'm pushing towards > > if (! ($_REQUEST['email'] =~ '/^[^@\s]+@([-a-z0-9]+\.)+[a-z]{2,}$/i')) { > $form->addError('Please enter a valid e-mail address.'); > } > > There's not much we can practically do about the punctuation density of > regular expressions, but we can make their use more widespread by changing > the syntax of how they're invoked.
Why would this make regular expressions more widespread? I would expect that regular expressions are used wherever necessary and otherwise not used, regardless of syntax. Or are you saying because regex matching is invoked via a function that you don't use regex? In such a case I'd have to ask what you use instead!? Cheers, Rob.
-- .------------------------------------------------------------. | InterJinn Application Framework - http://www.interjinn.com | :------------------------------------------------------------: | An application and templating framework for PHP. Boasting | | a powerful, scalable system for accessing system services | | such as forms, properties, sessions, and caches. InterJinn | | also provides an extremely flexible architecture for | | creating re-usable components quickly and easily. | `------------------------------------------------------------'

Adam Maccabee Trachtenberg

22 years ago
On 14 Oct 2003, Robert Cummings wrote:
> Why would this make regular expressions more widespread? I would expect > that regular expressions are used wherever necessary and otherwise not > used, regardless of syntax. Or are you saying because regex matching is > invoked via a function that you don't use regex? In such a case I'd have > to ask what you use instead!?
Following that logic, why would anyone prefer one Turing-complete language over another? They only differ in their syntax. :) Okay, this is an extreme example. However, I think it is safe to say that language syntax and grammar cause users to exhibit a tendency to use one set of techniques in comparison to another. As a general rule, PHP's operators (i.e. funny looking punctuation) are easily recognizable basics; its functions (i.e. letters with () at the end) are ones that need more clarify of identification at the expense of extra letters. (Not exactly, but I'm generalizing here.) I guess the real questions here (to me) is are regular expressions a common enough action and is "=~" a common enough symbol that it's worth a decrease in clarity? I don't know. Regular expressions have certainly shown their frequent usefulness. I've always hated =~ as a symbol, but with Perl's popularity, it doesn't really makes sense to choose anything else. In the 70s, regular expressions were expensive, so languages like C didn't include a regex operator. Why should we be bound by the decisions of K&R and strongly typed languages? At some point, languages moved away from the Lisp camp, where everything was a word. Lisp is super verbose, but I don't see people advocating for a return, even if we got rid of all the stupid (()())s. As Dave said, we added === when no other language in the world has that. Why would =~ be necessarily more confusing? -adam
-- adam@trachtenberg.com

Red Wingate

22 years ago
Why not just allow this as an optional feature, i don't see any reasons to keep this out as it doesn't affect any performace and can maybe help some old-skewl PERL-Coders to convert over to PHP. One of the best features of PERL is the tight connection between regular expressions and the scripting-language. Developers that don't want to use this kind of features might just skip it, but some others will surely like this option when coding.
> On Tue, 2003-10-14 at 14:14, David Sklar wrote: > > On Tuesday, October 14, 2003 1:10 PM, mailto:rasmus@lerdorf.com wrote: > > > > > You are pushing towards > > > > > > $_~=/^\.*?\$$/; > > > > > > This is not human-readable code and one of the basic characteristics > > > that sets PHP apart from Perl. > > > > Actually, I'm pushing towards > > > > if (! ($_REQUEST['email'] =~ '/^[^@\s]+@([-a-z0-9]+\.)+[a-z]{2,}$/i')) { > > $form->addError('Please enter a valid e-mail address.'); > > } > > > > There's not much we can practically do about the punctuation density of > > regular expressions, but we can make their use more widespread by
changing

Ilia A.

22 years ago
On October 14, 2003 02:14 pm, David Sklar wrote:
> if (! ($_REQUEST['email'] =~ '/^[^@\s]+@([-a-z0-9]+\.)+[a-z]{2,}$/i')) { > $form->addError('Please enter a valid e-mail address.'); > }
Why not use PERL if you are looking for this sort of functionality? As indicated by other developers PHP strength comes from ease of use & clarity, the regex syntax you propose is anything but. Is calling preg_match() that much of a problem? Ilia

Rasmus Lerdorf

22 years ago
On Tue, 14 Oct 2003, David Sklar wrote:
> On Tuesday, October 14, 2003 1:10 PM, mailto:rasmus@lerdorf.com wrote: > > > You are pushing towards > > > > $_~=/^\.*?\$$/; > > > > This is not human-readable code and one of the basic characteristics > > that sets PHP apart from Perl. > > Actually, I'm pushing towards > > if (! ($_REQUEST['email'] =~ '/^[^@\s]+@([-a-z0-9]+\.)+[a-z]{2,}$/i')) { > $form->addError('Please enter a valid e-mail address.'); > } > > There's not much we can practically do about the punctuation density of > regular expressions, but we can make their use more widespread by changing > the syntax of how they're invoked.
And why in the world would we want to do that? I think you are completely missing the point here.
> My motivation for this operator is to encourage regular expression use as > part of the core toolbox of PHP programmers. I think, especially in a web > context, where so much work has to do with data validation and manipulation, > that this is a reasonable goal. The features that the preg_* functions > provide are great -- I think we should explore ways to have an operator > syntax for regular expressions.
I would like to encourage less usage of regular expressions by encouraging people to use very targeted and fast string manipulation functions. I know how most of the core guys feel on this one, and trust me, there is no chance this will get implemented. A bunch of us would put up a very strong veto. -Rasmus

David Sklar

22 years ago
>> My motivation for this operator is to encourage regular expression >> use as part of the core toolbox of PHP programmers. I think, >> especially in a web context, where so much work has to do with data >> validation and manipulation, that this is a reasonable goal. The >> features that the preg_* functions provide are great -- I think we >> should explore ways to have an operator syntax for regular >> expressions. > > I would like to encourage less usage of regular expressions by > encouraging people to use very targeted and fast string manipulation > functions.
I am all for str_whatever() when appropriate (for clarity, performance, or whatever else). I think we just disagree on the meaning of "appropriate" in this context.
> I know how most of the core guys feel on this one, and trust me, > there is no chance this will get implemented. A bunch of us would > put up a very strong veto.
OK, thanks for the heads-up. David

Andrei Zmievski

22 years ago
On Tue, 14 Oct 2003, Rasmus Lerdorf wrote:
> You are pushing towards > > $_~=/^\.*?\$$/;
Oh, but Rasmus, you don't even need $_ here.. Simply /^\.*?\$$/; will do. My Perl kungfu increases every day. - Andrei

Hartmut Holzgraefe

22 years ago
David Sklar wrote:
> One thing that I suppose you get by using "case preg_match()" like George > does below (as opposed to my preg_case operator or Hartmut's callback) is > you can deal with arbitrary arguments to the preg_match() (or other) > function -- storing captured subpatterns from the regex, for example. >
> >>Aesthetics aside, how is that different in effect from: >> >>switch(true) { >> case preg_match('/foo.*?bar/i', $data): >> /** stuff **/ >> break; >> case preg_match('/baz/', $data); >> /** other stuff **/ >> break; >> default: >> break; >>}
the "switch(true)" trick is completely ugly IMHO and is just an obfuscated way to write an if-elseif-...-else chain my point of view is that you use switch whenever you have a series of tests against a single value whereas in the code snippet above you could also test against e.g. $date instead of $data in some of the cases whereas this is meant to be so or a typo does not become clear by extending the switch statement itself in the way i propoesd you gain extra functionality from switch while keeping the "tests against a single value" contract i was originaly just thinking about an optional flag that would switch to "===" behaviour instead of just "==" but the callback mechanism was just too obvious :) and if you need extra parameters you can always wrap up whatever test function you use in an extra wrapper function ;)
-- Hartmut Holzgraefe <hartmut@php.net>

Hartmut Holzgraefe

22 years ago
David Sklar wrote:
> I was thinking about adding one or two regex-related features to the engine: > > 1. "preg_case": this would behave just like case but instead of doing an > equality comparison, would match against a regular expression, e.g. > > switch($data) { > preg_case '/^\d{5}(-\d{4})?$/': > print "US Postal Code"; > break; > preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; > print "Canadian Postal Code"; > break; > default: > print "something else!"; > } >
i've started to play with a more general way to handle this: switch(mixed data [, callback compare_function]) { ... "compare_function" would default to the functionality of the "==" operator to show the current behavior "compare_function" will be called with two parameters: the switch expression and the case value and should return true if both match, false otherwise as a special case "compare_function" should also accept "===" this way we do not have to add new keywords and are not limited to preg expresions here, possible uses not possible now would include: switch($foobar, "===") { case false: ... case 0: ... case '': ... switch($foobar, "preg_match") { case '/^\d{5}$/': ... case '/(foo)?bar/': ... switch($foobar, "fnmatch") { case '*.gif': ... case '*.jpg': ... switch($foober, "my_custom_function") { ... switch($foobar, array($this, "compare_function") { ...
-- Hartmut Holzgraefe <hartmut@php.net>

netcat

22 years ago
Hartmut Holzgraefe wrote:
> David Sklar wrote: > >> I was thinking about adding one or two regex-related features to the >> engine: >> >> 1. "preg_case": this would behave just like case but instead of doing an >> equality comparison, would match against a regular expression, e.g. >> >> switch($data) { >> preg_case '/^\d{5}(-\d{4})?$/': >> print "US Postal Code"; >> break; >> preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; >> print "Canadian Postal Code"; >> break; >> default: >> print "something else!"; >> } >> > > i've started to play with a more general way to handle this: > > switch(mixed data [, callback compare_function])
seems the sanest thing up until now. slighly O.T.: would be and even better if lambda functions were available here
> > { > ... > > "compare_function" would default to the functionality of the > "==" operator to show the current behavior > > "compare_function" will be called with two parameters: > the switch expression and the case value and should > return true if both match, false otherwise > > as a special case "compare_function" should also accept > "===" > > this way we do not have to add new keywords and are not > limited to preg expresions here, > possible uses not possible now would include: > > switch($foobar, "===") { > case false: ... > case 0: ... > case '': ... > > switch($foobar, "preg_match") { > case '/^\d{5}$/': ... > case '/(foo)?bar/': ... > > switch($foobar, "fnmatch") { > case '*.gif': ... > case '*.jpg': ... > > switch($foober, "my_custom_function") { > ... > > switch($foobar, array($this, "compare_function") { > ... > > > >
-- NetCat -------------------------------------------------------- FREE 10MB email + Antivirus + AntiSpam + POP3 + more.... Get it at http://www.doal.co.il:81/free/?c=both

Hartmut Holzgraefe

22 years ago
netcat wrote:
>> i've started to play with a more general way to handle this: >> >> switch(mixed data [, callback compare_function]) > > > seems the sanest thing up until now. > > slighly O.T.: would be and even better if lambda functions were > available here
http://php.net/create-function
-- Hartmut Holzgraefe <hartmut@php.net>

netcat

22 years ago
Hartmut Holzgraefe wrote:
> netcat wrote: > >>> i've started to play with a more general way to handle this: >>> >>> switch(mixed data [, callback compare_function]) >> >> >> >> seems the sanest thing up until now. >> >> slighly O.T.: would be and even better if lambda functions were >> available here > > > > http://php.net/create-function >
Thanks for the link. It's just feels a little awkward to see something like: switch($x,create_function('$a,$b','return $a=="MY$b;"')) { } Again, it's a feeling, maybe it's just me. It doesn't feel right to me that inside definition of new function i have to escape quotes (that's because create_function is a function and not special construction on the other side if it would be a construct it can't return a value as in $x=create_function(...) ). Anyway, i have nothing better to suggest (yet) .... I hope one of you have some suggestion that will make create_function more usable.
-- NetCat ------------------------------------------------------ SPAM-Free 10mb Free email + Antivirus + POP3 + more... Get it at http://www.doal.co.il:81/free/?c=all-spam

George Schlossnagle

22 years ago
On Wednesday, October 15, 2003, at 07:32 AM, netcat wrote:
> I hope one of you have some suggestion that will make create_function > more usable.
Brought up and shot down historically (check the archives). George

Curt Zirzow

22 years ago
* Thus wrote David Sklar (sklar@sklar.com):
> I was thinking about adding one or two regex-related features to the engine: > > 1. "preg_case": this would behave just like case but instead of doing an > equality comparison, would match against a regular expression, e.g. > > switch($data) { > preg_case '/^\d{5}(-\d{4})?$/': > print "US Postal Code"; > break; > preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; > print "Canadian Postal Code"; > break; > default: > print "something else!"; > }
[...]
> > Some issues with adding these features: > > - It creates an engine dependency on the PCRE library. > - There would have to be some new opcodes and parser tokens > - Ideally the code that implements these operators could share as much as > possible with what's already been done in the PCRE extension -- is that > possible?
I'm thinking a more viable solution would be to do something like: mixed preg_match ( mixed pattern, string subject [, array matches [, int flags]]) if pattern is an array, preg_match will cycle through each one till a match is made; returns the index of the array passed (that matched) or false if not found. This should also be BC if you just pass one string. Curt
-- "I used to think I was indecisive, but now I'm not so sure."

Derick Rethans

22 years ago
On Tue, 14 Oct 2003, Curt Zirzow wrote:
> This should also be BC if you just pass one string.
http://docs.php.net/en/function.preg-grep.html we already have that too. Derick
-- "Interpreting what the GPL actually means is a job best left to those that read the future by examining animal entrails." ------------------------------------------------------------------------- Derick Rethans http://derickrethans.nl/ International PHP Magazine http://php-mag.net/ -------------------------------------------------------------------------

Kouber Saparev

22 years ago
"Curt Zirzow" <php-dev@zirzow.dyndns.org> wrote in message news:20031014165039.GP51024@bagend.shire...
> * Thus wrote David Sklar (sklar@sklar.com): > > I was thinking about adding one or two regex-related features to the
engine:
> > > > 1. "preg_case": this would behave just like case but instead of doing an > > equality comparison, would match against a regular expression, e.g. > > > > switch($data) { > > preg_case '/^\d{5}(-\d{4})?$/': > > print "US Postal Code"; > > break; > > preg_case '/^[a-z]\d[a-z] ?\d[a-z]\d$/i'; > > print "Canadian Postal Code"; > > break; > > default: > > print "something else!"; > > } > [...] > > > > Some issues with adding these features: > > > > - It creates an engine dependency on the PCRE library. > > - There would have to be some new opcodes and parser tokens > > - Ideally the code that implements these operators could share as much
as
> > possible with what's already been done in the PCRE extension -- is that > > possible? > > I'm thinking a more viable solution would be to do something like: > > mixed preg_match ( mixed pattern, string subject [, array matches [, int
flags]])
> > if pattern is an array, preg_match will cycle through each one till > a match is made; returns the index of the array passed (that > matched) or false if not found.
It's the same if you have the array with patterns and you make a simple foreach() cycle. I don't think that functions with internal loops should be created, when we have a clear solution without it.