Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » New escaped output operator »

New escaped output operator

php.internals

Михаил Востриков

10 years ago
Hello. I was thinking about a presence of escaped output operator in PHP and found this feature request: https://bugs.php.net/bug.php?id=62574. I think this is quite necessary feature. There are a lot of projects which is written without templating engine, and there are frameworks without built-in templating engine by default. All this projects require to write the code. Usually it is rather simple to switch to new version of language, but it is almost impossible to switch many and many templates on a templating engine. Most of output code is an output of properties of database entities, and only in some cases it's needed to concatenate HTML into string and then print it with unescaped output. Escaped output operator can be useful. Also we output data not into the void and not into simple text file, but into HTML-document which has a certain format (markup). Also this is logical - to have both forms, escaped and unescaped. I want to suggest the operator "<?~ $str ?>", which will automatically wrap output in htmlspecialchars(). It is mentioned in the feature request above. It is quite easy to type, and there is a small possibility to write "<?= ?>" instead. In PHP 7 there are new operators and other changes. I think, new echo operator also can be added. I can implement it myself.

Stas Malyshev

10 years ago
Hi!
> Most of output code is an output of properties of database entities, and > only in some cases it's needed to concatenate HTML into string and then > print it with unescaped output. Escaped output operator can be useful. Also > we output data not into the void and not into simple text file, but into > HTML-document which has a certain format (markup). Also this is logical - > to have both forms, escaped and unescaped.
This has been discussed on the list a number of times. Main issue with this kind of proposals is that escaping is context-dependent. E.g. htmlspecialchars() would not help you in many scenarios - e.g. it won't protect you from XSS if you ever place user-controlled data in HTML attributes. Having operator for each of the possible contexts does not really looks feasible, and having it for only one of them and not the others would be misleading people into thinking this operator is generic and can be used in all contexts safely.
-- Stas Malyshev smalyshev@gmail.com

Unnamed Person

10 years ago
you can simply add the context to the current output operator: <?=html $str ?> <?=attr $str ?> <?=text $str ?> (=strip_tags) <?=js $str ?> <?=css $str ?> Regards Thomas Stanislav Malyshev wrote on 17.06.2016 22:14:

Ryan Pallas

10 years ago
On Fri, Jun 17, 2016 at 2:23 PM, Thomas Bley <mails@thomasbley.de> wrote:
> you can simply add the context to the current output operator: > <?=html($str) ?> > <?=attr($str) ?> > <?=text($str) ?> (=strip_tags) > <?=js($str) ?> > <?=css($str) ?> >
Look at that. Add a couple parens and its completely implementable in userland now with no language changes required.

Unnamed Person

10 years ago
Sure you can implement that in userland, but people don't do it or make it too complicated, so you get every day code with unescaped stuff. Regards Thomas Ryan Pallas wrote on 18.06.2016 00:27:

Михаил Востриков

10 years ago
> e.g. it won't protect you from XSS if you ever place user-controlled data
in HTML attributes. As I've found, such an XSS can have a place in the code like this: $xss = "');your_code_here();//"; <div onmouseover="alert('<?php echo htmlspecialchars($xss, ENT_QUOTES, 'UTF-8') ?>')"> I think this is more architectural problem, not an escaping problem. This is very special case when we really need it.
> If you are in a HTML context you need different escaping than you need in
a CSS or JS block. For JS it's better to use json_encode(). And I've never met CSS+PHP output, this is some special case.
> would be misleading people into thinking this operator is generic and can
be used in all contexts safely. I don't think that many programmers can think so. Anyway, this can be menthioned in documentation.
> The escaping should also be aware of the content encoding.
For special cases - e.g. when we use one encoding and need to output a value in another encoding - htmlspecialchars() still can be used.
> Sure you can implement that in userland, but people don't do it or make
it too complicated,
> so you get every day code with unescaped stuff.
Yes. This is the main problem. Almost each echo operator is an output of data from database, usually this is an entity property if the ORM is used or an array key if isn't. I'm not talking about fully functional escaping operator for all cases, just for most often case - output a value into HTML document. If we have a shorcut for "<?php echo $value; ?>" then we also need a shortcut for "<?php echo htmlspecialchars($value, ENT_QUOTES); ?>", because PHP is a web-programming language. I think this operator can make many projects more safer. 2016-06-18 3:32 GMT+05:00 Thomas Bley <mails@thomasbley.de>:

Rasmus Schultz

10 years ago
> Add a couple parens and its completely implementable in userland
If we could autoload functions, I bet that's what everyone would be doing. At the moment, no one is able to commit to that pattern, because it doesn't scale - you can't just keep adding to a list of global functions (and files) that get aggressively loaded whenever you render a view, even if each view uses only one or two of them... So in practice, you minimally end up with something like this: <?php use My\Stuff\EscapeFunctions as e; ?> <?=e::html($str) ?> <?=e::attr($str) ?> <?=e::text($str) ?> ... But that isn't really practical either, since you can only cram so many functions into the same class - at which point you start adding more classes... <?php use My\Stuff\EscapeFunctions as e; ?> <?php use My\Stuff\OtherFunctions as o; ?> <?=e::html($str) ?> <?=o::stuff(...) ?> It quickly gets ugly, messy and confusing. Then I start thinking about crazy solutions like tokenizing the template file first and dynamically adding require_once statements for any functions discovered being used, which would be more convenient, but quite overly complex for such a small problem - and we're still talking about occupying the global namespace with lots of functions. And so you likely end up accepting that it's ugly and inconvenient, and you resign yourself to use-statements and static methods, or fully-static classes, which I've taken to referring to as "psuedo-namespaces", since we're really abusing classes as a kind of namespace for functions, just so we can get them to autoload. Functions just aren't all that convenient or useful in PHP, because they largely depend on manual use of require_once, which feels really ugly and old-fashioned (since everything else autoloads like it's supposed to) - and it isn't even always possible, since, for example, you can't (reliably) know where a Composer package is located relative to your project or package; it depends on whether your project is currently the root package (e.g. under test) or an installed package in the vendor-folder. I really like pure functions - they're neat, simple and predictable. In Javascript (and other languages) I always use functions first and resort to classes only when there's a real clear benefit. In PHP, I feel like I'm almost always forced into using classes for everything, mainly because that's what works best in PHP and creates the least rub. This has been bothering me for many years - and I wish that I could propose a solution, but I really don't have any ideas. Can we do something to improve and encourage the use of functions in PHP? On Sat, Jun 18, 2016 at 12:27 AM, Ryan Pallas <derokorian@gmail.com> wrote:

Niklas Keller

10 years ago
Rasmus Schultz <rasmus@mindplay.dk> schrieb am Sa., 18. Juni 2016, 17:44:
> > Add a couple parens and its completely implementable in userland > > If we could autoload functions, I bet that's what everyone would be doing. > > At the moment, no one is able to commit to that pattern, because it > doesn't scale - you can't just keep adding to a list of global > functions (and files) that get aggressively loaded whenever you render > a view, even if each view uses only one or two of them... > > So in practice, you minimally end up with something like this: > > <?php use My\Stuff\EscapeFunctions as e; ?> > <?=e::html($str) ?> > <?=e::attr($str) ?> > <?=e::text($str) ?> > ... > > But that isn't really practical either, since you can only cram so > many functions into the same class - at which point you start adding > more classes... > > <?php use My\Stuff\EscapeFunctions as e; ?> > <?php use My\Stuff\OtherFunctions as o; ?> > <?=e::html($str) ?> > <?=o::stuff(...) ?> > > It quickly gets ugly, messy and confusing. >
Did you know that you can alias namespaces, too? <?php use My\Stuff\Escape as esc; ?> <?=esc\html($str)?> You can always add more functions to a namespace even spread accross multiple files. Then I start thinking about crazy solutions like tokenizing the

Marco Pivetta

10 years ago
On 19 June 2016 at 09:53, Niklas Keller <me@kelunik.com> wrote:
> Rasmus Schultz <rasmus@mindplay.dk> schrieb am Sa., 18. Juni 2016, 17:44: > > Did you know that you can alias namespaces, too? > > <?php use My\Stuff\Escape as esc; ?> > <?=esc\html($str)?> > > You can always add more functions to a namespace even spread accross > multiple files. >
Pro-userland: quick reminder that a `composer update` is much quicker than a full system PHP version upgrade. I'd rather rely on an escaping package written in PHP, easier to maintain and quicker to upgrade, than something that will likely use some obscure shared library (or the PHP binary itself) that may not be upgraded for weird reasons (it's shared, remember?). I know that you put a lot of effort in security maintenance, but it's still easier to deal with this stuff in userland in any case, and most templating languages in common frameworks already inject helpers in the script context in order to achieve quick, effective and context-aware (no automatic context detection) escaping. Marco Pivetta http://twitter.com/Ocramius http://ocramius.github.com/

Михаил Востриков

10 years ago
Guys, wait please) I don't suggest escaping package for all contexts and for all cases. This is not what I described in my first letter. My point is that the main job of echo operator "<?= ?>" is output an unknown value from database to an HTML environment. So in all this places we should copy-pase the call of htmlspecialchars() to prevent XSS. There are many projects which is written on custom engines, or frameworks, or CMS, and they does not have any templating engine, and there is no possibility to rewrite many working PHP templates to Twig, or Smarty, or something else. I suggest new simple operator "<?~ ?>" which will automatically wrap the output value in htmlspecialchars(). It is intended specially for HTML, not for XML or JS. It does not require any php.ini settings, new classes or constants. The reason for implementing it is the same as for implementing "??", or "<=>", or "<?= ?>" operators - make better usual and often operations, descrease copy-paste, and increase security. I can implement it myself and send a patch. What do you think? 2016-06-19 12:59 GMT+05:00 Marco Pivetta <ocramius@gmail.com>:

Lester Caine

10 years ago
On 19/06/16 09:38, Михаил Востриков wrote:
> My point is > that the main job of echo operator "<?= ?>" is output an unknown value from > database to an HTML environment. So in all this places we should copy-pase > the call of htmlspecialchars() to prevent XSS.
The majority of XSS problems are created because the free format input INTO the application are not correctly handled. Simply banging htmlspecialchars() around totally unmanaged text is NOT the solution, and handling the correct filtering of the inputs is where this should be handled. I'm sure all of you see various attempts at XSS and SQL injections in your log files. About 20% of my overnight traffic is people trying to 'get in' but because I do not allow raw text to get through all it results in is errors in the log files. The packages that we have had problems cleaning up have tried using the 'clean the output' approach, but this STILL left holes which can only be fixed by cleaning the input ...
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

Marco Pivetta

10 years ago
On 19 June 2016 at 10:56, Lester Caine <lester@lsces.co.uk> wrote:
> On 19/06/16 09:38, Михаил Востриков wrote: > > My point is > > that the main job of echo operator "<?= ?>" is output an unknown value > from > > database to an HTML environment. So in all this places we should > copy-pase > > the call of htmlspecialchars() to prevent XSS. > > The majority of XSS problems are created because the free format input > INTO the application are not correctly handled. Simply banging > htmlspecialchars() around totally unmanaged text is NOT the solution, > and handling the correct filtering of the inputs is where this should be > handled. > > I'm sure all of you see various attempts at XSS and SQL injections in > your log files. About 20% of my overnight traffic is people trying to > 'get in' but because I do not allow raw text to get through all it > results in is errors in the log files. > > The packages that we have had problems cleaning up have tried using the > 'clean the output' approach, but this STILL left holes which can only be > fixed by cleaning the input ... >
This basically means that you lack basic understanding of how escaping and user input are to be handled. Most apps out there about getting a bunch of text from the user, then rendering it somewhere else in the app. Cleaning user input just leads to frustration and a big mess in most scenarios, which is why we're all talking about escaping output instead. This is not "cleaning" either, it's escaping, which is a non-destructive and reversible operation (which is why it works so well). Marco Pivetta http://twitter.com/Ocramius http://ocramius.github.com/

Unnamed Person

10 years ago
I think it's best to create a rfc and put it to vote: https://wiki.php.net/rfc/howto Having <?~ makes it a lot easier to do code reviews. I also think majority of use cases is <?~, other parts can use json_encode(), filter_var() and other filters/escapers. Regards Thomas Михаил Востриков wrote on 19.06.2016 10:38:

Михаил Востриков

10 years ago
Please give me RFC karma. My wiki account is "michael-vostrikov". I plan to create an RFC for this feature. 2016-06-19 21:09 GMT+05:00 Thomas Bley <mails@thomasbley.de>:

Rasmus Schultz

10 years ago
> Did you know that you can alias namespaces, too?
Yes
> You can always add more functions to a namespace even spread accross multiple files
Same problem: no autoloading. You would have to add require_one statements - which, as said, is not really possible with Composer packages... On Sun, Jun 19, 2016 at 9:53 AM, Niklas Keller <me@kelunik.com> wrote:

Marco Pivetta

10 years ago
On 19 June 2016 at 11:34, Rasmus Schultz <rasmus@mindplay.dk> wrote:
> > You can always add more functions to a namespace even spread accross > multiple files > > Same problem: no autoloading. > > You would have to add require_one statements - which, as said, is not > really possible with Composer packages... > >
You should look at packages that already do this: https://github.com/nikic/iter/blob/5527ca489bf151ceef17622f1c89114640f522d2/composer.json#L16 Ref: https://getcomposer.org/doc/04-schema.md#files Marco Pivetta http://twitter.com/Ocramius http://ocramius.github.com/

Rasmus Schultz

10 years ago
I am well familiar with this approach, and it does not scale - not only would you be aggressively loading every installed view-helper anytime you render a view, you would even be loading them when you're *not* rendering a view. I'm afraid the best we could do at this point, without changing the language, is a establish a convention for autoloading functions (and/or namespaces of functions) from files, based on static analysis of template files. But that is pretty complex - on the organizational side, it requires developers to agree on and adopt a convention, and on the technical side, you need static analysis and thereby most likely a cache layer as well. It's all possible, but most people aren't going to put up with this much complexity for something this simple. Hmm. What if we could import static methods into file scope and use them as functions? use My\Namespace::my_function; my_function(); // <-- effectively My\Namespace::my_function() This would leverage auto-loading at least... I mean, it's still effectively just abusing classes as pseudo-namespaces, so there is that - but it would work with e.g. Composer right away, and probably with many existing static classes? Yeah, it's still ugly... On Sun, Jun 19, 2016 at 11:37 AM, Marco Pivetta <ocramius@gmail.com> wrote:

David Muir

10 years ago
> On 19 Jun 2016, at 7:57 PM, Rasmus Schultz <rasmus@mindplay.dk> wrote: > > I am well familiar with this approach, and it does not scale - not > only would you be aggressively loading every installed view-helper > anytime you render a view, you would even be loading them when you're > *not* rendering a view. > > I'm afraid the best we could do at this point, without changing the > language, is a establish a convention for autoloading functions > (and/or namespaces of functions) from files, based on static analysis > of template files. > > But that is pretty complex - on the organizational side, it requires > developers to agree on and adopt a convention, and on the technical > side, you need static analysis and thereby most likely a cache layer > as well. > > It's all possible, but most people aren't going to put up with this > much complexity for something this simple. > > Hmm. What if we could import static methods into file scope and use > them as functions? > > use My\Namespace::my_function; > > my_function(); // <-- effectively My\Namespace::my_function() > > This would leverage auto-loading at least... I mean, it's still > effectively just abusing classes as pseudo-namespaces, so there is > that - but it would work with e.g. Composer right away, and probably > with many existing static classes? > > Yeah, it's still ugly... > >
Ugly, but brilliant! +1 David

Christoph Becker

10 years ago
On 18.06.2016 at 17:44, Rasmus Schultz wrote:
>> Add a couple parens and its completely implementable in userland > > If we could autoload functions, I bet that's what everyone would be doing.
FWIW, there is an respective RFC draft[1] "lying around". See also <https://bugs.php.net/72459>. [1] <https://wiki.php.net/rfc/function_autoloading>
-- Christoph M. Becker

Rasmus Schultz

10 years ago
> [1] <https://wiki.php.net/rfc/function_autoloading>
beauty! when can we have that?? :-)

Christoph Becker

10 years ago
On 20.06.2016 at 19:19, Rasmus Schultz wrote:
>> [1] <https://wiki.php.net/rfc/function_autoloading> > > beauty! when can we have that?? :-)
Maybe never, but at least somebody would have to pursue the RFC. See also the related discussion from 2013, starting with <http://news.php.net/php.internals/68693>.
-- Christoph M. Becker

Yasuo Ohgaki

10 years ago
Hi Thomas, On Sat, Jun 18, 2016 at 5:23 AM, Thomas Bley <mails@thomasbley.de> wrote:
> you can simply add the context to the current output operator: > <?=html $str ?> > <?=attr $str ?> > <?=text $str ?> (=strip_tags) > <?=js $str ?> > <?=css $str ?>
We need <?=uri $str ?> in addition. If we adopt this, we must document clearly that LDAP, SQL, etc are not supported. I like this idea a lot. Output context is clear and explicit. We may be better to consider "<?= $str" to be "<?php echo htmlspecialchars($str)" rather than "<?php echo $str", but this change would be for PHP 8. Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

Eli

10 years ago
On 6/18/16 12:56 AM, Yasuo Ohgaki wrote:
> We may be better to consider "<?= $str" to be "<?php echo > htmlspecialchars($str)" rather than "<?php echo $str", but this change > would be for PHP 8.
No, that would be highly inadvisable. As it's been pointed out, people use PHP templating for things besides just HTML. And if you made this change, then <?= could no longer be used in echoing out javascript, XML, RSS feeds, making custom API responses, etc. Eli
-- | Eli White | http://eliw.com/ | Twitter: EliW |

Niklas Keller

10 years ago
Hi, the issue is that things have to be escaped dependent on the context. If you are in a HTML context you need different escaping than you need in a CSS or JS block. The escaping should also be aware of the content encoding. All that makes it difficult for PHP to directly support such an operator. You can always alias "e" or something like that to be your default escape function. Regards, Niklas Михаил Востриков <michael.vostrikov@gmail.com> schrieb am Fr., 17. Juni 2016, 21:29:

Unnamed Person

10 years ago
using the default encoding from php.ini's default_charset should be no problem, htmlspecialchars() already does it if the encoding parameter is not provided. Regards Thomas Niklas Keller wrote on 17.06.2016 22:31:

Walter Parker

10 years ago
Thomas, are you actually reading and understanding what the others are saying? You seem to be answering questions that have not been asked or giving the simple, easy and wrong answer. Walter On Fri, Jun 17, 2016 at 1:37 PM, Thomas Bley <mails@thomasbley.de> wrote:
> using the default encoding from php.ini's default_charset should be no > problem, htmlspecialchars() already does it if the encoding parameter is > not provided. > > Regards > Thomas > > Niklas Keller wrote on 17.06.2016 22:31: > > > Hi, > > > > the issue is that things have to be escaped dependent on the context. If > > you are in a HTML context you need different escaping than you need in a > > CSS or JS block. The escaping should also be aware of the content > encoding. > > All that makes it difficult for PHP to directly support such an operator. > > > > You can always alias "e" or something like that to be your default escape > > function. > > > > Regards, Niklas > > > > Михаил Востриков <michael.vostrikov@gmail.com> schrieb am Fr., > > 17. Juni > > 2016, 21:29: > > > >> Hello. I was thinking about a presence of escaped output operator in PHP > >> and found this feature request: https://bugs.php.net/bug.php?id=62574. > I > >> think this is quite necessary feature. There are a lot of projects > which is > >> written without templating engine, and there are frameworks without > >> built-in templating engine by default. All this projects require to > write > >> the code. Usually it is rather simple to switch to new version of > language, > >> but it is almost impossible to switch many and many templates on a > >> templating engine. > >> > >> Most of output code is an output of properties of database entities, and > >> only in some cases it's needed to concatenate HTML into string and then > >> print it with unescaped output. Escaped output operator can be useful. > Also > >> we output data not into the void and not into simple text file, but into > >> HTML-document which has a certain format (markup). Also this is logical > - > >> to have both forms, escaped and unescaped. > >> > >> I want to suggest the operator "<?~ $str ?>", which will automatically > wrap > >> output in htmlspecialchars(). It is mentioned in the feature request > above. > >> It is quite easy to type, and there is a small possibility to write "<?= > >> ?>" instead. > >> > >> In PHP 7 there are new operators and other changes. I think, new echo > >> operator also can be added. I can implement it myself. > >> > > > > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
-- The greatest dangers to liberty lurk in insidious encroachment by men of zeal, well-meaning but without understanding. -- Justice Louis D. Brandeis