Progress or just 'a mess'?

On 17.09.2017 at 12:53, Rowan Collins wrote:

The specification in the PHP manual[1] appears to be more appropriate for our current implementation: | As a regular expression, it would be expressed thus: '[a-zA-Z_\x7f- | \xff][a-zA-Z0-9_\x7f-\xff]*' With regard to control characters: that depends on the chosen character encoding; for instance in Windows-1252 the ¢ character is mapped to \xA2. [1] <http://php.net/manual/en/language.variables.basics.php>

On 17/09/17 11:53, Rowan Collins wrote:

I think this is actually the problem. Unicode is simply NOT a general solution! Normalizing is another aspect, and that can result in differences between strings if one also 'case folds'. On top of which one has to add the collation one is using to provide sort order which is another can of worms? Sorting array keys in order depends on the character set used ... which is perhaps why there seems to be a drive to replace associative arrays with simple numeric ones? "U+0020-U+007F" gives the Basic Latin set of characters (ASCII) "U+0080-U+00FF" add the "Latin-1 Supplement" The problem is that the second 128 characters is avoiding overlaying the "U+0000-U+001F" control character block, while single byte character sets WOULD be more productive if they followed the extra character convention instead. One of the irritating compromises made by Unicode? It would perhaps also be nice if the file naming convention used 'nbsp' for spaces rather than 'sp' and eliminate the need for quotes around file and directory names, but adding quotes is used by SQL to indicate 'case-sensitive' strings, yet another convention to be given a nod to? If you get an associative key from a quoted field name it is NOT case-insensitive and while a second field with the same combination of characters would be 'silly' it is something that can happen for many reasons ... and explode() falls over in some instances as a result.

On 20/09/17 08:26, Stanislav Malyshev wrote:

Actually that HAS already been identified in this thread, and it is only the basic ASCII character set, but this is not actually specified anywhere?

Having established that the only characters that are case-insensitive in PHP7 ... the unicode basic latin set ... the discussion SHOULD be on either expanding that to cover all case folding or simply removing this rather limited case? Tony Marston is making an impassioned demand to retain this very limited case, and therefore expand it to cover all character sets, and as a fellow 'English only' coder, I can accept that argument. However many of my clients do not use English as a first language so any data handling has to be unicode based, and case in that data can be important, so is case-insensitive really as universal as Tony thinks? Certainly we need data case-insensitivity to handle unicode properly and not just a few english characters ( should I really add a capital 'E' to english just to please the spell checker? ) People are using their own languages when writing PHP variables and function names, and apart from a few edge cases this does seem to be working for them. As with SQL, the key programming words are in English, and I don't think anybody would suggest adding aliases for them, so restricting keywords to 'unicode basic latin set' can be defined, but does THEN making that case-insensitive add to the problems of making PHP more user friendly in handling unicode names elsewhere? I am seeing SQL field names coming in with unicode content, and these are then array keys in PHP ... the latin characters get lower cased at times and this DOES cause a problem if the metadata defines upper case and I suspect that is something that will never be changed now, but the actual rules applied would be nice to know?