Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » bugs.php.net downtime »

bugs.php.net downtime

php.internals

Rasmus Lerdorf

8 years ago
I need to move bugs.php.net to another server sometime today. I won't be able to do it without a little bit of downtime as I have to stop new activity on the existing box, copy the DB over and then point DNS to the new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for much longer than that. Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5 to 7.2 on faster hardware. -Rasmus

marius popa

8 years ago
Hello Rasmus, I can't access previous patches https://bugs.php.net/patch-display.php?bug_id=62300&patch=0&revision=latest On Tue, Jul 17, 2018 at 11:38 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

Niklas Keller

8 years ago
Hey Rasmus I just found this bug: https://bugs.php.net/bug.php?id=76553 Has this bug been like that before the migration, too? Or did something go wrong? Regards, Niklas Am Do., 19. Juli 2018 um 10:35 Uhr schrieb marius adrian popa <mapopa@gmail.com>:

Hoffman, Zachary Robert

8 years ago
On Thu, 2018-07-19 at 22:35 +0200, Niklas Keller wrote:
> Hey Rasmus > > I just found this bug: https://bugs.php.net/bug.php?id=76553 > > Has this bug been like that before the migration, too? Or did > something go wrong?
No, those used to be Unicode characters from the cyrillic block. https://web.archive.org/web/20180719203931/https://webcache.googleusercontent.com/search%3Fq=cache:https://bugs.php.net/bug.php%3Fid=76553 (https://tinyurl.com/it-used-to-look-like-this)
-- Zach Hoffman

Rasmus Lerdorf

8 years ago
On Thu, Jul 19, 2018 at 4:45 PM, Hoffman, Zachary Robert <zrhoffman@ku.edu> wrote:
> On Thu, 2018-07-19 at 22:35 +0200, Niklas Keller wrote: > > Hey Rasmus > > > > I just found this bug: https://bugs.php.net/bug.php?id=76553 > > > > Has this bug been like that before the migration, too? Or did > > something go wrong? > > No, those used to be Unicode characters from the cyrillic block.
This appears to be database-related. Something got messed up with encodings on the mysql dump/import from MySQL 5.1.73 into MariaDB 10.1.26: mysql> select sdesc from bugdb where id=76553; +----------------------------------------------------------------------------------+ | sdesc | +----------------------------------------------------------------------------------+ | Имя переменной может содержать управляющие | +----------------------------------------------------------------------------------+ 1 row in set (0.00 sec) MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553; +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | sdesc | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Ð˜Ð¼Ñ Ð¿ÐµÑ€ÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ может Ñ Ð¾Ð´ÐµÑ€Ð¶Ð°Ñ‚ÑŒ ÑƒÐ¿Ñ€Ð°Ð²Ð»Ñ ÑŽÑ‰Ð¸Ðµ | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) The dumped table schema from MySQL has: DROP TABLE IF EXISTS `bugdb`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `bugdb` ( `id` int(8) NOT NULL AUTO_INCREMENT, `package_name` varchar(80) CHARACTER SET latin1 DEFAULT NULL, `bug_type` varchar(32) CHARACTER SET latin1 NOT NULL DEFAULT 'Bug', `email` varchar(40) CHARACTER SET latin1 NOT NULL DEFAULT '', `reporter_name` varchar(80) CHARACTER SET latin1 DEFAULT '', `sdesc` varchar(80) CHARACTER SET latin1 NOT NULL DEFAULT '', `ldesc` text CHARACTER SET latin1 NOT NULL, `php_version` varchar(100) CHARACTER SET latin1 DEFAULT NULL, `php_os` varchar(32) CHARACTER SET latin1 DEFAULT NULL, `status` varchar(16) CHARACTER SET latin1 DEFAULT NULL, `ts1` datetime DEFAULT NULL, `ts2` datetime DEFAULT NULL, `assign` varchar(20) CHARACTER SET latin1 DEFAULT NULL, `passwd` varchar(64) CHARACTER SET latin1 DEFAULT NULL, `registered` tinyint(1) NOT NULL DEFAULT '0', `block_user_comment` char(1) DEFAULT 'N', `cve_id` varchar(15) DEFAULT NULL, `private` char(1) DEFAULT 'N', `visitor_ip` int(10) unsigned NOT NULL, PRIMARY KEY (`id`), KEY `php_version` (`php_version`(1)), KEY `status` (`status`), KEY `package_name` (`package_name`), FULLTEXT KEY `email` (`email`,`sdesc`,`ldesc`) ) ENGINE=MyISAM AUTO_INCREMENT=76637 DEFAULT CHARSET=utf8 PACK_KEYS=1; /*!40101 SET character_set_client = @saved_cs_client */; When I dump it from MariaDB I see: DROP TABLE IF EXISTS `bugdb`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `bugdb` ( `id` int(8) NOT NULL AUTO_INCREMENT, `package_name` varchar(80) CHARACTER SET latin1 DEFAULT NULL, `bug_type` varchar(32) CHARACTER SET latin1 NOT NULL DEFAULT 'Bug', `email` varchar(40) CHARACTER SET latin1 NOT NULL DEFAULT '', `reporter_name` varchar(80) CHARACTER SET latin1 DEFAULT '', `sdesc` varchar(80) CHARACTER SET latin1 NOT NULL DEFAULT '', `ldesc` text CHARACTER SET latin1 NOT NULL, `php_version` varchar(100) CHARACTER SET latin1 DEFAULT NULL, `php_os` varchar(32) CHARACTER SET latin1 DEFAULT NULL, `status` varchar(16) CHARACTER SET latin1 DEFAULT NULL, `ts1` datetime DEFAULT NULL, `ts2` datetime DEFAULT NULL, `assign` varchar(20) CHARACTER SET latin1 DEFAULT NULL, `passwd` varchar(64) CHARACTER SET latin1 DEFAULT NULL, `registered` tinyint(1) NOT NULL DEFAULT '0', `block_user_comment` char(1) DEFAULT 'N', `cve_id` varchar(15) DEFAULT NULL, `private` char(1) DEFAULT 'N', `visitor_ip` int(10) unsigned NOT NULL, PRIMARY KEY (`id`), KEY `php_version` (`php_version`(1)), KEY `status` (`status`), KEY `package_name` (`package_name`), FULLTEXT KEY `email` (`email`,`sdesc`,`ldesc`) ) ENGINE=MyISAM AUTO_INCREMENT=76650 DEFAULT CHARSET=utf8 PACK_KEYS=1; /*!40101 SET character_set_client = @saved_cs_client */; Other than the autoincrement they are identical. I normally use utf8mb4, but I figured I would play it safe and copy it over verbatim. I guess it wasn't safe. I'll do some research, but ideas welcome. -Rasmus

Rasmus Lerdorf

8 years ago
For future reference, here is what I did to fix the encoding problem: MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553; +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | sdesc | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Ð˜Ð¼Ñ Ð¿ÐµÑ€ÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ может Ñ Ð¾Ð´ÐµÑ€Ð¶Ð°Ñ‚ÑŒ ÑƒÐ¿Ñ€Ð°Ð²Ð»Ñ ÑŽÑ‰Ð¸Ðµ | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) MariaDB [phpbugsdb]> alter table bugdb drop index email; Query OK, 76298 rows affected (0.85 sec) Records: 76298 Duplicates: 0 Warnings: 0 MariaDB [phpbugsdb]> alter table bugdb modify sdesc varbinary(80) NOT NULL DEFAULT '', modify ldesc binary NOT NULL, modify email varbinary(40) NOT NULL DEFAULT ''; Query OK, 76298 rows affected, 65535 warnings (0.65 sec) Records: 76298 Duplicates: 0 Warnings: 76091 MariaDB [phpbugsdb]> alter table bugdb modify sdesc varchar(80) CHARACTER SET utf8mb4 NOT NULL DEFAULT '', modify ldesc text CHARACTER SET utf8mb4 NOT NULL, modify email varchar(40) CHARACTER SET utf8mb4 NOT NULL DEFAULT ''; Query OK, 76298 rows affected, 127 warnings (0.57 sec) Records: 76298 Duplicates: 0 Warnings: 127 MariaDB [phpbugsdb]> alter table bugdb add FULLTEXT INDEX `email` (`email`,`sdesc`,`ldesc`); Query OK, 76298 rows affected (1.56 sec) Records: 76298 Duplicates: 0 Warnings: 0 MariaDB [phpbugsdb]> select sdesc from bugdb where id=76553; +----------------------------------------------------------------------------------+ | sdesc | +----------------------------------------------------------------------------------+ | Имя переменной может содержать управляющие | +----------------------------------------------------------------------------------+ 1 row in set (0.00 sec) The trick was to convert the columns to binary first. When I went straight from latin1 to utf8 I got the utf8 equivalent of the latin1 characters. By telling it that the data was actually binary first, it converted from binary to utf8 which appears to have worked. There were some warnings, which I assume are invalid utf8 byte sequences somewhere. -Rasmus

Rasmus Lerdorf

8 years ago
Uh, ok, something obviously went wrong there. Checking. On Sat, Jul 21, 2018 at 8:30 AM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

Rasmus Lerdorf

8 years ago
Ok, should be fixed now. https://bugs.php.net/bug.php?id=76553 looks good. But I think between my backup and the conversion I lost a couple of bug comments from this morning. Sorry about that. -Rasmus On Sat, Jul 21, 2018 at 8:31 AM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

Yasuo Ohgaki

8 years ago
On Sat, Jul 21, 2018 at 10:14 AM Rasmus Lerdorf <rasmus@lerdorf.com> wrote:
> Other than the autoincrement they are identical. I normally use utf8mb4, > but I figured I would play it safe and copy it over verbatim. I guess it > wasn't safe. >
Right. There are risks. For example, encoding like SJIS contains \ as a part of valid char. When encoding is mixed, escape could be disabled and injections are possible. Even when UTF-8 is used, mixed invalid encoding handling can break security measures. e.g. Invalid UTF-8 encoding that is missing the last multibyte byte. When santaization is required, programmers have two choices. - Remove all bytes specified by MSB of UTF-8 first byte. i.e. Consume the last byte. - Remove only bytes that are invalid as UTF-8. i.e Leave last ASCII char, for example If these designs are mixed, encoding attack is possible also. DoS by invalid char is trivial. Current web browsers can refuse to render entire page that has badly broken encoding. The only good countermeasure against encoding attack is encoding validation with Fail Fast principle. i.e. Validate encoding at application software's outer most trust boundary. I'll do some research, but ideas welcome.
>
IMO, all data should be converted to valid UTF-8 encoding as we use UTF-8 as bugs.php.net encoding. Replace invalid date to "?" or something else. Some data will be lost, however valid char encoding is mandatory for correct data handling as described above. In order to replace invalid char to "?" (or something else), mb_convert_encoding() can be used. "mbstring.substitute_character" INI is for specifying replacing char. Default is none, so it removes invalid data by default. If you would like to keep original data, number of detected invalid chars are recorded and can be retrieved by mb_get_info()'s array. "illegal_chars" is "Total number of illegal chars in the script's lifetime". By checking this, invalid char existence can be checked. (Alternatively, simply comparing original and converted data works also.) You might want to count number of all illegal chars in the db before converting data. "illegal_chars" is handy for this. Old data may be added to converted data by using base64 if it's necessary. Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

Rasmus Lerdorf

8 years ago
It looks like the MDB2 -> PDO migration messed this up. There is a weird MDB2 feature fetchAll() feature that returns the first field as the associative key or something like that. Need to rewrite the code to not rely on that. The MDB2 query returns: array(2) { ["fix-71630.patch"]=> array(1) { [0]=> array(2) { [0]=> string(10) "1522170171" [1]=> string(11) "cmb@php.net" } } ["zero-data.patch"]=> array(1) { [0]=> array(2) { [0]=> string(10) "1521729404" [1]=> string(11) "cmb@php.net" } } } And the PDO query gives: array(2) { [0]=> array(3) { ["patch"]=> string(15) "fix-71630.patch" ["revision"]=> int(1522170171) ["developer"]=> string(11) "cmb@php.net" } [1]=> array(3) { ["patch"]=> string(15) "zero-data.patch" ["revision"]=> int(1521729404) ["developer"]=> string(11) "cmb@php.net" } } I will find some time to fix this in the next day or two unless someone beats me to it. -Rasmus On Thu, Jul 19, 2018 at 4:35 AM, marius adrian popa <mapopa@gmail.com> wrote:

Yasuo Ohgaki

8 years ago
Hi, On Wed, Jul 18, 2018 at 5:39 AM Rasmus Lerdorf <rasmus@lerdorf.com> wrote:
> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5 > to 7.2 on faster hardware. >
It feels faster now. Nice work. Anyway, it seemed UTF-8 char is not handled well. I tried to send reply with UTF-8 chars にほんご文字列 for https://bugs.php.net/bug.php?id=76635 and response is empty page with 500 code. ---- Request URL: https://bugs.php.net/bug.php?id=76635&edit=1 Request Method: POST Status Code: 500 Internal Server Error Remote Address: 206.189.200.141:443 Referrer Policy: no-referrer-when-downgrade ---- Invalid UTF-8 data stored in bug db can result in broken pages. How about convert invalid data into URL encoded data or something else? Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

Nikita Popov

8 years ago
On Tue, Jul 17, 2018 at 10:38 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:
> I need to move bugs.php.net to another server sometime today. I won't be > able to do it without a little bit of downtime as I have to stop new > activity on the existing box, copy the DB over and then point DNS to the > new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for > much longer than that. > > Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5 > to 7.2 on faster hardware. > > -Rasmus >
I'm getting a WSOD when trying to add a comment to a bug while logged in. Not sure what's different for me, as it seems to work for other @php.net users. Nikita

Christoph Becker

8 years ago
On 21.07.2018 at 10:46, Nikita Popov wrote:
> On Tue, Jul 17, 2018 at 10:38 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote: > >> I need to move bugs.php.net to another server sometime today. I won't be >> able to do it without a little bit of downtime as I have to stop new >> activity on the existing box, copy the DB over and then point DNS to the >> new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable for >> much longer than that. >> >> Hopefully the new box will be a bit quicker too. It is moving from PHP 5.5 >> to 7.2 on faster hardware. > > I'm getting a WSOD when trying to add a comment to a bug while logged in. > Not sure what's different for me, as it seems to work for other @php.net > users.
Same for me; it doesn't matter whether I'm logged in as cmb@ or just commenting as user. Interestingly, altough I get the 500, I can change the meta information (status, summary, etc.), but no comment is added.
-- Christoph M. Becker

Rasmus Lerdorf

8 years ago
That one should be fixed. It was actually posting the comments, but getting an error after. If you reload the bug you see it. On Sat, Jul 21, 2018 at 5:41 AM, Christoph M. Becker <cmbecker69@gmx.de> wrote:

Nikita Popov

8 years ago
Still doesn't work for me. I get a white screen and the comment does not appear even after I reload the page. Nikita On Sat, Jul 21, 2018 at 11:51 AM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

Christoph Becker

8 years ago
On 21.07.2018 at 11:51, Rasmus Lerdorf wrote:
> That one should be fixed. It was actually posting the comments, but getting > an error after. If you reload the bug you see it.
I've commented multiple times on bug 76637, but none of the comments are visible. I also commented on bug 76642 and successfully changed a typo in the title (summary), but the “meta info changed” comment is not there either.

Rasmus Lerdorf

8 years ago
Works fine for me on https://bugs.php.net/bug.php?id=76635&thanks=1 On Sat, Jul 21, 2018 at 6:06 AM, Christoph M. Becker <cmbecker69@gmx.de> wrote:

Christoph Becker

8 years ago
On 21.07.2018 at 12:10, Rasmus Lerdorf wrote:
> Works fine for me on https://bugs.php.net/bug.php?id=76635&thanks=1
I just commented “Testing”, but the comment isn't there. Don't the error logs give some useful hints?

Rasmus Lerdorf

8 years ago
Yeah, working on it. Amazingly it looks like it is an ipv6 issue. The old bugs didn't have an ipv6 address, but the new one does and we store the ip of the commenter using the MySQL function inet_aton() and not inet6_aton(). Need to change that, but also alter the table to make the column fit an ipv6 addr. On Sat, Jul 21, 2018 at 6:18 AM, Christoph M. Becker <cmbecker69@gmx.de> wrote:

Rasmus Lerdorf

8 years ago
By the way, the following users have ssh accounts on bugs.php.net: bjori derick felipe nikic rasmus sas scottmac tyrael the error log is in /srv/bugs.php.net/logs/error.log and the code is here: git@git.php.net:/web/bugs.git Feel free to log in and fix stuff as you find more issues. Right now the patch handling is still broken along with the latin1 columns that need to be converted to utf8. -Rasmus

Christoph Becker

8 years ago
On 21.07.2018 at 12:20, Rasmus Lerdorf wrote:
> Yeah, working on it. Amazingly it looks like it is an ipv6 issue. The old > bugs didn't have an ipv6 address, but the new one does and we store the ip > of the commenter using the MySQL function inet_aton() and not inet6_aton(). > Need to change that, but also alter the table to make the column fit an > ipv6 addr.
Works fine for me again. Thanks, Rasmus!

Pedro Magalhães

8 years ago
Hi Rasmus, I'm not able to attach a PR to a bug report, however I'm not sure if this wasn't already happening before the migration. When submitting the form, I get a 500. Thanks, Pedro On Tue, Jul 17, 2018 at 9:38 PM Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

Gabriel Caruso

8 years ago
> > Hi Rasmus, > > I'm not able to attach a PR to a bug report, however I'm not sure if this > wasn't already happening before the migration. When submitting the form, I > get a 500. > > Thanks, > Pedro > > On Tue, Jul 17, 2018 at 9:38 PM Rasmus Lerdorf <rasmus@lerdorf.com> wrote: > > > I need to move bugs.php.net to another server sometime today. I won't be > > able to do it without a little bit of downtime as I have to stop new > > activity on the existing box, copy the DB over and then point DNS to the > > new box. The DNS TTL is only 5 minutes, so it shouldn't be unavailable > for > > much longer than that. > > > > Hopefully the new box will be a bit quicker too. It is moving from PHP > 5.5 > > to 7.2 on faster hardware. > > > > -Rasmus > > >
Pedro This was previously reported as a bug https://bugs.php.net/bug.php?id=76079. Thanks
-- Gabriel Caruso

Pedro Magalhães

8 years ago
On Fri, Aug 3, 2018 at 6:44 PM Gabriel Caruso <carusogabriel34@gmail.com> wrote:
> Pedro > > This was previously reported as a bug > https://bugs.php.net/bug.php?id=76079. >
I knew I had read about it somewhere! There it is. Thanks!