Everything

php

JSON ampersand patch

php.internals

Tyler Lawson

19 years ago

Hello,

I hope this is the right place for this, as I'd like to post a patch for
your consideration to the way the JSON handles ampersands.  I have had
problems sending JSON data back and forth over POST requests, where an
ampersand separates variables.  This patch will convert "&" into "\u0026",
which will allow ampersands to travel freely and be parsed by any valid JSON
decoder.

This has been applied against php-5.2.1

--- json.c      2007-01-12 07:17:32.000000000 -0500
+++ my_json.c    2007-04-27 15:12:44.000000000 -0400
@@ -301,6 +301,11 @@
smart_str_appendl(buf, "\\t", 2);
}
break;
+            case '&':
+                {
+                    smart_str_appendl(buf, "\\u0026", 6);
+                }
+                break;
default:
{
if (us < ' ' || (us & 127) == us)


Tyler

Antony Dovgal

19 years ago

On 04/27/2007 11:39 PM, Tyler Lawson wrote:

> Hello,
>
> I hope this is the right place for this, as I'd like to post a patch for
> your consideration to the way the JSON handles ampersands.  I have had
> problems sending JSON data back and forth over POST requests, where an
> ampersand separates variables.  This patch will convert "&" into "\u0026",
> which will allow ampersands to travel freely and be parsed by any valid JSON
> decoder.

I believe a short reproduce case would explain it better.

> This has been applied against php-5.2.1
>
> --- json.c      2007-01-12 07:17:32.000000000 -0500
> +++ my_json.c    2007-04-27 15:12:44.000000000 -0400
> @@ -301,6 +301,11 @@
>                      smart_str_appendl(buf, "\\t", 2);
>                  }
>                  break;
> +            case '&':
> +                {
> +                    smart_str_appendl(buf, "\\u0026", 6);
> +                }
> +                break;
>              default:
>                  {
>                      if (us < ' ' || (us & 127) == us)
>
>
> Tyler
>

--
Wbr,
Antony Dovgal

Tyler Lawson

19 years ago

Alright, here goes.

post.php
<?php
$data="data=This is my data & it is really great";
$json_data=json_encode($data);

$request ="GET /test.php HTTP/1.0\n";
$request.="Host: something.com\n";
$request.="Content-Type: application/x-www-form-urlencoded\n"
$request.="Content-Length: ".strlen($json_data)."\n";
$request.="\n";
$fsock=fsockopen("http://server.com ",80);
fwrite($fsock,$request);

$response=fread($fsock,2048);
echo $response;
?>

test.php:
<?php
var_dump($_POST);
?>

Under php-5.2.1, the output would be:

array(4) {
["data"]=>
string(16) "This is my data "

["it is really great"]=>
string(0) ""
}

With this patch it would be:

array(4) {
["data"]=>
string(42) "This is my data \u0026 it is really great"
}


Tyler
-----Original Message-----
From: Antony Dovgal [mailto:antony@zend.com]
Sent: Friday, April 27, 2007 3:59 PM
To: Tyler Lawson
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] JSON ampersand patch

On 04/27/2007 11:39 PM, Tyler Lawson wrote:

> Hello,
>
> I hope this is the right place for this, as I'd like to post a patch for
> your consideration to the way the JSON handles ampersands.  I have had
> problems sending JSON data back and forth over POST requests, where an
> ampersand separates variables.  This patch will convert "&" into "\u0026",
> which will allow ampersands to travel freely and be parsed by any valid

JSON

> decoder.

I believe a short reproduce case would explain it better.

> This has been applied against php-5.2.1
>
> --- json.c      2007-01-12 07:17:32.000000000 -0500
> +++ my_json.c    2007-04-27 15:12:44.000000000 -0400
> @@ -301,6 +301,11 @@
>                      smart_str_appendl(buf, "\\t", 2);
>                  }
>                  break;
> +            case '&':
> +                {
> +                    smart_str_appendl(buf, "\\u0026", 6);
> +                }
> +                break;
>              default:
>                  {
>                      if (us < ' ' || (us & 127) == us)
>
>
> Tyler
>

--
Wbr,
Antony Dovgal

Rasmus Lerdorf

19 years ago

Well, your example is weird in many ways.  First, you don't ever send
$json_data over the wire as far as I can tell.  And if you did, are you
actually sending it in form-urlencoded format as the mime-type you
specified indicated?  As in, something like:

$request .= "json_data=".urlencode($json_data);

This will automatically get decoded by PHP because that's what
urlencoded form data is supposed to look like.

It is quite likely that you aren't actually sending valid urlencoded
data and just sending the raw jsonn data over the wire in which case you
should be using application/json as your mime type and fetch it directly
from the raw post data.

-Rasmus

Tyler Lawson wrote:

Tyler Lawson

19 years ago

That was just a crappy test case I put together really quickly, but I do see
you are correct.  I was so focused on ampersands being the issue that the
simple and correct solution eluded me.

Sorry to bother you.

Tyler Lawson

-----Original Message-----
From: Rasmus Lerdorf [mailto:rasmus@lerdorf.com]
Sent: Friday, April 27, 2007 4:26 PM
To: Tyler Lawson
Cc: 'Antony Dovgal'; internals@lists.php.net
Subject: Re: [PHP-DEV] JSON ampersand patch

Well, your example is weird in many ways.  First, you don't ever send
$json_data over the wire as far as I can tell.  And if you did, are you
actually sending it in form-urlencoded format as the mime-type you
specified indicated?  As in, something like:

$request .= "json_data=".urlencode($json_data);

This will automatically get decoded by PHP because that's what
urlencoded form data is supposed to look like.

It is quite likely that you aren't actually sending valid urlencoded
data and just sending the raw jsonn data over the wire in which case you
should be using application/json as your mime type and fetch it directly
from the raw post data.

-Rasmus

Tyler Lawson wrote:

> Alright, here goes.
>
> post.php
> <?php
> $data="data=This is my data & it is really great";
> $json_data=json_encode($data);
>
> $request ="GET /test.php HTTP/1.0\n";
> $request.="Host: something.com\n";
> $request.="Content-Type: application/x-www-form-urlencoded\n"
> $request.="Content-Length: ".strlen($json_data)."\n";
> $request.="\n";
> $fsock=fsockopen("http://server.com ",80);
> fwrite($fsock,$request);
>
> $response=fread($fsock,2048);
> echo $response;
> ?>
>
> test.php:
> <?php
> var_dump($_POST);
> ?>
>
> Under php-5.2.1, the output would be:
>
> array(4) {
>   ["data"]=>
>   string(16) "This is my data "
>
>   ["it is really great"]=>
>   string(0) ""
> }
>
> With this patch it would be:
>
> array(4) {
>   ["data"]=>
>   string(42) "This is my data \u0026 it is really great"
> }
>
>
> Tyler
> -----Original Message-----
> From: Antony Dovgal [mailto:antony@zend.com]
> Sent: Friday, April 27, 2007 3:59 PM
> To: Tyler Lawson
> Cc: internals@lists.php.net
> Subject: Re: [PHP-DEV] JSON ampersand patch
>
> On 04/27/2007 11:39 PM, Tyler Lawson wrote:
>> Hello,
>>
>> I hope this is the right place for this, as I'd like to post a patch for
>> your consideration to the way the JSON handles ampersands.  I have had
>> problems sending JSON data back and forth over POST requests, where an
>> ampersand separates variables.  This patch will convert "&" into

"\u0026",