Forum - proper encoding for POSTing to Rest API

proper encoding for POSTing to Rest API

Aldevinas Katkus

3-18-20

If I send a string like this "Some product name <>" via Rest API, then if I urlencode (in PHP) this string I get "Some%20product%20name%20%3C%3E" (or a bit different if I choose to encode as RFC1738) stored in the SERP, so it looks like the string is not decoded to normal text.
I have set header as 'Content-Type: application/x-www-form-urlencoded'
How to encode strings I send via rest api?

David Delač

3-18-20

Hi Aldevinas,

Can you please explain a bit more what the issue is? Is your problem that HAL doesn't decode properly or is hal encoding but some other place fails to decode?

Please also list any HAL functions that you are using in your workflow.

Best Regards,
David D.

Sanjay Parbhoo

3-18-20

Created byDavid Delač11:57 18 Mar 2020

Hi Aldevinas, Can you please explain a bit more what the issue is? Is your problem that HAL doesn't decode properly or is hal encoding but some other place fails to decode? Please also list any HAL functions that you are using in your workflow. Best Re

Hi Aldevinas

The content of your call should not be in the URL I think.

Aldevinas Katkus

3-18-20

I am POSTing in php:
$hwPostData['set_row_field.'.$rownr.'.Spec']='Some product name <>';

....
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($hwPostData,'','&',PHP_QUERY_RFC3986));

and this results in ORVc.Spec field value in SERP:
Some%20product%20name%20%3C%3E

So I can see the text is not decoded before storing to the SERP DB.

If I send plain text not urlencoded then it is saved as is:
I am sending
set_field.CustCode=100004&set_field.Comment=Some comment&set_field.OrderClass=WEB&set_field.CustOrdNr=WebRest 2&set_field.PlanShip=2020-04-05&set_field.ShipAddr0=ShipAddr0&set_field.ShipAddr1=ShipAddr1&set_field.ShipAddr2=ShipAddr2&set_field.ShipAddr3=ShipAddr3&set_field.Phone=+37061269996&set_row_field.0.ArtCode=10102&set_row_field.0.Quant=2&set_row_field.0.Price=123.45&set_row_field.0.VATCode=PVM1

and this results in Comment field value "Some comment" and CustOrdNr value "WebRest 2" with no spaces converted to plus signs or %20.

I don't use any hal functions, I am just trying to use rest api v.2.2 with basic http auth, because oauth crashes the server (bug is already reported)

David Delač

3-19-20

Hi Aldevinas,

We already have a wish documented for that:

API POST>>To be able to receive URL encoded format

I will make sure our Development Manager is aware of the need for it.

Let me know if I can assist you with anything else!

Best Regards,
David D.

Aldevinas Katkus

3-19-20

Thanks. An extra note: if we send an ampersand when inserting, e.g, a new customer named "some company &Co.", then ampersand is treated like a field separator and customer name will be stored as "Some company".

David Delač

3-20-20

Hi Aldevinas,

Thank you for your information. I added it to the wish.

Best Regards,
David D.

Gastón Salomone

5-25-20

Hi,

I would like to know if this data encoding issue is solved and If It is, in which release ?

Regards

David Delač

5-26-20

Hi Gastón

Issue is still present in the newest version but I have escalated it to our development manager now.

Best Regards,
David D.

Omar Dottin

6-12-20

Hi Gaston, this issue has been fixed in our development version. It will be available in our next release.

Best Regards,
Omar

Edgars Dzenis

6-13-20

Hi, Aldevinas!
Special symbols should be HTML URL encoded

Aldevinas Katkus

6-16-20

In my opinion the text I am sending is urlencoded https://www.w3schools.com/tags/ref_urlencode.ASP, but SERP does not understand urlencoded data.

Aldevinas Katkus

6-16-20

Created byAldevinas Katkus13:14 16 Jun 2020

In my opinion the text I am sending is urlencoded https://www.w3schools.com/tags/ref_urlencode.ASP, but SERP does not understand urlencoded data.

The first message on the top sends urlencoded data which, as I understand. is the standard/default way of sending data to the web server. It should be very very easy to decode the data on the serp end: check for header and if the "Content-Type" header value is "application/x-www-form-urlencoded" then just run the corresponding function.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST

Gastón Salomone

6-26-20

Hi,

I´ve downloaded and tested Release Candidate version 8.5 2020-06-22 (build 85342803) and this but is still there.

Please can you confirm when it would be fixed and in which version.

Regards

Omar Dottin

7-1-20

Hello! We have fixed the issue of reading urlencoded data in 06-22, but we have identified the issue where special characters such as ÑĮŌ are not being recognized. This is escalated as priority. Will update you when we have this fixed

Best Regards,
HW Support

Aldevinas Katkus

7-26-23

FYI not fixed yet in 8.5 2023-06-12 (build 85550736).
POSTing a string "inserted via Rest API ąčęėį & ž яшерты" results in
"inserted via Rest API ÄÄÄÄÄ‾ & Å¾ ÑÑÐµÑÑÑ" with RFC3986 encoding
or "inserted+via+Rest+API+ÄÄÄÄÄ‾+&+Å¾+ÑÑÐµÑÑÑ" with RFC1738 encoding, which is the default one in php function http_build_query().

Aldevinas Katkus

7-26-23

I figured out why this happened:
you must add charset=UTF-8 like this to fix characters and set argument separator to '&' for a workaround to work.

$headers[]='Content-type: application/x-www-form-urlencoded;charset=UTF-8';
$postfields=http_build_query($post,'','&',PHP_QUERY_RFC3986);//inserted via Rest API ąčęėį & ž яшерты

Problem solved (kind of a workaround).

Back to the list

New Message
Open Questions My Questions