Show a write's answer in the charset it names
Summary
A browser session on a public Japanese bid-information site reports the answer to each write the gate sends under writes[].response. The site answers Content-Type: text/html;charset=Windows-31J (Shift_JIS), and the report read:
// �ʋ�D
var OPEN_BID = "100201";
where the page says // 一般競争入札. Every Japanese word in the answer was unreadable.
Root cause
The gate takes the answer with Fetch.getResponseBody, which hands text over base64-encoded as the raw bytes the site sent. describeResponse decoded the base64 and then read the bytes as UTF-8 (strings.ToValidUTF8), whatever charset the answer named, so Shift_JIS and EUC-JP text came out as replacement characters.
Fix
In writegate/policy.go, describeResponse decodes the bytes from the charset the answer's content type names (htmlindex.Get, so every WHATWG label such as Windows-31J, Shift_JIS, or EUC-JP resolves), then hides sensitive values as before. An answer with no charset, an unknown one, or one handed over as text already reads as it did. gate.go is unchanged.
Tests
TestDescribeResponseReadsItsCharset: Shift_JIS HTML and EUC-JP JSON handed over as bytes read as Japanese, with the JSON's password still hidden; UTF-8 and an answer with no charset read as before; a body handed over as text is left as it is. The Shift_JIS and EUC-JP cases failed before the fix.
Checked with the built binary against the live site: the report now reads // 一般競争入札.
Summary by cubic
Fixes the write gate so it shows a write's answer in the charset the response names, instead of always reading it as UTF-8.
A browser session on a public Japanese bid site answers writes in Shift_JIS, and every Japanese word in the reported answer came out as replacement characters. The response bytes are now decoded from the charset in the answer's Content-Type (via htmlindex, so labels like Windows-31J, Shift_JIS, or EUC-JP resolve). Secrets are hidden both in the bytes as sent (before decoding) and in the decoded text, so a secret sent in another charset than the one named is still hidden. Answers with no charset, an unknown one, or handed over as text read as before.
Summary by CodeRabbit
- Bug Fixes
- Response details now display readable text for base64-encoded content using supported character sets, including Shift_JIS and EUC-JP.
- Sensitive values remain hidden in decoded response text.
- Responses that are already text continue to display without being treated as encoded content.