Bug #5730
open/parseline -b incorrectly UTF8 encodes text
Added by JD Byrnes over 1 year ago. Updated over 1 year ago.
0%
Description
Probably related to most of the other UTF8 issues where -u isn't implemented...
/parseline -obqn &binvar
encodes the &binvar as UTF8 unexpectedly.
Updated by Per Amundsen over 1 year ago
Probably, do you have an example with some text in the &binvar?
I am not able to implement -u without breaking some important scripts, but these scripts doesn't use binvars, so I could possible fix them.
Updated by JD Byrnes over 1 year ago
Per Amundsen wrote in #note-1:
Probably, do you have an example with some text in the &binvar?
I am not able to implement -u without breaking some important scripts, but these scripts doesn't use binvars, so I could possible fix them.
I've done a bit more testing.
alias test {
bset -ta &test 1 PRIVMSG #The\bLobby :©
parseline -obqpn &test
}
The /bset sets the variable correctly, respecting if the -a switch is used.
&test can be read, and the © character is correctly shown as 194 196 / 196 (depending on the -a)
The issue is when the parseline -bo is called with the &binvar. I'd expect the &binvar not to be encoded (it should already be if we want it to be).
However, it still attempts to ensure UTF8 encoding (even if it's invalid UTF8)
If the -a switch is not used (valid UTF8) it is correctly sent as a UTF8 encoded string: ©
If the -a switch is used (invalid UTF8) it is sent incorrectly: � (0xEFBFBD) also known as the UTF8 replacement character.
To be clear: If we were sending text, I'd expect it to maybe encode the characters, but since we're sending an &binvar, I wouldn't expect AdiIRC to do anything to it (except maybe add a crlf if -n is used)
Because it's trying to encode (and failing) the text, it makes AdiIRC incompatible with our authentication system.
Updated by JD Byrnes over 1 year ago
mIRC's behaviour:
The © is encoded as © irrespective of how it was set (bvar © and © are treated as equal) if /parseline -b is used
The © is encoded as © irrespective of how it was set (bvar © and © are treated as equal) if /parseline -bu0 is used
I suspect that this is a (unrelated) bug in mIRC as © should probably be sent as © when -u0 switch is used with /parseline. It seems mIRC is actual utf8decoding it.
Updated by Per Amundsen over 1 year ago
Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.
You probably then want to do something like -a0n0 to avoid any encoding/decoding of both the input and the output.
bset -ta &test 1 PRIVMSG #The\bLobby :© /parseline -a0u0qobpn &test
Updated by JD Byrnes over 1 year ago
Per Amundsen wrote in #note-4:
Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.
You probably then want to do something like -a0n0 to avoid any encoding/decoding of both the input and the output.
[...]
Yeah, that's pretty much what Khaled had said (although Khaled indicated it's just -a, which seems to work on mIRC)
https://forums.mirc.com/ubbthreads.php/topics/271646/parseline-always-mangles-data#Post271646
Updated by Per Amundsen over 1 year ago
Oh okay, was going by someone who tested it and they explained it like that, but I think they meant -a1u0 which would be the same as just -a. I think and -a0 is the same as no -a, while -a1 is the same as -a.
I'll figure it out.
Updated by JD Byrnes over 1 year ago
Just tested sending a PRIVMSG in &bvar format (example alias on mIRC forum post above) - Figured it might save you some time since I have a few aliases to test with.
- without -a
- Parseline out: PRIVMSG #Test :©
- with -a
- Parseline out: PRIVMSG #Test :©
- with -a0
- Parseline out: PRIVMSG #Test :©
- with -a1
- Parseline out: PRIVMSG #Test :©
Can confirm that with -u0 the server receives these exactly as specified.
This seems to match the -a that is available for /bset etc.
Updated by JD Byrnes over 1 year ago
Per Amundsen wrote in #note-4:
Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.
[...]
Are you able to confirm if this was added to b20230614? I couldn't find any reference to it
Updated by Per Amundsen over 1 year ago
It's not added yet, ran into some regressions. It will be in next beta for sure, sorry about that.
Updated by Per Amundsen over 1 year ago
I have managed to separate the utf8 from raw in most places now, you can check it out in this build https://adiirc.com/build/1688604936/AdiIRC.exe.
There is still work to do and things to test, but you should at least be able to send real raw data now.