Project

General

Profile

Actions

Bug #5730

open

/parseline -b incorrectly UTF8 encodes text

Added by JD Byrnes 12 months ago. Updated 10 months ago.

Status:
New
Priority:
Normal
Assignee:
Category:
Scripting
Target version:
Start date:
05/10/2023
Due date:
% Done:

0%

Estimated time:
Operative System:
All
Regression:
No

Description

Probably related to most of the other UTF8 issues where -u isn't implemented...

/parseline -obqn &binvar encodes the &binvar as UTF8 unexpectedly.

Actions #1

Updated by Per Amundsen 12 months ago

Probably, do you have an example with some text in the &binvar?

I am not able to implement -u without breaking some important scripts, but these scripts doesn't use binvars, so I could possible fix them.

Actions #2

Updated by JD Byrnes 12 months ago

Per Amundsen wrote in #note-1:

Probably, do you have an example with some text in the &binvar?

I am not able to implement -u without breaking some important scripts, but these scripts doesn't use binvars, so I could possible fix them.

I've done a bit more testing.

alias test {
bset -ta &test 1 PRIVMSG #The\bLobby :©
parseline -obqpn &test
}

The /bset sets the variable correctly, respecting if the -a switch is used.
&test can be read, and the © character is correctly shown as 194 196 / 196 (depending on the -a)

The issue is when the parseline -bo is called with the &binvar. I'd expect the &binvar not to be encoded (it should already be if we want it to be).
However, it still attempts to ensure UTF8 encoding (even if it's invalid UTF8)
If the -a switch is not used (valid UTF8) it is correctly sent as a UTF8 encoded string: ©
If the -a switch is used (invalid UTF8) it is sent incorrectly: � (0xEFBFBD) also known as the UTF8 replacement character.

To be clear: If we were sending text, I'd expect it to maybe encode the characters, but since we're sending an &binvar, I wouldn't expect AdiIRC to do anything to it (except maybe add a crlf if -n is used)
Because it's trying to encode (and failing) the text, it makes AdiIRC incompatible with our authentication system.

Actions #3

Updated by JD Byrnes 12 months ago

mIRC's behaviour:
The © is encoded as © irrespective of how it was set (bvar © and © are treated as equal) if /parseline -b is used
The © is encoded as © irrespective of how it was set (bvar © and © are treated as equal) if /parseline -bu0 is used

I suspect that this is a (unrelated) bug in mIRC as © should probably be sent as © when -u0 switch is used with /parseline. It seems mIRC is actual utf8decoding it.

Actions #4

Updated by Per Amundsen 12 months ago

Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.

You probably then want to do something like -a0n0 to avoid any encoding/decoding of both the input and the output.

bset -ta &test 1 PRIVMSG #The\bLobby :©
/parseline -a0u0qobpn &test 
Actions #5

Updated by JD Byrnes 12 months ago

Per Amundsen wrote in #note-4:

Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.

You probably then want to do something like -a0n0 to avoid any encoding/decoding of both the input and the output.

[...]

Yeah, that's pretty much what Khaled had said (although Khaled indicated it's just -a, which seems to work on mIRC)
https://forums.mirc.com/ubbthreads.php/topics/271646/parseline-always-mangles-data#Post271646

Actions #6

Updated by Per Amundsen 12 months ago

Oh okay, was going by someone who tested it and they explained it like that, but I think they meant -a1u0 which would be the same as just -a. I think and -a0 is the same as no -a, while -a1 is the same as -a.

I'll figure it out.

Actions #7

Updated by JD Byrnes 12 months ago

Just tested sending a PRIVMSG in &bvar format (example alias on mIRC forum post above) - Figured it might save you some time since I have a few aliases to test with.

  • without -a
  • Parseline out: PRIVMSG #Test :©
  • with -a
  • Parseline out: PRIVMSG #Test :©
  • with -a0
  • Parseline out: PRIVMSG #Test :©
  • with -a1
  • Parseline out: PRIVMSG #Test :©

Can confirm that with -u0 the server receives these exactly as specified.

This seems to match the -a that is available for /bset etc.

Actions #8

Updated by JD Byrnes 10 months ago

Per Amundsen wrote in #note-4:

Since we now know about the undocumented /parseline -aN .. I will implement it and -uN for &binvars for next beta.

[...]

Are you able to confirm if this was added to b20230614? I couldn't find any reference to it

Actions #9

Updated by Per Amundsen 10 months ago

It's not added yet, ran into some regressions. It will be in next beta for sure, sorry about that.

Actions #10

Updated by Per Amundsen 10 months ago

I have managed to separate the utf8 from raw in most places now, you can check it out in this build https://adiirc.com/build/1688604936/AdiIRC.exe.

There is still work to do and things to test, but you should at least be able to send real raw data now.

Actions

Also available in: Atom PDF