CJK characters in WASAPI audio device name is broken displaying.

• Jan 15, 2019 - 13:37
Reported version
3.0
Type
Graphical (UI)
Frequency
Once
Severity
S4 - Minor
Reproducibility
Always
Status
needs info
Regression
No
Workaround
No
Project
Tags

(This is repost of https://musescore.org/en/node/279203#comment-886726 )

Related problems is almost fixed by @ericfontainejazz's effort,
but only when set API to "WASAPI", CJK characters in device name is broken displaying.
(In the case of other settings "MME", "DirectSound" and so on for API, it is displaying correctly.)

Audio/MIDI device itself works.
Problem is only the button(drop down list) label displaying.

I think that although the cause is in the output of PortAudio,
if MuseScore could absorb it as in case of MME, DirectSound and so on,
I think it is better workaround at present.

My environmet is as follows:
OS: Windows 10 (10.0), Arch.: x86_64, MuseScore version (64-bit): 3.0.1.5087, revision: 06a66a2

device_name_broken.png


Comments

Gosh, I don't know how easy it will be for me to fix this issue, since I would have to set myself up with a windows computer with everything in japanese just in order to debug the problem.

Unfortuantely, I don't know much MuseScore can do to solve this problem, because MuseScore is simply just using portaudio/midi to request a list of devices from windows. If those strings have unicode characters from supplemental planes (i.e. they require more than 16-bits to represent), then what I'm guessing happens is either those strings aren't properly handed over, or maybe the QComboBoxes don't know to handle the supplemental planes.

Basically this seems like too hard of an issue for me to fix, and the problem itself is very minor. I'm sorry but you might just have to tolerate the improperly-rendered strings.

Status active won't fix

I'm going to mark as won't fix for now. If for some reason there are dozens of CJK users who experience this problem and would like it fixed, then I will give it a shot. But without a high demand to fix it I don't see it is worth the effort to debug for now.

In reply to by ericfontainejazz

Yes, this is conspicuous but minor problem. It is not indispensable.
Functional problems do not occur, so almost CJK users will tolerate it.
(But, they may be misunderstand that the quality of MuseScore app is slightly low.)

I am not agree to "won't fix" but could understand as policy,
MuseScore itself not want to convert character encoding of string from API/Low-Layer library,
simply use the string as-is.

The essential cause is in the PortAudio library.
If I had time to spare and economic margin for life,
I will fix PortAudio problem.
Unlike before, that project seems to be active recently.

Thank you so much for considering.
I agree to close this issue without fixed.

I can only looking at the code and guess what is wrong. I notice that both Portaudio::apiList() and Portaudio::deviceList() use QString::fromLocal8Bit() for converting the string received from the api into the string musescore will use:

al.append(QString::fromLocal8Bit(info->name));

That code was by lasconic on May 2016 and in commit bc9bc93 and the purpose was "use local encoding for Portaudio device and api". Previously that line just used the plain char* string info->name directly without using that conversion. I assume lasonic did this to avoid a problem with how strings in different langauges get displayed...and I see that function (https://doc.qt.io/qt-5/qstring.html#fromLocal8Bit) description is:

"Returns a QString initialized with the first size characters of the 8-bit string str." and "QTextCodec::codecForLocale() is used to perform the conversion." ...and the description for that function is "On Windows, the codec will be based on a system locale. On Unix systems, the codec will might fall back to using the iconv library if no builtin codec for the locale can be found."

Interestingly the portmidi code for receiving api device string names, PortMidiDriver::deviceInList() and PortMidiDriver::deviceOutList(), don't use that special QString::fromLocal8Bit function, but rather just use QString(info->name). I'm curious, do the port midi input and output devices look normal or messed up in japanese??? I would think should use the same char* to QString conversion function in both cases, but of course need to figure out what is best. Looking at Qt docs, all the different string conversion functions are:

QString fromCFString(CFStringRef string)
QString fromLatin1(const char *str, int size = -1)
QString fromLatin1(const QByteArray &str)
QString fromLocal8Bit(const char *str, int size = -1)
QString fromLocal8Bit(const QByteArray &str)
QString fromNSString(const NSString *string)
QString fromRawData(const QChar *unicode, int size)
QString fromStdString(const std::string &str)
QString fromStdU16String(const std::u16string &str)
QString fromStdU32String(const std::u32string &str)
QString fromStdWString(const std::wstring &str)
QString fromUcs4(const uint *unicode, int size = -1)
QString fromUcs4(const char32_t *str, int size = -1)
QString fromUtf8(const char *str, int size = -1)
QString fromUtf8(const QByteArray &str)
QString fromUtf16(const ushort *unicode, int size = -1)
QString fromUtf16(const char16_t *str, int size = -1)
QString fromWCharArray(const wchar_t *string, int size = -1)

Considering that the input string from port audio is a char*, that might narrow down the list...but without knowing the true original representation of the sting, it is hard to guess what the best function to us it. My guess would be that the string is sent as Utf8, which is how to represent unicode with 8-bit chars only (in which case unicode may use up to four 8-bit chars to represent a single unicode character.)

Interestingly, it seems the default QString constructor from char* according to https://doc.qt.io/qt-5/qstring.html#QString-7 is converted to Unicode using the fromUtf8() function

In reply to by ericfontainejazz

PS C:\Users\kenzi> Get-UICulture

LCID Name DisplayName
---- ---- -----------
1041 ja-JP 日本語 (日本)

PS C:\Users\kenzi> Get-Host

Name : ConsoleHost
Version : 5.1.17763.134
InstanceId : 34c18328-81f2-4a95-a90f-19d67765378a
UI : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture : ja-JP
CurrentUICulture : ja-JP
PrivateData : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled : True
IsRunspacePushed : False
Runspace : System.Management.Automation.Runspaces.LocalRunspace

In reply to by ericfontainejazz

AFAIK,
In the past, there was a problem with PortAudio(It used to be called PortMIDI) itself,
it was return the string by incorrect encoding.
Is it correct encoding at present?

In any case,
I think that MuseScore(or if Qt is supported) have to convert to CP932 not UTF-8.
The information of system encoding is CP932, it will be able to taken from some Windows System API, I think.

Again,
I agree to close this issue without fixed.
I do not expect any more.
Thank you for your considering and effort, and MuseScore dev team.

I recall this problem as well. I think you reported it via Twitter many years ago, and I posted it here somewhere in the issue tracker. So it's still here somewhere.

In reply to by Thomas

It is slightly differ.
In the past year, I reported about "MME MIDI device name" not about "WASAPI audio device name".
Although this is similar and related issue, slightly differ.
These are using other API.

Again,
this is very minor problem. As you saying. I think so, too.
Functional problems do not occur, so almost CJK users will tolerate it.

I agree to close this issue without fixed.
Please use your power and resource for other issues.
Thanks Thomas.

Anyone who wants to help me debug, please download the test build from:

https://ci.appveyor.com/api/buildjobs/pgrocfuvrvn1f9kr/artifacts/MuseSc…

And let me know if those strings look better or not.

I did add raw debug output to that build, but have to set the QT_LOGGING_RULES prior to executing. For instance in powershell run

$env:QT_LOGGING_RULES = 'portMidi.debug=true;portAudio.debug=true'

prior to running the test build. Then when open Edit->Preferences, should see a bunch of messages containing the word "appending" and a bunch of random-looking numbers. Those numbers are the raw hex representation of the strings that windows sends through portaudio/midi to musescore...and pasting them here for me to look at would be most helpful in diagnosing the problem.

Status won't fix needs info

And I realize there is low priority to fix this, but incase there are anyone else who experiences this problem, please it would be great to provide me the debug information. So I'll change status to "needs info".