[JW Lua] FCString:CreateEnigmaStrings doesn't work on long unicode strings, but on short ones

[Jari Williamsson]

Jan,

The Unicode issues you're experiencing are related to Lua's text 
handling, which is basically 8-bit internally.

You'll see a similar issue in this standard Lua code:
---
s = "töst"
print (string.len(s))
---
Result is 5, since "ö" occupies 2 characters in UTF-8.


I'd suggest that you use the FCString methods as much as possible when 
dealing with character manipulation in the Unicode range. For example:
"str:AppendCharacter(0xe0f3)" to add the character U+E0F3 to the string, 
or "str:GetLength()" to get the length of the string.

This advice will stay true even when JW Lua starts using Lua 5.3.


Best regards,

Jari Williamsson


On 2015-05-29 22:14, Jan Angermüller wrote:
> Jari,
>
> I noticed a weird problem with CreateEnigmaStrings and unicode fonts.
> It occured at a forte-fortissimo created from three "f"s with the
> unicode character U+E050.
>
> When calling FCString:CreateEnigmaStrings(true) the non-enigma commands
> were removed (!), when the string contained*3**or more* "f"s in a row.
> But they were split correctly when it were only *1 or 2* "f"s.
> Below is the code (Bravura is used just for demonstration purposes,it
> holds the violin clef on U+E050, not an "f"). The first two splits work
> fine, while the next two runs return a length of zero and an empty string.
> And of course, it also works fine for the non-unicode range (fifth run).
> The same bug occurs if you change the unicode value to any other
> character from the widespread U+EXXX range.
>
> Any ideas ?
>
> All the best,
> Jan
>
> function SplitEnigma(text)
>      --Create enigma strings and print all non-enigma commands (i.e. all
> texts)
>      print("String to split:",text.LuaString)
>      local splittext=text:CreateEnigmaStrings(true)
>      for i in each(splittext) do
>          if not i:IsEnigmaCommand() then
> print("Len:",string.len(i.LuaString),"String:",i.LuaString)
>          end
>      end
> end
>
> function UnicodeToUTF(number) --unicode to UTF conversion for numbers
> between 2048 and 65535
>      return string.char(bit32.bor(224,bit32.rshift(number,12))) ..
> string.char(bit32.bor(128,bit32.band(bit32.rshift(number,6),63)))..
> string.char(bit32.bor(128,bit32.band(number,63)))
> end
>
> local text=finale.FCString()
> text.LuaString="^fontMus(Bravura,8192)^size(24)^nfx(0)"..UnicodeToUTF(57424)
> SplitEnigma(text)
> text.LuaString="^fontMus(Bravura,8192)^size(24)^nfx(0)"..UnicodeToUTF(57424)..UnicodeToUTF(57424)
> SplitEnigma(text)
> text.LuaString="^fontMus(Bravura,8192)^size(24)^nfx(0)"..UnicodeToUTF(57424)..UnicodeToUTF(57424)..UnicodeToUTF(57424)
> SplitEnigma(text)
> text.LuaString="^fontMus(Bravura,8192)^size(24)^nfx(0)"..UnicodeToUTF(57424)..UnicodeToUTF(57424)..UnicodeToUTF(57424)..UnicodeToUTF(57424)
> SplitEnigma(text)
> text.LuaString="^fontMus(Bravura,8192)^size(24)^nfx(0)"..string.char(123)..string.char(123)..string.char(123)..string.char(123)
> SplitEnigma(text)
>
>
> _______________________________________________
> JWLua mailing list
> JWLua at jwmusic.nu
> http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu
>