[JW Lua] ForEachInteger processing speed ? Very slow on high ranges

[Jari Williamsson]

Jan,

Just a followup on this. I found the bottleneck. It was in the dynamic 
memory expansion code for the FCString class. I applied the same kind of 
fixes to the __FCCollection class as well, so LoadAll() for very large 
data collections should get a performance boost as well.


Best regards,

Jari Williamsson


On 2015-12-17 16:47, Jari Williamsson wrote:
> Jan,
>
> The issue seems to be triggered by the print() command when very large
> number of lines are used. 20000+ lines is a lot for a simple text control.
>
> I can see if I can optimize it a bit. One approach could be to have a
> limit on the number of lines the console can output. Perhaps an option
> to reset the output console after nnnn number of lines?
>
>
> Best regards,
>
> Jari Williamsson
>
>
> On 2015-12-16 14:20, Jan Angermüller wrote:
>> Jari,
>>
>> I noticed that ForEachInteger is extremly slow on loops with high ranges.
>> And that there is an*incredible speed boost* on these loops, if the
>> script is run a second time.
>>
>> With the simple print-loop code snippet at very bottom, I get the
>> following results for processing time, if I run the loops with these
>> values in this order:
>>
>> endloop=10000   -> 0.214s
>> endloop=20000   -> 0.813s
>> endloop=60000   -> *14.928s*
>> endloop=160000   -> *134.258**s* (!)
>> endloop=20000   -> 0.071s (run again with the same 20000   value)
>> endloop=60000   -> 0.221s (run again with the same 60000 value)
>> endloop=160000   -> 0.566s (run again with the same 160000 value)
>> endloop=80000 -> 0.282s (although this value was not yet calculated it's
>> now done very quickly. Probably because the value 80000 is below the yet
>> highest value of 160.000)
>>
>> If you look more deeply into these numbers, it looks like for example in
>> the 160.000 loop the 60.000 elements that were calculated in the
>> previous run of the script were calculated extremly fast and only the
>> remaining new 100.000 elements took very long.
>> So if I restart Finale and start with calling the script with 160.000,
>> it even takes longer than 134 seconds.
>> When running the script again with a lower loop end value than before,
>> the calculation time is linear and very fast.
>>
>> Are the Pentium prefetching optimizations really so clever that they
>> save 99% of the processing time ?
>> I think there must be problem in the implementation as I don't know why
>> this simple print loop should take 50s on only 180.000 iterations on a
>> high speed PC. Maybe it's an iterative memory allocation problem: only
>> the first time the memory is allocated and then kept free for further
>> loop calls.
>>
>> Jan
>>
>> function DoCallback(i)
>> print("Callback"..i)
>> end
>>
>> local it=finale.FCLuaIterator()
>> it.UseProgressBar=true
>> it.AbortableProgressBar=false
>> it.ProgressUpdateFrequency=1
>> local endloop=....  --(take value from above)
>> it:ForEachInteger(1,endloop,DoCallback)
>>
>>
>> _______________________________________________
>> JWLua mailing list
>> JWLua at jwmusic.nu
>> http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu
>>
>
>
>
> _______________________________________________
> JWLua mailing list
> JWLua at jwmusic.nu
> http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu
>
>