<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Jari,<br>
<br>
yes, I know the parallel processing concepts very well and have <br>
been working in that area for some time:<br>
I have been doing research in digital image processing for a <br>
few years and it was also the topic of my computer science diploma
thesis.<br>
Image processing is a topic where parallel processing is very
common.<br>
<br>
It's more or less the same concept that I was thinking of in
Finale:<br>
in digital image processing you very often process line per line <br>
in parallel processes. So if you have 1920 pixels per line (in
HDTV), <br>
each process has to repeat at least 1920 function calls, <br>
until it gets the next available line from a queue. <br>
So the overhead is minimized. <br>
Processing individual pixels in parallel processes is usually
avoided,<br>
as there would be too much overhead.<br>
<br>
In Finale you can view the score as an image with each entry per
measure<br>
being a pixel, or all entries in all measures on one staff being
an image line.<br>
All staves are independent as all lines in an image are
independent.<br>
Dependencies should more or less only be within its entries.<br>
<br>
For example voice 2 depends on voice 1 or a note is tied to the
next note.<br>
The same in image processing where you usually take pixels from
the nearest <br>
neighboring pixels for calculating the new pixel.<br>
<br>
There will definitely be no boost, if the selected area in Finale
is only <br>
a few measures or just one staff. But if you are processing an<br>
orchestra or big band score with many measures and many staves,<br>
this should give an incredible boost if you have 6 cores <br>
(i.e. 12 hyperthreading cores) and a script that runs over the
whole score <br>
and is very CPU-intensive.<br>
<br>
For example my "fix all layout issues" script which is a
combination<br>
of about 20 plugins takes about 10-30 seconds per staff for a 120
measures piece,<br>
depending on the complexity of the staves.<br>
This is quite some time for 40 staves - although a 10-minute
coffee <br>
break is also an alternative ;-)<br>
<br>
Making this script parallel is not so much effort:<br>
usually every functionality is a for loop on the region, <br>
followed by the core function.<br>
To have it parallel you would need to separate <br>
the region into staves and measures. Then parallelize the staff
loop,<br>
and have the measure loop and all core functions together.<br>
<br>
You mention the "buffered processes" as an area for parallelizing
- <br>
that's something similar that I already use in JW Lua:<br>
I load and store all required layout and metrics values in a local
array <br>
at the beginning of the plug-in, then run all "core" functions, <br>
and only save the local array to Finale at the very end of the
plugin -<br>
and not within each core function. <br>
So I know exactly about the data dependencies.<br>
<br>
I had many metrics issues when I was relying on the "live" values<br>
that Finale offered me. This array also saved some time:<br>
1.) only one save() call at the end of the plugin, instead of 20
saves <br>
(although I have no idea, how much overhead the Save() call has).<br>
2.) calculations like for example CalcStemUp() were only done<br>
once and then read from the array in each core function.<br>
(BTW CalcStemUp() also seems very critical with metrics
information -<br>
calculating it only once at the beginning saved me from that
trouble).<br>
<br>
I don't know about the storing concepts within Finale and the
Finale PDK,<br>
so I don't know if there are more bottlenecks or required
semaphores than I would expect.<br>
But I guess most layout and plugin functions are only staff or
even measure or entry related <br>
(i.e. in a more local context) and not in a global context.<br>
<br>
This is certainly a low priority feature,<br>
but it's definitely a nice-to-have for releasing CPU-intensive
scripts.<br>
<br>
Jan<br>
<br>
<br>
Am 03.09.2014 09:54, schrieb Jari Williamsson:<br>
</div>
<blockquote cite="mid:5406C942.4050801@mailbox.swipnet.se"
type="cite">Jan,
<br>
<br>
Processing data in multiple threads will not boost the CPU or
anything like that. However, if CPU cores are idle, calculations
can sometimes be processed faster with multiple threads running on
different cores.
<br>
<br>
But in the case of Finale, where the resource (the Finale
document) is in a single pipe, it would be very difficult to get
any speed boost since a semaphore in one thread could easily block
the other threads. Or it could also result in threads overwriting
data that other threads changed.
<br>
<br>
Multi-threading works well for separate buffered processes, like
downloading, playback, printing, screen painting, timers, etc, but
in my experience data processing tends to get slower and much more
complicated when distributed through multiple threads.
<br>
<br>
I'll investigate if I can get Finale's progress bar feature to
work with the JW Lua iterators.
<br>
<br>
<br>
Best regards,
<br>
<br>
Jari Williamsson
<br>
<br>
<br>
On 2014-08-13 15:32, Jan Angermüller wrote:
<br>
<blockquote type="cite">Jari,
<br>
<br>
just a nice-to-have-idea: would it be possible to speed up JW
Lua with
<br>
parallel processing ? I already suggested it to MakeMusic some
time ago
<br>
for some of their slow plugins or functions on huge orchestra
scores.
<br>
<br>
Most (not all) JW Lua scripts work staff-independent,
<br>
so making for ... loops on staves work parallel should
theoretically
<br>
be possible for many plugin tasks.
<br>
E.g. processing 30 staves with 60 measures in 12 parallel
processes
<br>
would speed up the plugin tremendously.
<br>
<br>
Jan
<br>
<br>
<br>
_______________________________________________
<br>
JWLua mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:JWLua@jwmusic.nu">JWLua@jwmusic.nu</a>
<br>
<a class="moz-txt-link-freetext" href="http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu">http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu</a>
<br>
<br>
</blockquote>
<br>
<br>
<br>
_______________________________________________
<br>
JWLua mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:JWLua@jwmusic.nu">JWLua@jwmusic.nu</a>
<br>
<a class="moz-txt-link-freetext" href="http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu">http://jwmusic.nu/mailman/listinfo/jwlua_jwmusic.nu</a>
<br>
<br>
</blockquote>
<br>
</body>
</html>