Parallel calculations: Difference between revisions

Takes the list of numbers and converts them to a <tt>HyperSeq</tt> that is stored in a raw variable. Anyand evaluated concurrently. <tt>HyperSeq</tt>s overloadsoverload <tt>map</tt> and <tt>grep</tt> to convert and pick values in worker threads. The runtime will pick the number of OS-level threads and assign worker threads to them while avoiding stalling in any part of the program. A <tt>HyperSeq</tt> is lazy, so the computation of values will happen in chunks as they are requested.

The hyper (and race) method can take two parameters that will tweak how the parallelization occurs: :degree and :batch. :degree is the number of worker threads to allocate to the job. By default it is set to the number of physical cores available. If you have a hyper threading processor, and the tasks are not cpu bound, it may be useful to raise that number but it is a reasonable default. :degreebatch is how many sub-tasks are parcelledparceled out at a time to each worker thread. Default is 64. For small numbers of cpu intensive tasks a lower number will likely be better, but too low may make the dispatch overhead cancel out the benefit of threading. Conversely, too high will over-burden some threads and starve others. Over long-running processes of multi hundreds/thousands of sub-tasks, the scheduler will automatically adjust the batch size up or down to try to keep the pipeline filled. For small batch sizes of cpu intensive tasks (such as this one) it is useful to give it a smaller starting batch size.