Ensuring worker processes have access to code and data 

When developing parallel computation, a beginner often runs into the following issues:

  • Functions not defined in the worker processes: This can be a symptom of a library package not being loaded, or a function that was only defined in the current process but not defined in the worker processes. Both issues can be resolved by using the @everywhere macro as shown in the preceding examples.
  • Data not available in the worker processes: This can be a symptom of the data being stored as a variable in the current processes but not passed to the worker processes. SharedArray is convenient because it is automatically made available to worker processes. For other cases, the programmer generally has two options:
    • Explicitly pass the data via function arguments.
    • If the data is in a global variable, then it can be transferred using the @everywhere macro, as follows:
@everywhere my_global_var = whatever_value

For more advanced use cases, the ParallelDataTransfer.jl package provides several helpful functions to facilitate data transfer among the master process and worker processes.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.218.127.141