Asynchronous and corrected-asynchronous finite difference solutions of PDEs on MIMD multiprocessors

A major problem in achieving significant speed-up on parallel machines is the overhead involved with synchronizing the concurrent processes. Removing the synchronization constraint has the potential of speeding up the computation, while maintaining greater computation flexibility (e.g. differences in processors speed; differences in the data input to processors). We construct asynchronous (AS) finite difference schemes for the solution of PDEs by removing the synchronization constraint. We analyze the numerical properties of these schemes. Based on the analysis, we develop corrected-asynchronous (CA) finite difference schemes which are specifically constructed for an asynchronous processing. We present asynchronous (AS) and corrected-asynchronous (CA) finite difference schemes for the multi-dimensional heat equation. Although our discussion concentrates on the Euler scheme it should serve only as a sample, as it can be extended to other schemes and other PDEs. These schemes are implemented on the shared-memory multi-user Sequent Balance machine. Numerical results for one and two dimensional problems are presented. It is shown experimentally that synchronization penalty can be about 50% of run time: in most cases, the asynchronous scheme runs twice as fast as the parallel synchronous scheme. In general, the efficiency of the parallel schemes increases with processor load, with the time-level, and with the problem dimension. The efficiency of the AS may reach 90% and over, but it provides accurate results only for steady-state values. The CA, on the other hand, is less efficient but provides more accurate results for intermediate (non steady-state) values. The results show the potential of developing asynchronous finite deference schemes for steady-state as well as non steadystate problems.