Tutorials

Enjoy This Site? Join Our Remote R/Bioinformatics Classes

Note: These tutorials are incomplete. More complete versions are being made available for our members. Sign up for free.

Multi-core Processor, SIMD and Shared Memory

If a bioinformatician needs to run BLAST searches on 20,000 genes against a genome, conceptually the easiest way to parallelize the calculation is to split it among 100 independent computers. Each computer will do the calculation for 200 genes and the user can gain 100x gain in speed. This mode of parallelization has several disadvantages. Storing 100 computers not only takes enormous amount of space, but also extracts very high electricity and cooling costs. The computers also need to be connected together by a reasonably fast local-area network for moving large data files around.

With continued miniaturization of electronic circuits as explained by Moore’s law, it is possible to find more cost-effective solutions. Going back to our picture of computer architecture, we notice that the real speed-up in computation is done within the ALU. Therefore, if it is possible to combine 100 ALU units together, the calculation can be effectively parallelized. Developments in computing world over the last 10 years are slowly taking us to such a solution, as allowed by microprocessor fabrication technology. Multi-procesor amd multi-core are two popular solutions leading us in that direction.

In case of multi-processor solution, multiple microprocessors are combined on to the same motherboard sharing the same hard drive. In terms of configuration, each microprocessor has own cache. They share part of the RAM and can have independent RAM. The hard drive is shared.

In case of multi-core solution, same microprocessor chip has multiple ALUs. They share part of cache and each also has independent cache. The RAM and hard drive are shared.

Threads, OpenMP and Shared Memory http://www.rhinocerus.net/forum/lang-fortran/675266-parallel-processing-threads-vs-mpi.html http://computing.ornl.gov/internships/rams/archive/rams06/websites/j_rann/images/Presentation.pdf Threading is software way of allowing programmers to run programs in multiple cores or processors. The programs share a memory block, but multiple processors operate on it. Issue of cache and shared memory. Issue of memory access bottleneck. http://en.wikipedia.org/wiki/OpenMP distributed memory, distributed shared memory Each processor works on its own memory. Copying problem + more memory needed. OpenMP is an implementation of multi-threading. http://en.wikipedia.org/wiki/File:AmdahlsLaw.svg SIMD Traditional ALU design is based on single processor and does not take parallelization or existence of multiple processors. SIMD is a set of parallelized operations built into the ALU control of processors. It allows identical processing of a data set by many processors. SIMD is hardware access to multiple cores. Inefficient --> compilers break down. Used mostly for graphics. No bioinformatics application. GPUs are better. http://en.wikipedia.org/wiki/Multi-core_processor http://stackoverflow.com/questions/2166425/how-to-structure-a-c-application-to-use-a-multicore-processor Multi-core in R http://www.rforge.net/doc/packages/multicore/multicore.html http://kochanski.org/blog/?p=326 http://stackoverflow.com/questions/1961682/how-can-i-make-my-perl-script-use-multiple-cores-for-child-processes http://stackoverflow.com/questions/1002440/which-scripting-languages-support-multi-core-programming