Security Level:

11 / 10 / 2017

Active objects for BSP (Work In Progress) Parallel And Distributed Algorithms Lab

Pierre Leca

SCALE team

Context 1st year industrial PhD as French CIFRE contract between Huawei and I3S Company supervisors : Gaetan Hains and Wijnand Suijlen University supervisors : Ludovic Henrio and Eric Madelaine Early work on mixing BSP with active objects

2

Content  Active objects  Bulk Synchronous Parallel  BSP active objects  Future work ideas

3

Content  Active objects  Bulk Synchronous Parallel  BSP active objects  Future work ideas

4

Active object A

Active objects

foo

foo()

bar

bar(1)

Active object B

fut = B.bar(1) result

fut.get()

result

Memory Represent asynchronous entities living in their own thread Object function call syntax for sending requests Result represented as future returned immediately Blocking access to future only when required (wait by necessity) 5

Active object A

Active objects

bar

bar(1)

Active object B

fut = B.bar(1) result

SOA vision

foo

foo()

fut.get()

result

Memory

Suited for task-parallel algorithms One request served at a time Partially deterministic execution

6

Content  Active objects  Bulk Synchronous Parallel  BSP active objects  Future work ideas

7

Superstep

Bulk Synchronous Parallel

Comp 0 Comp 1

Parallel execution model A program is a sequence of supersteps

Comp 2 Comp 3

Computation → communication → synchronization Synchronization ensure alignment of supersteps

8

Superstep

Bulk Synchronous Parallel

Comp 0 Comp 1

HPC vision Deterministic execution

Comp 2 Comp 3

Easy to avoid deadlocks Has a simple cost model Suited for balanced data-parallel algorithms 9

Content  Active objects  Bulk Synchronous Parallel  BSP active objects  Future work ideas

10

BSP active objects : coordination Active object model is a good match for task-parallelism BSP is a good match for data-parallelism Both are not adequate for the other kind of parallelism Both have interesting properties Our idea is to combine them into a single execution model

11

BSP active objects : execution model Multiple processes per active object Requests handled sequentially by one head process The head process can use other processes in BSP mode BSP mode can be used for processing data-parallel parts

12

BSP active objects : execution model Active object A

Process A.1

Process A.0 (1)

(4)

(2)

(4)

foo

foo()

bar

foo

bar

(3)

bsp_run(bar) BSP communications result

(5)

Memory

Memory 13

BSP active objects : example vector foo (const vector & v, int d) { int blocksize = v.size() / bsp_nprocs(); for (int i = 0; { bsp_send(i, bsp_send(i, bsp_send(i, }

void bar() { bsp_sync(); int blocksize, d; bsp_recv(0, &blocksize, sizeof(int));

i < bsp_nprocs(); ++i) &blocksize, sizeof(int)); &v[i * blocksize], blocksize * sizeof(int)); &d, sizeof(int));

int v[blocksize]; bsp_recv(0, v, blocksize * sizeof(int)); bsp_recv(0, &d, sizeof(int));

bsp_run(bar);

for (int i = 0; i < blocksize; ++i) { v[i] += d; }

vector res(v.size()); for (int i = 0; i < bsp_nprocs(); ++i) { bsp_recv(i, &res[i * blocksize], blocksize * sizeof(int)); } return res;

bsp_send(0, v, blocksize * sizeof(int)); bsp_sync(); }

}

14

BSP active objects : pros Programming sequentially by default is easy Possible to express multi-parallelism algorithms Task and Data parallel with deterministic properties Implemented algorithms are easy to reuse

15

BSP active objects : cons Active objects always communicate results Distributed data are always aggregated to a single process May be too complex for straightforward data parallelism

16

BSP active objects : implementation choices Combine into a single language : C++ Build on top of MPI Active objects use existing processes (vs dynamic creation)

17

Content  Active objects  Bulk Synchronous Parallel  BSP active objects  Future work ideas

18

Future work ideas Combine distributed vectors and lazy future update strategy Generic communication primitives between subsets Process sharing

19

Future work ideas Formalization Study of cost model

20

Thank You. Copyright©2016 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.