Skip to content

Commit 33b5a2d

Browse files
author
Ilya Ganelin
committed
Added code examples for java and python
1 parent 1fd59b2 commit 33b5a2d

File tree

1 file changed

+20
-0
lines changed

1 file changed

+20
-0
lines changed

docs/programming-guide.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1318,6 +1318,8 @@ of that each task's update may be applied more than once if tasks or job stages
13181318

13191319
In addition, accumulators do not maintain lineage for the operations that use them. Consequently, accumulators updates are not guaranteed to be executed when made within a lazy transformation like `map()`. Unless something has triggered the evaluation of the lazy transformation that updates the value of the accumlator, subsequent operations will not themselves trigger that evaluation and the value of the accumulator will remain unchanged. The below code fragment demonstrates this issue:
13201320

1321+
<div class="codetabs">
1322+
13211323
<div data-lang="scala" markdown="1">
13221324
{% highlight scala %}
13231325
val acc = sc.accumulator(0)
@@ -1326,6 +1328,24 @@ In addition, accumulators do not maintain lineage for the operations that use th
13261328
{% endhighlight %}
13271329
</div>
13281330

1331+
<div data-lang="java" markdown="1">
1332+
{% highlight java %}
1333+
Accumulator<Integer> accum = sc.accumulator(0);
1334+
data.map(x -> accum.add(x); f(x););
1335+
// Here, acc is still 0 because no actions have cause the `map` to be computed.
1336+
{% endhighlight %}
1337+
</div>
1338+
1339+
<div data-lang="python" markdown="1">
1340+
{% highlight python %}
1341+
accum = sc.accumulator(0)
1342+
data.map(lambda x => acc.add(x); f(x))
1343+
# Here, acc is still 0 because no actions have cause the `map` to be computed.
1344+
{% endhighlight %}
1345+
</div>
1346+
1347+
</div>
1348+
13291349
# Deploying to a Cluster
13301350

13311351
The [application submission guide](submitting-applications.html) describes how to submit applications to a cluster.

0 commit comments

Comments
 (0)