Asynchronous actions

Although most Spark actions are synchronous in nature, there are few actions that can be executed asynchronously. In certain scenarios one can execute different asynchronous actions on different RDDs simultaneously when the resources of a Spark cluster are not being utilized completely. There are countless scenarios where an asynchronous action can be utilized to its full potential as all asynchronous actions return FutureAction, which inherits its feature from the Future class of Java's concurrent package.

  • countAsync: The countAsync action returns a FutureAction having a value of the total number of elements in the RDD. The FutureAction class has a few important helper methods, like isDone() to check if the action is complete, the cancel() method attempts to cancel the executing action, and the get() method waits if necessary for the computation to complete, and then retrieves its result.
JavaFutureAction<Long> intCount = intRDD1.countAsync();
System.out.println(" The async count for "+intCount);
  • collectAsync: This is another asynchronous action that returns a future for retrieving all elements of this RDD. All the FutureAction helper methods discussed previously hold true for this action as well:
JavaFutureAction<List<Integer>> intCol = intRDD2.collectAsync();
for(Integer val:intCol.get()){
System.out.println("The collect val is "+val);
}
  • takeAsync: An asynchronous action takeAsync retrieves the first n elements of the RDD in a FutureAction:
JavaFutureAction<List<Integer>> takeAsync = intRDD.takeAsync(3);
for( Integer val:takeAsync.get()) {
System.out.println(" The async value of take is :: "+val);
}
  • foreachAsync: Applies a function to each element of the RDD asynchronously:
intRDD.foreachAsync(newVoidFunction<Integer>() {
@Override
public void call(Integer t) throws Exception {
System.out.println("The val is :"+t);
}
});

Similarly, foreachPartitionAsync action applies a function to each partition of the RDD.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.135.213.212