mapValues

mapValues is applicable only for pair RDDs. As its name indicates, this transformation only operates on the values of the pair RDDs instead of operating on the whole tuple.

Acting on a pair RDD, it runs a map operation on the RDD values. This transformation is really useful if the user is only interested in transforming the values on a pair RDD.

The following is an example of mapValues transformation:

Java 7:

pairRDD.mapValues(new Function<Integer, Integer>() {
@Override
public Integer call(Integer v1) throws Exception {
return v1*3;
}
});

Java 8:

pairRDD.mapValues(v1 -> v1*3);

Another useful fact of mapValues is that it preserves the partitioning information; that is, it instantiates com.javachen.spark.examples.rdd.MapPartitionsRDD with the value of preservesPartitioning set to true, which makes sense since this transformation is only operating on values.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.147.85.181