mapVertices

Similar to map transformation on RDD, mapVertices allows users to transform the vertices of graphs. The following is the signature of the mapVertices method:

mapVertices(scala.Function2<Object,VD,VD2> map, scala.reflect.ClassTag<VD2> evidence$3, scala.Predef.$eq$colon$eq<VD,VD2> eq)

To implement scala.Function2 in Java, scala.runtime.AbstractFunction2 can be used because implementing scala.Function2 in Java requires users to implement a large number of abstract methods and this generates a lot of byte code.

AbstractFunction0, AbstractFunction1, AbstractFunction2, ... were introduced to reduce the byte code generated by anonymous functions such as scala.Function0, scala.Function1, scala.Function2, ..... That is why they are kept in the scala.runtime package. We can leverage them while calling graph operations in Java. However, as these are abstract classes, we cannot implement these as Lambdas:

public class AbsFunc1 extends scala.runtime.AbstractFunction2<Object, string, string> implements Serializable { 
@Override
public string apply(Object arg0, string arg1) {
return "Vertex:"+arg1;
} }

Here the property vertex is transformed by adding Vertex: string in front of every vertex name.Notice, that AbsFunc1() implements Serializable as well which is mandatory as Spark, being a distributed processing system, requires objects to be serialized.

As described in the previous section, scala.reflect.ClassTag can be created in Java as follows:

ClassTag<string> stringTag = scala.reflect.ClassTag$.MODULE$.apply(string.class); 

scala.Predef.$eq$colon$eq is also a Scala object which can be defined in Java as follows:

$eq$colon$eq<string, string> tpEquals = scala.Predef.$eq$colon$eq$.MODULE$.tpEquals(); 

Hence, mapVertices() can be executed as follows:

Graph<string, string> mapVertices = graph.mapVertices(new AbsFunc1(), stringTag, tpEquals); 

To verify, all vertices can be printed as follows:

mapVertices.vertices().toJavaRDD().collect().forEach(System.out::println); 

mapvertices() is equivalent to executing maps on the vertices RDD in terms of the transformed output values of vertices, that is:

JavaRDD<string> map = graph.vertices().toJavaRDD().map(x->"Vertex:"+x);

However, do notice that it does not preserve graph indices or structures. It is returning JavaRDD, which needs to be used to construct the graph again. So, mapVertices() is recommended.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.15.74.25