Revisiting the graph format

Next, we have the GraphBuilder method, which is our own component:

    //when
val graph = GraphBuilder.loadFromFile(spark, path)

The following is our GraphBuilder.scala file for our GraphBuilder method:

package com.tomekl007.chapter_7

import org.apache.spark.SparkContext
import org.apache.spark.graphx.{Graph, GraphLoader}

object GraphBuilder {

def loadFromFile(sc: SparkContext, path: String): Graph[Int, Int] = {
GraphLoader.edgeListFile(sc, path)
}
}

It uses a GraphLoader class from the org.apache.spark.graphx.{Graph, GraphLoader} package and we are specifying the format.

The format that's specified here is edgeListFile. We are passing the sc parameter, which is the SparkContext and path parameter, which contains the path of where the file is placed. The resulting graph will be Graph [Int, Int], which we will use as the identifier of our vertices.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.89.2