Debugging Spark application on Eclipse as Scala debug

To make this happen, just configure your Eclipse to debug your Spark applications as a regular Scala code debug. To configure select Run | Debug Configuration | Scala Application as shown in the following figure:

Figure 17: Configuring Eclipse to debug Spark applications as a regular Scala code debug

Suppose we want to debug our KMeansDemo.scala and ask Eclipse (you can have similar options on InteliJ IDE) to start the execution at line 56 and set the breakpoint in line 95. To do so, run your Scala code as debugging and you should observe the following scenario on Eclipse:

Figure 18: Debugging Spark applications on Eclipse

Then, Eclipse will pause on the line you ask it to stop the execution in line 95, as shown in the following screenshot:

Figure 19: Debugging Spark applications on Eclipse (breakpoint)

In summary, to simplify the preceding example, if there is any error between line 56 and line 95, Eclipse will show where the error actually occurs. Otherwise, it will follow the normal workflow if not interrupted.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.133.138.134