It has been shown that scripts can be created in Notebooks in Scala, Python, or SQL, but it is also possible to use an IDE such as IntelliJ or Eclipse to develop code. By installing an SBT plugin into this development environment, it is possible to develop code for your Databricks environment. The current release of Databricks, as I write this book, is 1.3.2d. The Release Notes link, under New Features on the start page, contains a link to the IDE integration, which is https://dbc-xxxxxxx-xxxx.cloud.databricks.com/#shell/1547.

The URL will be of this form, with the section starting with dbc changed to match the URL for the Databricks cloud that you will create. I won't expand on this here, but leave it to you to investigate. In the next section, I will investigate the Databricks table data processing functionality.