The data for this project consists of three types: system-based medical history, automated dialog content, and physician observations. This is illustrated in the following diagram:

The system-based patient medical history columns within our data include common data points, such as sex, age, height and weight, medical conditions, past treatments, allergies, and so on (we will see more detail on this in a later section of this chapter, as we begin working with the data). The new information, the data that we are most interested in, is the content of the patient dialog and physician observations.

This information has been collected using a combination of voice-to-text technology, as well as information manually entered by the physician during the dialog session with the patient and perhaps after the dialog is completed.

The combination of these two nicely complementing datacapture methods provides us with a flexible structure to capture the information for later use without allowing the conversation to stray too far from the point, while also not forcing a rigid dialog, since it is imperative that the patient have the freedom of (almost) complete expression.

As we will see, the dialog content will include the patient's view of things, such as the symptoms that they feel they are experiencing, their level of discomfort, their description of their lifestyle—including their admission of whether or not they use alcohol or drugs—personal family health history, and even what they feel should be the prognoses and/or treatment.

As we've already mentioned earlier in this chapter, oftentimes, the ability to review and understand the data using IBM Watson Analytics requires various preprocessing activities. At some point, the ability to do all of that preprocessing in Watson Analytics may be possible, but at the time of writing, it's best to do this outside of Watson.

We'll start this Watson Analytics project with the assumption that minimal preprocessing has already taken place, and we are ready to load the data into Watson.

However, we will soon see that, although the data is somewhat preprocessed/preformatted for us, there will be additional refinement required once the data is loaded.