Fluorophores and fluorescent tags

A fluorophore is a chemical compound that can reemit light at a certain wavelength. These compounds are a critical tool in biology because they allow experimentalists to image particular components of a given cell in detail. Experimentally, the fluorophore is commonly applied as a dye to a particular cell. Figure 7-9 shows the molecular structure of a common fluorophore.

Figure 7-9. DAPI (4',6-diamidino-2-phenylindole) is a common fluorescent stain that binds to adenine-thymine–rich regions of DNA. Because it passes through cell membranes, it is commonly used to stain the insides of cells. (Source: Wikimedia.)

Fluorescent tagging is a technique for attaching a fluorophore to a biomolecule of interest in the body. There are a variety of techniques to do this effectively. It’s useful in microscopy imaging, where it’s common to want to highlight a particular part of the image. Fluorescent tagging can enable this very effectively.

Fluorescent microscopy has proven a tremendous boon for biological research because it permits researchers to zoom in on specific subsystems in a given biological sample, as opposed to dealing with the entirety of the sample. When studying individual cells, or individual molecules within a cell, the use of tagging can prove invaluable for focusing attention on interesting subsystems. Figure 7-10 shows how a fluorescent stain can be used to selectively visualize particular chromosomes within a human cell nucleus.

Figure 7-10. An image of a human lymphocyte nucleus with chromosomes 13 and 21 stained with DAPI (a popular fluorescent stain) to emit light. (Source: Wikimedia.)

Fluorescence microscopy can be a very precise tool, used to track events like single binding events of molecules. For example, binding events of proteins to ligands (as discussed in Chapter 5) can be detected by a fluorescence assay.

The Mesosome: An Imaginary Organelle

The process of fixing a cell for electron microscopy introduces a crucial artifact, the mesosome in gram-positive bacteria (see Figure 7-11). Degradations in the cell wall, caused by the process of preparing the sample for the electron microscope, were originally thought to be natural structures instead of artifacts.

Be warned that similar artifacts likely exist in your own samples. In addition, it’s entirely possible that a deep network could train itself to detect such artifacts rather than training itself to find real biology.

Figure 7-11. Mesosomes are artifacts introduced by preparation for electron microscopy that were once believed to be real structures in cells. (Adapted from Wikimedia.)

Implementing cell counting in DeepChem

This section walks through the construction of a deep learning model for cell counting using DeepChem. We’ll start by loading and featurizing a cell counting dataset. We use the Broad Bioimage Benchmark Collection (BBBC) to get access to a useful microscopy dataset for this purpose.

Before you can load the dataset into DeepChem, you’ll first need to download it locally. The BBBC005 dataset that we’ll use for this task is reasonably large (a little under 2 GB), so make sure your development machine has sufficient space available:

wget https://data.broadinstitute.org/bbbc/BBBC005/BBBC005_v1_images.zip
unzip BBBC005_v1_images.zip

With the dataset downloaded to your local machine, you can now load this dataset into DeepChem by using ImageLoader:

image_dir = 'BBBC005_v1_images'
files = []
labels = []
for f in os.listdir(image_dir):
 if f.endswith('.TIF'):
  files.append(os.path.join(image_dir, f))
  labels.append(int(re.findall('_C(.*?)_', f)[0]))
loader = dc.data.ImageLoader()
dataset = loader.featurize(files, np.array(labels))

This code walks through the downloaded directory of images and pulls out the image files. The labels are encoded in the filenames themselves, so we use a simple regular expression to extract the number of cells in each image. We use ImageLoader to transform this into a DeepChem dataset.

Let’s now split this dataset into training, validation, and test sets:

splitter = dc.splits.RandomSplitter()
train_dataset, valid_dataset, test_dataset = splitter.train_valid_test_split(
	  dataset, seed=123)

With this split in place, we can now define the model itself. In this case, let’s make a simple convolutional architecture with a fully connected layer at the end:

learning_rate = dc.models.tensorgraph.optimizers.ExponentialDecay(0.001, 0.9,
                                                                 250)
model = dc.models.TensorGraph(learning_rate=learning_rate, model_dir='model')
features = layers.Feature(shape=(None, 520, 696))
labels = layers.Label(shape=(None,))
prev_layer = features
for num_outputs in [16, 32, 64, 128, 256]:
 prev_layer = layers.Conv2D(num_outputs, kernel_size=5, stride=2, 
 							in_layers=prev_layer)
output = layers.Dense(1, in_layers=layers.Flatten(prev_layer))
model.add_output(output)
loss = layers.ReduceSum(layers.L2Loss(in_layers=(output, labels)))
model.set_loss(loss)

Note that we use L2Loss to train our model as a regression task. Even though cell counts are whole numbers, we don’t have a natural upper bound on the number of cells in an image.

Training this model will take some computational effort (more on this momentarily), so to start, we recommend using our pretrained model for basic experimentation. This model can be used to make predictions out of the box. There are directions on downloading the pretrained model in the code repository associated with the book. Once you’ve downloaded it, you can load the pretrained weights into the model with:

model.restore()

Let’s take this pretrained model out for a whirl. First, we’ll compute the average prediction error on our test set for our cell counting task:

y_pred = model.predict(test_dataset).flatten()                           
print(np.sqrt(np.mean((y_pred-test_dataset.y)**2)))

What accuracy do you get when you try running the model?

Now, how can you train this model for yourself? You can fit the model by training it for 50 epochs on the dataset:

model.fit(train_dataset, nb_epoch=50)

This learning task will take some amount of computing horsepower. On a good GPU, it should complete within an hour or so. It may not be feasible to easily train the model on a CPU system.

Once trained, test the accuracy of the model on the validation and test sets. Does it match that of the pretrained model?

Implementing cell segmentation in DeepChem

In this section, we will train a cellular segmentation model on the same BBBC005 dataset that we used previously for the cell counting task. There’s a crucial subtlety here, though. In the cell counting challenge, each training image has a simple count as a label. However, in the cellular segmentation task, each label is itself an image. This means that a cellular segmentation model is actually a form of “image transformer” rather than a simple classification or regression model. Let’s start by obtaining this dataset. We have to retrieve the segmentation masks from the BBBC website, using the following commands:

wget https://data.broadinstitute.org/bbbc/BBBC005/BBBC005_v1_ground_truth.zip
unzip BBBC005_v1_ground_truth.zip

The ground-truth data is something like 10 MB, so it should be easier to download than the full BBBC005 dataset. Now let’s load this dataset into DeepChem. Luckily for us, ImageLoader is set up to handle image segmentation datasets without much extra hassle:

image_dir = 'BBBC005_v1_images'
label_dir = 'BBBC005_v1_ground_truth'
rows = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L',
		'M', 'N', 'O', 'P')
blurs = (1, 4, 7, 10, 14, 17, 20, 23, 26, 29, 32, 35, 39, 42, 45, 48)
files = []
labels = []
for f in os.listdir(label_dir):
 if f.endswith('.TIF'):
  for row, blur in zip(rows, blurs):
   fname = f.replace('_F1', '_F%d'%blur).replace('_A', '_%s'%row)
   files.append(os.path.join(image_dir, fname))
   labels.append(os.path.join(label_dir, f))
loader = dc.data.ImageLoader()
dataset = loader.featurize(files, labels)

Now that we have our datasets loaded and processed, let’s hop into building some deep learning models for them. As before, we’ll split this dataset into training, validation, and test sets:

splitter = dc.splits.RandomSplitter()
train_dataset, valid_dataset, test_dataset = splitter.train_valid_test_split(
	  dataset, seed=123)

What architecture can we use for the task of image segmentation? It’s not just a matter of using a straightforward convolutional architecture since our output needs to itself be an image (the segmentation mask). Luckily for us, there has been some past work on suitable architectures for this task. The U-Net architecture uses a stacked series of convolutions to progressively “downsample” and then “upsample” the source image, as illustrated in Figure 7-14. This architecture does well at the task of image segmentation.

Figure 7-14. A representation of the U-Net architecture for biomedical image segmentation. (Adapted from the University of Freiburg.)

Let’s now implement the U-Net in DeepChem:

learning_rate = dc.models.tensorgraph.optimizers.ExponentialDecay(0.01, 0.9, 250)
model = dc.models.TensorGraph(learning_rate=learning_rate, 
							  model_dir='segmentation')
features = layers.Feature(shape=(None, 520, 696, 1)) / 255.0
labels = layers.Label(shape=(None, 520, 696, 1)) / 255.0
# Downsample three times.
conv1 = layers.Conv2D(16, kernel_size=5, stride=2, in_layers=features)
conv2 = layers.Conv2D(32, kernel_size=5, stride=2, in_layers=conv1)
conv3 = layers.Conv2D(64, kernel_size=5, stride=2, in_layers=conv2)
# Do a 1x1 convolution.
conv4 = layers.Conv2D(64, kernel_size=1, stride=1, in_layers=conv3)
# Upsample three times.
concat1 = layers.Concat(in_layers=[conv3, conv4], axis=3)
deconv1 = layers.Conv2DTranspose(32, kernel_size=5, stride=2, in_layers=concat1)
concat2 = layers.Concat(in_layers=[conv2, deconv1], axis=3)
deconv2 = layers.Conv2DTranspose(16, kernel_size=5, stride=2, in_layers=concat2)
concat3 = layers.Concat(in_layers=[conv1, deconv2], axis=3)
deconv3 = layers.Conv2DTranspose(1, kernel_size=5, stride=2, in_layers=concat3)
# Compute the final output.
concat4 = layers.Concat(in_layers=[features, deconv3], axis=3)
logits = layers.Conv2D(1, kernel_size=5, stride=1, activation_fn=None, 
					   in_layers=concat4)
output = layers.Sigmoid(logits)
model.add_output(output)
loss = layers.ReduceSum(layers.SigmoidCrossEntropy(in_layers=(labels, logits)))
model.set_loss(loss)

This architecture is somewhat more complex than that for cell counting, but we use the same basic code structure and stack convolutional layers to achieve our desired architecture. As before, let’s use a pretrained model to give this architecture a try. Directions for downloading the pretrained model are available in the book’s code repository. Once you’ve got the pretrained weights in place, you can load the weights as before:

model.restore()

Let’s use this model to create some masks. Calling model.predict_on_batch() allows us to predict the output mask for a batch of inputs. We can check the accuracy of our predictions by comparing our masks against the ground-truth masks and checking the overlap fraction:

scores = []
for x, y, w, id in test_dataset.itersamples():
 y_pred = model.predict_on_batch([x]).squeeze()
 scores.append(np.mean((y>0) == (y_pred>0.5)))
print(np.mean(scores))

This should return approximately 0.9899. This means nearly 99% of pixels are correctly predicted! It’s a neat result, but we should emphasize that this is a toy learning task. A simple image processing algorithm with a brightness threshold could likely do almost as well. Still, the principles exposed here should carry over to more complex image datasets.

OK, now that we’ve explored with the pretrained model, let’s train a U-Net from scratch for 50 epochs and see what results we obtain:

model.fit(train_dataset, nb_epoch=50, checkpoint_interval=100)

As before, this training is computationally intensive and will take a couple of hours on a good GPU. It may not be feasible to train this model on a CPU system. Once the model is trained, try running the results for yourself and seeing what you get. Can you match the accuracy of the pretrained model?