Index

[SYMBOL][A][B][C][D][E][F][G][H][I][J][K][L][M][N][O][P][R][S][T][U][V][W][Y][Z]

SYMBOL

* function

A

absolute value
academic papers
accuracy
activation functions
  adding to layer
  computable
  continuous and infinite in domain
  defined
  hidden-layer
  installation instructions
  monotonic
  nonlinear
  output layer
  similar inputs
  slope
  softmax computation
  upgrading MNIST network
activation input parameter
actual error
__add__ function
addition backpropagation
additional functions, adding support for
algorithms
alpha
Anaconda framework
AND operator

arbitrary length
  backpropagation with
  challenge of
  forward propagation with
  weight update with

architecture of neural networks
  importance of visualization tools
  language
  overview of
artificial neural networks
attenuation

autograd (automatic gradient computation)
  adding cross entropy to
  adding indexing to
  general discussion
  implementing LSTM with
  to train neural network
  upgrading to support multiuse tensors
  used multiple times
automatic optimization.
    See also autograd (automatic gradient computation).
  adding
  adding support
  for additional functions
  for negation
  addition backpropagation
  deep learning framework, 2^nd
  dynamic computation graph
  layers
  adding support for layer types
  containing layers
  cross-entropy layer
  embedding, 2^nd
  in Keras or PyTorch.
  loss-function layers
  nonlinearity layers
  recurrent neural network (RNN) layer
  tensors
  defined
  that are used multiple times
averaged word vectors, RNN

B

Babi dataset

backpropagation
  addition
  in code
  iteration of
  recurrent neural network (RNN), 2^nd
  truncated
  disadvantages of
  iterating with
  weighted average delta
  weighted average error
.backward( ) method, 2^nd, 3^rd, 4^th, 5^th
bash commands
batch gradient descent, 2^nd
batch_loss
batch_size, 2^nd
batch_size/alpha pair
Bernoulli distribution
blogging, to teach deep learning
Bongo Java
bptt variable, 2^nd, 3^rd

C

calculus
cell-update vector
character language modeling.
    See also LSTM (long short-term memory).
ciphertext
cluster labels

comparing
  mean squared error, 2^nd
  measuring error
computable activation functions
computation graphs, 2^nd
concatenation
conditional (sometimes) correlation, 2^nd
continuous functions
convolutional neural networks
  convolutional layer
  implementation in NumPy
  reusing weights in multiple places
corners.
    See convolutional neural networks.

correlation
  coefficients
  creating
  indirect
  input/output
  learning
  negative
  searching for
  selective
correlation summarization
counting-based learning.
    See nonparametric learning.
creation_op add
creators attribute
cross communication, 2^nd
cross-entropy class, 2^nd, 3^rd, 4^th
cross-entropy layer
cryptography
curves

D

data, grouping
data patterns
datapoints, 2^nd

datasets
  Babi
  clustering into groups
  IMDB movie reviews
  learning whole
  MNIST, 2^nd, 3^rd
  preparing data
  streetlight problem
  transforming
debugging frameworks
decoder weight matrix

deep learning
  adapted for beginning learners
  analogies and
  defined
  difficulty level for learning
  ongoing education
  overview
  project-based teaching method
  reasons for
  incremental automation of intelligence
  potential for automation of skilled labor
  stimulates intelligence and creativity
  requirements for
  high school–level mathematics
  Jupyter Notebook
  NumPy Python library
  personal challenge to solve
  Python knowledge
  subfield of machine learning
  teaching
  textbook for
  to understand frameworks and code libraries
deep learning framework, 2^nd.
    See also automatic optimization.
deep neural network
  backpropagation
  in code
  iteration of
  weighted average delta
  weighted average error
  batch gradient descent
  building
  correlation
  creating
  indirect
  learning
  full gradient descent
  importance of
  learning whole dataset
  linear versus nonlinear networks
  making predictions
  matrices and matrix relationship
  importing matrices into Python
  patterns
  streetlight problem
  overfitting
  preparing data
  running program
  sometimes correlation, 2^nd
  stacking
  stochastic gradient descent, 2^nd
  weight pressure
  conflicting pressure
  weight update
defeat_delta variable
delta, multiplying by slope
delta variable, 2^nd, 3^rd, 4^th, 5^th
deniability, 2^nd

derivatives
  calculating
  defined
  example
  relationship between weight and error and, 2^nd
  using to learn
  weight_delta
diagonal
diff variable
direct imitation
direction_and_amount variable
discorrelation
divergence, 2^nd
dot products, 2^nd, 3^rd, 4^th
  neural prediction
  visualizing
double negative
down pressure

dropout technique, regulation
  in code
  evaluated on MNIST
  general discussion
dynamic computation graph
DyNet framework

E

early stopping, 2^nd
edges.
    See convolutional neural networks.
ele_mul function, 2^nd
elementwise addition, 2^nd
elementwise multiplication, 2^nd
elementwise operation
embedding layers, 2^nd, 3^rd
encryption
error curve

errors
  error attribution
  mean squared error, 2^nd
  measuring
  positive
  reducing
Euclidian distance
execution and output analysis, RNN
expand function, adding support for

exploding gradient problem
  countering with LSTM
  general discussion
  toy example

F

federated learning
  general discussion
  hacking into
  homomorphically encrypted
  overview of, 2^nd
fill-in-the-blank task, neural networks, 2^nd
for loop, 2^nd, 3^rd, 4^th
forward( ) method, 2^nd
forward propagation
  finishing with .index_select( ) method.
  importance of visualization tools
  linking variables
  with multiple inputs
  how it works
  runnable code
  weighted sum
  with multiple outputs, 2^nd
  predicting with multiple inputs
  using single input
  neural networks
  defined
  purpose of
  simple
  stacking
  NumPy Python library
  overview of, 2^nd, 3^rd, 4^th, 5^th
  predicting on predictions
  prediction
  recurrent neural network, 2^nd
  side-by-side visualization
frameworks, debugging
full gradient descent
functions

G

gates
generalization, regulation
get_parameters( ) method
global correlation summarization
goal_pred variable
goal_prediction variable
GPUs
gradient descent.
    See also neural learning.
  breaking
  general discussion
  iteration of
  with multiple inputs
  freezing one weight, 2^nd
  single-weight neural network versus
  turning single delta into three weight_delta values
  with multiple inputs and outputs
  gradient descent generalizes to arbitrarily large networks
  neural networks making multiple predictions using single input
  with multiple outputs
  visualizing dot products (weighted sums)
  weights
graphs, 2^nd
grouping data, 2^nd, 3^rd
Gryffindor

H

hidden variable
hidden-layer activation functions
hidden-to-hidden layer
hidden-to-output layer
high-low pattern
homomorphic encryption, 2^nd

hot and cold learning
  characteristics of
  example
  general discussion

I

ICML (International Conference on Machine Learning)
identity matrices, 2^nd, 3^rd
identity vectors
image classification
images matrix
IMDB movie reviews dataset
imitation
inceptionism
.index_select( ) method
indices array, 2^nd
indirect correlation
indirect imitation
infinite functions
infinite parameters
input -> goal_prediction pairs, 2^nd
input data pattern
input datapoints
input datasets, 2^nd
input layers
input node
input values, 2^nd
input variable, 2^nd
input vector, 2^nd, 3^rd
input/output correlation
inputs
  gradient descent with
  freezing one weight
  general discussion
  generalizes to arbitrarily large networks
  neural networks making multiple predictions using only single input
  single-weight neural network vs.
  turning single delta (on node) into three weight_delta values
  how it works
  overview of
  runnable code
  weighted sums
input-to-hidden layer
installation instructions, activation functions
intelligence targeting
intermediate dataset
intermediate predictions
International Conference on Machine Learning (ICML)

J

Jupyter Notebook

K

Keras framework, 2^nd, 3^rd
kernel_output
kernels
knob_weight variable, 2^nd

L

labels, 2^nd, 3^rd, 4^th, 5^th
language, neural networks understanding
  embedding layer
  fill-in-the-blank task
  general discussion
  IMDB movie reviews dataset
  interpreting output
  loss function
  meaning of neuron
  natural language processing
  neural architecture
  predicting movie reviews
  supervised natural language processing
  word analogies
  word correlation
  word embeddings
Lasagne framework
Layer class
layer_0_delta variable
layer_1_delta variable, 2^nd
layer_2_delta variable

layers
  adding activation functions to
  adding support for layer types
  containing layers
  cross-entropy layer
  dimensionality of matrices and
  embedding, 2^nd
  embedding layer translates indices into activations
  in Keras or PyTorch
  loss-function layers
  nonlinearity layers
linear neural networks
linear nodes
list objects
local correlation summarization
log function
logical analysis
logical AND
long short-term memory.
    See LSTM (long short-term memory).
loss function, 2^nd, 3^rd, 4^th
loss.backward( ) function
loss-function layers
lossless representation
lower weights
LSTM (long short-term memory)
  character language model
  training
  tuning
  upgrading
  countering vanishing and exploding gradients with
  gates
  using autograd system to implement

M

machine learning
make_sent_vect function
matrices and matrix relationship
  importing matrices into Python
  layers and
  patterns
  streetlight problem
matrix multiplication, adding support for
max pooling
mean pooling
mean squared error, 2^nd, 3^rd, 4^th, 5^th, 6^th, 7^th
measuring error
memorization, regulation
memorizing neural network code
mini-layers
missing values

MNIST (Modified National Institute of Standards and Technology) dataset
  overview of
  three-layer network on
  upgrading
MNIST digit classifier
MNISTPreprocessor notebook
monotonic activation functions
multi-input gradient descent
multiple inputs
  gradient descent with
  freezing one weight
  general discussion
  generalizes to arbitrarily large networks
  neural networks making multiple predictions using only single input
  single-weight neural network versus
  turning single delta (on node) into three weight_delta values
  how it works
  runnable code
  weighted sums
multiple outputs
  gradient descent with
  generalizes to arbitrarily large networks
  neural networks making multiple predictions using only single input
  how it works
  predicting with multiple inputs
  using single input
multiplication function, adding support for

N

n linear layers
n output neurons
n_batches
n_hidden parameter
n_layers input parameter
Nanodegree
NaNs (not-a-numbers)
natural language processing.
    See NLP.
n-dimensional tensors
__neg__ function
negation, adding support for
negative correlation
negative derivatives
negative labels, 2^nd, 3^rd
negative numbers
negative reversal attribute, 2^nd, 3^rd
negative sampling
negative sensitivity
negative weight
neural architecture.
    See architecture of neural networks.
neural learning
  alpha
  calculus and
  comparing
  mean squared error, 2^nd
  measuring error
  derivatives
  calculating
  defined
  example
  relationship between weight and error and, 2^nd
  using to learn
  weight_delta
  divergence
  error attribution
  functions
  gradient descent
  breaking
  general discussion
  iteration of
  hot and cold learning
  characteristics of, 2^nd, 2^nd
  memorizing
  overcorrections, 2^nd
  visualizing
  reducing error
  steps of
neural networks.
    See also deep neural network; ; neural learning.
  backpropagation
  in code
  iteration of
  weighted average delta
  weighted average error
  batch gradient descent
  building
  correlation
  creating
  indirect
  learning
  defined
  full gradient descent
  importance of
  learning whole dataset
  linear versus nonlinear networks
  making multiple predictions using single input
  making predictions
  matrices and matrix relationship
  importing matrices into Python
  patterns
  streetlight problem
  overfitting
  preparing data
  purpose of
  running program
  simple
  sometimes correlation, 2^nd
  stacking, 2^nd
  stochastic gradient descent, 2^nd
  visualizing
  architecture
  correlation summarization
  importance of visualization tools
  side by side
  simplifying, 2^nd
  vector-matrix multiplication
  weight pressure
  conflicting pressure
  weight update
neural prediction
  with multiple inputs
  how it works
  runnable code
  weighted sum
  with multiple outputs, 2^nd
  predicting with multiple inputs
  using single input
  neural networks
  defined
  purpose of
  simple
  stacking
  NumPy Python library
  predicting on predictions
  prediction
neurons, 2^nd, 3^rd
NLP (natural language processing), 2^nd, 3^rd
nodes
noise, 2^nd, 3^rd.
    See also regulation.
nonlinear activation functions
nonlinear neural networks
nonlinearities.
    See also activation functions.
nonlinearity layers, 2^nd

nonparametric learning
  counting-based methods
  parametric learning versus
normalization
normalized variants
normed_weights matrix
NOT operator
not-a-numbers (NaNs)
np.dot function
NumPy Python library, 2^nd, 3^rd

O

objective function
one_hot utility matrix
one-dimensional tensors
one-hot encoding
on-the-job training
open source project
OpenMined, 2^nd
OR operator
output data pattern
output datapoints
output datasets, 2^nd

output layer activation functions
  choosing
  configurations
  no activation function
  sigmoid
  softmax

outputs
  converting to slope
  gradient descent with
  generalizes to arbitrarily large networks
  neural networks making multiple predictions using only single input
  how it works
  neural networks
  predicting with multiple inputs
  using single input

overcorrections
  alpha
  visualizing

overfitting
  causes of
  general discussion
  overview of
overshooting

P

parameters, 2^nd, 3^rd

parametric learning
  nonparametric learning versus
  supervised
  unsupervised
patterns
peer support, for deep learning
perplexity metric
pip install phe command
pixels, 2^nd, 3^rd, 4^th
plaintext
plausible deniability, 2^nd
pooling
positive errors
positive labels, 2^nd, 3^rd
positive sensitivity
practice, importance of
pred variable

predictions
  deep neural networks
  with multiple inputs
  how it works
  runnable code
  weighted sum
  with multiple outputs, 2^nd
  predicting with multiple inputs
  using single input
  neural networks
  defined
  purpose of
  simple
  stacking
  NumPy Python library
  predicting images
  predicting movie reviews
  predicting on predictions
  prediction
privacy.
    See also security and privacy.
  federated learning
  general discussion
  hacking into
  homomorphically encrypted
  homomorphic encryption
  privacy
  secure aggregation
  spam
private key
probabilities.
    See activation functions.
project-based teaching method
propagation
  finishing with .index_select( ) method
  importance of visualization tools
  linking variables
  with multiple inputs
  how it works
  runnable code
  weighted sum
  with multiple outputs, 2^nd
  predicting with multiple inputs
  using single input
  neural networks
  defined
  purpose of
  simple
  stacking
  NumPy Python library
  overview of, 2^nd, 3^rd, 4^th, 5^th, 6^th
  predicting on predictions
  prediction
  recurrent neural network, 2^nd
  side-by-side visualization
public key
pure error, 2^nd

Python
  creating sentence embeddings using identity matrices in
  forward propagation in
  learning
  NumPy Python library, 2^nd, 3^rd
Python Codecademy course
PyTorch, 2^nd, 3^rd

R

random subsections
randomized response
randomness
raw error, 2^nd
recurrent embeddings
recurrent matrix
recurrent neural network.
    See RNN.
reducing error
regularization, 2^nd, 3^rd, 4^th
regulation
  batch gradient descent
  dropout technique
  in code
  evaluated on MNIST
  general discussion
  early stopping
  generalization
  memorization
  overfitting
  causes of, 2^nd
  predicting images
  three-layer network on MNIST
relu function, 2^nd, 3^rd, 4^th, 5^th, 6^th, 7^th, 8^th, 9^th
relu2deriv function, 2^nd, 3^rd
reviews2vectors matrix
RNN (recurrent neural network)
  arbitrary length of data
  backpropagation with
  challenge of
  forward propagation with
  weight update with
  averaged word vectors
  Babi dataset
  backpropagation
  character language modeling
  comparing sentence vectors
  execution and output analysis
  forward propagation in Python
  overview of, 2^nd
  sentence embeddings
  setting up
  vanishing and exploding gradients
  word embeddings
  how information is stored in
  limitations of
  neural networks use of
  summing
RNNCell class
runnable code, neural prediction
running.backward( ) method

S

sampling output
scalar multiples
scalar-matrix multiplication
scalars
scaling attribute, 2^nd, 3^rd
security and privacy
  federated learning
  general discussion
  hacking into
  homomorphically encrypted
  homomorphic encryption
  privacy
  secure aggregation
  spam
selective correlation
self.children counter
self.data array
self.w_hh layer
self.w_ho layer
self.w_ho.forward(h)
self.w_ih layer
self.weight
sensitivity, 2^nd, 3^rd
sent2output layer
sentence embeddings, RNN
  sentence vectors, 2^nd
  transition matrices
sentiment dataset
Sequential( ) method
SGD class
shape, 2^nd, 3^rd
sharpness of attenuation
short-term memory
sigmoid( ) function, 2^nd, 3^rd, 4^th, 5^th, 6^th, 7^th
signal, 2^nd.
    See also regulation.
similarity
simple neural networks
simplifying visualization, 2^nd
single-input gradient descent
single-weight neural network

slope
  converting output to
  multiplying delta by
  overview of

softmax computation
  activation functions
  output layer activation functions
softmax function, 2^nd, 3^rd, 4^th, 5^th
sometimes (conditional) correlation, 2^nd
sox_delta variable
spam
square matrix
squiggly line
stacked convolutional layers
stacking neural networks, 2^nd
stacks of layers
static computation graph
step_amount variable
stickiness
stochastic gradient descent optimizer, 2^nd, 3^rd, 4^th
stopping attribute, 2^nd, 3^rd

streetlight problem
  datasets
  matrices and matrix relationship
subtraction function, adding support for
sum function, adding support for
sum pooling
.sum(dim) function
summing embeddings
supervised machine learning
supervised natural language processing (NLP)
supervised parametric learning

T

tanh( ) function, 2^nd, 3^rd, 4^th, 5^th
target labels, meaning of neuron based on
tasks, NLP
Tensor class, 2^nd
Tensor.backward( ) function
TensorFlow, 2^nd

tensors
  adding nonlinear functions to
  automatic gradient computation (autograd)
  defined
  that are used multiple times
test( ) function
test_images variable
test_labels variable
testing accuracy
Theano
three-layer networks, 2^nd
topic classification
train( ) function
training accuracy
Training-Acc
transition weights
transpose function, adding support for
trial and error, 2^nd.
    See also parametric learning.
true signal

truncated backpropagation
  disadvantages of
  iterating with
  overview of
Twitter
two-dimensional tensors
two-layer networks, 2^nd

U

<unk> tokens
unsupervised machine learning, 2^nd
up pressure
utility functions

V

validation set

vanishing gradient problem
  countering with LSTM
  general discussion
  toy example
variable-length text

variables
  linking
  multiplying
variants
vect_mat_mul function
vectors, 2^nd, 3^rd, 4^th, 5^th
vector-scalar addition and multiplication
virtual graph
visualizing neural networks
  architecture
  correlation summarization
  importance of visualization tools
  side by side
  simplifying, 2^nd
  vector-matrix multiplication
  defined
  letters can be combined to indicate functions and operations
  linking variables
  using letters instead of pictures
volume, 2^nd

W

w_sum function, 2^nd

weight pressure
  conflicting pressure
  weight update
Weight Pressure table, 2^nd
weight values, 2^nd
weight variable
weight vector
weight_delta variable, 2^nd, 3^rd, 4^th, 5^th
weighted average delta
weighted average error
weighted sums, 2^nd
  neural prediction
  visualizing
weights.
    See also multiple inputs.
  batch gradient descent
  convolutional neural networks
  freezing one weight
  full gradient descent
  MNIST dataset
  stochastic gradient descent
  turning single delta into three weight_delta values
  visualizing weight values
  weight update with arbitrary length
weights variable, 2^nd, 3^rd, 4^th, 5^th
weights vector
wlrec predictor
word analogies
word correlation, capturing in input data

word embeddings
  comparing
  recurrent neural network
  how information is stored in
  limitations of
  neural networks use of
  summing
word vectors, 2^nd

Y

y. creation_op

Z

z .backward( ) function