TfidfTransfomer computes tf-idf weights from a document-term matrix of token counts, such as the one produced by the CountVectorizer.

TfidfVectorizer performs both computations in a single step. It adds a few parameters to the CountVectorizer API that controls smoothing behavior.

TFIDF computation works as follows for a small text sample:

sample_docs = ['call you tomorrow',
               'Call me a taxi',
               'please call me... PLEASE!']

We compute the term frequency as we just did:

vectorizer = CountVectorizer()
tf_dtm = vectorizer.fit_transform(sample_docs).todense()
tokens = vectorizer.get_feature_names()
term_frequency = pd.DataFrame(data=tf_dtm,
                             columns=tokens)

  call  me  please  taxi  tomorrow  you
0     1   0       0     0         1    1
1     1   1       0     1         0    0
2     1   1       2     0         0    0

Document frequency is the number of documents containing the token:

vectorizer = CountVectorizer(binary=True)
df_dtm = vectorizer.fit_transform(sample_docs).todense().sum(axis=0)
document_frequency = pd.DataFrame(data=df_dtm,
                                  columns=tokens)
   call  me  please  taxi  tomorrow  you
0     3   2       1     1         1    1

The tf-idf weights are the ratio of these values:

tfidf = pd.DataFrame(data=tf_dtm/df_dtm, columns=tokens)
   call   me  please  taxi  tomorrow  you
0  0.33 0.00    0.00  0.00      1.00 1.00
1  0.33 0.50    0.00  1.00      0.00 0.00
2  0.33 0.50    2.00  0.00      0.00 0.00