Now we know that the fluentd-cloud-logging-kubernetes pods are sending the data to the Google Cloud, but why do we need FluentD? Simply put, FluentD is a collector.
It can be configured to have multiple sources to collect and tag logs, which are then sent to various output points for analysis, alerting, or archiving. We can even transform data using plugins before it is passed on to its destination.
Not all provider setups have FluentD installed by default, but it is one of the recommended approaches to give us greater flexibility for future monitoring operations. The AWS Kubernetes setup also uses FluentD, but instead forwards events to Elasticsearch.
Exploring FluentD: If you are curious about the inner workings of the FluentD setup or just want to customize the log collection, we can explore quite easily using the kubectl exec command and one of the pod names from the command we ran earlier in the chapter. First, let's see if we can find the FluentD config file: $ kubectl exec fluentd-cloud-logging-kubernetes-minion-group-r4qt --namespace=kube-system -- ls /etc/td-agent.
We will look in the etc folder and then td-agent, which is the fluent sub folder. While searching in this directory, we should see a td-agent.conf file. We can view that file with a simple cat command, as follows: $ kubectl exec fluentd-cloud-logging-kubernetes-minion-group-r4qt --namespace=kube-system -- cat /etc/td-agent/td-agent.conf.
We should see a number of sources, including the various Kubernetes components, Docker, and some GCP elements. While we can make changes here, remember that it is a running container and our changes won't be saved if the pod dies or is restarted. If we really want to customize, it's best to use this container as a base and build a new container, which we can push to a repository for later use.
We will look in the etc folder and then td-agent, which is the fluent sub folder. While searching in this directory, we should see a td-agent.conf file. We can view that file with a simple cat command, as follows: $ kubectl exec fluentd-cloud-logging-kubernetes-minion-group-r4qt --namespace=kube-system -- cat /etc/td-agent/td-agent.conf.
We should see a number of sources, including the various Kubernetes components, Docker, and some GCP elements. While we can make changes here, remember that it is a running container and our changes won't be saved if the pod dies or is restarted. If we really want to customize, it's best to use this container as a base and build a new container, which we can push to a repository for later use.