Using Apache Mesos to Run Diverse Workloads

Apache Mesos provides a platform for managing a cluster, using frameworks, and running tasks. Whereas we can run frameworks for Hadoop, Spark, Storm, and Jenkins, we can also use Marathon to run specific commands. For example, we can use Apache Mesos to launch Docker containers, as well as workloads like Spark, Cassandra, or Kafka alongside Docker Swarm on the same cluster.

The Kubernetes and Swarm teams have also created Mesos frameworks for launching Kubernetes or Swarm on Mesos. As a framework, this offers much better integration than using Marathon to launch a Docker Swarm cluster on Mesos. Even though you can use Marathon or Aurora to schedule Docker containers, there can be some benefits to deploying Docker Swarm on Mesos. Docker Swarm exposes the same API used by developers to run containers, making it easy to get started, and they continue to add new features like software-defined networking that can be leveraged.

Using Apache Mesos to manage our application services removes the need to provision servers for specific services. Nodes are merged under a unified, logical interface. This makes it easy for frameworks like Marathon applications to assign themselves to the necessary resources without having to worry about load balancing, redundancy, scalability, or integrity.