In-memory

MongoDB storage in-memory is a risky task with high rewards. Keeping data in-memory, can be up to 100,000 times faster than durable storage on disk.

Another advantage of using in-memory storage is that we can achieve predictable latency when we write or read data. Some use cases dictate for latency that does not deviate from normal no matter what the operation is.

On the other hand, by keeping data in-memory we are open to power loss and application failure where we can lose all of our data.

Using a replica set can safeguard against some classes of errors but we will always be more exposed to data loss if we store in data as opposed to storing on disk.

There are, however, some use cases in which we may not care that much about losing older data.

In the financial world, we may have the following:

  • High-frequency trading/algorithmic trading where higher latency in the case of high traffic can lead to transactions not being fulfilled.
  • Fraud detection systems. In this case, we are concerned about real-time detection being as fast as possible and we can safely store only the cases that require further investigation or the definitely positive ones to durable storage.
  • Credit card authorizations, trade ordering reconciliation, and other high-traffic systems that demand a real-time answer.

In the web applications ecosystem, we have the following:

  • Intrusion detection systems. Like fraud detection, we are concerned with detecting intrusion as fast as possible without so much concern for false positive cases.
  • Product search cache. In this case, losing data is not mission-critical but rather a small inconvenience from the customer's perspective.
  • Real-time personalized product recommendations. Again, a low-risk operation in terms of data loss, we can always rebuild the index even if we suffer data loss.

A major disadvantage of an in-memory storage engine is that our dataset has to fit in-memory. This means that we must know and keep track of our data usage so that we don't exceed the memory of our server.

Overall, using the MongoDB in-memory storage engine may be useful in some edge use cases but lacking durability in a database system can be a blocking factor in its adoption.

In-memory storage is part of MongoDB Enterprise Edition, the paid offering by MongoDB.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.147.49.252