Capped collections

Non-functional requirements are often related to the application's response time. Especially nowadays when we are connected to news feeds all the time and we want fresh information to be available in the shortest response time.

MongoDB has a special type of collection that meets this non-functional requirement, capped collections. Capped collections are fixed size collections that support high read and write throughput. This is because the documents are inserted in their natural order, without the need for an index to perform the write operation.

The natural insertion order is guaranteed by MongoDB, which writes the data like it is written on the disk. Therefore, updates that increase the document size are not allowed during the document's lifecycle. As soon as the collection reaches maximum size, MongoDB automatically cleans old documents so that new documents can be inserted.

One very common use case is the persistence of application logs. MongoDB itself uses the replica set operation log, oplog.rs, as a capped collection. In Chapter 8, Logging and Real-time Analytics with MongoDB, you will see another practical example of this.

Another very common usage of MongoDB is as a publisher/subscriber system, especially if we use tailable cursors. Tailable cursors are cursors that remain open even after the client reads all the returned records. So, when new documents are inserted into the collection, the cursor returns it to the client.

The following command creates the ordersQueue collection:

db.createCollection("ordersQueue",{capped: true, size: 10000})

We used the util command createCollection to create our capped collection, passing to it the name ordersQueue and a collection with the capped property with the value true and size with a value of 10000. If the size property is less than 4,096, MongoDB adjusts it to 4,096 bytes. On the other hand, if it is greater than 4,096, MongoDB raises the size and adjusts it to be a multiple of 256.

Optionally, we can set the maximum document number that a collection can have by using the max property:

db.createCollection(
   "ordersQueue",
   {capped: true, size: 10000, max: 5000}
)

Note

If we need to convert a collection into a capped collection, we should use the convertToCapped method as follows:

db.runCommand(
   {"convertToCapped": " ordersQueue ", size: 100000}
)

As we have already seen, MongoDB keeps the documents in a natural order, in other words, the order in which they are inserted into MongoDB. Consider the following documents, inserted in the ordersQueue collection as shown:

{
   "_id" : ObjectId("54d97db16840a9a7c089fa30"), 
   "orderId" : "order_1", 
   "time" : 1423539633910 
}
{
   "_id" : ObjectId("54d97db66840a9a7c089fa31"), 
   "orderId" : "order_2", 
   "time" : 1423539638006 
}
{
   "_id" : ObjectId("54d97dba6840a9a7c089fa32"), 
   "orderId" : "order_3", 
   "time" : 1423539642022 
}
{
   "_id" : ObjectId("54d97dbe6840a9a7c089fa33"), 
   "orderId" : "order_4", 
   "time" : 1423539646015 
}
{
   "_id" : ObjectId("54d97dcf6840a9a7c089fa34"), 
   "orderId" : "order_5", 
   "time" : 1423539663559 
}

The query db.ordersQueue.find() produces the following result:

{ 
   "_id" : ObjectId("54d97db16840a9a7c089fa30"), 
   "orderId" : "order_1", 
   "time" : 1423539633910 
}
{ 
   "_id" : ObjectId("54d97db66840a9a7c089fa31"), 
   "orderId" : "order_2", 
   "time" : 1423539638006 
}
{ 
   "_id" : ObjectId("54d97dba6840a9a7c089fa32"), 
   "orderId" : "order_3", 
   "time" : 1423539642022 
}
{ 
   "_id" : ObjectId("54d97dbe6840a9a7c089fa33"), 
   "orderId" : "order_4", 
   "time" : 1423539646015 
}
{ 
   "_id" : ObjectId("54d97dcf6840a9a7c089fa34"), 
   "orderId" : "order_5", 
   "time" : 1423539663559 
}

If we use the $natural operator as shown in the following query, we will have the same result as shown in the preceding output:

db.ordersQueue.find().sort({$natural: 1})

But if we need the last insertion first, we must execute the command with a -1 value on the $natural operator:

db.ordersQueue.find().sort({$natural: -1})

We must be careful when creating a capped collection as:

  • We cannot shard a capped collection.
  • We cannot update a document in a capped collection; otherwise, the document grows in size. If we need to update a document in a capped collection, then we must make sure that the size will remain the same. And for better performance, we should create an index to avoid a collection scan when updating.
  • We cannot delete a document in a capped collection.

Capped collections are a good tool when we have a high read/write throughput as a non-functional requirement, or we need to limit the collection size in bytes or by document number.

Nonetheless, if we need to automatically expire data, based on a time frame, we should use the time to live (TTL) function.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.191.139.42