© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2023
N. TolaramSoftware Development with Gohttps://doi.org/10.1007/978-1-4842-8731-6_12

12. Epoll Library

Nanik Tolaram1  
(1)
Sydney, NSW, Australia
 

Building an application that processes a huge amount of network processing requires a special way of handling connections in a distributed or cloud environment. Applications running on Linux are able to do this thanks to the scalable I/O event notification mechanism that was introduced in version 2.5.44. In this chapter, you will look at epoll. According to the documentation at https://linux.die.net/man/7/epoll,

The epoll API performs a similar task to poll: monitoring multiple file descriptors to see if I/O is possible on any of them.

You will start by looking at what epoll is and then move on to writing a simple application and finish off looking at the Go epoll library and how it works and also how to use it in an application.

On completion of this chapter, you will understand the following:

  • How epoll works in Linux

  • How to write a Go application to use the epoll API

  • How the epoll library works

  • How to write a Go application using the epoll library

Source Code

The source code for this chapter is available from the https://github.com/Apress/Software-Development-Go repository.

Understanding epoll

In this section, you will start by looking at what epoll is all about from a system perspective. When you open a socket in Linux, you are given a file descriptor (or FD for short), which is a non-negative value. When the user application wants to perform an I/O operation to the socket, it passes the FD to the kernel. The epoll mechanism is event-driven, so the user application is notified when an I/O operation happens.

As shown in Figure 12-1, epoll is actually a data structure inside Linux that is provided to multiplex I/O operations on multiple file descriptors. Linux provides system calls for user applications to register, modify, or delete FDs from the data structure. Another thing to note is that epoll has Linux-specific features, which means applications can only be run on Linux kernel-based operating systems.

A block diagram has 2 blocks of user space and kernel space. A user-space includes a user app to register, modify, and delete linked to 2 rectangular blocks of interest and a ready list.

Figure 12-1

epoll data structure

The data structure contains two sets of lists:

  • Interest List: This list/set contains FDs that applications are interested in. The kernel will only send events related to a particular FD that applications are interested in.

  • Ready List: This list/set contains a subset of reference FDs from the Interest List FDs. The FDs in this list are in the *ready* state that the user application will be notified of.

The following are the system calls used by applications to work with the data structure. In the later sections, you will look closely at how you are going to use them in application and also inside an epoll library.

  • epoll_create: A system call to create a new epoll instance and return a file descriptor.

  • epoll_ctl: A system call to register, modify, and delete a FD from the Interest List.

  • epoll_wait: A system call to wait for I/O events or another way the system call is called to fetch items that are ready from the Ready List.

To use epoll effectively in application, you need to understand how event distribution is performed. Simply put, there are two different ways events are distributed to applications:

  • Edge triggered: A monitored FD configured with edge will be guaranteed to get one notification if the readiness state changed since the last time it called epoll_wait. The application will receive one event and, if it requires more events, it must perform an operation via a system call to inform epoll that it is waiting for more events.

  • Level triggered: A monitored FD configured with level will be batch together as a single notification and an application can process them all at once.

From the above, it is obvious that edge triggered requires an application to do more work compared to level triggered. By default, epoll operates using a level triggered mechanism.

epoll in Golang

In this section, you will write a simple application that uses epoll. The app is an echo server that receives connections and sends responses to the value that is sent to it.

Run the code inside the chapter12/epolling/epollecho folder. Open your terminal to run the following command:
go run main.go
Once the app runs, open another terminal and use the nc (network connect) tool to connect to the application. Type in something in the console and press Enter. This will be sent to the server.
nc 127.0.0.1 9999

The sample app will respond by sending the string that was sent by the client. Before diving into the code, let’s take a look at how epoll is used in an application.

Epoll Registration

As you can see in Figure 12-2, the application creates a listener on port 9999 to listen for incoming connections. When a client connects to this port, the application spins off a goroutine to handle the client connection.

A block diagram has 2 blocks. A client app is reversibly linked to the user app in the user space block with an F D value of 2000. The user app is linked to interest list blocks in kernel space.

Figure 12-2

Listener epoll registration

Now, let’s take a more detailed look at how the whole thing works inside the app. The following snippet shows the application creating a socket listener using the syscall.Socket system call and binding it to port 9999 using syscall.Bind:
   ...
   fd, err := syscall.Socket(syscall.AF_INET, syscall.O_NONBLOCK|syscall.SOCK_STREAM, 0)
   if err != nil {
       fmt.Println("Socket err : ", err)
       os.Exit(1)
   }
   defer syscall.Close(fd)
   if err = syscall.SetNonblock(fd, true); err != nil {
       ...
   }
   // prepare listener
   addr := syscall.SockaddrInet4{Port: 9999}
   copy(addr.Addr[:], net.ParseIP("127.0.0.1").To4())
   err = syscall.Bind(fd, &addr)
   ...
   // listener
   err = syscall.Listen(fd, 10)
   ...
   ...
On successfully listening on the port, the app creates a new epoll by calling syscall.EpollCreate1. This instructs the kernel to prepare a data structure that the application will use to listen for I/O events for file descriptors that it is interested in.
   ...
   epfd, e := syscall.EpollCreate1(0)
   if e != nil {
       ...
   }
   ...

Once the data structure successfully creates the application, it proceeds by registering the socket listener file descriptor, as seen in the following code snippet. The code uses syscall.EPOLL_CTL_ADD to specify to the system call that it is interested in doing a new registration.

The registration is done based on the information provided in the event struct, which contains the file descriptor and the event that it is interested in monitoring.

The application uses the EPOLLIN flag to indicate that it is only interested in reading the event. The epoll documentation at https://man7.org/linux/man-pages/man2/epoll_ctl.2.html provides details on the different flags that can be set for the event.Events field.
  // register listener fd to Interest List
   event.Events = syscall.EPOLLIN
   event.Fd = int32(fd)
   if e = syscall.EpollCtl(epfd, syscall.EPOLL_CTL_ADD, fd, &event); e != nil {
       ...
   }

Epoll Wait

The last step after registering is to call syscall.EpollWait to wait for an incoming event from the kernel, which is wrapped inside a for {} loop as shown in the following snippet. The -1 parameter passed as the timeout parameter to the system indicates the application will wait indefinitely until an event is ready to be delivered by the kernel.
  for {
       n, err := syscall.EpollWait(epfd, events[:], -1)
       ...
   }
When the application receives events, it start processing by looping through the number of events it receives, as shown here:
   for {
       n, err := syscall.EpollWait(epfd, events[:], -1)
       ...
       // go through the events
       for ev := 0; ev < n; ev++ {
           ...
       }
   }

The event received contains the event type generated by the system and the file descriptor that it is for. This information is used by the code to check for a new client connection. This is done by checking whether the file descriptor it received is the same as the listener; if it is, then it will accept the connection by calling syscall.Accept using the listener FD.

Once it gets a new FD for the client connection, it will also be registered by the code into epoll using EpollCtl with EPOLL_CTL_ADD flag. Once completed, both listener FD and client connection FD are registered inside epoll and the application can multiplex I/O operations for both.
   for {
       n, err := syscall.EpollWait(epfd, events[:], -1)
       ...
       // go through the events
       for ev := 0; ev < n; ev++ {
           // if it is the same as the listener then accept connection
           if int(events[ev].Fd) == fd {
               connFd, _, err := syscall.Accept(fd)
               ...
               // new connection should be non blocking
               syscall.SetNonblock(fd, true)
               event.Events = syscall.EPOLLIN
               event.Fd = int32(connFd)
               // register new client connection fd to Interest List
               if err := syscall.EpollCtl(epfd, syscall.EPOLL_CTL_ADD, connFd, &event); err != nil {
                   log.Print("EpollCtl err : ", connFd, err)
                   os.Exit(1)
               }
           } else {
               ...
           }
       }
   }

As a final step, when the code detects that the FD received from the event is not the same as the listener FD, it will spin off a goroutine to handle the connection, which will echo back data received from the client.

Epoll Library

You looked at what epoll is all about and created an app that uses it. Writing an app that uses epoll requires writing a lot of repetitive code that takes care of accepting connections, reading requests, registering file descriptors, and more.

Using an open source library can help in writing better applications because the library takes care of the heavy lifting required for epoll. In this section, you will look at netpoll (http://github.com/cloudwego/netpoll). You will create an application using the library and see how the library takes care of epoll internally.

The code can be found inside the chapter12/epolling/netpoll folder. It is an echo server that sends requests received as a response to the user.
import (
   ...
   "github.com/cloudwego/netpoll"
)
func main() {
   listener, err := netpoll.CreateListener("tcp", "127.0.0.1:8000")
   if err != nil {
       panic("Failure to create listener")
   }
   var opts = []netpoll.Option{
       netpoll.WithIdleTimeout(1 * time.Second),
       netpoll.WithIdleTimeout(10 * time.Minute),
   }
   eventLoop, err := netpoll.NewEventLoop(echoHandler, opts...)
   if err != nil {
       panic("Failure to create netpoll")
   }
   err = eventLoop.Serve(listener)
   if err != nil {
       panic("Failure to run netpoll")
   }
}
   ...

This code snippet shows the creation of a socket listener using CreateListener from the library to listen on port 8000. After successfully opening the listener, the code proceeds to configure the netpoll by specifying the timeout and specifying the echoHandler function to handle the incoming request. The code starts listening to incoming requests by calling the Serve function of netpoll.

The echoHandler function handles reading and writing from the client socket connection using the passed-in parameter netpoll.Connection. The function reads using the connection.Reader() and writes using connection.Write().
func echoHandler(ctx context.Context, connection netpoll.Connection) error {
   reader := connection.Reader()
   bts, err := reader.Next(reader.Len())
   if err != nil {
       log.Println("error reading data")
       return err
   }
   log.Println(fmt.Sprintf("Data: %s", string(bts)))
   connection.Write([]byte("-> " + string(bts)))
   return connection.Writer().Flush()
}
You can see that the code written using the netpoll library is easier to read than the code that you looked at in the previous section. A lot of the heavy lifting is performed by the library; it also provides more features and stability when writing high-performance networking code. Let’s take a look at how netpoll works behind the scenes. Figure 12-3 shows at a high level the different components of netpoll.

A flow chart of the net poll server and new connection represents the Epoll load balancer including epoll blocks under the Linux kernel and finally to the goroutine pool.

Figure 12-3

netpoll high-level architecture

The library creates more than one epoll and it uses the number of CPUs as the total number of epolls it will create. Internally, it uses a load balancing strategy to decide which epoll a file descriptor will be registered to.

The library will register to the epoll when it receives a new connection or when the netpoll server runs for the first time, and it decides which epoll to use by using either a random or round-robin load balance mechanism, as shown in Figure 12-4. The load balancer type can be modified in an app using the following function call:
netpoll.SetLoadBalance(netpoll.Random)
netpoll.SetLoadBalance(netpoll.RoundRobin)

A block diagram of Epoll load balancer includes two small blocks named round-robin and random.

Figure 12-4

netpoll load balancer

The library takes care of a high volume of traffic by using goroutines. This is performed internally by utilizing a pool of goroutine pooling mechanisms. Developers just need to focus to ensure that their application and infrastructure can scale properly.

Summary

In this chapter, you looked at different ways of writing applications using epoll. Using your previous learning from Chapter 2 about system calls, you build an epoll-based application using the standard library. You learned that designing and writing epoll network applications is different from normal networking applications. You dove into an epoll library and learned how to use it to write a network application. Also, you looked at how the library works internally.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.142.171.64