Coding a data simulator

Coding a simulator that spits out data at a defined rate can help simulate real load.

This recipe assumes that the reader's machine is installed with MySQL.

Getting ready

  1. Make sure the MySQL production database server is up and running.
  2. Open a command line MySQL client shell as the root user.
  3. Create a database for this recipe called recipe63 as well as a user with permission to access it.
  4. Exit the shell.
    Getting ready

How to do it...

With these steps, we will explore coding a test simulator:

  1. Create a test generator script called recipe63.py that uses various Python libraries.
    import getopt
import random
import sys
import time
from network import *
from springpython.remoting.pyro import *
  2. Create a usage method that prints out command-line options.
    def usage():
    print "Usage"
    print "====="
    print "-h, --help           read this help"
    print "-r, --rate [arg]     number of events per second"
    print "-d, --demo           demo by printing events"
  3. Use Python's getopt library to parse command-line arguments.
    try:
    opts, args = getopt.getopt(sys.argv[1:], "hr:d", ["help", "rate=", "demo"])
except getopt.GetoptError, err:
    print str(err)
    usage()
    sys.exit(1)
        
rate = 10
demo_mode = False

for o, a in opts:
    if o in ("-h", "--help"):
        usage()
        sys.exit(1)
    elif o in ("-r", "--rate"):
        rate = a
    elif o in ("-d", "--demo"):
        demo_mode = True
  4. Add a switch so when NOT in demo mode, it uses Spring Python's PyroProxyFactory to connect to a server instance of the network management application defined in Chapter 7, Building a network management application.
    if not demo_mode:
    print "Sending events to live network app. Ctrl+C to exit..."
    proxy = PyroProxyFactory()
    proxy.service_url = "PYROLOC://127.0.0.1:7766/network"
  5. Code an infinite loop that creates a random event.
    while True:
    hostname = random.choice(["pyhost1","pyhost2","pyhost3"])
    condition = random.choice(["serverRestart", "lineStatus"])
    severity = random.choice([1,5])

    evt = Event(hostname, condition, severity)
  6. If in demo mode, print out the event.
        if demo_mode:
        now = time.strftime("%a, %d %b %Y %H:%M:%S +0000",
                                          time.localtime())
        print "%s: Sending out %s" % (now, evt)
  7. If not in demo mode, make a remote call through the proxy to the network app's process method.
        else:
        stored_event, is_active, updated_services, 
             updated_equipment = proxy.process(evt)
        print "Stored event: %s" % stored_event
        print "Active? %s" % is_active
        print "Services updated: %s" % updated_services
        print "Equipment updated; %s" % updated_equipment
        print "================"
  8. Sleep a certain amount of time before repeating the loop.
        time.sleep(1.0/float(rate))
  9. Run the generator script. In the following screenshot, notice there is an error because we haven't started the server process yet. This can also happen if the client and server have mismatched URLs.
    How to do it...
  10. Create a server script called recipe63_server.py that will run our network management app connected to MySQL using SQL script recipe62_network.sql from Targeting the test server.
    from springpython.database.factory import *
from springpython.database.core import *
from springpython.remoting.pyro import *

from network import *

import logging
logger = logging.getLogger("springpython")
loggingLevel = logging.DEBUG
logger.setLevel(loggingLevel)
ch = logging.StreamHandler()
ch.setLevel(loggingLevel)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
ch.setFormatter(formatter)
logger.addHandler(ch)

# Initialize the database
factory = MySQLConnectionFactory("user",      "password",
                                 "localhost", "recipe63")
dt = DatabaseTemplate(factory)
sql = open("recipe62_network.mysql").read().split(";")
for statement in sql:
    dt.execute(statement + ";")
  11. Add code to expose the app using Pyro.
    # Create an instance of the network management app
target_service = EventCorrelator(factory)

# Expose the network app as a Pyro service
exporter = PyroServiceExporter()
exporter.service_name = "network"
exporter.service = target_service
exporter.after_properties_set()
  12. Run the server script in a different shell.
    How to do it...
  13. The default rate is 10 events/second. Run the generator script with a rate of one event/second. In the following screenshot, notice how the script generated a clear, fault, and then another fault. The service started at Operational, moved to Outage, and stayed there.
    How to do it...

How it works...

Python's random.choice method makes it easy to create a range of random events. By using the time.sleep method, we can control the rate at which the events are created.

We used Pyro to connect the test generator to the network management application. This isn't the only way to connect things together. We could have exposed the application through other means, such as REST, JSON, or perhaps by communicating through a database table. That's not important. What is important is that we built an independent tool that fed data into our application as if it came from a live network.

There's more...

We built a test generator. It's easy to run multiple copies of it in different shells, at different rates. We have an easy way to simulate different subnets producing different volumes of traffic.

We could also add more command-line options to fine tune the events. For example, we could make the event condition a parameter, and emulate different rates for different types of events.

Why does the server script initialize the database?

A production version of the server wouldn't do this. For demonstration purposes of this recipe, it is convenient to put it there. Every time we stop and start the server script, it relaunches the database.

Why MySQL instead of SQLite?

SQLite has some limitations when it comes to multithreading. Pyro uses multithreading and SQLite can't pass objects across threads. SQLite is also relatively lightweight and probably not well suited for a real network management application.

See also

  • Targeting the test server
  • Chapter 7, Building a network management application
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.225.234.28