Characteristics of a Large-Scale Software System

The biggest difference between a large-scale software system and a small-scale one revolves around the software’s life cycle model . It refers to all the phases of a software product, from initial development, through exploitation, maintenance, evolution, and retirement. To make the discussion simple, our notion of large-scale is a software system that cannot be implemented by a small team; is part of a complex, socio-technical, software-intensive system; and whose retirement would have a huge business impact. Software maintenance and evolution is usually the longest phase in a large-scale software system’s life cycle; for some safety-critical software systems, it may span a couple of decades. This means that most activities will happen after the deployment of the first production version.

It is instructive to use a tournament chess game as an analogy to the software life cycle model. As with software, such a chess game is preceded by an initial preparation and planning phase (e.g., analyze your opponent’s style of play and opening repertoire, recall past experience with the opponent, compare your current standings, decide whether you want to aggressively push for a win, etc.). You cannot approach the game as amateurs do by simply making moves based on the current position of the chess pieces. During the game you balance resources (material, time, position, etc.), apply theory, use experience, and progress through stages (opening, middle game, and end game). The stages of a chess game are mapped to the phases of the software life cycle model in Figure 3-1.

To keep the cycle rolling, a holistic approach to software engineering is required. You must evaluate a broad spectrum of possibilities to properly judge the potential vectors of changes. These dictate where your software must be flexible in order to accommodate future additions and improvements. In each round, you must also address waves of change requests; this is similar to chess, where you must constantly monitor what your opponent is doing and alter your strategy accordingly. Finally, as in chess, if you mess up the requirements or devise a wrong architecture, then it is very difficult to make corrections later. In chess, a bad opening would turn your game into a struggle for a draw.

While in chess everything ends in the last stage, this is completely different in the development of a large-scale software system. The cycles are chained and encompass developments of new features. There is even a view that software maintenance and evolution is a continuation of the development phase, and in this respect, we talk about perpetual evolution (successful improvements and extensions of a product). In some sense, developing a small software product is like a single chess game, while developing a large-scale software system is akin to a very long tournament.

Evidently, the ratio is 3:1 between maintenance and initial development. Consequently, we should beware of being fast at the expense of quality. Many immature companies nurture a false belief that tooling is the silver bullet. They will both speed things up and compensate for human deficiencies (lack of knowledge and experience, inattentiveness, etc.). As you will later see, the only solution is to invest in people before anything else.

Rules, Principles, Conventions, and Standards

All the terms in the title of this section are interrelated and differ mostly in their level of rigor. Conventions are agreements between parties, while standards are formalized specifications that are managed by appropriate standardization bodies (for example, IEEE 754-2019 - IEEE Standard for Floating-Point Arithmetic; see https://standards.ieee.org/standard/754-2008.html ). Unlike physics, the software engineering discipline usually doesn’t have laws, although exceptions exist. Rules and principles are in the middle of the spectrum between conventions and standards. All of them are intended to impose constraints on system design to boost interoperability, integration, and productivity. Maybe it sounds weird, but judicious limitation of choices actually channels creativity; the objective is to avoid reinventing the wheel all the time. Furthermore, rules and principles synthesize knowledge of a field in a straightforward manner that helps attain uniformity and quality. Python is also bundled with a set of rules called The Zen of Python, as shown in Figure 3-2. The standard quality model is attached to the list to remind you how these items are important.

As you can see, according to The Zen of Python, it is crucial to eschew the temptation of treating rules and principles as dogmas. This is superbly illustrated by contrasting the first and ninth rules (which I have connected with a curved arrow). So, even The Zen of Python warns us about this problem. My commentary inside the box on the left provides an example of why vehemently trying to satisfy a rule could cause problems. On the other hand, there is an important meta-rule (the eighth rule, shown with a border) to remind us that it is equally wrong to blatantly ignore rules. You must know what you’re doing before departing from any sort of edict. “Readability counts” has a central location, with good reason. Software engineers spend most of their time during maintenance and evolution on understanding code.

This chapter implicitly follows the rules listed in Figure 3-2; the exposition of topics relies on simple examples (the Zen mentions implementation details instead of narrative). An absence of such examples could indicate a vague topic, an inept lecturer, or both. The illustrations that follow are indeed as simple as possible to keep your focus on the core subject. There is a key Agile principle (see https://agilemanifesto.org/principles ) about simplicity that states “Simplicity--the art of maximizing the amount of work not done--is essential.” Ironically, this is the most misapplied tenet. The work required to realize what should be expelled is everything but simple.

The preceding tools calculate quantitative proxies for qualitative aspects of a code base. Consequently, they can only report and act upon efficiently measurable indicators. Often, the most important traits of a software development endeavor are hard or impossible to measure. People-related elements belong to this class. The next section explains this phenomenon in more detail.

Context Awareness and Communicative Abilities

In general, we may discern three major personality types among software engineers¹:

• Context and knowledge oblivious

  The recipient will blindly follow instructions without seeking out help for further clarification. This type of person will guess instead of basing his decision on facts. Moreover, due to lack of knowledge, he will make up ad hoc rules/principles, as he judges appropriate. You may try excessive micromanagement, which is equally troublesome for both parties. This behavior is depicted in Figure 3-3.

• Knowledgeable

  This class is very similar to the previous one, the only difference being that this type of person has proper knowledge and thus will stick to major rules and principles of the discipline. This behavior is depicted in Figure 3-4.

• Context aware and knowledgeable

  This type of person seeks to maintain a constant feedback loop by proactively asking questions, as needed. This behavior is depicted in Figure 3-5.

Software engineers must be conscious of the language of business and talk to management in terms of money flows. Otherwise, there is a danger of destroying the precious feedback loop between development and management, resulting in two isolated factions (something that frequently happens in organizations). Such separation has detrimental consequences on success of the business.

Asking the proper question (the question itself is the treasure on the island in Figure 3-5), properly formulating the problem, and expressing choices in the language of business are most important to keep the valuable feedback loop active. There are two preconditions for attaining quality via feedback loops: having time to ask questions and create alternative designs, and knowing how to talk to business (management). Both conditions are usually neglected by software professionals.

Handling Legacy Code

A software system may transform into legacy code for many reasons: it uses an outdated technology; it is utterly non-evolvable due to its deteriorated quality; it is pushed out by novel business solutions; and so forth. Nevertheless, sometimes you will need to handle such code. The biggest conundrum is to reverse engineer it in an attempt to recover the necessary information to grasp its design and implementation. Don’t forget that legacy code is sparsely covered by automated tests and isn’t properly documented. All that you have is the bare source code. This section emphasizes the importance of documentation (test cases are a type of documentation) to avoid developing code that is essentially legacy code after its first release. The next chapter is devoted to JupyterLab, which is all about this matter.

The Importance of APIs

An API must efficiently communicate the intended usage and behavior of some piece of a system. Furthermore, it should enable a client to gradually comprehend the target system, as needed. An API that forces you to understand everything before being able to interact with a system is a failure. Equally bad is to be forced to look into the source code to understand what a function or class performs. Structuring an API should revolve around the entropy model regarding the matching system.

Suppose you don’t know anything about a system S. Therefore, your confusion and uncertainty are capped out. The level of entropy may be expressed as

(1)

where s_i is an assumed state of the system.

The API is a set of artifacts (usually abstractions); in other words, API = {a₁, a_2,…, a_m}. Knowing some element of an API should generate an information gain formulated as

IG(S, API) = H(S) – H(S|API)(2)

(3)

A good API should order its elements according to the law of diminishing returns . This entails knowing that, for example, a package name should have a higher impact on your information gain (according to (2) and (3)) than stumbling across an internal method parameter of a deeply hidden class. Also, after seeing some function’s name and establishing a sound understanding about its behavior, none of its parameters should oblige you to reevaluate your original conception. The next subsection will illuminate this issue.

There will always be emergent properties and empirically observable behaviors of a system not specified in an API. Such “holes” may be occluded via consumer-driven contracts (CDCs) . These are augmentations of the API that are created by clients (they may be shaped as a set of test cases). Once those are attached to an API, then the cumulative information gain due to CDCs is defined as

IG(S, CDC) = IG(S, API U CDC)(4)

All this semiformal reasoning entails that you must carefully name your abstractions. The names must not contradict the abstractions themselves. The next section gives a concrete example.

The Socio-* Pieces of Software Production

This conundrum is tightly associated with data science. Data science is a team sport, relies on software solutions, and executes in a broader sociological context (this is also the primary focus of systems engineering). Consequently, the following example equally applies both to systems and software engineering and to data science. The objective is to debunk the myth that being purely data-driven is both a necessary and sufficient condition for success.

There are two-primary socio-* parts of software: socio-economic and socio-technical. These have to be in balance. The socio-economic component ensures healthy business, while the socio-technical aspect keeps people happy. The crucial problem with data-driven approaches is that they are good for things that can be measured (preferably with tool support) but are not good for the many equally important characteristics that are hard to quantify.

Funny Elevator Case Study

Suppose you have a funny elevator that during the daytime ascends U(p) meters, while during the night descends D(own) meters. It must reach some total height H(eight) in meters. You need to implement the function num_days(h, u, d) to calculate the number of days for attaining the desired height. You can assume that 0 ≤ D < U < H < 10³⁰. For example, calling num_days(6, 5, 1) should yield 2. Listing 3-14 shows the initial version (see the elevator0.py file in the optimization folder). The running time is O(h / (u – d)) (for a good recap of Big-O notation, visit http://bigocheatsheet.com ).

def num_days(h, u, d):

    total_days = 1

    curr_height = 0

    while h - curr_height > u:

        curr_height += u - d

        total_days += 1

    return total_days

Listing 3-14

The First Unoptimized Variant of Our Function

The preceding code literally simulates the climbing process with a conditional loop. Here are some timings of executions (notice the bold lines, which you shouldn’t try to run):

>> %timeit num_days(6, 5, 1)

301 ns ± 7.52 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>> %timeit num_days(1000000000, 4, 2)

>> %timeit num_days(998998998, 123461, 123460)

>> %timeit num_days(1000000000, 2314601, 2314600)

Clearly, the performance is unacceptable, and the software must be enhanced. Before any optimization work, you must unambiguously define what fast enough means. To aid testing, you must specify three values:

• Minimum performance: Below this threshold, the product is not acceptable. In our case, this value is 5 seconds for all test cases (i.e., the worst-case running time).

• Target performance: This is what you are aiming for. In our case, 1 second.

• Maximum performance: Above this threshold, there is economic motivation to spend resources for further improvement. In our case, 0.1 second.

Without this range, testers may reject your work just because execution time is negligibly below the target. Most of the time, the product is performing quite well even without exactly hitting the target. Of course, you must establish the baseline (our initial code) and track progress over time.

First Optimization Attempt

We can avoid many iterations by observing that the total number of days isat least . We just need to account for the nightly plunges. Listing 3-15 shows the faster version of our code.

def num_days(h, u, d):

    total_days = 1

    height_left = h

    while u < height_left:

        days = height_left // u

        total_days += days

        height_left -= days * (u - d)

    return total_days

Listing 3-15

The Next Variant of Our Software

Following are the new execution times (the bold line is still out of reach of this program):

>> %timeit num_days(6, 5, 1)

321 ns ± 10.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>> %timeit num_days(1000000000, 4, 2)

6.96 μs ± 438 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

>> %timeit num_days(998998998, 123461, 123460)

235 ms ± 2.49 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>> %timeit num_days(1000000000, 2314601, 2314600)

3.18 s ± 104 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>> %timeit num_days(10 ** 20, 2314601, 2314600)

This is a remarkable improvement compared to the original version. After all, the first four test cases are within acceptable limits. Data (running times) should reassure us that we are on the right track. Observe that the code is a bit more complex, which is a natural consequence of performance optimization.

Note

We are now moving in the realm of socio-economic aspects, since we are trying to optimize a business-critical part of the code base. We measure improvements and let data drive us toward better variants. This is a typical route in mature software companies. You should avoid the temptation to make changes based on guessing about problems.

Second Optimization Attempt

It is possible to apply the divide and conquer paradigm, since we can combine the results from two subproblems of size H/2 to handle size H. This may drive us toward a logarithmic algorithm, provided that we find an efficient way to merge subcases. Listing 3-16 shows this approach in action; it applies many software engineering techniques: the divide and conquer paradigm, memorization, and hybrid approach.

from functools import lru_cache

@lru_cache(maxsize=32)

def _partial_num_days(height_left, u, d):

    total_days = 1

    while u < height_left:

        days = height_left // u

        total_days += days

        height_left -= days * (u - d)

    return total_days

H_LIMIT = 1000000

def num_days(h, u, d):

    if h > H_LIMIT:

        days = 2 * (num_days(h // 2, u, d) - 1)

        height_left = h - days * (u - d)

        return days + _partial_num_days(height_left, u, d)

    else:

        return _partial_num_days(h, u, d)

Listing 3-16

The Very Fast Variant of Our Code

Notice that the _partial_num_days function is essentially our previous num_days function. It covers low-dimensional cases. The threshold H_LIMIT controls when recursion should happen. This arrangement of selecting algorithms based on some condition is the hallmark of the hybrid approach. Below are the execution times:

>> %timeit num_days(6, 5, 1)

298 ns ± 14 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>> %timeit num_days(1000000000, 4, 2)

5.6 μs ± 155 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

>> %timeit num_days(998998998, 123461, 123460)

6.01 μs ± 56.9 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

>> %timeit num_days(1000000000, 2314601, 2314600)

6.38 μs ± 998 ns per loop (mean ± std. dev. of 7 runs, 1 loop each)

>> %timeit num_days(10 ** 20, 2314601, 2314600)

29.9 μs ± 1.19 μs per loop (mean ± std. dev. of 7 runs, 1 loop each)

All running times are way below our imaginary 0.1 second. Should anyone question this achievement? Solely from the socio-economic viewpoint, this version is perfect. It is a business enabler, since all use cases are working within acceptable performance limits. What about the socio-technical aspect? This is exactly what many organizations miss due to blindly being data-driven. This version is an abomination from the socio-technical perspective. It is simply awful!

Teammate- and Business-Friendly Variant

List 3-17 shows the final version, which balances both socio-economic and socio-technical aspects of a solution. It is a straightforward, ultra-fast, and short program (the extra variables further clarify the partial expressions).

def num_days(h, u, d):

    import math

    height_left_until_last_day = h - u

    daily_progress = u - d

    return 1 + math.ceil(height_left_until_last_day / daily_progress)

Listing 3-17

Overall Best Version of Our Code with O(1) Running Time

Is it possible to land here after publishing the previous incomprehensible variant? Well, this is a tough question. You will need to convince management that something that enabled the business to roll now must be rewritten. Your only chance is to apply software economics and justify your intention by expressing how much the business will save through such maintainable software. It will be very hard to explain in monetary terms the beautiful code’s positive impact on team morale and happiness. For this you must be a lucky person who is surrounded by unselfish management.

Exercise 3-1. Draw The Cone of Uncertainty

Look up the Cone of Uncertainty at https://www.construx.com/books/the-cone-of-uncertainty and try to reproduce it using the matplotlib framework (which is introduced in Chapter 6). The outcome of such a visualization would be a conceptual diagram. It doesn’t contain actual values, but rather artificial ones just to showcase the phenomenon.

Hint: You may want to read the excellent tutorial for matplotlib at https://realpython.com/python-matplotlib-guide , which contains examples of conceptual graphs.

Summary

This chapter has tried to illuminate the importance of being context aware and knowing the sociological aspects of software production. Both are indispensable attributes in large-scale software development and data science. They cannot be treated separately once you recognize that computing is at the heart of scaling toward Big Data problems. Most topics discussed in this chapter are barely touched upon in classical courses, and thus practitioners typically learn them the hard way.

Python is the most popular language of data science thanks to its powerful frameworks, like those found in the SciPy ecosystem. Acquaintance with reuse-based software engineering is essential to properly apply those frameworks. Part of this effort revolves around topics touched upon in this chapter (APIs, readable code bases, adequate documentation, judicious use of abstractions, usage of tools, etc.).

As a data scientist, you will surely work under pressure with strict deadlines. This is the main difference between working as a professional in a software company and volunteering on open-source projects. This chapter’s goal was to prepare you for working in large enterprises.

References