The site isn’t down, but an awful lot of alarms are going off. We trace the problems to changes someone made to a third-party API we use. The changes are causing major data problems on our side; namely, we don’t know what the changes are and we can’t find anyone who can tell us. It also turns out the system that talks to the API uses legacy code, and the only guy who knows how to work on it happens to be away on vacation. This a big system: 500,000-lines-of-Java-and-OSGI big.
Support calls are flooding in, lots of them. Expensive support calls from frustrated customers. We need to fix the problem quickly. I start up a Clojure REPL and use it to poke around the problem API.
My boss pokes his head into my office. “How’s it going?” he asks. “Working on it,” I say. Ten minutes later, my grandboss pokes his head into my office. “How’s it going?” he asks. “Working on it,” I say. Another ten minutes pass by when my great-grandboss pokes his head into my office. “How’s it going?” he asks. “Working on it,” I say. I get a half hour of silence before the CTO pokes his head into my office. “Working on it,” I say before he opens his mouth.
An hour passes, and I figure out what’s changed. I whip up a way to keep the data clean until the legacy developer gets back and can put together a proper fix. I hand my little program off to the operations team, which gets it up and running in a JVM, somewhere safe. The support calls stop coming in, and everyone relaxes a bit.
A week or so later at an all-hands meeting, the great-grandboss thanks me for the Java program I wrote that saved the day. I smile and say, “That wasn’t Java.”