Chapter 6
The Technology Domain

Technology is the foundation upon which all of the rest of your infrastructure depends. The business depends on information, information depends on applications, and applications depend on technology. Poorly managed information domain functions will kill a company within a few years. Poorly managed application domain functions will kill a company within a few months. Poorly managed technology functions can kill a company within hours.

I’ve mentioned several times in this book the precedence of the architectural domains: Business, Information, Application, and then Technology. That makes it sound as if the architectural process should be managed as a traditional “waterfall” workflow, where each domain completes their work in isolation, then throws their work product over the fence to the next domain. That’s clearly unwise. Architecture is an agile process. Yes, there are some decisions that come before others, but those decisions are often made in a group effort.

For example, the business may decide there’s a regulatory requirement that is going to require encrypted data. Before the information and application architects spend a bunch of time and effort designing encryption solutions, the technology architect needs to be able to point out that the storage solution you’re already using automatically encrypts data at rest. Architecture is a team effort.

Two kinds of people – deep focus versus broad focus

There are two kinds of people in this world, those that have demonstrated deep mastery of one particular subject area, and those that demonstrate a broad understanding of the bigger picture.

Most of us have worked at companies where tenure plays a larger part than performance when determining promotions and raises and, assuming we were the ones with lack of tenure, we complained bitterly about this. It seems obvious to all that the person who gets promoted to the level four job description should be the one who’s doing level-four work, not necessarily the one who’s been there 15 years. If you’re trying to attract and retain the best people, not just the oldest, then promotions should be based on ability, not tenure.

The problem is that there are lots of different abilities. When you hire a developer fresh out of school, pretty much all they can really do is code subroutines to specifications you give them. As their coding skills grow stronger, they start to develop extended skills. You can give them specifications at a higher level, and they can figure out the subroutines they need to orchestrate. Given even more time, most developers will take on other challenges, such as the ability to work with the business to get their own requirements, or developing written communication skills, or bettering their understanding of the business model or their understanding of emerging software development tools and techniques.

Becoming a better software developer isn’t increasing mastery of one “software development” skill; it’s mastery of more and more related skills. There’s always another skill to master, another mountain to climb. No one climbs them all, and no one climbs them in the same order or to the same height. Some people can’t manage the public speaking mountain. Some hit the wall on project management.

Fortunately, there’s a lot of room in software development for people with different strengths, as long as their weaknesses aren’t critical to their job role.

Unfortunately, I often see cases where a developer, based on exceptional skills in one area, is promoted to a position that requires other strengths. There seems to be an assumption that the natural progression of any developer’s career is to eventually graduate into management of a team of developers, or a position in strategic architecture.

A manager needs to understand the job that they are managing, and an architect needs to understand the process they are architecting, but having come up through the ranks isn’t enough, by itself, to justify a promotion to those positions. Architecture and manager roles require the candidate to master additional skills. There comes a point in the career of a developer when they ask, “Do I want to climb higher on these skills I use today, or do I want to start climbing new skill-slopes?” They have to make the decision of whether to developer deeper development skills or widen their skillset to include those skills necessary to another role. In ten years’ time, would they rather be a ninja level coder or a mid-level manager or an architect?

Different developers are going to make different choices. Your best programmer may really rather continue to sit in their cube getting better and better at what they do, and have no interest in giving that up to jump to another career track. This is why your best developers may actually make incredibly poor managers of developers, and really struggle if pushed into an architect position. You have to make all of those options available as viable career tracks. Don’t top out your developer pay classes so that great developers have to give up development if they want a raise. Don’t promote purely because of tenure, but if a developer continues to improve, providing increasing value to the company as a developer, that should be a reasonable career option.

Both manager and architect positions require strategic thinking, communications skills, and leadership ability to degrees that are not really all that necessary to a developer. In order to jump tracks into management or architecture, you need to demonstrate some mastery of skills that weren’t really all that important to you as a developer.

I think most companies have figured out that jumping tracks into management requires new skills, although I’ve certainly seen some questionable internal promotions there, too. I think it’s far more common for a company to think of an architect as simply the next level of programmer (or data modeler, or technician), a continued mastery of the same skills that have gotten you where you are today. The architect role isn’t yet common enough for companies to see the patterns of their promotion decisions emerging. Most times, when these architects fail, they bring down the EAG program with them, and companies believe that the EAG program itself, not the staffing, was the problem.

When I’m looking for an EAG architect, I look for people who have developed a specific set of skills:

• They are already demonstrating experience and skill at managing processes in one or more domains. Don’t make an EAG architect responsible for setting up architecture and governance over IT processes they have never mastered in the real world.
• They are constantly saying “if they had only asked me, what we should have done was…” Not in a derogatory way, but in a way that demonstrates insight into the larger picture and an understanding of long-term strategy.
• They always have a well-reasoned opinion, but are willing to listen to opposing viewpoints and be persuaded. They have strong personalities, but they never get caught up in dominance games, wanting to be the one who won the argument, rather than wanting the company to win.
• They are always thinking big-picture, always trying to put things in the larger context. They are the ones with the reputation for addressing a problem by saying, “The way I see it, we have three options…,” listing the pros and cons of each, and ending by naming their preferred option. They don’t jump on one solution; they see the larger context, thinking through all possible approaches before picking the best.
• They are always taking every opportunity to explain things clearly, with easy to understand pictures. They are always drawing on the whiteboard. The have a knack for making complex ideas easier to understand.
• They clearly understand the business model, not just IT, and understand that IT exists to support the business, not the other way around.

Perhaps the worst mistake I’ve ever made personally in recommending a person for an architecture role was hiring a person we’ll call Andy. Andy was an outside hire who had been a consultant for more than a decade, and had a fantastic resume that checked all the boxes. Not only did he have years of experience with the necessary IT skills, he had that experience in our industry, working at companies just like ours on projects just like the ones we were staffing up for. He only worked a couple of years at each location, but he was a consultant, and people don’t stick with one job their whole careers as they used to, so we didn’t count that against him. We confirmed his work history, and offered him a job.

Big mistake! The question we didn’t ask was what part of the project lifecycle he worked on at all those positions. It turned out that his whole career was based on coming in during the planning phase, setting up “the way things ought to be,” and handing it over to someone else to implement. Despite twelve years of experience in the position we were hiring for, he had never had to implement his conceptual architecture and governance, and deal with them in the real world.

You don’t want to hire EAG architects who’s only experience is being an architect. Make sure you hire architects who have actually been there and done that in the real world, and understand what works and what doesn’t. They need to understand that we aren’t building a perfect architecture to hand in to a professor as an assignment in a college class. We are building an EAG program to support a business in the real world, where things are not perfect, and we don’t have all the time and all the money to make them perfect. We have to create architecture and governance that can work in that environment.

Hardware lifecycle functions

As I’ve said before, your hardware lifecycle functions are undoubtedly the most mature architecture and governance in your company. The other domains have much more work to do before they reach that same level of maturity. The problem with the hardware domain is that it has become the victim of its own success.

The hardware domain has existed in a silo for decades, making decisions on their own, without consulting other domains. Why? Because at the enterprise level, there were no other domains to consult with. The main thing you’re going to have to do with your hardware IT functions is to work on integrating them with the emerging enterprise-level architecture and governance of the other domains.

The goal of the hardware domain isn’t to fill the computer room with the latest, shiniest, fastest devices on the market. The goal of the hardware domain is to support the needs of the business as efficiently as possible. This shouldn’t be a Field of Dreams, “build it, and they will come” approach. The purpose of the hardware is to support the applications that support the data that supports the business.

Think again about that Charles Dickens bookkeeper sitting on a high stool all day, entering and totaling numbers in a ledger book with a quill pen. This is how people ran business for centuries before the advent of computers. Computers were adopted by businesses solely to automate the retention and manipulation of business information. They aren’t an end to themselves. Computers exist in the data center only to automate those quill-pen business processes and let Bob Cratchit go home for Christmas.

New IT skills had to be brought into the business to support all the new hardware and software that was purchased to automate the business processes, but, over time, it seems we IT people have gotten so focused on the hardware and software that we’ve forgotten about Bob. While we sit in the warm computer room admiring the shiny new hardware, Bob Cratchit is back at his cold desk trying to get his job done, and Tiny Tim is home waiting.

Old Ebenezer Scrooge was very good at his job, but he had forgotten the true meaning of Christmas. If you want to integrate your technology management into a larger EAG program, you’re going to have to remember the true meaning of Information Technology.

Hardware acquisition

The first function in the technology lifecycle, the first ghost to visit old Hardware Scrooge, is hardware acquisition. Hardware acquisition is much the same as application acquisition (page 181). It’s a complicated subject, with intricate vendor pricing discounts and bundling, complex integration requirements, and a deep, specialized understanding of rapidly changing technology. The main message from the ghost of IT past is that you re-examine the way you collect requirements. The main modification you’re going to need to make to a functioning hardware acquisition process is to integrate the business, information, and application domains into the mix. Those are your customers. They don’t exist so that you can buy hardware. You buy hardware to support them! If you think you don’t have to involve those groups in your planning and purchasing decisions, then you are definitely not treating them as your customer.

It’s going to be difficult, politically, to stop the technology tail from wagging the EAG dog. They key is for everyone to understand that the technology team is not being put in their place. You have to approach this from the standpoint of opening up the doors to one of the most mature processes in your infrastructure, so that, rather than pushing the technology domain down, you are pulling the other domains up, and integrating them into something even broader in scope, leveraging the technology team’s process maturity as a platform on which to model and integrate the other EAG domains.

Hardware licensing

Hardware licensing is a form of asset inventory. It needs to integrate with the inventory of your information and application assets. There are relationships between these different domain assets, and capturing those relationships is just as important as capturing the assets themselves. You need to know which information assets and which application assets are associated with which technology assets.

Asset inventory is usually accomplished these days through applications: network scanners and software agents running on the hardware. Those applications generate triggers and alerts, and have business logic that must be configured. The application domain will have a great deal to say about the architecture of these applications.

All the asset inventory information will be collected in some central repository, integrating information, application, and technology assets and their relationships in one data store, exposed through data services, with an audit trail, information security, quality control, analytics, and information delivery. Contract information from the contract management system will also need to be integrated into this enterprise asset inventory repository, so that contractual license compliance can be measured. This centralized enterprise asset inventory repository will be architected and governed by the information domain.

No one domain exists in isolation. Decisions that affect the enterprise have to be made at the enterprise level. Enterprise architecture and governance must have equal involvement from all domains.

Hardware security

Hardware security is a bit of a misnomer. There is no need to secure hardware. What you secure is the information that resides on, flows through, or is gained access through the hardware. Hardware security exists to protect information.

The main thing that needs to happen in the hardware security function to make it part of an integrated EAG program is to coordinate with the other domain security initiatives to understand what information sits on and flows through the hardware, and what applications on the hardware provide access to or alter that information. You cannot implement a security program over technology components without understanding the relationships between each piece of technology and the specific information it protects.

Hardware security is critically important. Just make sure you understand what you’re securing, and why.

Hardware standup, configuration, and maintenance

There are quite a few other technology domain functions, all of which are quite mature in most organizations. The main areas to examine when integrating these functions into an EAG program are the ones that have information and application components.

Hardware maintenance, for example, can be due to security issues in the microcode on the hardware. You need to make sure the relevant microcode release levels are accurately and automatically collected in your asset inventory information by the asset inventory applications, so that the extent of risk across the enterprise can be measured. Again, you are not measuring the number of hardware platforms that are at risk, you are measuring the extent of business information residing on or flowing through those platforms that is at risk.

Older hardware release levels will eventually no longer be supported by the manufacturer. As with security, this release level and expiration date needs to be tracked in the asset inventory system, so that the hardware supporting the business is always supported in turn by the manufacturer, assuming always that the risk to the business is outweighed by the cost of support. Whether to keep hardware under vendor support or not is a business decision, not an IT decision.

Many aspects of hardware configuration are actually a configuration of application code embedded in the technology. It’s quite common for the technology domain to own this embedded software, but when that is the case, they need to carefully consider how much of the application domain software ownership lifecycle architecture and governance needs to apply, and the lines between software owned by the application domain and software owned by the technology domain need to be clearly defined and agreed upon. For many appliance solutions, this line is going to be quite fuzzy. It is the job of an EAG program to clarify that line. The functional framework offers a good platform for that clarification.

Hardware tuning

Hardware tuning is a vast, extremely complicated subject that changes rapidly as new technologies are constantly introduced. No one person can master all aspects of this subject. I wouldn’t even consider myself to know enough to qualify as a summer intern. I have great respect for the men and women who do this well. My hat’s off to them.

From the standpoint of an integrated, cross-functional enterprise-level program to manage IT complexity, the important thing to remember is that hardware tuning can’t be done in isolation. No matter how much money you have to throw at the solution, there’s no one hardware solution that’s going to be a silver-bullet for every possible workload. Hardware selection and tuning must be performed with the information and application requirements in mind. You can’t just tune hardware; you have to tune it for some purpose. That understanding seems to be missing within many organizations today. Different business purposes will require different tuning even if they use the same hardware components.

Still without claiming to be an expert, I want to call out a few hardware selection/tuning areas that seem to consistently come up as challenges. In every case, the issues arise when hardware selection and tuning is performed in isolation, without waiting for or taking the time to understand the nature of the workload the hardware will be asked to serve.

Tune the entire stack, not just CPU and memory

Proper selection, configuration, and monitoring of the hardware stack are crucial to the success of an organization. Many organizations stand up all new equipment using the factory recommended infrastructure including CPU, memory, and network cards. When business function performance degrades, they mainly look at CPU and memory. Rarely is anything else considered until those two are first maxed out. Every three years or so, they will budget for a hardware refresh that basically buys the latest version of the equipment you already have, with no real analysis of how well that stack is working. With even a little bit of thought, it’s clear that this isn’t the right approach. Nevertheless, in my experience this is the way most organizations work. There are two problems with this approach.

The first problem is that there’s a lot more to the stack than memory and CPU. Typically, the application software and the application database will be implemented on separate, dedicated hardware servers. In this case, your stack will include things like:

• Application Server
  • CPU
  • Memory
  • Ram Disk
  • Application Software configuration
  • Security/encryption software
  • Number and throughput of network cards
• IP Network
  • Cabling
  • Routers
  • Firewalls
• DB Server
  • Number and throughput of network cards
  • CPU
  • Memory
  • Configuration of database software
  • Number and throughput of Host Bus Adapter (HBA) cards (to connect to SAN network)
• SAN Network
  • Fiber channels
  • Switches
• Disk
  • Disk controller cards
  • Caching
  • Number and size of disks
  • Read and seek time of disk

The throughput of the business processes you are serving depends on all of these factors and more. A bottleneck with any component limits the throughput of the entire stack. You need to be able to proactively monitor the entire stack, so that you know ahead of time where you’re approaching capacity, and so that, in the event of a problem, you have the tools in place to immediately identify the bottleneck. How many times have you experienced degraded performance, only to be told that the CPU and memory utilization did not show them maxed out? Why don’t we have the tools in place to proactively monitor the rest of the stack?

In many cases, software licensing is based on CPU specifications. If you keep throwing CPU at a problem, it can have significant impact on your software license costs, when the CPU may, in fact, not even be the problem.

Know whether you are tuning for TPS or MBPS

Say you purchase a new database server. Odds are that you’ll configure it with the RAM, CPU, network cards, and HBA cards that come recommended by the factory, or perhaps by the database software. However, that isn’t always appropriate.

By far, most database installs are intended for operational applications. These applications have an activity footprint that consists of a very high volume of very small transactions. This is practically the definition of Online Transaction Processing (OLTP). Since most databases will be set up to support this type of activity, the configuration recommendations for the hardware and software will be designed to maximize the transactions per second (TPS) throughput. You would look at each layer of your hardware stack, above, and determine the TPS it will support, making sure there are no bottlenecks.

However, sometimes the database will need to support Online Analytical Processing (OLAP), with a small number of very large transactions. Optimizing the stack for TPS is not appropriate in this case. Rather than look at the TPS specification for a component, you need to look at the megabytes per second (MBPS) each component in the stack supports, and make sure that the appropriate capacity is supported throughout the stack.

One advantage of purchasing an analytical appliance solution such as Teradata or Netezza is that it is a stack-in-a-box, pre-tuned for analytical of access. If you’re going to build the stack yourself from components, you must understand how to configure and monitor it for the type of activity you want to support. You can’t just tune hardware; you have to tune hardware for a particular application activity footprint.

SMP versus MPP storage

Databases can also be purchased in SMP or MPP configurations. Some, such as DB2, can be configured using either technology, depending on your need. What need is better served by each technology?

SMP, or Symmetrical Multi Processing, takes the approach of giving every CPU access to all of the data in the repository. This is also known as a shared-all configuration, because access to all of the information is shared by all of the processors. Technically, it’s the underlying hardware that is SMP, but the software that runs on that hardware must be written in such a way as to take advantage of it. Oracle, Sybase, SQL Server, and other databases are examples of database applications written to take advantage of SMP hardware, distributing workload into threads that can run on different processors, all accessing the entire data store.

MPP, or Massively Parallel Processing, takes the approach of distributing large workloads to many different processors as well, but in this case the processors (or groups of processors) are truly independent, each working on a separate slice of the data in what is known as a shared-none configuration. In a MPP enabled database application, the data is hashed into as many slices as there are independent hardware nodes. Each node works only on the slice of data it can see. The results are passed up to an administration node, which integrates the data and returns it to the requestor.

Both SMP and MPP are powerful solutions, each of which is more appropriate for certain kinds of activity.

When speaking of database solutions, MPP is most powerful when the data can be sliced and distributed in such a way that the query will find all the information that must be joined together all on the same node. Back in 2005, I worked on a call-detail data mart for a large telecommunications company that loaded 4.5 terabytes of data daily, maintained a full 60 days of history, and required a response time of less than 5 seconds to any query. This database was used to monitor call activity for fraud, and to alter the switching paths between cell towers to minimize cost of routing calls from tower to tower. Because the call detail data was basically one large table, it could be hashed in such a way that the queries would run very, very efficiently. The queries never needed to join detail data that was on one node with data from another node. This data mart was credited with saving the company more than $8 million in expenses each month. When the queries and the data allow the workload to be split in such a way as to be run in isolation on each node, MPP is extremely scalable, with very little loss of efficiency. If you want the solution to run twice as fast, you can simply double the number of nodes.

However, there are times when the workload can’t be split this way. The example above worked well because it was one large table. If you have several large tables, but they all have some field that ties them together, you can still hash them on that unifying variable so that all the rows in each table that belong together to be co-located on the same node, and still run efficiently. Order header and order line could both be hashed on the order number. A claim header, claim line, adjustments to the line, and the diagnosis and procedure codes that go with each line, can all be hashed on the claim key so that all the information for a single claim, across all of these tables, is always located on the same node.

But if you need to join those claim tables to enrollment tables and provider tables, it becomes impossible to hash everything on one common hash key across all the tables so that the provider and the enrollment and the claim information that go together are always guaranteed to be located on the same node. In this case, the MPP use-case falls apart rapidly. The MPP engine has to analyze the query, and pick one or more tables and collect all the detail information for those tables together in one place, either on the administration node, or, worse, place a complete copy of the table on each node. Suddenly, you are spending a great deal of time shuffling data around, then running almost the full workload on each of the smaller nodes.

MPP is a powerful solution, but only when you can guarantee co-location. If you can’t, then SMP is probably going to work better. Guaranteeing co-location involves either a very simplistic data model, or being able to guarantee ahead of time the exact nature of the queries. If you have ad-hoc queries running over a complex data model composed of many large tables that cannot all be collocated, MPP is not going to work well for you. Unfortunately, one of the main applications people use MPP solutions for these days is data warehousing. MPP can be an excellent choice for analytical reporting data marts, but is usually a poor choice for ad-hoc reporting in the operationally modeled layer of a warehouse.

Big data platforms are, in many ways, very similar to MPP architecture, because they distribute data across an array of independent commodity servers. They work best when the workload can be split into many different units of work that can run in isolation using the data on each node of the big data array. However, if the workload involves joining data that isn’t co-located on a single node, big data suffers the same challenges as MPP architecture. Some NoSQL implementations, such as MongoDB, also use an array of commodity servers and share the same limitations.

You can’t select the most appropriate hardware (technology domain) without a good understanding of the data itself (information domain) and the kinds of queries that will be used to access it (application domain). Appropriate hardware selection requires involvement of all three IT domains.

Row-based versus column-based storage

Another area where inappropriate decisions are often made because the workload isn’t well understood is row versus columnar database engines.

Traditionally, database engines have used a row-based storage approach. If you were to examine the data on the database disk, you would see each database row stored all together in one place. If you examined each block of data (the unit of I/O used for database disk reads and writes) you would see a number of rows from a database table, stored back to back, with a unused free area at the end of the block typically sized to allow the insertion of several additional complete records in that block before requiring the block to be split in two. If the table has a large number of fields, then the block may actually only have a handful of records. I can recall one instance where the most used table in an application was so wide that only seventeen records were stored in each block.

A columnar database, on the other hand, stores data by column, not by row. While not precisely accurate, you can think of this as taking the data in a traditional table and splitting it into a number of tables, each with the primary key and one non-key field.

Now imagine a query that is summing the contents of one field across all the rows in a very large, very wide table. In a row-based table, each block you read will give you only a handful of records, of which only a very small fraction of the data is relevant to your query. Unfortunately all of that data has to be read from the disk, sent to the database application server, loaded into memory, and processed by the CPU just to extract the one field you want. In the one example above where each block contained only 17 rows, you would be reading 16K (the block sized that tablespace used, which is pretty typical), for each 17 rows.

In a columnar solution, that same block would contain an array of very small chunks of data, each containing only the information you are looking for. Instead of processing 17 rows per block, you’ll probably process closer to 1000 rows per block. If you’re joining data from multiple tables, this advantage is multiplied. Columnar solutions often run an average of 600 times faster than row-based solutions. Not 600 percent faster, but 600 times faster.

However, here again, columnar isn’t a perfect solution for every kind of workload. Imagine now a query that selected all the fields from that same table, a “SELECT *”. In a row based table, a single read pulls together all the fields, where, in a columnar solution, each of the non-key columns are instantiated separately, and must be joined together to produce the desired output. In this case, the columnar solution would take far longer than the row-based solution to retrieve the same data.

Columnar solutions work best when the query returns a small number of columns from a very wide table. If there are only a few rows, columnar won’t make much of a difference. If the table only has a few columns, then columnar won’t make much difference. When queries return most of the fields in the table, columnar is going to hurt, not help the performance. But given the right conditions, columnar-organized databases can produce amazing results, performing several orders of magnitude faster than row-based databases.

I went to an industry conference once where I sat in a breakout session that was a case study of a company that replaced the hardware infrastructure of the database for a major application with columnar technology. In their presentation, they showed sample query results both before and after the migration. They concluded their presentation saying they were disappointed to have only gotten a fourteen percent average increase in performance rather than the significant lift they had expected. Some queries actually ran much slower. However, the slides that showed the before and after performance were showing queries that selected a very large number of fields. I was very embarrassed for the presenters, who clearly chose a technology without understanding their application workload or the types of workload for which the technology was suited.

On the other hand, I worked on a project that produced a dimensional model for reporting through a high-end business intelligence tool. The fact and dimension tables in our star schema were quite large, but any one query would only return a small number of fields to be displayed on the screen. Because of pre-existing infrastructure, the platform where all of this was built was a row-based database with no failover or redundancy, with large batch windows where the data was unavailable. The dimensional data was created on this platform, and then copied to a smaller columnar database that had much higher availability. Our business analysts could point their BI tool to the row based solution to verify the data before it was moved to the columnar solution where our end users accessed it. The difference in performance between the row based solution (running on beefier hardware) and the columnar solution was almost exactly 600 times performance improvement. Because the access was all through a BI reporting tool that typically only brought a small number of columns to the screen, the query footprint for this database was ideally suited to a columnar database’s sweet spot.

Here again, this is a good example of why the hardware (technology domain) can’t be selected and tuned until the data (information domain) and query (application domain) requirements are understood. In an enterprise-wide, holistic IT management solution, the technology domain must be developed subject to the requirements of the workload it will serve. Technology decisions can no longer be made in a vacuum.

Analytical disk solutions

Disk performance and flexibility increases every year. We are getting to the point where you really don’t have to worry too much about disks at all, as they’re all a virtualized commodity.

Back in the day, for an analytical database disk array, we would purchase lots of small disks, and only use about half the tracks on each. This had the effect of minimizing seek time, the time it takes to move the read head from one track to another. We would dedicate the disk to the analytical database, because if another app was using the disk, then the read heads kept getting pulled off “our” data. We would deliberately locate data table spaces and index table spaces on different spindles, so that the disk did not have to seek back and forth between index and data. We would configure this stack for MBPS, optimized with caching for reads, because we do a lot more reading than writing, and seldom in mixed workloads. And so forth.

Our technology teams don’t do this much anymore. Nearly every part of that setup goes against the principles of SAN management. The modern trend is to have fewer, larger disks, to use 100% of the disk, and to share that disk virtually with many applications, not allowing them to control what data is on what disk.

I understand the advantages of SAN management, but I wonder if we are truly to the point where all the stuff we used to do back in the day is really irrelevant. Are our SAN management principles optimizing SAN cost at the expense of business analytics performance? I don’t know. Not my field. Moreover, I seem to create a lot of ill will whenever I ask those kinds of questions. It seems to me that if the technology truly had surpassed the need for all those things we used to do, as it someday must, then we could talk about it openly, and not treat the subject as the elephant in the middle of the room.

OK, maybe I’m a little bitter about that one…

Hardware end of life

A recent study47 by Forrester Research among IT leaders at more than 3,700 companies estimated that an average of 72% of a company’s total IT budget is spent on sustaining existing infrastructure, while just 28% is applied to new investments.

With our goal of efficiently managing IT complexity in support of the business, we need to consider whether or not we’re being good stewards of the company’s financial resources by upgrading every hardware device as soon as the vendor tells us to. Hardware vendors and grocery stores have a vested interest in keeping us terrified of expiration dates.

There are several things to keep in mind when discussing hardware end-of-life:

• Performance. If the hardware platform is not meeting the business service level needs, it may need to be replaced. In this case, the hardware may not be at the end of life – just the end of usefulness for its original purpose. The hardware may be repurposed for other solutions with lower demands, perhaps a lower environment (DEV, TEST) for the same application. Note that the decision of whether the hardware is fast enough or not is a business decision, not an IT decision. Hardware should not be replaced simply because there is a new, faster version if there is no business need.
• Support. You define end-of-life, not the vendor. Most vendors actually have two dates for end-of-life, one date for when they stop marketing and selling the product, and another date for when they stop supporting it. The difference between those dates is often as much as five years, and can sometimes be negotiated for longer at a rate far below the cost of upgrading hardware that is meeting your needs just fine. Even when the product has no vendor support, you still need to consider whether the risk of extended outage while a replacement is found is worth the cost of replacement. The information system the technology supports may be non-critical, or may be about to be replaced. This is a business decision.
• Security. In many cases, hardware needs to be replaced even if the performance is fine, because a security flaw has been uncovered that cannot be addressed with maintenance. These security flaws may relate to increased risk for information breach, or they may simply expose a risk for non-compliance with some regulatory standard. Depending on the data that the hardware exposes, this may need to be a priority replacement. However, if the hardware does not expose any sensitive data, there may be cases where the minimal risk exposure is outweighed by the cost of upgrading the hardware. This is a business decision, not an IT decision. Even if the security flaw is significant and disruptive, you may still be able to use the technology for a lower environment, if those environments use deidentified or test data.

We also need to consider the option of replacing an on-site solution with a hosted solution. Most cloud-based technology is managed by the third-party vendor, including the task and cost of keeping the solution current. If 72% of your non-payroll IT budget is spent on keeping the solution current, isn’t it worth at least exploring whether that function can be delegated to the external vendor?

Trend: The virtualization of hardware

The final spirit to visit old Ebenezer was the Ghost of Christmas Yet-to-Come. I think the spirit of hardware yet-to-come would show our Scrooge a vision of an empty data center, quiet and cold; a vision of a time when most of the hardware and network we support today is in the cloud; a vision of a day where the corporate firewall has fallen, and our corporate infrastructure spilled out into the web.

Hardware is increasingly a virtual commodity. Hardware virtualization within the corporate network (i.e. SAN arrays and virtual servers) is commonplace today. It’s so pervasive that it would be a very unusual datacenter that does not support a significant volume of virtual disk and virtual servers. You also find more and more hardware implemented as a third-party virtualized service. To the average user sitting at their desk, even a server in the corporate data center is, for all intents and purposes, a virtual implementation. The actual server is hidden away somewhere in a remote location where dedicated experts manage its existence. To the average user, there’s no real difference between a server in a local data center and a server in a cloud solution somewhere, as long as the performance, availability, and security meet the business need.

In 1950, Alan Turing developed a test to determine if a machine could exhibit intelligent human behavior. If a human used only a keyboard and monitor to conduct a conversation with a computer in another room, would the human be able to tell that the other side of the conversation wasn’t human? I’m suggesting that the increasingly connected world we live in has extended the Turing test in many interesting ways. If the business user is unable to distinguish whether the service they’re using is local or remote, can it be said that the remote service is exhibiting the character of a local service to the extent that it can be replaced? Are cloud-based hardware solutions one day going to completely replace the corporate data center?

It’s not just the data center hardware that’s moving farther away from the corporate network. The traditional network and workstation is leaving as well. Remote access through web portals and mobile devices is eating away at the need to stand up and support large farms of desktop workstations in cubicles. Telecommuting is on the rise. I once did some work for a company where about half the cubicles were shared. Five different employees would share each desktop, each coming in to the office one day a week and working from home the rest. There are some service industry jobs (e.g. firefighter, police) that aren’t going to be working remote any time soon, but many service roles (e.g. doctors) are increasingly able to work remotely. Thirty-four states now have laws in place setting guidelines for telemedicine.48 In many of those states, there’s no requirement for even one physical consultation before a doctor can begin treating the patient remotely. Mobile devices are investing billions of dollars to provide the accessories and protocols necessary to support this transition.

For most of us, the necessary technology to do our work remotely exists today. If we aren’t already working remotely, the roadblock is probably a matter of corporate preferences, not a matter of technology. As yesterday’s old-school executives are replaced by young men and women who grew up tethered to smart phones, corporate attitudes are going to shift, and shift quickly. Walls are going to fall, and the presentation layer of the corporate hardware infrastructure will quickly spill out into the web. Your organization may only be one retirement away from embracing large scale telecommuting.

Even five years ago, many companies were very reluctant to trust their infrastructure to the cloud. The cloud was viewed as very high risk from a security, scalability, performance, and availability standpoint. Any discussions in that direction were shut down quickly. Comparing that to today, the difference in attitude is nothing short of astounding. If the back-end goes to the cloud, and the front end goes to the web, what will be left in the managed corporate infrastructure in ten years?

In this new world, the amount of corporate hardware maintained on premise by IT to support the business is dwindling. I’m not suggesting you ignore the technology domain. You’ll be managing corporate hardware assets for many years to come. In addition, even cloud-based services require management on your part. Rather, my point is that you need to take a close look at your long-term strategic vision for your technology infrastructure, and begin putting the architecture and governance in place within your functional framework to manage this trend at an enterprise level.