Chapter 10. Technology Innovation

“Innovation is the central issue in economic prosperity.”

Michael Porter

“Innovation has nothing to do with how many R & D dollars you have. When Apple came up with the Mac, IBM was spending at least 100 times more on R & D. It’s not about money. It’s about the people you have, how you’re led, and how much you get it.”

Steve Jobs

“Because, you know, resilience—if you think of it in terms of the Gold Rush, then you’d be pretty depressed right now because the last nugget of gold would be gone. But the good thing is, with innovation, there isn’t a last nugget. Every new thing creates two new questions and two new opportunities.”

Jeff Bezos

“I believe in innovation and that the way you get innovation is you fund research and you learn the basic facts.”

Bill Gates

“Learning and innovation go hand in hand. The arrogance of success is to think that what you did yesterday will be sufficient for tomorrow.”

William Pollard

Have you ever been at a conference and attended a really cool presentation? It focused your attention on a new area you were not familiar with, and it captured your interest. Later in the week, you noticed more sessions that were focused on different perspectives of the same technology or approach.

This kind of clustering of conference presentations—or magazine articles or blog posts—is usually a signal that a new trend is emerging. It may not have wide appeal yet, and it may not emerge into anything concrete, but it sounds fascinating and seems to address a real issue you are dealing with. Your ability to recognize these types of new trends can open up opportunities for your business and enable you to be a leader in the implementation of a new technology within your organization. On the other hand, the new opportunity could lead you down a rabbit hole with technology that has little support or interest from the wider development community internally or externally.

This chapter unveils one of the essential skills needed by a software architect: the ability to identify, assess, and infuse new and potentially disruptive technologies in a business-centric fashion.

Technology Innovation Defined

Technology innovation is the ability to

• Look where research, technology, and your industry are heading

• Identify trends and opportunities that are aligned with your business’s strategic goals and objectives

• Partner with the business and customers to discover solutions

• Make recommendations about the adoption of new technologies into the organization and their timing

Technology innovation is linked to the theory of Moore’s Law, which predicts the doubling of computing hardware capacity every two years. Many digital devices are impacted by Moore’s Law as processing power, memory, and other technical capabilities improve over time, speeding exponential growth and technology innovation.

Technology innovation is best done in partnership with the business to align with the company’s strategies and goals. It also helps ensure that technologies are not selected simply for their “cool” factor (see Figure 10.1).

Figure 10.1. Technology innovation

Trend Awareness

Trend awareness is your window into where interesting technologies are heading. It provides you with a glimpse of the possible new disruptive technologies, business models, processes, and so on that can either give you a leg up on the competition or put you at a disadvantage.

The trend diagram in Figure 10.2 shows the typical peaks and valleys in popularity that new technologies tend to follow over time. Observing the popularity of technologies, business models, processes, and related areas can give you a good sense of what is happening in the industry at the macro scale. Companies such as Gartner, GigaOM, and Forrester Research typically gather and publish this kind of information on technology topics such as big data, mobile, gamification, and others as they start to gain critical mass and awareness.

Figure 10.2. Trend diagram

Your ability to quickly identify promising new technology trends and predict their eventual success or failure will help you avoid spending time and resources on those technologies that end up toiling in obscurity and eventual failure.

Areas of Trend Awareness

A wide variety of trends are potentially applicable to your architectural areas of responsibility and should be observed on a regular basis. These include the following:

• Industry. What are the economic conditions that affect your industry? Is it in a cycle of growth, stability, or decline? The economics that are pervasive in your industry can and will affect the degree to which you will be able to attract technology investment from the business.

• University research. What areas are universities researching that may be of interest to your business and lead to potential partnerships? Knowing what schools are doing in research areas that are of interest to your business partners can be a great source of new information and talent in the form of interns and new hires as well as an area for potential joint research projects.

• Open-source tools. What tools and technologies are gaining or declining in popularity and community support, especially those that have more business-friendly licensing arrangements? These are potential candidates for adoption if they are growing in popularity and community support or candidates for elimination or replacement if the community interest is declining.

• Conferences. Conferences can be a great way to get a quick glimpse of what the new and emerging “hot” technologies are. The presentations at these conferences can give you a jump start into background information about new technologies and some of the early experiments and results. This will help you become aware of which areas are working well and which areas are still rough around the edges and need more time and testing.

• Regulatory compliance. Being aware of the types of changes that regulatory bodies are considering can help prepare you to plan the aspects of the architecture that may need to remain flexible to account for new regulatory requirements.

• Hardware capabilities. Hardware is constantly changing. Taking the time to keep up with key vendors and their roadmaps can help you prepare for hardware that may enable better performance or less expensive solutions in the near future.

• Cloud capabilities. The area of cloud computing is evolving quickly. Understanding the capabilities provided by cloud vendors can enable elastic growth with little or no effort. The key challenges are to understand the costs and data lock-in that may be associated with the varying solutions.

• Customers’ competitive knowledge. One of the best ways to keep up with your competitors is to closely partner with your customers. They can be an excellent source of information about what your competitors are up to and what they are promoting as strategic advantages and differentiators.

• Vendor capabilities. A large number of vendors make capabilities available for a price. Some vendors are leaders in their area of expertise. They often have a solid understanding of what others are doing and what is working well and what is not. Working with these types of vendors can be a great partnership. The key is to ensure that the value being added by the vendor is greater than the cost of using their products.

• Intellectual property. Working with your intellectual property counsel or in-house general counsel can help you keep up to speed on areas where the business may want to go after the development of patents or other intellectual property.

• E-mail alerts. Many online sources allow you to set alerts based on keywords. When you hear of a new technology or development, you can put an alert on it using the keywords associated with the technology. Then when there are new updates or recent news, you’ll be one of the first to know, which will help you consistently monitor the space.

Applying Trend Awareness

Trending information can be used as a source of guidance when you are trying to determine if a particular area is worth diving into in more detail. It can serve as an early warning that there is a technology-related change in direction that could impact the business, it can serve as an alert that there is a new opportunity for your business, or it may just serve as a confirmation that there is no cause for concern in this area, at least for now.

Remember that the rate of change is exponential, and you should expect that things will change quickly in a given technology focus area. Your ability to check in on a regular and frequent basis will help you stay on top of the trends that signal technological changes that could impact your industry and disrupt your business.

Business Alignment

Business alignment is a critical success factor to technology innovation.

Paying Attention to Trends on Customer Inquiries

If you are receiving a fair number of inquiries from customers about a particular new trend, it may well be worth your time to consider investing in this area. It is quite possible that they are using this new capability as a differentiator and a means of comparing and contrasting different solutions.

Although the capability may be not be a primary use case for what they want to use the product for and it may not even be used that often, it is something they can put on a checklist and use to make a decision.

Such items can help them gain support and excitement within their organization about why using the product they are considering is a good choice. It can help them justify the cost for the change or the purchase to begin with.

Getting Customer Feedback

Your customers are one of your best sources of information. There are different kinds of customers:

• Fans. They always have great things to say. Listen for patterns. The key is to find out what is behind the patterns. What is driving their adoption, and what maintains their enthusiasm for your products?

• Utilitarians. They are using your product because it is convenient, it is easy, it is practical, and it makes financial sense. The key here is to understand the key value proposition that keeps them coming back.

• Captives. They are using your product because they have to. Either they are not the decision makers about buying your product or there are no competing products in the marketplace. The key here is to understand who drives the purchasing decisions and what influences them.

• Followers. They are using your product because others said it was a great product. The recommendation can come from many different sources; it could be from a friend, or it could come from online comments. The key here is to find out what people are recommending and why. This is your real value; it’s an endorsement. In a similar fashion, hearing the negative things others are saying about your product can give you insights into what you need to improve.

No matter who your customers are, you want to make it easy for them to provide feedback. If you use a survey, keep it short, provide some multiple-choice questions, and provide a place for them to type in their comments in their own words. They may praise your product or roast your product, but getting a wide variety of feedback from your customers can give you insights into where you need to go and what improvements you should pursue. It may be as simple as adding link on the product’s website to solicit ways to improve the product.

Analyzing Customer Feedback

Once you get the customer feedback, ensure that internally everyone has access to the information, whether it is good or bad. The more access everyone has to this information, the more it can help influence their daily decisions about the work they do (see Figure 10.3).

Figure 10.3. Customer trend analysis: customer feedback can give you investment insights.

Look for common patterns in what is being said. You are looking to tell a story that can be used to help understand how your product is doing in the marketplace and help drive where investments should be made.

When to Be Cautious about Trends

There are a variety of situations when following or looking to follow new trends is simply not a good idea. For example:

• There is a great new technology and you want to reimplement a system that already works and that customers love.

• You are coming to the close of a major release and the team is working to stabilize the system.

• Your current system is already experiencing significant operational challenges, and the root cause is not known.

• Switching costs outweigh any perceived or realized gains.

One of your major goals as a technology architect is to manage risk, not introduce it. You need to ensure that you have adequate time to recover from destabilizing the system after the new technologies, approaches, or patterns have been introduced. As the idiom goes, “Discretion is the better part of valor.”

The last thing you want to do is harm your reputation unnecessarily or be viewed as reckless by the executives who are overseeing the work that you do. To mitigate this risk, always go in with your eyes wide open and be willing to listen to other people’s perspectives, whether positive or negative. Seek to understand their concerns and make sure you’re considering feedback from all levels.

Any change is difficult, and you don’t want to make it harder by pushing something your company and its people or customers aren’t ready for. Patience and persistence are valuable skills at this time. If you are right, as mass awareness grows eventually people will come around to seeing the value of your approach and recommendations. Remember that being too early can be just as bad as being too late. Timing is everything.

When to Embrace a Trend

There are a variety of situations that naturally lend themselves to exploring trends and bringing new and innovative approaches to the problems that need to be solved. These include the following:

• The business wants to do a technology refresh and modernize the technology stack for an important application or system.

• The business wants to establish a presence in a new market with a new set of applications or systems.

• The current set of technologies, approaches, and processes has been determined to be inadequate for solving the problems at hand, and a new approach is not only warranted but welcomed by everyone involved.

If you seek to introduce new ideas at the appropriate times, you will gain the trust of the business that you can take a pragmatic approach to moving the business forward. This form of partnership will serve you well.

Strategic Research

“Research is formalized curiosity. It is poking and prying with a purpose.”

Zora Neale Hurston

At any given time there are only a limited number of research projects or areas that can be undertaken. Several factors should be considered when determining what areas are the ones that should be explored. These include

• Alignment with strategic business goals. Research should be focused on an area that both the business and technology need to learn about in order to be able to move forward with a particular set of projects or areas that the business wants to pursue or invest in.

• Agreed-upon prioritization. Both technology and the business should agree that this area of research is needed, that the limited funding available for this type of research would be well spent, and that this is a high priority for both areas.

• Agreed-upon purpose. Both technology and the business should be in agreement about the purpose and desired outcomes of the research. Is it to determine the feasibility of pursuing a particular area? Is it to reduce the risk? Is it to help appropriately size the investment request? Is it to better understand the threat that this technology poses to the business? Is it to better understand the opportunities a particular area may offer? Are there legal and regulatory issues that need to be fleshed out?

• Agreed-upon time and resource allocation. Both technology and the business should agree upon a time box that this research will fit into. Ideally, this is measured in days. The goal should be to do the research in a cost-effective manner, and to the greatest degree possible limit the amount of time, money, and resources being used to pursue it. Are there other areas in the business with which you can share the cost of this research if it turns out to be a larger endeavor?

• Agreed-upon report-out mechanism. As the research is pursued, how will the progress and lessons learned be reported out to the business? Are there natural points at which these report-outs can help determine if this area of research is worth continuing to pursue? Are there other areas of the business that would be interested in the outcomes of the research? Can you post these findings in an internally accessible collaboration area to let others know how the research is progressing? If necessary, can you control who has access to the research findings, depending on the nature of the research?

The key is to partner with the business to determine the parameters that surround strategic research.

Research Approaches

Several different approaches to pursuing research can be taken, such as the following:

• Business unit driven. This research is done internally within a business unit. The resources working on it typically are not devoted 100% to research.

• R & D. This research is done by a dedicated research and development team. It is often staffed with resources who have PhDs in specific areas of interest to the business. They also are typically actively involved in the broader academic research community.

• University partnership. This research is done in partnership with a university. It allows the university to get exposure to challenging business problems and information that is generally not publicly available, and the business gets access to professors and students who specialize in a particular area of interest.

• Open innovation challenges. This is a research challenge that typically is constructed in partnership with a company such as InnoCentive. Working with you and your team, they will help frame the research problem or question into a challenge. They will also handle any intellectual property treatments and any awards for the winners of the challenge. The challenge itself can be open to the public or open only to those within your business. A successful challenge typically results in a crowdsourced solution to a specific business problem.

• Corporate innovation labs. These are effectively start-up groups that are formed within the overall business. They focus on rapid experimentation and validated learning through interacting with real customers. They often employ A/B testing to help drive decisions about which alternative solution is working better.

The goal is to pick the approach or mix of approaches that makes the most sense for your business situation. The key is to find ways to inject new ideas and approaches into the business that will enable it to grow.

Technology Innovation Principles

Technology innovation has the ability to bring disruptive technologies to the business. The challenge is to bring these technologies forward at the right time and in partnership with the business to enable the best possible chances for success.

Seek Approved but Minimal Time and Funding to Explore

Ideally, you should be able to get a limited amount of business-approved work time to explore new technologies and trends. This may take the form of

• Small research projects

• Exploratory prototypes

• Reading research papers

• Attending conferences that are focused on the area of business interest

• Reading blogs, magazines, or books that may enable new capabilities

Citing successful and innovative companies such as Google, 3M, and HP that allow employees “innovation time” to focus on and research new areas may also help in your efforts to obtain approval.

Ideally, your research request would be done in conjunction with your business partners in an area that

• They are looking to move into

• Is currently extremely hard for them to solve

• Requires a significant amount of manual effort to deal with

Make Small Bets

The goal is to survive to live another day. You don’t want the entire business riding on a single investment if you are unsure of its long-term future. Being prudent with these kinds of decisions allows you to fail quickly, with the least amount of disruption to the business, and learn how to improve.

Use Technology Scouting to Scan and Track the Trends Regularly

The goal with scouting technology trends is to learn about new things on a regular basis (see Figure 10.4) and find out

Figure 10.4. Trend scouting

• What is applicable to the business

• What things need to wait on the back burner while they mature

• What things warrant further investigation

Keep an eye on new and emerging trends using the scouting method of scanning the landscape to identify what’s new and interesting, then track those trends that look promising and apply them to your business. Scanning and tracking technology trends is of less value if the information obtained is not shared, so try to find a venue to share your findings and continue to make updates so your research is up-to-date and relevant.

Unless you are in some special circumstance, trends and trend investigation should not consume a large portion of your time. Major trends change less frequently—you may see the direction change every six to 12 months—while minor trends change more rapidly.

By quickly scanning an area of research interest and tracking any major changes, you’ll build up a knowledge base that will make your efforts more efficient the next time you scout the area. Catching these changes, whether the trends are gaining momentum or fading, gives you an indication of whether or not it is worth spending additional time and resources to learn more about a particular area and report out your findings.

Have a Lab Area

Having a lab environment can allow you play around with new technologies without a lot of visibility from other areas. For the areas in which you find success, you can bring them forward and have them progress toward a production environment.

Setting up a small lab typically doesn’t cost too much and allows you to do things that would scare the data center folks to no end.

Use Rapid Experimentation with User Feedback Loops

Experiments allow you to get firsthand knowledge of what is working or not working and a sense of what the real problems are. When experiments are combined with user feedback, you gain the ability to navigate toward customer needs. In many respects, this is similar to driving a car. You use the information that surrounds you to navigate toward your desired goal.

Show the Business and Customers Prototypes

As you have success with new technologies, show the business what is possible. A working example is worth an amazing amount to the business:

• It demonstrates initiative on your part.

• It reduces risk on the part of the business if it chooses to pursue the opportunity.

• It engages your mind in thinking of new ways and approaches to solving problems.

• Small innovations can grow into large enterprises and help sustain the business for the long haul.

• It engages the development team and shows them that working in your company will give them the opportunity to work on cool things and stay current with the industry.

Introduce New Technologies at the Edge

As you look to introduce new technologies into existing systems or even new systems, try to find areas near the edge of the systems that will have minimal impacts on the core (see Figure 10.5). This enables you to operationalize the new technology in a quiet manner.

Figure 10.5. Keeping trendy areas near the edge of your activities will minimize the overall risk of trying out new things and the visibility of failures.

If it fails, the impact will be relatively small and the executives will be less likely to take notice.

On the other hand, if you take out the core revenue-generating software for any period of time, you are likely to get a direct call from senior executives and have some very unpleasant meetings in which your judgment is questioned.

If this does occur:

• Be the first to approach the executives.

• Let them know what the situation is.

• Let them know what the possible resolutions are.

• Let them know what path you recommend.

You are much better off being proactive in this situation than reactive. It gives you an opportunity to be in control and to some degree manage the message that is delivered.

Pragmatic Technology Innovation

The key to pragmatic technology innovation is not so much what is here today; the key is to see what is emerging and anticipating the future needs of your business partners and their customers. Watching Gartner’s Hype Cycles, attending leading conferences, and following blogs can help give you a sense of where the industry investments are being made.

Take the time to play around with these technologies; if possible, find a low-profile project to test them on. The important point is to understand what problems are being solved or what business needs will be met.

How does this apply to your business? Think about these questions:

• Where is your business going?

• Where are your chief competitors going?

• Where are your competitors investing?

• Is the business landscape changing?

• Are there new emerging competitors?

• Is your customer base changing? Is it expanding? Shrinking? Aging?

• What is happening to your customers’ business model?

• Do you know why your customers’ business model is changing?

• What affects your market adoption?

• Do you need to be more aligned with trendy marketing and product appearance?

Taking the answers to these questions into account can help you navigate the tricky waters of what trends to be aware of, what trends to act upon, and what trends you should ignore.

Today, large sweeping trends are influencing technology. These include the following:

• Big data. The big data movement is transforming people’s thoughts about data analytics, data visualization, and content processing. Hadoop and MapReduce are becoming widely used technologies across all industries. An ever-growing set of technologies is integrating with the Hadoop Distributed File System (HDFS) and making access to big data ubiquitous. Your ability to understand customers and craft user-centric experiences has never been more important. Incorporating information life-cycle semantics into your architecture can enable significantly improved user experiences as your understanding of the users increases.

• Mobile. Expectations of mobile as a primary application access point are becoming commonplace. Understanding the user experience within your application as being lean forward (highly interactive) versus lean back (consuming content) is critical to ensuring that the user has a great experience. This understanding along with anticipating users’ tendencies to multitask can drive your architectural approaches for the best ways to deliver content.

• Networked platforms. With the rise of cloud computing and the proliferation of devices and sensors, platforms are slowly becoming more and more interconnected. Understanding the need to interact with users as well as systems can help drive the architectural layering and the secure API development that are needed to enable networked platforms.

The trends are constantly changing. The key is to be aware of them and to understand their ability to impact your areas of technology innovation. The trends tend to move relatively slowly, but your awareness of them and your ability to apply them appropriately to your business at the right time are critical to keeping your architecture current and relevant to the business.

Summary

The road to technology innovation begins with

• Being aware of the trends

• Aligning with the business

• Engaging in strategic research

• Using innovation principles

• Being a pragmatic technology innovator

Watching and following trends can be fun and exciting. They can easily distract you from the job at hand. However, they are also the eyes into the future that may provide opportunity or peril. Architects need to be familiar with the trends that are swirling around them and approach them with caution.

Technology innovation is a critical aspect of software architecture. Learning when and where to introduce new and potentially disruptive technologies into the business is essential for business growth and operational stability.

References

Blank, Steve. 2013. “Spotlight on Entrepreneurship: Why the Lean Start-Up Changes Everything.” Harvard Business Review, May.

Chesbrough, Henry William. 2005. Open Innovation: The New Imperative for Creating and Profiting from Technology. Harvard Business Review Press.

Davenport, Thomas H., and Jeanne G. Harris. 2007. Competing on Analytics: The New Science of Winning. Harvard Business Review Press.

Howe, Jeff. 2009. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Business.

Ries, Eric. 2011. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Press.