CHAPTER 18
Notes on the Actual Implementation
“There are no good surprises.”
—Pete, the guy who built my house
The actual deployment period for any project, including an IAM framework, can be an unhappy one for a couple of unavoidable reasons. First, it’s a lot of work. Your internal resources are being pulled from their regular duties. There are constant issues and interruptions. And of course, none of this effort is revenue-generating.
Second, there’s nothing to show for a while. You are putting effort in, and not seeing any results. Even if you pick those low-hanging fruit, like simple authentication or basic provisioning, it’s not instantaneous (although little stuff can be put in place within days). But you have made your plans and, more importantly, your commitments, and therefore you execute.
The point is to stick to that plan, and keep bad surprises to a minimum while you’re in the middle of building your IAM framework. Just remember that no matter what you do, you will never keep everybody happy. Things will come up. Bad things will happen. Factor them in and deal with them. Don’t try to hide things. And stick to the plan—until you can’t.
Keep People in the Loop (Sort Of)
If you tell everybody everything, you will regret it, guaranteed. Yes, that sounds blunt, but it’s true. Provide the information to the appropriate parties as necessary, or else you will invite questions and requests that will threaten to derail your timeline. Despite your best efforts at educating the organization as to the plan, there will be those who simply haven’t paid much attention, and once they start to comprehend what is being undertaken, they will suddenly want to get in on it. I have seen this in multiple projects.
When one of my companies bought another company, we were asked 19 times a day, “What’s going to happen with all those people you acquired? And what’s the roadmap look like? Which products are you going to keep? Can I get any of their stuff cheap now?” And so on. Many of us could speculate as to how it would all shake out, but we couldn’t say for sure, and even if we knew, we weren’t allowed to say.
My company put up a web page explaining the latest and greatest, along with all the usual disclaimers. It was terribly handy pointing outsiders to that page and explaining, “I can’t tell you, and even if I could, I don’t know anything. Go look at the web site.”
To keep the mob happy and off your back, or to at least avoid the endless redundant questions during the rollout, provide an internal document or site explaining the rollout, the plan, the effort, the expectations, and maybe, only maybe, some notion of the timeline. As with any large project, you can’t guarantee a precise delivery date. And whether or not you’re trying to be informative, an information page or wiki, or even simple HTML, will at least make it appear that you’re making the effort. Just make sure you’ve got the project plan well in hand before you do this, of course. Here’s why.
Near where I live, there’s a vacant lot in a very nice neighborhood with a sign proclaiming that soon they’ll be building a shopping plaza. That sign’s been there since 1988.
Years ago, a customer of mine put up a single web page about a coming project that became drastically delayed because of dueling budget initiatives. The page itself became a running joke, as the days and weeks dragged on and the project languished. Someone even put up a second page making fun of the first page. Now, if you advertise, and subsequently run into reasonable delays, it’s perfectly okay to post the why’s and wherefore’s, unless the dueling budget requesters decide to kill you.
You don’t need to expose every single glitch, otherwise you’ll be exposing something every other day. But neither do you expose all your successes. If you have something to show off, especially if it’s in keeping with the timeline, that’s fine. We’ve already discussed being able to demonstrate those low-hanging fruit to validate the effort to management. Showing progress is a good thing. If you wish to demonstrate basic authentication and a landing page, perhaps with SSO to at least one application, then by all means, show this to a couple of critical stakeholders. But don’t go crazy showing things off, since this might skew expectations. Yes, show progress. But don’t look cocky.
Governance
In the movie “Toys,” Robin Williams inherits his father’s toy factory, and an uncle (played by Michael Gambon of “The Singing Detective” and “Harry Potter”) comes in to run the place. Suddenly, the uncle has all sorts of secret projects going on that even the owners aren’t allowed to know about.
You think that can’t happen to you and your project? If you’ve got a huge implementation, that doesn’t mean you just hand over the keys to the integrator. I’ve seen this happen. This is how completed items don’t get unit-tested. It’s how deliverables slip. It’s how you suffer scope creep as your own staff starts communicating their requests to the integrators directly without review. It’s how the billable hours go through the roof. Because it’s so big and complex, and you’ve got your regular duties to perform, you stop asking questions or keeping up with the governance. Too bad. You asked for the project and the funding, so pay attention.
It’s not always a matter of not trusting people. It’s that stuff happens. Did you ever participate in one of those social engineering tests in school, where the teacher lines everybody up, then whispers something into the first kid’s ear and says, pass it on? The first kid whispers to the second, the second to the third, and so on. So it starts out as “my dog has fleas” and ends up as “Grandma’s made out of cheese.” Things get lost in translation over time. You might occasionally need to reset certain strategic or tactical goals. It’s almost like the reconciliation process we discussed a while back. When reality begins parting from policy, restate the policy and get everybody back on track.
Occasionally, when a project starts veering off for technical or resource reasons, the goal gets restated so as to match up with the actual results. This is changing the facts to fit the theory.
If you have multiple parties (lines of business, partners, corporate customers) involved with your project, either appoint or make them appoint one or more representatives who will serve as the messengers, troubleshooters, and blame-eaters for their respective organizations. They will be the ones with whom you sit down in a room when there’s an issue, and there will be an issue. Trawling-net meetings, where you pull every single body from every single party into one room, are not necessarily great ways to efficiently troubleshoot problems. Therefore you designate individuals from the stakeholder groups to represent the respective interests.
Set responsibilities. Stay involved. If there’s a dispute, do not let the different external parties try to resolve it without your direct intervention. It will take five times longer, and will not be properly documented. You need to mitigate any disagreements, and document them (for reasons we’ll go over shortly). This also keeps you aware of the dynamics between the parties.
A very large integrator took over a very large project after a previous integrator had failed. They began to show the customer how a more professional organization did things, including daily design reviews. Whenever a review was called, literally two dozen of the integrator’s personnel would leave their keyboards and crowd into a room. Individual projects and solutions were chronically behind because so many workers dropped their duties for an hour or more at a time. Whenever there was the slightest disagreement, the project lead from the integrator would say, “Everybody into the conference room.” |
As the software vendor’s representative, I attended several of these during onsite visits. After the last of three meetings on a particular day, during which several of the attendees spent more time checking messages or e-mail than actually contributing, one of my engineers asked the customer’s project manager, “Do you have any idea how much that meeting just cost you?” |
After some simple math, she found that this latest short gathering had cost just under $2000 U.S. That afternoon, she issued a memo specifying that from that day forward, only essential personnel would attend the reviews. A year later, another integrator ably took over the project, and with far fewer meetings. |
In this case, the customer shouldn’t have needed three consecutive integrators. But they handed the keys to somebody else and invested lots of trust but little governance. This is not to say that a third party will take advantage of you every time. I have worked with numerous integrators over the years who made my customers successful, and who very capably deployed my products. Just remember, “Trust, but verify.”
Mid-Stream Changes
Part of governance is ensuring that you stick to the plan. But here’s where I tell you I was lying. “Stick to the plan” sounds easy. You tell anyone who interrupts you with requests that are not part of the initial deployment that they will simply have to wait. But stuff happens.
During a years-long rollout at a global manufacturer, line of business owners who felt left out of the initial rollout kept petitioning for earlier inclusion in the framework. Their users and resources were not scheduled to be integrated for over a year, as systems deemed more critical were first addressed. But influence stemming from long-standing relationships allowed these parties to affect the schedule in their favor. Some of the requests made no sense from a business perspective, but they were put into priority anyway, and then the floodgates opened.
The project managers did the smart thing. With every one of these reprioritized requests, they prepared a document outlining
How the overall timeline would be affected
The impact on the other users and resources being bumped from the schedule
The realignment of internal resources as subject matter experts were juggled around
In fact, their most powerful argument against various reprioritized items was the need to reschedule internal experts. “We expected to pull the PeopleSoft team in October, but now we have to push that to January, and draft the LDAP guys.” These movements can have a ripple effect that involve other line of business owners.
The project managers made upper management sign off on all changes, and aid them in communicating changes to those departments who were getting bumped. In several cases, they were able to hold the line, based on the potential impact.
Since we’re talking about a framework, in which each piece helps lay the foundation for the next, the other item to consider is, what will a delay of any one component do to the overall goal? You can’t have cross-domain single sign-on until you have single domain single sign-on. You can’t have federation until you have authentication. There is no provisioning without an authoritative source.
Compliance is often a big driver for at least some of the functionality within a framework. But it’s only in the very last few years that audit requirements have been significant drivers of IAM. Even in the early years of SOX, it wasn’t a big consideration. These days, it’s common to see organizations putting in at least some aspect of IAM in order to pass their next scheduled audit. That is a powerful argument for staying on schedule.
Phased Deployment
As you had already long surmised, trying to service all users and all resources in the first round is completely impossible. Even if you have a vanilla environment (for example, all you provision to is database tables or LDAP groups), you still likely have lots of these vanilla targets to map out. Therefore you will be rolling out in segments.
As you do this, be sure to record your results. Some things will go smoothly, because of all your intense planning, and all the helpful suggestions you got from your committee members. But there will also be obstacles, unforeseen issues, last-minute requests, unexpected disagreements, and technical problems.
Don’t have too many individual projects going at once. Don’t let anyone rush you, unless they’re willing to learn how to help you expedite the timelines and take responsibility. In some cases, I’ve seen the need or desire to establish business application connections (for provisioning and/or SSO) prompt local admins to learn the process of creating connectors or adapters so that they could get their own departmental apps online sooner.
And before you move on to the next phase, make sure that the previous one is complete. This means tested and signed off on. As we discussed in Chapter 7, it helps to have agreement with individual departments or resource owners as to what “complete” means to them. Getting sign-off serves two purposes. First, you’ve satisfied the requirements, done the job. Second, you’ve covered yourself, allowing you to not only report that you’ve hit another milestone, but that you’re okay to move on to the next task in the framework construction timeline.
If audit support is one of your drivers, then obviously this should be out front in the schedule.
Cut-Overs
When the time comes to move a resource from its current state to a protected one, there are several things you can do to lessen the impact. These include having both old and new access methods available for a short period, in case anything strange does happen. You can also use the framework to provide both old and new access for a period as well, with the presumption being that once everything looks good, you turn off the old.
For example, a standard OOTB function of Oracle Access Manager is the ability to use secondary user stores for authentication. If you are deploying a new directory as part of your new framework, OAM can attempt to authenticate to the new directory and, failing that for unforeseen reasons, authenticate to additional directories. Fallback authentication methods can be employed as well. Once the kinks are worked out, and those secondary methods are no longer needed, the configuration can be changed to ignore them.
During a cut-over, it’s also advisable to monitor the performance of the components as they come online and begin to get hammered with traffic to see if there is a need to make adjustments in the architecture. As we discussed with regard to OAAM, you don’t always know how users will engage new resources in large numbers. Hopefully load testing has already helped you in validating your system sizing. In the next chapter, we’ll discuss the use of Oracle Enterprise Manager for monitoring all the components for a point solution, such as identity, access, and federation.
In Chapter 13, we discussed the Deployment Manager in Oracle Identity Manager. The purpose of this tool is to migrate configuration data between environments, such as between development, test, and production, and in doing so, minimize the most common issues that arise when moving from one server to another. The Deployment Manager governs the migration of the definitions involving:
Organizations
Resource, form, field, rule, and email definitions
Password and access policies
Lookups, tasks, and error codes
Adapters
When it’s time to cut over from test to production, Deployment Manager is a powerful tool with various functions designed to keep you from making mistakes during migration; but it should be used in accordance with very well documented best practices that are available from Oracle.
Training
Training should start taking place in the weeks prior to the actual launch. And training isn’t just teaching people how to use the software. It’s comprehending the procedures, the processes, and the options. Make sure everyone understands their responsibilities and their escalation procedures when things break, because they will.
If a partner is providing the training on a vendor’s product, make sure the vendor signs off on the content. Is the partner certified to provide this training? Or are they simply regurgitating the training that they themselves have previously attended?
Let’s poke a small hole in a commonly-used industry term: “train-the-trainer.” It’s the notion that, instead of training the entire IT department, you train some key people, and they’ll spread the wisdom around. And of course they’ll remember perfectly what they heard in class, and not miss a beat when they pass it on.
This works to a degree. But you will still need access to resources, human or otherwise, who truly know the material. Those trained trainers will have a clear head start, and will no doubt be of great use to the organization. But they will not be perfect resources once their initial training is done. That perfection only comes with time and experience.
Partners are occasionally guilty of misplaced optimism over train-the-trainer as well. And why are they in a hurry to do this? Because they’re getting paid for it, naturally. Now, this is not to say that integrators are not capable of delivering training on somebody else’s product. Oracle has many partners who know its products to an incredibly granular level, for both delivery and training purposes. Just verify that your software vendor signs off on the integrator’s ability to deliver that training.
If train-the-trainer is the way you’re going to proceed as well, consider performing that training in compartmental fashion. Instead of designating members of your staff to know the whole nine yards, designate target members instead to be subject matter experts (SMEs) on a limited set of features or functions so that they will have greater retention of their lesser scope, at least in the short term. Eventually everybody will absorb more content, but don’t let anything fall through the cracks because you made their brains too full right off the bat.
Multiple levels of training will be required. Administrators, end users, and non-employees will all require a transfer of knowledge. If you have an arrangement with customers or partners to manage their own users or policies via delegated administration, then they definitely need to share the burden of knowing how to perform these functions, or else you will be taking a lot of phone calls.
Developers will also require training. They will no longer be building security directly into their apps… probably. If you are making use of Oracle Entitlements Server for fine-grained entitlements within your code, then your developers may still be at least setting the table for security. However, what they are doing is simply providing the hooks for the framework to do its job, and OES will still be used for configuring, storing, and evaluating centralized policies. Developers will be opening up the pipe to the policy engine to make authorization decisions and provide responses. In most other cases, security and identity will be handled in the framework as a separate layer. Still, there are best practices for inserting these hooks into the code. Another point to make about Oracle Entitlements Server is that it is not enough to simply know the API. There is an entire design aspect to OES, regarding the definition of resources and rules, that well serves the framework. Therefore don’t simply point programmers to the manual and ask them to bang it out.
If you are following the excellent advice of not boiling the ocean, then you will be migrating additional user groups and/or resources into the framework. This means needing to build integrations in a way that is compliant, accurate, and well-performing. In the case of Oracle Identity Manager, this may entail learning the Adapter Factory, which is used for constructing connections to targets or sources for which a connector does not exist.
Rounding Up the Stragglers
During a discovery session at a public sector client, I was astounded at the number of side projects and barely documented applications within the organization. People were given lots of leeway for creating their own databases, web sites, and other assets that were barely or not-at-all secured. We quickly identified these areas as potential security disasters, since so many of these smaller databases were created from data extracts of corporate data. Our first observation was, the auditors will kill you. Our first suggestion was, grab these resource owners and make them get on board the identity platform.
You will be instructing any owners or developers of skunkworks projects on how to join the security framework. You won’t want them managing their own access, and they shouldn’t want to do it anyway. You are providing authentication (including SSO) and authorization as a service, simultaneously relieving them of the burden and preventing them from creating a security hole. Placing their security needs in the hands of the framework is both a benefit and a responsibility to these folks.
Lastly, training includes teaching users and administrators new procedures:
Troubleshooting Options are available for diagnosing and fixing problems.
Attestation Resource owners and managers will need to know what it means when they are called to perform this process, and how to do it.
Problem reporting Users and administrators will need to know what links, documents, and other resources are available when something breaks or is just plain confusing (because they weren’t paying attention to the first round of training).
Responding to notifications A raft of new messages will be showing up on screens and in mailboxes, and they’re there for a reason.
Self-service You can now do a whole bunch of things for yourself, so take advantage.
Make Sure Everybody Plays Nice
Your staff, your vendors, your consultants… they’re all your children. And I do mean children. If you haven’t seen one of these big projects before, you simply will not believe how childish, vindictive, or territorial the parties can become. Vendors want to get paid, but they want a happy, referenceable customer. Integrators want the same thing, although their costs are often not as fixed as the vendor’s. Your staff wants a successful deployment, but they also want to maintain control and remind the vendors and the integrators who the customer is, while simultaneously protecting their own jobs. You want all this done as quickly, efficiently, and cheaply as possible. Everybody wants to deflect blame when something, inevitably, goes wrong.
Remember the 57 authentication policies? Besides being a cautionary tale of how to abuse a product’s API, it’s also an example of how the customer raised an issue, and a consultant deflected blame and pointed at the product vendor. The result was a far longer road to resolution, and a damaged relationship between two parties vital to the success of the IAM project.
Sanity-check everything. Make sure any deployment partner is telling you what you need to hear, and not just enough to keep you happy so they can keep billing you. If it doesn’t sound right, or probable, or possible, then it probably isn’t.
And at all times, make sure that your staff is involved, and that they feel that they are involved. Don’t let them think that they’re only spectators, especially when there’s outside parties tracking mud on the carpet. Your staff will own this thing when it’s done. Don’t think that adult, professional system administrators can’t have their feelings hurt. Assign them duties that give them oversight. It’s a good thing to do, and a smart thing. A sense of ownership will give them an incentive to keep a watchful eye.
Control the Communications
You’re in charge, right? Even if you put a consultant or other partner in charge of the project, maintain connectivity to all parties. It’s common to have one consultant managing other consultants. They are literally managing others who compete with them. This means that any feedback you get on those underlings may be colored by opinion or what the managers want you to think or know. “Oh sure, they’re doing a good enough job, I guess, but when we get to the next phase, we might consider another party for the coding.” Uh-huh. I’ve seen consultants as well as vendors mess with each other in order to weasel their way more deeply into the customer’s affections. So even in situations where you’re allowing one group to manage the other groups, stick your head in once in a while and get everybody’s opinions on how it’s going, and do it in private. They might play nicely in front of you, but behind the scenes, watch out.
For each party involved, identify the head person, the one who will own an issue or problem. When in doubt, you hit up that person, and say, “Fix this.”
If it’s one of the auditing/consulting companies, you’ll be assigned a “partner,” which is an actual term they use. Think in terms of a law firm. When somebody hits a certain level, they’re a partner. The ones who will cost you the most, if they are part of the billable team, are the “senior partners.” But if you need somebody to jump and get something fixed, have that person’s phone number handy, because they likely have the influence to move things around in a hurry, and hopefully they have their lofty position because of their ability to make things happen.
Everybody Protects Everybody Else
While it’s a good assumption that everyone involved with an IAM project will keep their lips zipped about anything proprietary they learn, it’s always good to have it on paper.
A lot of times you simply don’t like to be used as a reference. Bear in mind that when you offer a reference for a vendor or integrator based on two things, a discount up front and a happy outcome on the back end, you will most assuredly receive reference calls. If you have not agreed up front to provide references, then you’re done. But that’s not to say that your name might not be used for selling purposes. If you absolutely don’t want a vendor or integrator to use your name under any circumstances, you need to specify that in the paperwork in advance.
If you thought there was a lot of information floating around during the sales process, that’s nothing compared to what’s available during implementation. Make sure everybody is comfortable with the protections in place during this phase, including those that keep implementation partners (who would normally be competing against each other) from each other’s throats.
Another important tip: If there are any issues, don’t blow them out of proportion. There will be multiple parties involved. If any one of them gets heavily flogged for a mistake, they can end up being a scapegoat for everyone else. Fix things and move on.
Educate the Masses
Training is one thing. But understanding the good citizenship that comes with the new framework is another. Think of this as an opportunity to coach your user community not just on software, but on their corporate responsibilities. Make sure they understand not only what you are launching, but what is expected of them. This entails two concepts:
Best practices This is not to say that everybody has to do things differently just because you’ve put a new system in place. In fact, the system should adjust to the business, not the other way around, although hopefully you will be affecting in a positive way everybody’s day-to-day activities through greater automation and transparency. What I am saying is, there may very well be better ways for them to do their jobs. In fact, the framework should just about guarantee that. If you have automated provisioning, for example, not only do you not want managers to manually provision as they’ve been doing forever, but you need them to know that the first time reconciliation runs, those manual provisioning efforts will be erased anyway. Managers and users need to take advantage of the new functionality. It’s there for a reason.
Required policies So let’s repeat. That manual provisioning should now be forbidden. It’s against policy. A lovely landing page, with your stated corporate policy, should pop up the first time everybody logs in under the new framework. At the very least, provide an obvious link to those policies. Have a “lunch and learn.” Host a call or web conference. Make sure everybody knows that things are different. And better. And that there are new requirements. As long as you’ve got the framework, also install the mindset. Part of that good citizenship means things like not creating unregulated data extracts. This is especially important if you don’t mask or encrypt your sensitive fields at the database level. Don’t share accounts. In other words, an IAM framework can prevent an awful lot of bad things, but a little common sense can help with the rest.
Establish Ownership and Responsibilities
An IAM framework is a powerful and necessary tool. It will provide benefit to management and end users alike. That said, it comes with a price beyond just hardware, software, and salaried hours. It requires care and feeding. Your procedures will be different. This goes without saying (but I just said it anyway). Policies must be maintained.
Across the Oracle identity suite, you have the power of delegated administration. You can create those sandboxes in which the administrators can do their jobs. It’s a benefit, certainly, in that they are empowered to manage their own resources, users, and policies. But this includes the responsibility to perform that management. Departmental managers should not be calling you to create, modify, or delete much of anything. Delegated administration gives them that power, that right, and that responsibility. Make sure they understand this. Training tells them how to do it, but you tell them to actually do it.