Lifting the fog
Many libraries struggle to obtain real value from the data they have, and this can result in a vicious cycle where they collect more data in the hope that the additional data will yield previously unobtainable insights. This chapter shows how to break out of this cycle by recognizing the emotions that underpin the decisions to continue to collect useless data, and advocating for an objective and simple set of tests to determine whether you should keep, change, or stop collecting specific datasets. This chapter outlines the importance of managing this change as a project, outlining key issues, considerations, roles and tasks.
Keywords
Decision making; emotional decisions; useless data; project management; objective tests; data selection criteria
Imagine your house is in shambles, clothes piled up in random containers tucked away in dark corners, shoe boxes collecting dust balancing precariously on the top of wardrobes, and you are sick of the state of mess. There is a sensible way to go about cleaning, and an irrational way. It is quite possible that the reason you have a mess is because you have more than you need. That dress may have looked great on you in your early twenties, but it is never going to fit again. And those pair of shoes you wore to your first job have gone out of style along with other relics that should stay in the past, like mullet haircuts. So, if you are serious about cleaning, this means letting go of some things. Easy said, not so easy to do.
The same applies to data. You might have some wonderful time series data that makes a pretty chart, or you might have some stats that staff have been collecting since the Stone Age, or you might have some statistics to which staff feel emotionally attached. Just because you collected it in the past does not mean you should have ever collected it, or even if it once was a legitimate collection from a business perspective, it does not mean that it is now. Just like the messy house, a bloated collection of irrelevant data, is counterproductive. At the very best irrelevance distracts from the data that is useful. At worst, the good data gets tainted by the bad data, with staff becoming cynical or disconnected with data. If the numerical literacy at your workplace is low, then chances are this will provide comfortable validation for those staff that want nothing to do with numbers.
When you are cleaning your house, the last thing you should do is rush off and buy more storage, and perhaps buy more clothes and shoes. This would only make the mess worse. The same applies to data. If you are not happy with the state of affairs with your data, don’t rush off and create new spreadsheets, sign up to new data vendors, or collect more data. Useful things become useless if they are hidden in a sea of rubbish. Indeed, this is meant to be one of the key value propositions of the library – they are a gateway to quality resources. Unfortunately, many professions don’t practice what they preach. However, if you are worried about the long term viability of your business model, then you will need good data; and to get good data you need to be disciplined and focused.
What is the first sensible thing to do when cleaning your house? You decide on criteria for determining whether to keep something or not, then assess whether the things you have meet those criteria. You would at the very least have three piles, one pile for stuff to keep, one to give away, another to chuck. Your criteria might be simple – it might be I will keep it if it fits me, and I will allow myself to keep five items for sentimental value.
When you are cleaning your data it is essential that you determine the criteria before you start. Cleaning data can be an emotional exercise, and if you don’t determine the criteria first, chances are you will inadvertently allow emotion to make the decisions. Of course, emotions for data are quite different to clothes. The emotional response might be something like:
1. I really don’t know why we ever collected this, but what happens if I chuck it and we need it later
2. I don’t understand this data, and I don’t want to admit that, so let’s just hang onto it
3. No one understands what we do, and if we don’t collect that data people will think we are not important, or not busy
4. I don’t know what I need to collect because I have no idea how to use the data, so let’s just keep collecting as much as possible and hope that the avalanche of statistics will somehow morph into something useful
5. It is all too hard, it is easier just to keep collecting it, besides, it does not take that long
None of the above are sound business reasons for keeping data, and therefore if you allow these types of reasons to unconsciously determine your choices, then chances are you will dispose of nothing. Now, if you have been lucky, and all your data is good data, then well done, go and buy a lottery ticket while the gods are smiling on you. However, chances are you have some bad data, which means anything you do to try to improve the good data will deliver slim value.
You might be asking what do you mean by good data. The sole purpose of data is to prove a point to an unconvinced audience, or help you to make a decision. If your stakeholders are worried about the value your library is delivering to clients, then you will need data to help put their minds at ease. If you want to create a new service, but are not sure whether there is the demand for such a service, then you need data to help identify the business case. Good data allows an organization to thrive, it can be used to build strong positive perceptions about your library, it can be used to drive continuous improvement, and occasionally it can be used to assess business cases for new ventures. Bad data is the fog that obscures the use-value of good data.
So what sort of criteria should you use to help you with your data de-cluttering? Broadly speaking, you should be collecting data to answer one of four questions:
1. How much effort am I putting into producing a given service/product
2. What is the demand for services/products I am producing
If your data does not answer any of the above questions, or answers them very poorly, then why keep collecting this data? The only possible valid answer is because you are required to collect that specific dataset by law. There are peak bodies that require the collection of statistics that are not of much use locally. The question you will need to answer is whether the cost of providing that data to the peak body is worth the goodwill. Here the cost is not just the time spent collecting and reporting on the data, but the contribution it makes to the fog of irrelevant information.
Some people may think these four questions are arbitrary, and certainly they are from the perspective of the terminology I used. There are a lot of key performance indicators out there, and a lot of tools for organizing them, such as the balanced scorecard. But focusing too much on the terminology at this stage runs the real risk of driving the data renewal program irretrievably into a semantic bog.
When I first started facilitating planning sessions at an academic library, I was amazed by how much energy staff were putting into crafting the right words for strategic goals. Eventually, I became a bit tired of the exercise and said quite loudly, “stop polishing stones.” My strange statement stunned a few people into silence, and when everyone turned to look at me I continued:
Your strength is also your weakness. Everyone here is great at crafting sentences, they are like finely polished stones. But before we start polishing stones we need to make sure that we have the right stones to begin with. If you have chosen the wrong course of action for the library’s strategic direction, then no amount of word smithing is going to help. In fact, it will hinder progress, because you will have these bright shiny stones that no one is going to want to let go of.
The same logic applies to your criteria. Focus on choosing the right criteria first, not crafting words first. The meaning of course needs to be clear, but you are not writing the constitution for a newly formed nation state. Lofty words and lengthy criteria will make the de-cluttering exercise more difficult, and wherever difficulty exists, emotion can sneak into the decision making process.
On the subject of decision making, one of the first things you will need to do before you starting de-cluttering is to identify who is authorized to make the decisions. One option is to do the following:
• If the data is never used outside the team, then the team leader takes authority for disposal
• If the data is never used outside the division, then the division manager makes the decision
This is a nice way to devolve responsibility; however, if you have a strong culture of data hoarding, then some people are quite likely to make emotional decisions with their data, despite your best efforts at communicating the criteria. In this situation a simple audit of everything might be required, with more central decision making. Otherwise, you risk going through a big exercise without culling much, which only risks further spreading the perception that data is irrelevant. Consequently, the best approach is to manage your data culling exercise as a project.
Projects have been around for a very long time, and there are many fine books written on the subject. There is a strong body of knowledge and scholarship on the principles of good project management, principles that have been refined over the centuries. As this is meant to be a practical handbook, and not an academic text, I am not going to discuss theory of project management, only what you need to consider in the library context for this specific project.
First steps – project management
Some libraries can struggle a bit with projects, so managing the change with a tool with dubious success might seem to some like saddling up for double barrelled failure. It does not have to be that way. If people struggle with projects, then don’t call it a project. But make sure you run it like one. Briefly, you need a project when you are doing something for the first time, and you require the coordination of several people, people that are dispersed across several teams.
If you are doing something for the first time, then you cannot use existing policy or procedures to manage the work. So you have two options. Make it up as you go along, with frequent team meetings to give the illusion of organization. This option is the no-win option. If you succeed, then it is down to luck – and if you fail, then it is your fault for not managing the thing properly. The other option is to plan what you are going to do before you start doing something. If you do this, then odds on you will be managing it as a project.
If you agree that the first step to improving the usefulness of your data is to de-clutter, and you agree that you need to adopt a project management approach to doing this, then the VERY first step you need to take is to ensure that you have full executive support. If you are the Director, then wonderful; if you have read this far chances are you are committed to changing data management. If you are not the Director, then you will need to ensure that you can sell the benefit of doing this to the executive.
A lot of people don’t like changing things they think are working, and if you attempt this project with only half-baked executive support, then at the first sign of any hardship, staff will default to saying they are too busy to help. The message needs to be clear with the executive team marching in locked step – we are going to improve our data management, here is the reason why, here is what we will do, here is when it will be done, and here is who is responsible. If you have that level of executive support, then you are well on your way to success.
Before you start communications, you need to sit down with the executive to obtain clear and realistic expectations on what you hope to achieve, and what sort of resourcing the project will need. Once everyone has agreed to this, you need to document this agreement, then identify a project sponsor and a project manager. These two roles can be confused a bit, so the following analogy might help.
Imagine you have just bought a wonderful block of land. It has great ocean views, and a lot of potential for gardening, which incidentally you happen to love! The land is empty, and you want to build your dream house on it. Imagine you have a rough vision for what you want your house to look like, but you don’t have the expertise, knowledge, and/or experience to translate that vision into an actual house. So you need to contract an architect and builder to do this for you. Now, unless you are very naive, or have some good reason to leave the architect and builder alone to their own devices, chances are you will want to catch up at certain stages to make sure everything is going along well. It’s your house, and you don’t want to run the risk of ending up with something awful at the end of it, when it is all too late. On the other hand, you are not a builder or an architect, so if you start attempting to manage the project as if you were a builder or an architect, then chances are you will only cause bottlenecks and miscommunications. This can only result in cost blow outs, delays, and poorer quality of work.
A project sponsor is like the home owner. They have a vested interest in ensuring the project is finished on time, on budget, and to specification. So, they manage this vested interest by checking in occasionally with the project manager. They might do this at certain predetermined milestones, or on a monthly basis. However, this checking in is not about managing the project.
A sponsor that takes over the project management is about as useful as a manager that does the job of their staff. If you are a project sponsor, and you do tend to micromanage, then be aware of it and have strategies to deal with it. If you cannot help yourself, get someone else to be the sponsor. Occasionally there will be roadblocks that are too big for the project manager to navigate by themselves. In these instances they may need to have the project sponsor weigh in, and throw their support behind the project. However, this should be a last resort, not the modus operandi of the project.
The same logic applies to the project manager. The project manager will have a team of people that will take responsibility for completing specific tasks by a due date to a specific standard. If the project manager starts micromanaging their team, then they will end up creating bottlenecks, delays, and by taking away responsibility from their team, it is likely that the quality of the team’s work will decline. When you hand a task to someone, it then becomes their responsibility to get that task done, and if they cannot, it is their responsibility to communicate any problems to the project manager as soon as possible.
When you have identified who will be the project manager, and who will be the project sponsor, then the project manager will need to draft up a project plan, and get sign off on the plan from the project sponsor. The project plan needs to identify what you are aiming to achieve. This will be the deliverable. If you were building a house, and you commissioned an architect to design a house, without any further input, then it is highly unlikely that you will end up with a design that remotely resembles anything you hoped for. The same applies to this project. If you are vague about what you hope to achieve, i.e., the deliverables are not clearly specified, then chances are no one is going to be satisfied with the project upon completion. This cannot be empathized enough. The deliverables for a project specify exactly what you hope to deliver at the end of the project – don’t assume everyone shares the same vision, because they will not. A clear and specific set of project deliverables will keep the project manager accountable and focused, and will ensure that the scope of the project stays contained.
This second point is also very important, because Librarians can tend to be perfectionists. If you try do to everything perfectly, then the scope of the project will grow and grow, to the point where it becomes so big it is impossible to complete. Everyone has their own vested interests, their own priorities, and their own view on how things should be done. All these things can and will place pressure on the project. There will be some pressure to reshape the project, some pressure from other corners to reduce its scope, and most likely, much pressure from many areas to expand its scope. This is so common it has a name, scope creep. As project manager you have to manage expectations tightly, both at the beginning and throughout the lifecycle of the project.
Once the project plan has received executive support, the next step is for the project manager to get the team together, to introduce them to the plan, each other, and their roles. Depending on the project team, you might wish to get them more involved in fleshing out the details of the plan, and identifying any gaps. If you have a small library, then the whole project could be left to one person. However, even if it is only a team of one, you still need to document what is expected from you, and plan ahead. Otherwise you are just making things up as you go along and therefore leaving success to chance.
I have witnessed many projects where it was not clear why something was being done. In the absence of information, staff make up their own minds. This can be dangerous, as it can lead to misinformed views which create real roadblocks. For example, at one place I worked, management decided to run an Activity Based Costing exercise, and did not do a very good job communicating why. This resulted in a lot of defensive posturing from some staff, with the result that the data quality was probably nowhere near as high as could have otherwise been achieved.
Library staff tend to be older, they have witnessed and been involved in many changes. Many are skeptical and cynical about change, particularly when you frequently see changes that after a decade end up coming round full circle. So it is absolutely imperative that the Director makes it very clear to all staff at the start of the project why they have commissioned this project. The Director needs to provide a compelling case for change. For example, a more theatrical Director might say something like this:
Over the past decades there have been many changes that have had a great impact on libraries. Even though we have embraced digital technologies, our fundamental business model has remained in large parts unchanged for centuries. The value we provided in the past does not guarantee that we will continue to provide value in the future. We are at the moment largely adrift on the sea of technological change. We have adopted technologies to our existing business model as these technologies have emerged. We have let go of some services, and created new ones. But we have for the large part been consigned to the role of reacting to change. If the tide and wind were with us, then we could have the option of continuing to drift. But it is not. We need to focus on and grow the new services that meet new demands. But we cannot strike out in the right direction when we are surrounded by fog. Data can tell us where we have been successful, and it can tell us where we can improve. Data can also tell us changes in trends, and potential new directions we could take. The problem is we are collecting so much data that does not need to be collected; and the data that is relevant, is out of date, inaccurate, and/or is too difficult to obtain in a timely manner. This is our fog. We are adrift in a data fog. For our long term success, we have to lift this fog. If we can clear the data fog we will be better able to determine whether we have been successful in our effort to reach new shores, and we will be better placed to know what we need to change when we drift off course.
With this in mind, I have asked Josephine Bloggs to review the data we are collecting, make recommendations on what we need to let go of, what we need to improve, and what new information we need to collect. That’s stage one of the project. The second stage is to recommend and implement new ways of managing this data. This is an important project. This project must succeed. I am sure you will give Josephine your full support. She will be forming a project team shortly, and will keep us informed of updates on the staff intranet.
I do not believe for a moment that any Library Director on this planet would give such a speech. I have worded it this way partly for some light hearted entertainment. But I also used the unlikely words of a Library Director as a device to draw sharp attention to a theme that underlies much of this book, the importance of using data to be good. This theme will be more fully explored as you progress through this book.
Now, while I am in full analogy swing, I would like to reiterate that data is not magical. It will not tell you exactly were to head. Most data is lagging. It tells you how things were. The Vikings used to trail rope behind their boats to tell if they were going in a straight line. The rope would tell them if they had shifted course recently, but it certainly would not help them to identify if their course was heading in the right direction. It only told them if they had stuck to their chosen course. Much of your operational data is like that rope.
After you have convinced staff of the need for change, the first task of de-cluttering then becomes a relatively simple one of auditing all the data being collecting by the organization, and assessing the value of the data against the predetermined selection criteria. This will need to be done in conjunction with the data owners. The person making the assessment might come to a different opinion on the value of the data being collected, but that difference should be informed by the application of the selection criteria, rather than ignorance of what, how and why the data is being collected. I suggest you start first with a friend, hone your technique and tools, before consulting more widely within the library.
The purpose of the audit should be to collect enough information to help you make informed decisions about which datasets to delete, improve, change or add. Once you have finished the audit process, you need to write up recommendations for your Library Director. Depending upon the complexity of the situation, some further consultation may be required before making a decision on a particular dataset. Before you get to this point, you will also need to think about how you are going to go about telling someone that their data will no longer be collected, or will be substantially changed. Stick to the facts, be friendly and respectful, but keep it away from the personal level. Try to think of a way of phrasing constructive feedback beforehand, for example:
As you know I have been reviewing the data being collected by the Library. I have reviewed a few datasets with you, and I greatly appreciate your help. We could see that the data was definitely useful in the past, and has been included in management reports many times. After careful consideration we think we probably do not need to collect this data from this point forward. There is however, other data that we will need from your team, and I will definitely need your advice and help with this at a later point. Meanwhile, how do we go about decommissioning this dataset?
Some staff may be happy to let go of the data, but others might take it a bit personally, particularly if they have been collecting it for years, and perceive it to be more valuable than it actually is. Just imagine someone with authority in your library told you that everything you were doing was rubbish. Well, obviously in this case it’s not everything, but you will get the idea. Telling staff they need to stop something could potentially be quite hurtful, so you will just need to be mindful of this to avoid unnecessarily bruising egos.
You will also need to consider who to consult in decommissioning data. It will probably be the team leaders, but you should have sorted this out as part of your earlier recommendations to the Director.
At the end of this process you should have a list that contains:
• All significant datasets collected by your library
• A very brief description of each dataset, including how it is currently used
• A very brief assessment of how each dataset meets the data assessment criteria. For example, how well does it currently answer any of the questions below:
• How much effort am I putting into producing a given service/product
• What is the demand for services/products I am producing
• What is the perceived value of my efforts to clients
• What is the outcome of my efforts for clients
• Brief comments on how the datasets’ use-value could be improved
Once you have collected this information, you will be ready to discuss the findings with the Director and Project Sponsor. This is a very critical stage, and the success of the data re-invigoration project will hang off how willing management is to make hard decisions, and stick to them.
Once you have your slimmer data, you will be ready to reshape the management of your data, so that information can be easily, reliably, and accurately retrieved. Before you can do that, however, you need to identify the best way to organize your data. The following chapters will show you how.