Chapter 2

Architecture and Capacity Planning

WHAT’S IN THIS CHAPTER?

• SharePoint products and licenses
• Critical non-SharePoint servers
• Hardware specifications
• Terminology
• Tools for controlling your deployment

SharePoint 2013 has greatly expanded its functionality from previous versions. New features include the following:

• An enhanced social experience, including microblogging, enhanced social experience, communities, and the capability to “follow” people, items, and sites
• More flexibility regarding how web applications consume services through service applications
• The Distributed Cache service, which helps relieve the workload on SQL, increase performance, and ease the technical networking complexity of multi-SharePoint server farms
• A new Request Management service that helps to distribute specific workloads

Applications that have undergone significant change include the following:

• Office Web Apps — Is now required to be installed on a separate server and is a shared service between SharePoint 2013, Exchange 2013, and Lync 2013.
• FAST Search Server — No longer exists as a unique product (or SKU) which can be purchased from Microsoft – it has been fully integrated into SharePoint.
• 2013 Workflow — While all the goodness you came to know and love in SharePoint 2013 is still available to you, SharePoint 2013 Workflow contains new capabilities that require the Azure Workflow Manager.

Those and numerous other new features in SharePoint 2013 are reasons why people are so excited about the product. Of course, all that new functionality means that users will deploy SharePoint for more tasks than ever before — and that increased traffic leads to increased demands from a hardware perspective. To complicate matters, the Office Web Apps and SharePoint 2013 Workflow services have now been spawned into their own application, which is a shared service between multiple Microsoft products that connect to SharePoint through the service application proxy. (Don’t get excited about a new GUI to explore – these server applications install and are configured completely through PowerShell.)

As a result of all these changes, administrators can anticipate a large increase in the number and size of servers in their farms. You can anticipate that the same user from SharePoint 2010 will make far more requests per second (RPS) to a SharePoint 2013 farm. In addition, the Office Web Apps service is designed to take requests from both Lync 2013 and Exchange 2013. Therefore, you have to consider the entire “collaboration” scenario of these services in planning your SharePoint infrastructure.

In terms of scaling your hardware, you can apply the same general benefits and rules used for SharePoint 2010. SharePoint service applications not only provide the capability to build unique servers in the SharePoint farm that provide a single service, they can be built into unique farms that provide a published service, to which farms can subscribe. Most service applications can coexist with one or more other service applications, just as they did in SharePoint 2010. There is one notable exception found in OWA. Here are a few curious facts about OWA:

• It requires its own server.
• It cannot be installed on a SharePoint server.
• It doesn’t need to reside in the same Active Directory (AD) domain as the SharePoint server.
• It can be shared as a resource between more than one SharePoint farm which could reside in different AD domains.
• It can be consumed by both Exchange 2013 and Lync 2013 to provide in-browser Office document editing.
• It is free for all read-only scenarios but requires a license for edit capabilities.
• It can’t be installed on a server that already utilizes Port 80, 443, or 809.

This chapter begins with a primer on the different versions of SharePoint 2013 you can expect to see, including an overview of SharePoint in the cloud. If you have spent any time with SharePoint 2007 and 2010 (and who wouldn’t want to), you will find the consolidation of the product and licensing model riveting! There are some significant changes. Then, because a SharePoint DVD isn’t good for much more than a coaster until you have the supporting servers on which to install it, you will learn all the software requirements you need to bring to the table.

Armed with the necessary software knowledge, the discussion turns to hardware, including the amount of metal you will need, ways you might consider scaling up versus out, and where virtualization might come into play. The chapter also delves into the misty atmosphere of cloud-based computing and how that might fit into your architecture decisions. Finally, the chapter describes some of the clever new tools available that will help you keep SharePoint running smoothly in your environment.

While this might not be as exciting as a blockbuster Hollywood movie, it will turn you into what every good summer blockbuster needs: a superhero. Therefore, read carefully and make sure you pick up your cape with the “S” on it from the dry cleaners — you will look much better as you soar through the clouds.

NAMES, NAMES, MY KINGDOM FOR A CONSISTENT NAME!

In terms of the core SharePoint naming convention established in SharePoint 2010, nothing has changed. Don’t get too comfortable, however, there are other changes. Specifically, the products formally known as Search Server, Search Server Express, and FAST Search Server no longer exist. Search Server and Search Server Express have been discontinued as an available product. Fast Server technologies have been fully integrated into SharePoint.

SharePoint Foundation

SharePoint Foundation will continue the legacy that SharePoint 2010 Foundation and WSS have established over the years, including that friendly price point that makes it so attractive. The name SharePoint Foundation is a perfect match for what the program brings to the table. As an administrator, it is easy to think of the product only in terms of the features you readily see in the browser — such as creating team sites and collaborating on content within lists and libraries, or features such as blogs, wikis, RSS feeds, alerts, and easy browser-based customizations.

Yet underneath all that great functionality is where some of the true power of SharePoint is hidden. Here, the foundation provides developers with a great platform on which to build. Out of the box, it handles storage, web presentation, authorization, user management, and has an interface into the Windows Workflow Foundation — and because all this functionality is easily accessible through the object model, APIs, and web services, it can greatly accelerate a developer’s job.

At this point, you might be thinking, “Wait. I thought you said that SharePoint 2013 Workflow required a new server; so why are you talking about Windows Workflow Foundation?” Simple. Windows Workflow Foundation continues to exist. If you don’t want to install and configure the SharePoint 2013 Workflows, that’s fine. You only need to install Azure Workflow Manager / Client applications if you want to use workflow capabilities. To help keep you out of trouble in SharePoint Designer 2013, the workflows are kept separate.

Therefore, rather than build all those infrastructure pieces for every web-based product, developers can leverage SharePoint Foundation and concentrate on just building the solution. Many companies have utilized these core SharePoint collaboration capabilities for many years. It is actually most advantageous to start with SharePoint Foundation, as these core collaboration capabilities generally meet the requirements of most new SharePoint users. This enables you to avoid additional licensing costs until features associated with those licensed versions are actually required by users.

SharePoint Server 2013

SharePoint Server 2013 is considered the premium SharePoint product. Compared to SharePoint Foundation, it offers additional collaboration and social capabilities and extends the use-case scenarios. Its robust tools enable better aggregation and displaying of content, which makes building grandiose thing, such as, portals much simpler, while better enabling end users to create specific line-of-business solutions for their departments. It also introduces additional web content management tools that enable developers to use Server as a platform for building Internet-facing websites.

This is achieved by building on the capabilities introduced by SharePoint Foundation. Anytime you install SharePoint Server, the Foundation product is installed automatically as well (though you will not see SharePoint Foundation installed in Programs and Features). Keep this in mind as you manage your environments, making sure you keep current regarding both Foundation and Server issues, as you really have both products. As you perform tasks such as applying service packs or adding third-party applications, this knowledge might affect how you go about these tasks.

Standard and Enterprise

As in the past, SharePoint Server 2013 is available primarily in two flavors, Standard or Enterprise. Standard introduces core functionality such as social, search, and advanced web and enterprise content management. Enterprise focuses primarily on adding functionality through new service applications, business intelligence, line-of-business integration, reporting, and some Office client services such as Visio and InfoPath Forms services.

This functionality is provided through one of two licensing models: a client access license (CAL) or a subscriber access license (SAL). A CAL is an individual license that you buy for a single user to access SharePoint. The person who buys it owns it forever. Also available is an option to pay an additional yearly fee for what is called Software Assurance (SA). Software Assurance grants the licensee the permission to use any version of SharePoint that is currently supported by Microsoft. A Software Assurance License (SAL) is an individual license that is rented, or leased, from a service provider, usually a hosting provider. A SAL includes Software Assurance by default but is not owned. This can be useful for scenarios in which a “lifetime” license isn’t necessary. For example, a contractor might require access to SharePoint for a defined period of time for a specific project. Table 2-1 summarizes SharePoint 2013 licensing options.

TABLE 2-1: SharePoint 2013 Licensing Options

You only need to run one setup program, which puts all the binaries on the server; and based on the license key you enter, either the Standard features will be available or both the Standard and Enterprise features will be available. You are required to have an appropriate CAL or SAL for each user accessing SharePoint Server 2013. To be clear: these licenses are for individual named users, not concurrent users.

Hosted SharePoint aka SharePoint for Internet Sites (FIS)

SharePoint first added the capability to host public Internet sites in SharePoint 2007, with the integration of Content Management Server. A few brave souls ventured to trust their sites to this new technology. With the release of SharePoint 2010, many more companies began using this platform — some for single, public Internet sites, and some for an intranet site publishing platform for many brands. It was clear to Microsoft that companies wanted to use its technology in this way, as most of their intranets were running on SharePoint — a logical extension of their approved tools and technologies.

As the conversation about SharePoint 2013 continues, it becomes evident that Microsoft had two obstacles to overcome in relation to SharePoint for Internet Sites:

• It was clearly too expensive. A company with licensing requirements for a “medium-size” farm that included four SharePoint servers, for which the per-server license fee is in the neighborhood of $45K per server, would be facing a $180K fee for SharePoint licensing alone.
• With the rise of SharePoint Online, as delivered by Office365, Microsoft had to clean up its product naming conventions.

As a result, Microsoft has changed the way it charges for “purchased” licenses (CAL model) and changed the naming of the product. What was once called “SharePoint for Internet Sites” is now called “hosted SharePoint.”

Another change related to the SharePoint FIS license is the removal of hosted SharePoint Standard Edition. It’s all (Enterprise) or nothing with hosted SharePoint now.

One very important thing to understand about hosted SharePoint is where it is applicable. Remember that everyone authorized to access the SharePoint Server site needs a CAL. When building an intranet portal, it is easy to count how many employees you have and to purchase a CAL for each one of them; but when you stand up http://www.company.com and make it available to the world, now how many CALs do you need? There are roughly 1.8 billion people on the Internet, and potentially every one of them can visit your website. That’s a lot of CALs to buy. Luckily, this is where hosted SharePoint comes into play. It allows unlimited non-employee access to your SharePoint Server. The reason why non-employee is emphasized is because this license does not cover any company employees, which has caused a lot of confusion in the past. The proper hosted SharePoint license can help you control the cost of your SharePoint Server deployment, but great care should be taken to use it properly.

You might be thinking to yourself, “That says non-employee when talking about my Internet site. Does that mean I could extend my intranet site into an extranet site, keep an authentication mechanism in place, and use the hosted SharePoint license?” The answer is a resounding yes! There is no reason to get all worked up about whether users will authenticate into the farm or not. Hear me now: authentication has nothing to do with the license that is required. Licensing has everything to do with who the users are. If they are employees (in the U.S., receiving a W-2 tax form at the end of the year), then only one licensing vehicle is available to them: the CAL or the SAL. If they are non-employees, then they can use either the per-user model or the per-server model. You choose the one that makes the most financial sense to you. Note one point that is relatively new to the SharePoint licensing story: if a “non-employee” is a contractor or agent using SharePoint as any of your employees would use SharePoint, he or she must have a CAL. The Hosted license will not work for you.

When you are licensing SharePoint in the traditional “on-premises” fashion, you actually require two licenses: one license for each user authorized to use SharePoint and one license for each server instance on which SharePoint is running. This second license is called, oddly enough, the server license. You can now use this server license as the licensing vehicle for any “extranet” users of SharePoint; impressive — a price drop from Microsoft. This is also the license that you would use for an on-premises deployment of a public-facing web page. Taken together, these changes enable Microsoft to delve deeper into the WWW space and simplify the jungle we have known as SharePoint licensing.

Search Server Express

In 2008 Microsoft introduced a product called Search Server Express. It was all of the good of SharePoint Server 2007 Search bolted on top of WSS v3 for the same price of WSS, free! This carried over to the SharePoint 2010 product cycle as well. It was a very popular flavor of SharePoint because who doesn’t like more free stuff?

With the SharePoint 2013 product cycle there is no more Search Server. You may think you need to go to your Congressman and complain for such an injustice but you don’t. The reason Search Server Express existed was because WSS/Foundation search was awful so Microsoft was compelled to offer the enhancements. Fortunately they solved the problem differently with SharePoint 2013. This time around Foundation has the same great Search architecture as SharePoint Server. No need for a product to bridge the gap so no more Search Server. Check out Chapter 13, “Configuring and Managing Enterprise Search,” for all of the fun details on Search.

FAST Search Server 2010

Oh, the casualties just keep coming. In 2008 Microsoft bought the Norwegian company FAST Search. With the release of SharePoint 2010, Microsoft took its first steps in integrating the product. As such, it was only half-baked into SharePoint. In SharePoint 2013, FAST is now been fully integrated, and no longer available as a standalone product.

SharePoint Online

Another push for SharePoint from Microsoft is SharePoint in the cloud, hosted by Microsoft. If you are interested in deploying SharePoint using this model, then you can probably stop reading the book at the end of this section because in SharePoint Online, the entire server infrastructure is hosted and maintained for you. This model removes the administrative overhead of SharePoint, enabling a business to focus on using SharePoint’s out-of-the-box power. (While this might be great for the business, it does eliminate the need for a SharePoint System Administrator, so many of us will consider this license option the enemy.)

Microsoft has made so many improvements to the usability, functionality, and application model of SharePoint, they would probably argue that you don’t need no stinking servers! For many use cases this is true. Software as a Service (SaaS) is a strong model that takes managing SharePoint out of the equation, enabling the business focus previously mentioned. Noble, for sure, but not necessarily practical in the long term for all customers. There are still certain payloads and use cases that make the Software as a Service model impractical, including the following:

• The need to deploy farm-level solutions. This will never happen in a traditional SaaS offering.
• Business intelligence solutions often require fast and secure access to back-end data warehouses or line-of-business systems.
• True Active Directory integration doesn’t currently exist in SharePoint Online in a non-synchronized implementation. That means a time lag between when you change your personnel in your AD and when those changes are available in SharePoint Online.
• External data connections. As with BI, fast and secure access to back-end data systems is required. While this can be done in part in SharePoint Online, many companies aren’t willing to allow this type of access to systems in their data center from a SaaS provider.
• Limited support. At the time of this writing, SharePoint Online (and Office365 for that matter) does not offer the escalation path to Premier Support that other product lines have.

This is not to say that SharePoint Online isn’t a great platform. It absolutely is and meets a strong need in the greater SharePoint community; one where you just need out-of-the-box SharePoint in a collaboration scenario. If this is your only goal of SharePoint, then Online is probably the way to go.

There are actually two models to consider with SharePoint Online: shared and dedicated. The shared model provides you with a slice of a shared farm and enables you to use SharePoint out of the box. Server-deployed code and customizations are not permitted but sandbox solution and the new app deployment/consumption are available to developers. The dedicated model enables you to run your own farm, and you are allowed to make approved customizations to the server. Any change must be packaged in a solution package and validated by Microsoft before being deployed to the server. All licenses are bought per user.

This offering still contains some newer licensing options added in 2010. One of these is the concept of the “deskless worker.” These are users you can add at a lower price point, and they have mostly read-only access to SharePoint. This is called the Kiosk subscription option. Also available are models that support hosting a partner collaboration site and public-facing Internet sites.

An interesting point to make around SharePoint Online and On Premises SharePoint is that Microsoft has been doing a lot of work around their ability upgrade and enhancing SharePoint Online on the fly. The promise here is that upgrades and enhancements could come to hosted SharePoint at Microsoft before they are made available to the general public. This could provide a distinct advantage with regards to receiving either fixes to SharePoint or new features before they are received by on-premises customers.

ADDITIONAL SERVER PLANNING

As a SharePoint administrator, you are also now responsible for a whole host of software. Because SharePoint isn’t an operating system (yet!), you need to have the right OS in place in order to deploy it. Additionally, SharePoint stores 99% of its content and configuration in a database, so SQL Server has to enter the conversation sooner, rather than later. Finally, most deployments want to take advantage of SharePoint’s ability to send notification e-mails, and some even take advantage of its ability to receive e-mail. Even though you may not directly be responsible for these products, they will affect your livelihood. Users don’t call to complain that SQL Server isn’t working; they call to complain that they cannot access SharePoint, that SharePoint isn’t sending alert e-mails, or that the data in the BI Portal of SharePoint is out of date. It is your job to determine whether that is because SQL Server (or any other server or service) isn’t responding. This section covers the ins and outs of these various components.

Windows Server and Required Additional Software

SharePoint is available only as 64-bit software (it’s been exclusively 64-bit since 2010), so by extension it can only be installed on servers with 64 bits or more. (Don’t bother looking — there is no 32-bit “test” version hiding out there. The authors have looked under every rock on the Internet and inside Microsoft; and like unicorns, it doesn’t exist.)

For production deployments, you will be installing on the 64-bit edition of Windows Server 2008 R2 Service Pack 1 (SP1) Standard, Enterprise, or Datacenter, or the 64-bit edition of Windows Server 2012 Standard or Datacenter.

Noticeably absent is the Server Core installation of Windows, which unfortunately does not allow all the components necessary for SharePoint’s operation to run, so it is not supported. In addition, the Web Edition is not supported, which is probably a good thing — thanks to its limited memory capacity, it would not perform very well.

After installing Windows Server, the server needs to be included as a member of an Active Directory (AD) domain. SharePoint does not support local machine accounts for any type of farm deployment, and the configuration wizard will issue an error if you try to use a local account.

Most administrators realize that something like IIS needs to be installed on the Windows Server in order for SharePoint to render web pages. They are often tempted to install this manually, which is safe to do but probably a waste of time. The server also has roughly a dozen other prerequisite software packages that need to be installed, including the Web Server (IIS) role. Thankfully, you can use the SharePoint Products and Technologies Preparation Wizard, aka the prerequisite installer, to install and configure all these components when the time comes. That tool, and all of its intricate details, is covered in Chapter 3, “Installing and Configuring SharePoint.” Please note that an active Internet connection is required by the server to use this tool. If one is not present then you will need to download all prerequisites and install them manually.

Windows Vista, 7, and 8

In order to appease your friend the developer, Microsoft has introduced the capability to install SharePoint using a standalone install, for development purposes, on certain versions of Windows Vista x64 and Windows 7 or 8 x64. In these instances, a Developer Site must be created. The editions that are supported are:

• Windows Vista SP1 and later:
  • Business edition
  • Enterprise edition
  • Ultimate edition
• Windows 7 RTM and later:
  • Professional edition
  • Enterprise edition
  • Ultimate edition
• Windows 8 RTM and later:
  • Professional edition
  • Enterprise edition
• The N and KN editions of the preceding software will also work.

Absolutely not supported is using a standalone installation for a production farm. Standalone installations should only be used for developers who wish to do SharePoint development locally on their own machine. If development is done in these environments, then it is highly recommended that developers have a test environment to validate their solution before deploying to production. These types of deployments are a little more tedious in the initial configuration and are discussed in more detail in the next chapter.

SQL Server

Get used to it: SQL Server just became your best/worst friend. Because everything inside of SharePoint, including all your content, lives inside a SQL Server database, as SQL goes so does your farm. Specifically, an unhealthy SQL Server means an unhealthy SharePoint farm. For example, the most common performance bottleneck in SharePoint is SQL Server. Therefore, in order to do your job effectively, at a minimum you need to understand what is going on in SQL Server. Ideally, you will start sucking up to your resident database administrator (DBA) to ensure that your SharePoint databases are well cared for.

As with the Windows Server requirement, SharePoint also requires 64-bit SQL Server; 32-bit SQL Server is not supported. The 64-bit editions of SQL Server that are supported are the 64-bit edition of Microsoft SQL Server 2012 or the 64-bit edition of SQL Server 2008 R2 Service Pack 1. In case you are considering trying to sneak an unsupported edition of SQL Server by SharePoint, know that SharePoint checks before you attempt to create your farm.

E-mail Servers and SMS Options

SharePoint comes with a handy piece of built-in functionality that enables it to send e-mail. This is often used to notify users that they have been granted access to a particular site. Users can also subscribe to an alert whereby they are notified when items are modified on a particular list or library. Additionally, with a little extra work, SharePoint workflows (2010 or 2013) can be configured to e-mail users as necessary.

In order for SharePoint to send this e-mail, it needs to be configured with an outbound e-mail server. The SMTP server you point SharePoint at needs to allow anonymous relay from SharePoint. Unfortunately, SharePoint cannot be configured to provide authentication information when sending e-mails. (Also unfortunately, this hasn’t changed since 2010.) In most environments, anonymous relay is not permitted, because for years evil spammers have used anonymous relays to avoid detection as they flood you with offers for low-cost medicines and opportunities to invest in dubious banks. In this case, you can ask the e-mail administrator to add the IP addresses (or subnet) of all SharePoint servers to the list of servers that are allowed to anonymously relay mail. If this is not acceptable, then your second option is to install the SMTP service on one of your SharePoint servers and then configure it as necessary. You need to ensure that it can correctly send outbound e-mail and that it allows all anonymous relay from all SharePoint servers in the farm. This isn’t difficult for a Windows admin, as it just involves adding a role to one or more of the Windows Servers.

Another requirement for outgoing e-mail is that port 25, the default SMTP port, is not blocked between the SharePoint servers and your e-mail servers. Such a blockage can happen at the firewall level or at the local server level. Some antivirus vendors configure their software to block port 25 outbound on all machines. This will stop SharePoint from sending e-mail, so be on the lookout. This also emphasizes the need to have a thorough understanding of what SharePoint is doing, what e-mail is doing, what the networking environment is doing, and how to troubleshoot all those components. In the end, if “Bob” doesn’t get his announcement, it’s not SharePoint’s fault. It’s your fault!

A lesser-known feature of SharePoint is its ability to receive incoming e-mail and then route that e-mail to the appropriate list or library based on the To: address. This enables scenarios such as having salespeople in the field e-mail their expense report to a special e-mail address. That e-mail would be routed to the SharePoint server and then the attachment could be extracted and uploaded to the appropriate document library. From there, whatever business process needs to take place could be invoked. Yes — this means workflow. A simpler scenario might be setting up an e-mail address for a discussion forum. Then, anytime you send an e-mail to that address, the e-mail becomes a discussion item in the list, enabling a conversation that begins in e-mail, which is notoriously poor for capturing intellectual property, to be forwarded to SharePoint. Once in SharePoint, it is easily indexed so it can be discovered later; and because it is now a normal list item, the discussion can continue, as all good SharePoint users would expect.

Configuring this functionality requires the help of the e-mail administrator; note, however, that it does not require the use of Exchange. This is a multi-step, complex process that touches several pieces, but the core steps are as follows:

With these steps completed, e-mails will be sent to [email protected]. Your corporate e-mail server will relay that mail to the SMTP service running on the SharePoint server. The SMTP service then takes that e-mail and puts it in a mail-drop folder. The SharePoint timer service checks that folder once a minute by default, looking for e-mail. When it finds a message, it routes it to the appropriate list or library based on the address.

While that is a simple scenario, many configuration options are available. You can, for example, configure Exchange Server and Active Directory to allow users to create their own e-mail addresses. This is done through the creation of an additional organizational unit in your domain. This is a more complex scenario, but it eliminates the administrative burden of having to set up e-mail contacts each time a new list or library requires mail functionality.

You can find detailed configuration information, with multiple scenarios and troubleshooting steps, on TechNet (http://technet.microsoft.com/en-us/library/cc262947(office.15).aspx).

Sending messages via e-mail isn’t the only way to inform SharePoint users. SharePoint has become so cool that it can even send text messages; and because SharePoint still isn’t old enough to drive, you don’t even have to worry about it texting and driving. Once the service is configured, users can choose to have alerts sent to e-mail or text message or both.

The service is pretty straightforward to set up from within Central Administration and can be scoped at either the farm or the web application level. You will need to provide the URL of an SMS sending service. If you don’t have one handy, you can click the link on the page to find one based on your preferred wireless provider. Just watch out for this functionality: it can easily become a runaway cost depending on the expenses incurred from your SMS provider for each message.

HARDWARE REQUIREMENTS

Build it and they will come. Underpower it and they will complain. (No user has ever complained that SharePoint is too fast.) Of course, with budgets being very tight, you will feel the pressure to keep hardware costs as low as possible. This tension between functionality and cost creates a fine line to walk.

Perhaps the easiest way to start thinking about hardware is to do a comparison of the minimum recommended requirements from MOSS 2007 through SharePoint Server 2013 (see Table 2-2).

TABLE 2-2: MOSS 2007 versus Server 2013 Recommended Minimum Hardware Requirements for a 2-tiered farm

Note that part of the wide gulf between minimum server requirements from 2007 through 2013 indicates that Microsoft has done a better job of setting the minimum bar this time. Note also that the need for SharePoint resources is growing exponentially. It could be argued that attempting to run a single server with all SharePoint resources running on that one server isn’t practical anymore for a full production installation of SharePoint.

Why show SharePoint 2007 requirements in a book about SharePoint 2013? Simple — there is still a ton of SharePoint 2007 running in production; and if it’s still running in production, it could be running at the original hardware specifications from Microsoft. That’s why it is incredibly important that you — our trusty SharePoint administrator — understand what will be required of your new farm in order to run SharePoint 2013.

Experience has shown that SharePoint farms tend to range from vastly undersized desktop-class machines running thousands of users, slowly, to supercomputer-class machines that on their best day use 20 percent of their resources to serve 100 users. Therefore, if you are going to make hardware assumptions at least in part based on your 2007 or 2010 environment, make sure you understand how that hardware is used today. The next few sections describe the different server types and how their hardware considerations vary.

Web Servers

Often referred to as web front-end (WFE) servers, these are the machines ultimately responsible for rendering of the SharePoint pages. They typically do not have a high CPU load because they attempt to cache as much content as possible to avoid doing the same work repeatedly (there are a lot of configuration capabilities here). To do caching properly, the server consumes quite a bit of RAM, so be sure to dedicate a substantial portion of your spending on this server to RAM.

A key consideration when determining how much memory you might need is the number of application pools you plan to have. Application pools are a deep topic, but in a nutshell they comprise the various IIS processes that listen for incoming web traffic and then handle it accordingly. In Task Manager, you will see each application pool as w3wp.exe. For example, when you create a new SharePoint web application and choose a new application pool, a new instance of this process runs. Now when you access SharePoint, this process is actually receiving your request and coordinating with SharePoint to render your page. When SharePoint is caching content in memory, it is being stored in RAM associated with this process.

Part of this consideration, though, is that every application pool has a certain amount of overhead associated with it, the process, and the memory it needs to do its job. Therefore, for each new application pool you create, your RAM requirements will increase, so plan accordingly.

RAM is also affected by the “search-based nature” of all things SharePoint 2013. In SharePoint 2013 you can configure the SharePoint Index process to index content at the time it is uploaded into SharePoint — known as “Continuous Crawl.” If this is done you have two primary results:

• Content shows up in search results nearly immediately
• Your memory requirements on the server increase (aka: thank you NodeRunner.exe)

This role requires very little local storage and does not need to be optimized in any way. The only storage this machine is handling is the SharePoint root, all the local ULS and IIS logs, and possibly some disk-based BLOB caching. In other words, don’t get carried away here and create a 500GB C: drive. SharePoint occasionally needs to have extra space for temporary files, maybe to unpack a solution or to deploy a service pack, so an 80GB or 100GB C: drive is reasonable for your WFE. Microsoft recommends that you allow for five times the amount of free space as RAM on the SharePoint Server. Not only do they recommend it, they have set a health analyzer rule to help remind you of the fact (it’s a nasty red warning).

Application Servers

Application server is the generic name for any server that is responsible for providing resources for the various service applications. These are servers that have the SharePoint code installed on them. They are added to the farm using the configuration wizard or PowerShell. They could take SharePoint customer traffic if the load-balancer were configured that way. The tricky part of sizing these boxes is that each service application has a different usage profile, so the requirements vary according to what is running on the box and how heavily that functionality is being used. In addition, when building out an application tier, you should consider scaling out versus scaling up: Is it better to have one large application server with a lot of resources but a single point of failure, or several smaller boxes running the same services that provide fault tolerance but require more administration? The following sections describe some of the key types of application servers and their individual considerations.

Search Server

This section begins with the term “Search Server” as opposed to either Query Server or Index Server because there is so much new to Search in SharePoint 2013. These additional components are the result of fully integrating FAST Search into SharePoint. Of particular interest is how SharePoint consumes everything related to FAST’s inclusion. For instance, nearly the entire web content management experience is one that is served by SharePoint Search. That same story continues to other areas of SharePoint with the addition of the Content Search Web Part (CSWP). Each CSWP is associated with a search query and shows the results for that search query in a way that is easy to style. The CSWP can be used to display any content that is stored in the Search Index.

Search is divided into the following component parts:

• Crawl Component — Crawls content sources to collect crawled properties and meta data from crawled items and sends this information to the content processing component.
• Content processing component — Transforms the crawled items and sends them to the index component. This component also maps crawled properties to managed properties and interacts with the analytics processing component.
• Analytics processing component — Analyzes the crawled items and how users interact with the search results. The analyses are used to improve the search relevance and to create search reports and recommendations.
• Index component — Receives the processed items from the content processing component and writes them to the search index. This component also handles incoming queries, retrieves information from the search index, and sends back the result set to the query processing component.
• Query processing component — Analyzes incoming queries, which helps to optimize precision, recall, and relevance. The queries are sent to the index component, which returns a set of search results.
• Search administration component — Runs the system processes for search, and adds and initializes new instances of search components.

To support these new components of Search in SharePoint 2013, the following databases are created:

• Crawl database — Stores tracking information and details about crawled items such as documents and URLs. It also stores information such as the last crawl time, the last crawl ID, and the type of update (add, update, delete) during the last crawl.
• Link database — Stores unprocessed information that is extracted by the content processing component and information about search clicks. The analytics processing component analyzes this information.
• Analytics Reporting database — Stores the results of usage analysis, such as the number of times an item has been viewed. It also stores statistics from the different analyses. These statics are used to create the usage reports.
• Search Administration database — Stores the settings for the Search service application, such as the crawl rules, topology, query rules, and the mapping between crawled and managed properties.

Needless to say, a lot of planning goes into an architectural plan for a SharePoint farm. Gone are the days of putting the Query Server on the SharePoint Web Server and using a dedicated Index Server. Microsoft’s guidance for server specs for the Search components are described in Table 2-3.

TABLE 2-3: Minimum Search Server Requirements by Role

Table 2-4 summarizes the requirements for the database server hosting search databases:

TABLE 2-4: Database Server Minimum Hardware Requirements

Table 2-5 summarizes Microsoft’s guidance for the volume of items that trigger breaking Search components onto their own servers.

TABLE 2-5: Search Volume Thresholds

You need to keep a close eye on these servers for utilization. So much of the overall “SharePoint experience” is bound to the performance and health of Search that this single set of component services needs to be monitored and managed on a frequent basis.

Managing the Search service involves three primary tasks:

• Managing the default search topology
• Managing search components
• Managing the index component

Be aware that any buttons you used to click in SharePoint 2010 for these tasks, such as Manage Search Topology, are gone. No more pretty GUIs. PowerShell is used nearly exclusively to manage SharePoint. Repeat the following mantra after me: I will learn PowerShell, I will learn PowerShell, I will learn PowerShell....

Distributed Cache

The Distributed Cache service provides in-memory caching services to several features in SharePoint Server 2013 to provide fast retrieval of data, and it has the subsequent effect of lowering the load on SQL Server. Like so many other new features in SharePoint 2013, Distributed Cache is a feature that will be used mostly by large SharePoint farms such as those found at large hosting companies. Because the Distributed Cache service holds context- and application-specific information in memory, it eliminates the need to set the “sticky session” (or server affinity) flag on the network load balancer for SharePoint farms that have many SharePoint web Servers.

The Distributed Cache service in SharePoint is actually an extension of the AppFabric of the underlying Windows.

Following are some of the features that use the Distributed Cache service:

• All social features
• Collaboration features
• Authentication
• OneNote client access
• Security trimming
• Page load performance

Note that the Distributed Cache does not replace other “caches” within SharePoint, such as BLOB cache and Output cache.

Servers that are running the Distributed Cache service are termed cache hosts. A cache host can be joined with other cache hosts to create a cache cluster. A cache cluster has a “total cache size,” which is the sum of the memory allocated to the Distributed Cache service on each cache host. Note that while the data in the cache is stored in memory on any one of the cache hosts, the cache host or cluster can’t be made highly available. There is only ever one copy of an item in the cache.

When you plan to implement the Distributed Cache service, consider that it can be deployed in two modes: dedicated mode or collocated mode. In dedicated mode, all services other than the Distributed Cache service are stopped on the application server that runs the Distributed Cache service. In collocated mode, the Distributed Cache service runs together with other services on the application server. Dedicated mode is the recommended mode in which to deploy the Distributed Cache service.

As you can imagine, the Distributed Cache service is a memory-intensive service. It is recommended that the following services not run on the same server that is functioning as a cache host:

• SQL Server
• Project Server
• Excel Services
• Search Services

When SharePoint is installed the Distributed Cache service assigns 10% of the total memory for the server. If additional memory is added, administrators have to manually adjust the memory assigned through PowerShell: Update-SPDistributedCacheSize -CacheSizeInMB <CacheSize>. If a cache host’s server utilization reaches 95%, it will throttle itself by no longer accepting new read/write requests until the utilization falls below 70%. This doesn’t mean that customers don’t get their data — it just means the request times out and SharePoint then requests the information from the SQL server.

Because there is never more than one copy of any item in the cache, it is very important to carefully add and remove servers from the Distributed Cache cluster. To join or rejoin a server to a cache cluster, use the SPDistributedCacheServiceInstance cmdlet. To remove the Distributed Cache service from a server on which it was running, don’t simply remove the service. It is recommended that you first shut down the service, and then remove it using the following two PowerShell cmdlets:

Stop-SPDistributedCacheServiceInstance –Graceful
Remove-SPDistributedCacheServiceInstance

Please take note of the –Graceful tag shown in the cmdlet above. The “graceful shutdown” of a Cache Server in a Cache Cluster transfers any cache data to another Cache Server in the Cache Cluster from the server being removed.

Request Manager

The Request Manager is another new bit of functionality in SharePoint 2013 and represents another feature that will only be used by extremely large farms or more specifically large hosted SharePoint farms. The purpose of this feature is to help SharePoint understand more about the requests that it receives and how to handle or throttle them. This feature allows additional “intelligence” (or rules) to be applied to specific types of traffic. In SharePoint farms with multiple servers which are hosting all web applications, a network load balancer would be used to distribute the traffic across all servers with the intent of creating a “balanced” load across all servers as well as avoiding servers which are unavailable. Figure 2-1 shows how these components are arranged in a network and how traffic flows through the environment.

FIGURE 2-1

The Request Management service creates a request routing bus between SharePoint web servers to ensure that specific types of traffic go to a specific server or collection of servers. Examples of the advanced routing capabilities available for various SharePoint requests include the following:

• Based on Server Health Score
• Specific HTTP elements (e.g., host name, service type, etc.)
• Source IP
• Type of traffic (user vs. service)
• Resource availability

An interesting use of this service can be to aid with troubleshooting. As an administrator, there is nothing worse than the error that is received “sometimes.” The “sometimes” is usually associated with one server in a large group of SharePoint servers that is experiencing issues. Using rules to direct specific traffic to just one server enables the SharePoint engineer to observe an issue as it is being experienced, which helps with its identification and resolution.

One other important activity that the Request Manager performs is using rules to identify harmful requests to SharePoint. Should the service encounter traffic that fits that specified pattern, the Request Manager doesn’t route that traffic at all, enabling SharePoint to avoid activity that could jeopardize the health of the farm. It should also be noted that the Request Management service is intended to work with Host Header site collections as the service changes the request header when redirecting traffic (e.g., http://portal.contoso.com to http://spwfe1).

Why not just use a network load balancer for all of this, you ask? Well, an NLB is great at more generic requests. To be fair, the NLB generally has no idea what SharePoint is, let alone what specific types of traffic it receives. The Request Management service adds additional intelligence to direct traffic. Please note that this doesn’t mean it replaces traffic management of the NLB. The NLB is still very much a required component of your large-farm architecture.

To take this to the level of absurdly sized use cases, it is even possible to build dedicated request management farms. These farms are used to direct traffic (based on rules of your choosing) to their own unique farms. This is obviously a relatively unique scenario and one that applies only to large hosting companies. For example, say that you are a hosting company that continuously builds multi-tenant SharePoint farms as the previous one reaches a certain level of utilization. Such a hosting company could manage a database of users and their associated farms. In this case, all customers are directed to the request management farm when logging into SharePoint. The Request Manager determines, based on the customer, to which content farm they should be sent. An enterprise scenario for which this might be a good fit is when the legal department for a company needs to access SharePoint and store their data uniquely separate from all others. A resource management farm could evaluate the user’s identity based on their credentials and direct them to the correct content farm. Very cool, if complex, stuff!

There are three main functions of the Request Manager:

• Throttling and routing
• Prioritization
• Load balancing

These features run as part of the Request Manager, which runs on SharePoint Servers in the farm, which in turn are running the SharePoint Foundation Web Application Service (SPFWA). This is the core SharePoint service that runs on every web server in the SharePoint farm.

The Request Manager configuration includes five elements that determine how the rule logic is evaluated and applied before these elements are routed and to which server they are routed:

• Routing targets/servers — A routing target (or machine) is any server running the SharePoint Foundation Web Application Service. Targets have a weight applied, which grades them as a viable target for traffic. This weight can be a static amount and/or a health weight.
• Machine pool — A collection of routing targets
• Routing rules — A definition of the criteria by which traffic is routed (or abandoned)
• Throttling rules — Rules used by a routing target to evaluate whether it will refuse a request
• Execution groups — A collection of routing rules that controls the precedence of rule evaluation and that manages routing rules in batches. There are three groups (0, 1, 2) that are evaluated in order. If the rule isn’t associated with an execution group, then it is evaluated with those in Group 0.

In the context of the Request Management service, throttling rules are the natural evolution of the throttling introduced in SharePoint 2010, which provided the capability to throttle the amount of content returned from large lists. This myopic approach only considered the current farm. Conversely, request management in SharePoint 2013 allows for “servers based on load” to be avoided to help ensure the perception of a happy-healthy SharePoint farm for your users. For instance, assume it is the first of the month and the new sales figures have been posted via Excel Services. Every salesperson and executive is going to log into SharePoint to see how the company did, thereby putting the servers that serve Excel Services under extreme load. A load balancer can distribute all users across all servers; but in terms of ensuring that the Excel Services servers are providing data quickly, the Request Management service helps balance the Excel Services traffic — and the server can throttle new requests if it starts to struggle under the load.

One last comment about administering the Request Management service: it’s all handled with PowerShell. There is no GUI in Central Administration or anywhere else for that matter that will assist with configuring Request Management. Repeat after me: I will learn PowerShell, I will learn PowerShell, I will learn PowerShell....

Excel Services

Excel Services and the other service applications that are focused on Office client tasks and compatibility features are generally more CPU heavy. This is because they typically do not have any storage and are only being used to offload processing from the clients to the server. Business units utilize these services much differently than other services with SharePoint. Consider the server load when a user opens a Word document on his or her computer from SharePoint to edit. SharePoint only has to work when the file is opened or saved. In the context of Excel Services, when the user opens the file and every time the data is refreshed, the server application has to “compute,” or work. This work all occurs in memory with little to nothing going to disk. To that end, it’s important to consider how your users (or if your users) will use Excel Services. Therefore, don’t overscale for this functionality until you confirm that business adoption will accelerate demand. The three keys to success in managing the scaling of service applications such as Excel Service or Search Query Service? Monitor, monitor, monitor.

Shredded Storage

Despite its unfortunate name, Shredded Storage may turn out to be your favorite new feature in SharePoint 2013. The purpose of Shredded Storage is to bring to SharePoint item versioning the behavior a lot of us thought was there from the beginning. For example, suppose you spend weeks creating a wonderful 10MB PowerPoint presentation that surpasses anything you have ever done before. Unfortunately, your manager likes to get his hands into everything and wants to provide “his input.” In order to ensure that your artistic integrity isn’t damaged, you turn versioning on in your document library so you can easily see what was changed and roll back anything that isn’t to your liking. After several changes on both your parts, you have 10 versions of your 10MB file. In all versions of SharePoint up through 2010 that support versions, you would have consumed 100MB of SQL Server storage. That’s because every version that is saved is a complete copy of the entire document. The magic behind Shredded Storage is that when a version is saved to SharePoint 2013, only the delta (or change) is saved.

It’s time for a quick primer of how SharePoint stores documents. When a file (of any type) is stored into SharePoint, it is actually housed in SQL Server. The way SQL Server accomplishes this is by converting that file into what is called a binary large object, or BLOB. The advent of Shredded Storage doesn’t change this other than saving the BLOB that represents the delta from the last version, which might be only a small portion of the entire file.

A lot is going on behind the scenes to capture all these fragments of files in order to keep track of them and reassemble them upon request by a user (or Index service). We won’t cover all those details here — that’s what TechNet is for — but we will describe the methodology employed to accomplish this Herculean feat.

In SharePoint 2010 Microsoft introduced the Cobalt protocol, or file synchronization via SOAP of HTTP. This was introduced into SharePoint and the Office Client applications with the purpose of limiting the amount of bandwidth required to send incremental changes of Office documents back to SharePoint. Cobalt did a great job of this and end users everywhere rejoiced in the network performance (sarcasm included free with the purchase of this book). Even though in SharePoint 2010 the incremental change was sent from the Office client to SharePoint, the SharePoint server still reassembled the entire document before committing the document back to SQL Server. The reality is this was the building block for Shredded Storage.

Welcome to SharePoint 2013 and Shredded Storage, which now has the ability to only save the delta to SQL Server. A lot of additions were made to the DocStreams table in the Content Database schema inside of SQL Server to facilitate the storage, tracking, and reassembly of various Office documents into one complete document for you to enjoy and show to your friends and neighbors. The goal of Shredded Storage is to limit the amount of data that needs to be transmitted between the client and SharePoint, as well as limiting the amount of data that needs to be stored in the SQL Server content database(s).

Back to the example of the PowerPoint document that began at 10MB and underwent 10 edits and saves. If those edits were just text or slide ordering, then the total amount of storage consumed would be less than 11MB. This feature alone doesn’t significantly change your SharePoint configuration from a service or feature perspective. You won’t be building Shredded Storage resource farms in support of rich document editing. That’s just silly. What you will do as you plan storage for your farm(s) is include much less SQL Server disk space, as the version storage requirements for Office documents will be dramatically less.

One last point before moving on: notice that the preceding refers to Office document storage. Because Microsoft chose to implement this feature on top of Cobalt, it only affects Microsoft documents out of the box. If you are using SharePoint for other document or image types that don’t have Cobalt implemented, then every version saved of that document will consume the total size of that document in SQL Server.

Usage and Health Data Collection

The Usage and Health Data Collection service application might be the most data-intensive Service in SharePoint. It enables the collection of all the diagnostic and usage data from your entire SharePoint farm into the logging folder and the logging database. This database can then be used for reporting, and it is even flexible enough to accommodate custom reporting. Interestingly enough, this service application can be configured in partitioned mode — meaning it is configured for multi-tenancy. The benefit here is that all the reporting information is gathered and can be reported on by tenant. These tenants could be individual companies consuming a hosted SharePoint offering or they could be departments within a large enterprise that are consuming SharePoint in a “charge-back” model.

Results have shown that in large environments, this feature creates a very large SQL Server load. Therefore, in order to fully utilize it, you may want to consider putting it onto its own SQL Server or at least creating a mirror of the reporting data so that you can pound against it with your reporting tools and not create contention for the active database. Check out Chapter 20, “Monitoring and Analytics,” for full details, but be sure to factor the amount of usage of this functionality into your farm planning.

SQL Servers

There are entire multi-book series on this one topic, and even if you read all of them you still wouldn’t have a definitive answer about sizing your SQL Server. Therefore, as you approach sizing this particular box or boxes, don’t be afraid to ask your friendly neighborhood DBA for help. In addition, keep in mind that over the years, the main bottleneck in most SharePoint farms is SQL Server performance. As a side note, this has been true since SharePoint started saving all of the user data in SQL Server. If anything has changed, it has changed for the worse as more service applications are created that rely on SQL Server to store their information.

The key thing to remember is that all the standard SQL Server hardware best practices are important. SQL Server loves memory and will utilize every bit it can get its hands on, so you should plan accordingly — 16GB of memory should be the absolute minimum you consider, and 32GB or even 64GB might be appropriate in a heavily used production environment. CPUs require the same consideration; a quad-core processor might get you started, but boxes with multiple hex-core processors are more common.

Even if you buy enough CPU and RAM, you still are not out of the woods. Disk configuration has as much, if not more, to do with performance. You need to plan for the number of spindles your SQL Server has access to and how they will be configured; and to do this properly, you must consider the amount and shape of data you plan to store in SharePoint. You should be following the SQL Server best practices specifying that the data (*.mdf) and log (*.ldf) files are on different disks, and that the log files are optimized for write. When you are considering which databases to optimize first, the order is as follows:

While tempdb should clearly always be the first database optimized, your needs for the Search databases and content databases will vary according to your specific scenario. For example, if you have created a content database for collaboration that is excessively large (greater than 200GB), then in order to minimize locking issues you may need to move that database to optimized disks instead of the typical content database that performs adequately on a basic RAID 5 volume. The key here is to make sure either you understand all your SQL Server disk requirements before you purchase the box or you have access to a flexible solution, such as a SAN, for storing your databases.

If you are dealing with an inordinately large amount of users and/or WFEs, consider having more than one SQL Server (or cluster). This enables you to spread your content databases across multiple SQL servers and helps to limit the bottleneck of a single SQL server. Bear in mind that this solution is for the extreme scenario of tens of thousands of users in a single SharePoint farm.

Finally, when it comes to SQL Server, SharePoint doesn’t really care how you set it up. As long as SQL Server is running a supported version and can serve databases back to SharePoint, it doesn’t matter whether SQL Server is dedicated to SharePoint or is shared with other applications in the company. Nor does SharePoint care whether SQL Server is clustered or doing database mirroring or even transparent encryption. SharePoint simply calls to a SQL Server instance for a database, and if data is returned it is happy. Your users just might not be happy with the responsiveness of SharePoint if SQL Server is architected incorrectly.

Mixing and Matching Servers

Now that you have an understanding of the different types of servers, you need to consider how they will be deployed onto actual hardware. As you combine them, you need to consider the hardware profile of each, and how the server will need to support the aggregate of SharePoint roles.

One Server

This is a configuration you will typically see only for demonstration and evaluation purposes. In the example shown in Figure 2-2, all SharePoint server roles and the SQL Server are configured to run on one machine.

FIGURE 2-2

Microsoft’s recommendation for a “one box wonder” whereby SharePoint and SQL are installed on a single machine is as follows:

Two Servers

A two-server configuration is generally considered the minimum point of entry for a small SharePoint deployment. In this scenario, shown in Figure 2-3, all the SharePoint services run on one server, and SQL Server runs on a separate server.

FIGURE 2-3

Following are Microsoft’s recommendations for installing SharePoint as a standalone server. Note that these are minimum requirements. As mentioned previously, memory is cheap compared to angry users because SharePoint is slow. Do yourself a favor: double the memory.

Three Servers

Adding a second server with SharePoint installed enables the creation of a high-availability solution (see Figure 2-4). By putting some type of network load-balancing (NLB) device in front of SharePoint, you can ensure that the WFE services are fault tolerant. Then, by configuring the service applications to run on both machines, you can avoid one server crashing and ruining your day. Note that everything you add to the SharePoint farm affects the performance of that farm. It was just pointed out that by adding a second WFE to the farm you increase the fault tolerance of the farm. Conversely, putting a substandard NLB in front of SharePoint might lower its performance even though you have increased the farm’s resiliency. When a quality NLB (and it is suggested that you use a hardware-based NLB dedicated to the single role of load balancing) is placed in front of your SharePoint farm, performance can actually increase. The NLB manages the traffic coming through itself and balances the load across each WFE.

FIGURE 2-4

In terms of hardware requirements, as a general rule they remain the same for the two-server configuration described in the preceding section.

Four or More Servers

This is where you start making choices. Figure 2-5 shows a scenario in which the environment has been optimized for performance and availability for WFE and Query, but the downside is that the application tier does not provide high availability. This is generally not a good idea, so you may want to jump straight to introducing a fifth server to the farm in order to bring high availability to the other service applications. You will not need another NLB device because service applications handle their own load balancing.

FIGURE 2-5

At this point, you probably get the idea that you can scale out any of the various service applications as necessary to meet your needs, which leads to our next topic: server groups.

Server Groups

Server groups refer to the logical concept of grouping similar SharePoint service applications together on the same logical server. This enables you to add servers, which means additional capacity, to each tier as demand increases. This also segregates the performance impact of the various service applications. Figure 2-6 shows an example.

FIGURE 2-6

This example isolates the web, search, business intelligence (BI), and all the other service applications. Now, if business adoption of the BI increases beyond the current capacity, it will not affect the performance or stability of the rest of the farm. It is also simple to purchase another server, install SharePoint onto the box, and then add it to the farm. Once it is a member, you would then add it to the BI group by configuring it to run only the Excel and PowerPivot services. Note that you will not see the term “server group” in Central Administration anywhere; it is only a logical concept.

Notice also in Figure 2-6 that SQL Server has been extended into various logical groupings. The performance characteristics and demands of different databases can vary greatly, and in large environments it can be very helpful to configure and manage each one separately. It should be noted that for every SharePoint and/or SQL server added to a farm, you increase your licensing exposure. Carefully choose server additions based on verified demand by your business units within your company.

Other Hardware Notes

Now that you are familiar with the hardware and server configuration options, the following sections describe a few more important considerations as you plan your farm’s architecture.

The Network

Network connectivity between the servers in your farm is hugely important. At a minimum, all servers in the farm should be connected through gigabit connections. The hard requirement here is that each server should be connected by a gigabit connection with less than one millisecond of latency. Can you have greater latency between servers? Yes. Will your users like you for this? No. This precludes most companies from having a SharePoint farm with servers in multiple, geographically dispersed data centers attempting to act as one farm.

For many companies, the sheer volume of network traffic generated between the members of the farm is overwhelming. In order to better control this traffic, they move all inter-farm traffic to a dedicated virtual local area network (VLAN). This is like the server groups discussed earlier. Grouping all this traffic makes it easier to monitor and administer in the case of any issues. A dedicated VLAN is not a requirement for SharePoint, but in a large farm it is often recommended. This is also a way to create a secure boundary between the SharePoint environment and the rest of your network — provided it is a requirement by your security department to create that level of server-based isolation.

Network Load Balancers

In order to achieve high availability of the SharePoint web applications, it is necessary to introduce a tool to do network load balancing. This can be either a hardware-based tool, such as an F5 device, or an external software solution, such as Microsoft’s reverse proxy server solutions, or even something as simple as the built-in Windows Network Load Balancing (NLB) feature.

Hardware-based solutions are generally best, as they offer the most configuration options and usually the best performance, but they are also typically very expensive. The software-based solutions such as TMG provide a happy middle ground, especially if you are already using them in your environment. They include just enough options and monitoring to make the cut. Although using the Windows NLB is free because Windows provides it out of the box, it is a very rudimentary feature. It cannot do tasks such as confirm whether the server is serving valid pages, and instead only confirms that the server responds to ping traffic. For mission-critical scenarios this is not an ideal solution. At the end of the day, SharePoint is a web-based application. It is generally recommended that you use whatever standard your company has for NLB solutions.

Prior to SharePoint 2013, SharePoint cached too many requests and didn’t share that cache across WFEs, so if users were constantly moving from server to server, it was possible for them to have erratic results. In the bright shiny morning of 2013 and the new reality of distributed cache — specifically, by caching the FedAuth token — it is no longer necessary to use “sticky session” on the load balancer.

If you are adding an NLB to your farm because a high-availability solution is required for your environment, then ensure that you make every aspect redundant — including the NLB! Having redundant WFEs is of no benefit if your NLB fails, thereby blocking network traffic to your SharePoint servers. The NLB provides additional value in that it is listening /verifying a server is “alive” and can also monitor additional HTTP headers like the SharePoint Health Score, or listen for 503 errors that are produced in the event of an unhealthy server.

Server Drives

When you configure SharePoint, it is generally a good idea to remove everything possible from the C: drive. For example, navigate to Central Administration and change the diagnostic logs to be hosted on the E: drive instead of the C: drive. Remember that this is a farm-wide setting, so all servers in your farm must have an E: drive or they will get errors and stop logging. This is inconvenient but not the end of the world. The end of the world happens if you try to add a server without an E: drive to this farm after changing this setting. You will get file I/O errors running the configuration wizard and it can take a long time to figure out the cause. That is why it is recommended that all the SharePoint servers in your farm have standard drive letters. A simple design choice like that can greatly reduce your headaches going forward. It is also very important to configure your disk drives with some form of redundancy, such as mirroring (also known as RAID 1). This gives your data security should one of the hard drives fail and also has the ability to provide performance increases on reads and writes to the drive.

Virtualization

Now that you have learned about hardware considerations, the question that logically follows is, “Which servers can I virtualize?” After looking at some of those server groups, it is very easy to see some opportunities. Typically, when it comes to virtualization, it is recommended that you start at the top of the farm and work your way down. Web front ends have almost no disk requirements and generally are consuming RAM and some CPU. They virtualize very well. How well application servers virtualize depends on what service applications they are hosting. For example, if you have a server group that is hosting only services such as Office Web Apps or the Managed Metadata service, they would virtualize easily — again, because they have almost no disk requirements.

Your query and index servers can be virtualized but the benefit will depend on your performance expectations. Crawling 10 or 20 thousand items once a day is a pretty light load, and will virtualize without issue. However, trying to crawl 100 million items virtualizing and still getting the performance you need would be extremely difficult; but just because it’s difficult doesn’t mean that it can’t be attained. If your implementation is one that has a user community in the same geographic location, then you can assume that your farm will sleep for half of the day. If this is the case, then you can be a little more liberal about virtualizing the index server. Just make sure you configure the indexing to occur when no one is using the farm. You will also want to work with your virtualization engineers to have that server configured to be allowed to use additional physical resources if they aren’t being consumed by other processes.

The moral of the story when it comes to virtualizing SharePoint servers is that you should first understand how hard that server is working and where its first bottlenecks occur. If you can safely virtualize without reducing performance, then do so. Also, if you are building test and development environments for which performance is not a critical factor, then virtualization is the way to go.

Should you virtualize SQL Server? That question generates almost as much passionate debate as Mac versus PC. Virtualizing SQL Server and achieving acceptable performance is possible; unfortunately, your average virtualization administrator, in partnership with your average SQL Server database administrator, cannot do it well. It is too complex a configuration for someone to stumble through. As a SharePoint administrator, you already know that SQL Server is going to be the factor holding your farm’s performance down; do you really want to gamble with virtualization, which is likely to further decrease that performance? Remember disk I/O on the SQL server is king.

One other point to make about virtualizing SharePoint server is the impact it can have on hosted (remember FIS) licensing. Hosted licenses license the entire “processor socket” in a Services Provider License Agreement (SPLA) model. That socket has many cores that can be divided up among several SharePoint servers. For instance, in a traditional on-premises installation for which you had two SharePoint WFEs for a Hosted (FIS) site, you would be required to purchase two separate licenses even if these were virtualized. In a hosted SPLA model whereby the license is applied to the processor socket and not the server, one license could be applied to two SharePoint VMs that each had four virtual CPUs (cores) of an 8-core processor.

TERMINOLOGY

One of the biggest challenges for SharePoint administrators new and old is the vocabulary. SharePoint is littered with words, such as “site,” that have about a dozen different meanings (no one is ever really sure what a site actually is, and many consider it apt that site is a four-letter word). To clarify this confusion, Figure 2-7 is here to save the day. Once you can speak to the entire hierarchy from top to bottom your job is half complete, you have practically conquered SharePoint — so study hard.

FIGURE 2-7

Starting from the top you see “Farm = Configuration Database.” This means that each SharePoint Server can belong to only one farm. A farm can be a single server (refer to Figure 2-2) or something as complex as what is shown earlier in Figure 2-6. A farm refers to all the servers using the same configuration database. When you run the configuration wizard, you choose to connect to an existing configuration database (join a farm) or create a new configuration database (create a farm). All servers in a farm, therefore, share everything, including the Central Administration site that controls all servers in the farm.

Below this, in the column on the right, is “Services.” These are the actual services on the server that run to provide functionality to the “Service Applications.” For example, the Excel Service Application you create in Central Administration from Manage Service Applications is a Service Application Connection point. That Service Application Connection point is the proxy to the instances of the Excel Service that are running on the server(s) in the farm. Don’t worry if that isn’t completely clear; Chapter 4, “Understanding Service Applications,” is dedicated to the inner workings of service applications.

Finally, at the bottom of the right-hand column is “Service Application Databases.” Some service applications require database(s) to store information in order to work, while others do not. This is just one of the many reasons why using SharePoint 2013 means you will be getting to know SQL Server.

“Web Applications,” at the top of the left column, are the actual SharePoint websites you visit. Because they appear in IIS, you will hear people refer to them as sites, websites, IIS sites, and other creative things. However, it is very important that you refer to them as web applications, or web apps for short, as everything in the SharePoint management interface and all the documentation always refers to them as web applications. Examples are http://portal.company.com, http://www.company.com, https://extranet.company.com, or http://team.

Between Web Applications and Service Applications is a double-headed arrow labeled “Many to Many.” This is your reminder that this is the only place on the hierarchy with a many-to-many relationship — in other words, one web application can consume multiple service applications, and service applications can also service multiple web applications. This is one of those seemingly infinite configuration options that makes SharePoint so much fun to architect.

Every time you create a new web application, SharePoint automatically creates a new “Content Database” for you and associates it with the web application. This will be the default location for storing content from the web application. You can also create additional content databases to associate with the web application. This is done to help scale. Two unique web applications cannot share a content database.

That brings you to the most important concept in SharePoint: the site collection. Site collections are the unit of scale in SharePoint. The easiest way to think of a site collection is as a bag, because they are really just a boundary or container. They are not actually content users can touch. The reason why this “bag” is so important is because it determines a lot about how your information is stored.

Site collections are a storage boundary and are stored in one and only one content database. They cannot span multiple databases. When you create a site collection it is created in a database and that is where it stays unless you manually move it. If, for example, you want to limit all your content databases to 40GB because that is the largest size you are comfortable with, then you need to ensure that no site collection is larger than 40GB. Similarly, if you have multiple site collections (and everyone does), then you would need to apply quotas to those site collections to ensure that the sum of the site collections doesn’t exceed your 40GB database limit. For instance, if you have 10 site collections, then you set your quotas to 4GB per site collection.

Normally we would state that the strongest security boundary (other than a new farm) would be at the Web Application layer. When your farm is deployed in multi-tenant mode, the site collection is also a security boundary. In this mode a customer (or tenant) is assigned a subscription ID. This ID becomes the identifier for everything that stores data in SharePoint. From content DBs to Search DBs to managed meta data — all of that data is segregated inside of the common databases by this subscription ID. This way, content from one tenant doesn’t “leak” into another tenant’s site or search data or term store.

Site collections are the only objects in SharePoint to which you can apply a storage quota. If you want to limit a user to storing only 10GB of content in a particular document library, there is no way to do that. You would have to set that entire site collection to a 10GB limit. If you have two document libraries and you want to give each one 10GB of storage, then you have to ensure that each document library is in its own site collection.

Even if you have no intention of holding users to limits, quotas are generally recommended for all site collections, as they serve as a checkpoint and help prevent runaway site collections. When users call to say that they are getting warnings or errors because they have met their quota, it is a simple process for you to increase the quota, and it gives you a chance to ask, “So what are you doing with SharePoint that you need so much storage space?” It would be good to know if someone is just backing up an entire MP3 collection to SharePoint.

Site collections also serve as an administrative boundary. Site collection administrators are a special group of users who have complete power over the site collection without necessarily having any access to other site collections. The Site Settings page contains an entire menu of configuration options that only a site collection administrator can modify (see Chapter 8, “Configuring SharePoint for Business Intelligence”). For example, suppose you have two groups, such as HR and Accounting, in the same site collection. If one of them comes to you because they need to administer one of these special settings, you will have to do some rearranging. If you make Nicola from Accounting a site collection administrator, then she can fully administer the account site as needed, but she also has full control over the entire site collection, including the HR web. You need to instead move the Accounting web to its own site collection and then make Nicola an administrator there.

Site collections are also boundaries for out-of-the-box functionality such as navigation and the various galleries. This can be a drawback of many site collections. Out of the box, it is impossible to enforce consistent, self-maintaining navigation across site collections. The galleries such as the themes, Web Parts, lists, and solutions are all scoped at the site collection level. For example, if you need a list template to be available to multiple site collections, then you have to manually deploy it to each.

Site collections also serve as security boundaries. The All People list and the various SharePoint groups are all scoped at the site collection level, and are not accessible for reuse outside of the site collection.

Inside of site collections are one or more webs. A web is the object that is referred to throughout the user interface as a site. It can also be called a subsite or a subweb. Again, because the term “site” can be very confusing, whenever possible refer to these as webs. This is the first object users can actually touch. You can apply security to it, and it contains all of the user content. Each web has its own lists (libraries are just a special type of list). and all those lists store items, which refers to the actual content, such as documents and contacts.

As you look at the hierarchy from web applications to items, remember that it is a one-to-many relationship going down but a one-to-one relationship going up. That is, an item can belong to only one list, a list can belong to only one web, a web is part of only a single site collection, a site collection lives in only one content database, and a content database can be associated with only one web application.

Still a little fuzzy? Try this analogy to understand how these pieces work together: Web applications are the landfill. Content databases are giant dumpsters. A site collection is a big, black 50-gallon garbage bag. Webs, lists, and items are pieces of trash. Your users spend all week creating garbage, continuously stuffing it in the garbage bags, with each piece of trash occupying only one garbage bag at a time. Each garbage bag can hold only 50 gallons of trash (quotas) before it is full, after which the user has to either ask for a new garbage bag or get a bigger garbage bag. That full garbage bag is placed in a dumpster, and it is not possible to put a garbage bag in more than one dumpster without destroying it. Dumpsters are serviced only by one landfill but that landfill can handle thousands of dumpsters without issue. How was that? Clear as mud?

CONTROLLING DEPLOYMENTS

SharePoint 2013 ships with more than a handful of tools that help you keep it under control. First take a look at the different throttling capabilities of SharePoint and then things to consider when it comes to SharePoint’s recycle bin. With all of those tools in hand you then you can review the primer on the boundaries of SharePoint to make sure you keep performance in check.

HTTP Throttling

A potential challenge SharePoint administrators have faced in the past and are certain to see again is lack of resources and the odd behaviors it produces. One scenario is an overworked WFE server. As a WFE is processing requests, it can reach a point where it is not immediately responding to a request due to a lack of resources. It will then begin to queue the requests, but it has a limited capacity for storing requests also. If the queue fills up, then it will just start indiscriminately dropping requests until it catches up. While this is not a big deal for a typical GET request, what if you are a user who has just spent an hour taking a survey or filling out an application? If that PUT request is dropped, your hour was spent in vain and you will have no option but to start over.

To avoid this issue, Microsoft introduced HTTP throttling to protect a server during peak load. By default, this feature monitors the available memory in megabytes and the ASP.NET requests in queue. As it monitors these counters, it generates a health score for the server on a scale from 0 to 9, with 0 being the best. The monitor checks every five seconds by default. If the score is 9 for three consecutive tests, then the server will enter a throttled state. In this throttled state, SharePoint will return a 503 server busy message to all GET requests, including the crawler if you happen to be indexing. In addition, all timer jobs are paused, which enables the server to concentrate on finishing existing requests and hopefully makes room for anyone doing a PUT request, like that user who just spent an hour filling out a form. The monitoring continues every five seconds, and throttling is disabled after one occurrence of a score below 10.

You can configure this feature using Central Administration, to be enabled or disabled per web application. Using Windows PowerShell, you can go a step further and view and edit the thresholds using the following cmdlets:

• Get-SPWebApplicationHttpThrottlingMonitor
• Set-SPWebApplicationHttpThrottlingMonitor

The health score is exposed to all HTTP requests. If you use a tool that enables you to inspect your web traffic, such as Fiddler (www.fiddler2.com), you will see in the header under Miscellaneous the value X-SharePointHealthScore. Where this truly comes into play is with the Office clients. The Office 2010 and 2013 client programs are aware of the score and can use it to adjust their behavior. For example, Word uses it to determine how often updates should be requested when handling live co-authoring of a document.

Large List Throttling

Compared to SharePoint 2010, a few refinements have been made to throttling in SharePoint 2013. SharePoint 2013 supports lists of up to 30 million items. Previous versions of SharePoint did recommend not exceeding 2,000 items in a list view because of the strain it put on your farm’s performance. Think about what happened behind the scenes when a user tried to view 3,000 items in a list. First, the SQL Server had to generate a query to return all 3,000 items at once. Next, that information had to be sent to the WFE server and added to the page. Finally, the user had to download the page with its 3,000 items and wait on Internet Explorer to render all that content. It could literally take minutes to return the page. Sadly, there was no way to stop users from doing this or even to monitor that activity until now. SharePoint 2013 improves this scenario.

SharePoint 2013 provides controls that you can configure to prevent the bad behavior listed previously. Figure 2-8 shows the Resource Throttling screen in Central Administration. You can access this screen by navigating to Application Management ⇒ Manage web applications. Then select your web application, click the drop-down for General Settings, and select Resource Throttling. All default settings are shown.

FIGURE 2-8

The List View Threshold, which is set to 5000 by default, represents the maximum number of items a standard user can return in a view. As users approach the limit, they will see the dialog shown in Figure 2-9, which indicates how many items they have and where the throttling limit is set.

FIGURE 2-9

The following relevant settings are available:

• Object Model Override — This setting specifies whether a developer can override the throttling through the object model code to allow their code to run.
• List View Threshold for Auditors and Administrators — This setting is used to grant special power users a larger threshold. You can set a user up as an auditor through the Manage web applications screen. You first add a permission policy and enable the Site Collection Auditor permission policy level. Then, using User Policy, also on the Mange web applications Ribbon, select the new permission level you created.
• List View Lookup Threshold — This setting controls the number of lookups that can be specified.
• Daily Time Window for Large Queries — This setting is also referred to as happy hour. It allows you to set a time of day when throttling is disabled and views are unrestricted.
• List Unique Permissions Threshold — This setting limits the number of unique permissions a given list can have. This is a good idea, as you can run into performance problems if a list has too many unique permissions coupled with too many items. Security trimming is a great but sometimes expensive feature.

The remainder of the settings are not part of the list-throttling feature.

When users exceed this limit, they will see a warning message in the browser stating, “Displaying only the newest results below. To view all results, narrow your query by adding a filter.” This will show the last 1,000 modified items.

Recycle Bin Architectural Implications

The Recycle Bin? Are you serious? Isn’t that like designing a kitchen around the garbage can? Not really. When a site is deleted in SharePoint 2013, it isn’t truly deleted — it’s sent to the recycle bin. The implication for you, our dazed SharePoint administrator, is that you have to account for SQL Server disk storage of potentially very large sizes. If a user deletes a site that contains 100GB of storage, your SQL server still has to store that data. It will of course be removed when the recycle bin flushes itself or you manually remove it.

One key with the Recycle Bin is keeping its scope in mind. If you delete a list item, folder, a whole list, or even a web (aka subsite), those items will all appear in the site collection Recycle Bin and can be recovered using your browser. If you delete a site collection then you have to go to the SharePoint server and use your friend PowerShell. The cmdlet Get-SPDeletedSite will help you get started. Or better yet go take a look at Chapter 9, “Configuring SharePoint for High Availability Backups,” for the full scoop on the Recycle Bin.

Software Boundaries and Limits for SharePoint 2013

Trying to keep up with all of the boundaries, thresholds, and limits for SharePoint is probably very similar to keeping up with the US tax code from year to year. Everything you could ever hope to know is there right beside bizarre things you don’t even understand why you should care about. All of the information is there, but generally speaking you should focus on the information most relevant to you.

As you dive into these different considerations for SharePoint one key piece of advice is try to think about why a specific threshold might be in place and what they are trying to tell you with that limit. Some examples:

• 5 zones per web app — SharePoint has been this way since SharePoint 2007 and isn’t configurable or exceedable. SharePoint law!
• 20 managed paths per web app — The idea is to have as few as possible because SharePoint has to check the whole list every time you make a web request. If you introduce 3 or 4 to meet a specific need, you open Pandora’s box. All of the sudden 3 or 5 becomes 50 and all of SharePoint suffers. So even though the threshold is 20 really just using the out-of-the-box managed path might make the most sense.
• 10 application pools per server — This one is trying to protect you from having 1TB of RAM. If money is no object and you feel like your server has too much RAM stretch this limit as far as you want.
• Database size limits — Is it 200GB or 4TB or infinite? The answer is it depends. 200GB works well if you don’t like to think. But the more important question is not how big of a database can SharePoint support but instead how big of a database can you support? Can you really back up, maintain, and move back and forth to test a 1TB database? If not then why are you worried about SharePoint’s upper bounds?

Keep the preceding in the back of your mind as you read through the limits of SharePoint. Some limits are financial, some are because SharePoint cannot handle it, and some are trying to keep you out of trouble. Also, if you do try to test a limit with your initial solution consider the possibility that growth over time might cause you to blow by the limit and create problems in the long term.

This topic is really a doozy. Microsoft has a great article on TechNet that offers guidance on all things related to the boundaries and limits of SharePoint. This document is also updated on an ongoing basis based on new findings, patches, and service packs. Therefore, instead of killing trees to copy and paste content that will be changing over time, here is the link to the web page: http://technet.microsoft.com/en-us/library/cc262787.aspx.

The key point is: don’t ignore this page. You cannot build a farm with a 10-terabyte site collection and then be mad at SharePoint when nothing works. If you read this page early and often you can avoid known issues.

SUMMARY

Wow! What a fun ride. As you can see now, SharePoint’s architecture is a vast topic that seems to get more vast with each version and even each patch. After reading this chapter you should feel confident in laying out a farm topology and all of the components that go into that decision. Those components are everything from the role of SQL Server and email to how distributed cache and Request Manager allow you new flexibility.

And of course as you plan all of that out don’t forget that maybe Cloud is part of or even your whole solution. A few years ago that conversation wasn’t even on the table but in today’s world it is very real. So make sure you run it up the flag pole. The same way virtualization used to only be for testing yet today almost every SharePoint farm has some portion virtualized. Cloud is coming and the more you know the better off you are.

All set? Great. Now that you know the pieces of the puzzle dive into the next chapter and start assembling the puzzle.