← Main Project Page

Technical Project Charter

This document describes the Enterprise Portal technical design and provides technical descriptions that will be implemented throughout the Enterprise Portal rollout. The first step to achieve this, is the development of an architectural design. Microsoft Office SharePoint Server (MOSS) 2007 was selected for the next generation Workspace collaboration technology.

This document will be provided to Microsoft to achieve design validation.

Technical Risk Items

Technical risk items include

• Ability to hardware load-balance
• Sufficient storage size and proper design
• Backup and Recovery of sites and item-level
• Ability to CAC-authenticate
• Extranet users Smart Card authentication and aackup authentication method
• Integrating eTools Launch Site with SharePoint without dependency
• Challenges with custom web part development.

Technical Assumptions

It is assumed that SharePoint will eventually replace the current BEA (Plumtree) Portal. It is not intended to replace File Shares. The goal of the design is to create a highly available (dependable) portal for collaboration of documents. Secondary to that are Web Content Management, Publishing and Business Intelligence.
To achieve the design goals, the SharePoint architecture will employ the use of multiple front-end servers, a dedicated front-end for the extranet portion, a powerful indexer and several databases servers to spread the load. Virtualizing the infrastructure will give this company the benefits that Exchange and other architectures have.

Project Personnel

See POC list.

Overall Architecture

There will be a SharePoint Farm that consists of Web Front-End & Query servers, Index server, Application servers, and SQL Database servers. Additionally, to support the SharePoint Farm there will be several ISA 2006 servers (CAC Authentication) and SnapManager servers for backup and restore.

In order to properly explain how the SharePoint Farm hardware is scaled, we must define the Application Architecture first.

Application Architecture

Site Collection hierarchy will have a top level URL of https://[TBD].domain.com. All Site Collections belonging to the portal will be prepended with /sites and use a human-readable site name (e.g.: Boeing_Atlanta).

Assumptions based on data gathered by eBusiness and the Web team indicate we need to plan for

• 250 Site Collections that will hold about
• 4.6 TB of data over 12 months.
• Intranet Access for 11,000 Internal Users (CONUS/OCONUS).
• Extrnet Access for 5,000 External (Public) Users.
• This will not replace www.company.com
  

Hardware Configuration

The hardware architecture consists of the following components and configurations:

Manufacturers Minimum Specifications

Microsoft has determined the following minimum requirements.

Server

Vendor Notes

• You must download an update for Windows Server 2008 before you run Setup, or Setup will not run. For Windows Server 2008 with SP2, see FIX: A hotfix that provides a method to support the token authentication without transport security or message encryption in WCF is available for the .NET Framework 3.5 SP1 (http://go.microsoft.com/fwlink/?LinkID=160770).
• Uninstall PowerShell 1.0 before running Setup.
• For the database server, install Cumulative Update 2 with Cumulative update package 2 for SQL Server 2008 Service Pack 1 (http://go.microsoft.com/fwlink/?LinkId=165962). This cumulative update consists of two parts; ensure that you download both files. When you install Microsoft SQL Server 2008 SP1 on Windows Server 2008 R2, you might receive a compatibility warning. You can disregard this warning and continue with your installation.

IIS

• Keep IIS in its default location. Service updates most likely will look for INETPUB in the default location, so those folders can not be deleted. See IIS.NET

Servers

Server Roles

Proxy (ISA Server 2006)

Web Front-End (plus Query)

Crawl (Index) Server

Application (plus CA Web Front-End)

Database Server (SQL)

Security

The following are considerations for security:

• CAC Authentication per JTF/GNO will be provided by ISA 2006 and Kerberos Constrained Delegation.
• Personally Identifying Information (PII). Fidelis will be used to send TCP resets to any clients attempting to load documents or other content containing PII.
• Sensitive information. SharePoint Search will be configured to attempt to identify this.
• Extranet authentication can use DoD Smart Card with a failover option of Forms-Based Authentication using ISA 2006. The Design Team is currently validating several options.

Service Accounts

• Review: Todd Klindt's recommendations
• Resource: Service Accounts & Permissions Excel Workbook
• Resource: Managed Service Accounts from Gary LaPoint

Sample Service Account List

Active Directory Groups

The following lists "administrative" Active Directory Groups used for the SharePoint Infrastructure:

External Access Control (Firewall)

Review Firewall Ports for SharePoint.

• Additional resource: UK SharePoint Blog
• Additional resource: Joel Oleson

1. The firewall must be configured to allow inbound traffic on port 443 to the ISA 2006 servers. The SharePoint Web Front-Ends will not be accessible from the internet. The ISA server acts as a proxy for SharePoint and protects it at the application level.
2. The SharePoint Web Front-Ends will reside in the Authorized Web DMZ and will traverse the backend firewall to access SQL data.

Logging

1. SharePoint has several log files. They are the Trace Logs (for developer level error tracing), Usage Logs (to collect site usage and analysis) and Diagnostic Logs (to collect information for the Administrators).
2. Existing Log Aggregation software will be leveraged for SharePoint logs as well as the OS Event Logs.
3. Uncontrained logging will not be turned on and SharePoint Logs will be set to expire (auto-delete) after they have been collected.

Patch Management

1. SharePoint is an application 'platform', rather than another Windows application. Therefore, it requires careful planning to deploy patches. Many Windows and Office patches will require special SharePoint tools (such as the SharePoint Products and Technologies Wizard or psconfig) to run after the patch has been applied to clean up the database.
2. An exemption to the current process will be required to properly implement a patch management plan that does not break SharePoint services.

Authentication

Authentication is provided by Active Directory using the Kerberos protocol. See Authentication

Authorization

1. Authorization is provided via SharePoint Groups. Each SharePoint Group defines a role that specifies to what level a user can access any given content. SharePoint Groups will have a 1 to 1 relationship with Active Directory groups.
2. This will allow the Helpdesk and Operations to add and remove users in SharePoint

Encryption

The web application will be encrypted using an SSL certificate.

User Management

Generally, SharePoint sites have the following roles:

• Owner - Can design the site (create libraries) and change permissions
• Contributor - Can edit and delete files in existing libraries.
• Reader - Can only view content.

All roles will be explicitly granted. This means that if a user is not in any group, they will be denied access, including read access to that site. They will have the option to request access through SharePoint.

Each of these roles will be defined by a SharePoint Group (for example, SiteCollection1_Owner, SiteCollection2_Contributor, etc). A SharePoint Group is like a container that will hold one or more Active Directory security groups.

• We will leverage existing infrastructure and business processes to maintain these groups. For example, the Helpdesk will have rights to add and remove users from Active Directory security groups that correlate to SharePoint Groups.
• Additionally, the Helpdesk will have rights to create groups in a specific OU for the purposes of assigning site permissions in SharePoint.
• These Active Directory groups will map to CMO's, Divisions, Centers and like organizational units that are easily distinguishable.

Roles

The Owner role will be tightly controlled by IT Operations. This will prevent self-service site creation which can lead to orphaned sites, objects, files, permissions and prevent users from creating security holes.

Follow-On Actions

• The naming convention within Active Directory must make it obvious which groups belong to which sites.
• Active Directory groups will be cleaned up and fine tuned.

Storage Considerations

SharePoint makes heavy use of SQL databases and SQL logs. These components make up most of the content. However, additional components that will use storage are the index, search, query and administrative databases. The databases should be separated in different volumes to increase performance. Separating databases and logs will also significantly increase performance.

Capacity Projection

The Capacity projection detailed in this section is based on maximum system capacity using the basic requirements from E-Business. Using the basic requirements and the size of each Site Collection (projected via data collected from E-Business regarding CMOs and Projects), the capacity required is calculated as below:

1. Content Sizing
   • Base Requirement from E-Business: 250 Site Collections
   • Size of each Site Collection: between 5GB and 25GB

   (25GB will be used for size calculations)

   • Maximum size of a Content Database: 50GB Data, 15GB Logs

   (100GB is the recommended limit by Microsoft, this number is scaled down based on recommendation from NetApp's best practices to maximize performance)

   • Number of Site Collections per Content Database: 2
   • Number of Content Databases required: 125
   • Maximum content size per SQL Server: 20 Content Databases

   (based on NetApp's best practices to maximize performance)

   • Number of SQL Servers required: 8

   (The actual number based on requirements is 125/20 = 6.25, which means 7 servers is the minimum number of servers required with some growth capacity. To make sure there are ample growth capacity, the initial build out will be 8 servers.)

   • Storage requirements for Content: 7.8TB Data, 2.3TB Logs
2. Index, Query, and Search Sizing
   • Number of Front End servers: 4

   (Each Front End server contanis its own set of Query data)

   • Projected size of Query data: 9.6TB

   (30% of 7.8TB of Content Data x 4 Front End Servers, based on Microsoft best practices)

   • Projected size of Index data: 2.4TB

   (30% of 7.8TB of Content Data, based on Microsoft best practices. This can be hosted on SATA)

   • Projected size of Search Data: 1.3TB
   • Storage requirements for Index, Query and Search: 13.3TB Logs

   (9.6TB + 2.4TB + 1.3TB = 13.3TB)

3. Backup Sizing (based on NetApp's Snapmanager for SMOSS/SMSQL)
   • SQL Snapshot info: 18.4TB
   • SMOSS Media Index: 1.1TB
   • Storage Requirements for Backup: 19.5TB

   (Based on NetApp's best practices. Both can be hosted on SATA)

Capacity Proposed

The table below are numbers derived from what the company is estimated to use as of deployment.

The capacity calculation is as below

1. Content Size
   • Number of Content Databases required: 92
   • Number of SQL Servers: 8

   (This number is kept at 8 for growth)

   • Number of databases per SQL Server: 12

   (92/8 = 11.5, rounded up to 12)

   • Storage requirements for Content: 4.7TB Data, 1.4TB Logs
2. Index, Query, and Search Sizing
   • Number of Front End servers: 4

   (Each Front End server contanis its own set of Query data)

   • Projected size of Query data: 5.6TB

   (30% of 7.8TB of Content Data x 4 Front End Servers, based on Microsoft best practices)

   • Projected size of Index data: 1.4TB

   (30% of 7.8TB of Content Data, based on Microsoft best practices. This can be hosted on SATA)

   • Projected size of Search Data: 1.3TB
   • Storage requirements for Index, Query and Search: 8.3TB Logs

   (5.6TB + 1.4TB + 1.3TB = 8.3TB)

3. Backup Sizing (based on NetApp's Snapmanager for SMOSS/SMSQL)
   • SQL Snapshot info: 11.2TB
   • SMOSS Media Index: 1.1TB
   • Storage requirements for Backup: 12.3TB

   (Based on NetApp's best practices. Both can be hosted on SATA)

The total capacity proposed moving forward is 13TB FC and 13.7TB SATA

Storage Best Practices for SharePoint

• Distribute Content Databases – ~100GB (75GB if using NetApp storage + VMWare)
• Defrag SQL Database(s) – http://support.microsoft.com/kb/943345
• Use DNS alias for SharePoint SQL Server Instances (SQL Alias)
• Monitor Database & Log Disks -Disks Reads/Writes, Disk Queues
• Use Dual Fiber Channel Paths to storage
• Use RAID 10 for DB & RAID 1/10 for LOGS (RAID-DP if using NetApp)
• Use VMware HA
  • Does not require Microsoft Clustering
  • Uses VMware Host Clusters
  • Automatically restarts failed SQL virtual machine in minutes
  • Heartbeat detects hung virtual machines
• Use VMware DRS
  • Will monitor state of virtual machine resource usage
  • Can automatically and intelligently locate virtual machine
  • Can create a dynamically balanced SQL deployment
• Use VMware VMotion
  • Reduce virtual machine planned downtime
  • Relocate SQL virtual machines without end-user interruption
  • Perform host maintenance anytime of the day

This section must detail all anticipated storage requirements for the design. How much space is needed initially for each component and what is the anticipated growth? Is this SAN storage, NAS, or cheap archive storage? Is the root, database, and log location different? If not, should it be?

• Storage footprint of components
  
• Anticipated growth rate
  
• Storage performance requirement
  
• COOP considerations
  
• Storage footprint of backups
  

Disaster Recovery

SharePoint will potentially host mission critical files so Disaster Recovery is a key consideration.

Backup Components

The following components will be backed up:

• Content Databases
• Search Databases
• Index
• IIS data and Hive files
• Administrative (configuration) Databases
  

The following components will not be backed up:

• Query Databases (Index will hold master copy)
• SharePoint Logs (Log Aggregator will collect these)

Off-site backup and recovery

SnapMirror will replicate data between datacenters

Recovery Point Objectives (RPO)

4 hours

Recovery Time Objectives (RTO)

8 hours; The SnapManager must index all items which could take this long with millions of documens.

Availability

Designing a SharePoint solution that is highly available has been made more easily possible due to excellent scaling options. SharePoint is designed to be an Enterprise platform that scales up and scales out. SharePoint servers can hold one or more roles and can have multiple instances of a role.

Availability Requirements

• SharePoint will be available 24 hours a day, 365 days a year.
• High Availability will be achieved through the use of multiple load-balanced servers.
  • There will be 2 ISA 2006 servers to handle authentication.
  • There will be 4 Web Front-Ends that will render and serve content including search queries.
  • There will be 9 SQL Database servers to read and write data to the SAN.
• The SAN is broken up into many volumes; data and log files are kept separate.

Additionally, high availability will be ensured by:

1. COOP Considerations
   1. We will use NetApp's SnapMirror to copy data to another Data Center. SnapMirror has the ability to copy only changes and therefore significantly save on bandwidth costs.
   2. Additionally, SnapMirror is cabable of restoring entire SharePoint sites or just single items (1 document).
2. Maintenance Windows
   1. There will be 2 hour maintenance window each month for patches and updates.
   2. Updating one server at a time and allowing the others to pick up the load will minimize downtime.
3. Monitoring for outage / availability
   1. The Design Team is validating the detail of monitoring provided by Toad and SystemEdge.
   2. Server uptime will be monitored by the NOSC. Additionally, web service availability will be monitored.
4. Fault Testing (failing different components of the overall architecture)
   1. Web Front-End and SQL servers are load balanced and will fail over to the other servers in the Farm.
   2. There is a single Index server that will NOT fail over. The server must be restored within 24 hours or search results will no longer be accurate.
   3. Restoring the Index server can be done quickly with NetApp SnapManager since we are planning to backup this component on its own schedule.

Performance

The following metrics will be used to gauge performance

• Page Load Time
• Application Throughput (in RPS)
• Search Throughput
• Index Throughput

The following lists the expectations for each metric

• Page Load Time: 5 Seconds
• Application Throughput (in RPS): 100 Requests Per Second served
• Search Throughput: 5 Seconds
• Index Throughput: 50 GB/hr (Full)

To achieve maximum performance in our environment, we'll use

• multiple ISA 2006 and SharePoint Web Front-End servers. These can be scaled out if performance degrades by installing additional Front End servers in the Farm.
• The Index server is CPU intensive and is therefore allocated 4 vCPU and 16GB of RAM.
• Finally, we will be setting database size and quantity limits on a per SQL Server basis to achieve optimal performance. SQL Servers can be increased as needed.

Load Testing will be accomplished using Microsoft simulation tools and is detailed in the Test Plan.

Configurable Items

At a high level, all SharePoint Servers will be configured with a "complete" installation. See the Hardware Configuration section for a breakout of roles for each server.

SharePoint stores configuration data in a database. This is used when adding additional Web Front-End servers or other type of servers to the SharePoint Farm. However, specific configurable items that must be tracked are the deployment of Solutions, custom site definitions, or other custom code.

All other "configurable items" or "tweaks" will be documented in the As-Built documentation. Once the system is stood up, configuration changes will be approved by a council or board and documented in the Wiki.

Testing

The Test Plan will be a separate document that will contain specifics on what will be tested and how. The test MUST reflect concurrency and usage requirements. Some considerations are

• Load Testing
• Fault Testing
• Disaster Recovery Testing
• Usability Testing (Concurrency)

Integration Points

The following are dependancies that are essential to the functionality of SharePoint:

• Virtual Infrastructure - All systems will be running as a VM.
• Active Directory - All users and service accounts depend on AD availability.
• SQL backend - The content, indices, search information and active logs are stored in the SQL backend.
• SAN backend - All data needed to render pages and web applications are stored in SQL databases housed on the SAN.