Qualitem - Large List Support - SharePoint Saturday
Download as PPTX, PDF1 like1,312 views
This document discusses strategies for managing large lists in SharePoint when the number of items exceeds thresholds. It begins by explaining how the List View Threshold of 5,000 items can cause errors when exceeded. It then provides ways to manage large lists, such as creating indexed columns, using folders, search, and filtering views on indexed columns. The document also cautions against exceeding limits on lookup columns and demonstrates tools and techniques for resolving issues with large lists.
Qualitem - Large List Support - SharePoint Saturday
- 1. Gold Sponsors Bronze SponsorsSilver Sponsors Large List Support List Throttling Rick Rosato & Benjamin Wong
- 2. Agenda • Large List support • What this means? (Throttling/Threshold/Limits) • Strategies for managing large lists in SharePoint • Not covering • Office 365 • Amount of content • Size considerations • Content Type management • Archiving • Storage strategies • If you have any of these issue, come see me afterwards!
- 3. What usually happens? • Using SharePoint happily • Upload or add to the list • Listview throws an error, you have insufficient privileges to view the page content. • You panic, your world is imploding • You call the helpdesk • They scramble to solve the issue
- 4. SharePoint 2013 Limits • Lists and Libraries – limits (SharePoint 2013) • 50 million (items) • 2 GB (item or attachment) • Threshold Limit is 5,000, why 5,000? • Minimize database contention (row-level locking) • When we do get errors, what’s going to happen?
- 5. These things stop if I exceed the threshold? • List issues • Datasheet view breaks - permission error • Add/remove/update a list column/content type • Create/remove an index • Manage files which have no checked-in version • Non-indexed recursive queries • Cross list query • Lookup columns that enforce relationship behaviour • Deleting a list/site • Save list as template with data • Showing totals in list views • Enable/disable attachments in a list • Folder issues • Delete/copy/rename • Queries that filter on non- indexed col • Setting item-level security • Open with Explorer
- 7. Resource throttles and limits Threshold or Limit Default value Description List View Threshold 5,000 (warning at 3,000) Specifies the maximum number of list or library items that a database operation, such as a query, can process at one time. Operations that exceed this limit are blocked. Unique permissions limit 50,000 (breaking the inheritance is counted as 1 unique permission) Specifies the maximum number of unique permissions allowed for a list or library. Row size limit per list (internal rows) 6 (8,000 bytes) (related to column Specifies the maximum number of table rows internal to the database used for a list or library item. To accommodate wide lists with many columns, each item is wrapped over several internal table rows,. Continue…
- 8. Resource throttles and limits (cont..) Threshold or Limit Default value Description List View Lookup Threshold 12 (joins) Specifies the maximum number of join operations, such as those based on lookup, Person/Group, or workflow status columns. List View Threshold size for auditors and administrators 20,000 Specifies the maximum number of list or library items that a database operation, such as a query, can process at one time when performed by an auditor or administrator with appropriate permissions. This setting works in conjunction with Allow Object Model Override. Allow Object Model Override Y Specifies whether or not developers can perform database operations, such as queries, that request an override of the List View Threshold to the higher limit specified by the List View Threshold for auditors and administrators. Daily time window None Specifies a time period during which resource thresholds and limits are ignored. An administrator can configure a time window during "off-peak" hours in 15 minute increments and up to 24 hours, such as, 6:00 PM to 10:00 PM or 1:30 AM to 5:15 AM.
- 9. Ways to manage large lists and libraries • Create SharePoint Indexed Columns • Create Filtered Views Based on Column Indexes • Organizing Items into Folders • Controlling the display of data on the page • Using offline synchronisation • Using Search to find list or library items • Using indexed and fallback queries • Using External Data
- 10. Create SharePoint indexed Columns • Help improve performance, up to 20 columns • Only add to those you need, does add overhead Supported Column Types Unsupported Column Types Single line of text Multiple lines of text Choice (single value) Choice (multi-valued) Number Calculated Currency Hyperlink or Picture Date and Time Custom Columns Lookup (single value) Lookup (multi-valued) Person or Group (single value) Person or Group (multi-valued) Yes/No External data External data Managed Metadata
- 11. Create Filtered Views Based on Column Indexes • First filter MUST be indexed (to return less than List View Threshold) • Other filtered columns not countered (except for devs using compound indexes) • Multi-filter with AND operator to limit results • Recycle Bin may skew results set • Some operations might still be blocked (requires entire list) • adding or deleting an index • creating a sort in the view definition; • displaying a column total • adding, updating, or deleting calculated fields.
- 12. Organising Items into Folders • Folders - are an internal index • Subfolders count as an item (but not items in that subfolder). • Good quick strategy for fixing List View Threshold issues. • Moving between folders is difficult
- 13. Controlling the display of data on the page • Setting an item limit on a page • Display items in batches of the specified size (for paging), • Limit the total number of items returned to the specified amount • Using the Preview Pane view style • Using the Group By view style • Using RSS (turn on Site Collection and List) • Configure List RSS - Based on modified date – limit results • Set the maximum number of days to include
- 14. Using offline synchronisation • Access 2013 • Local copy, synchronises when on online (upto 2GB) • 50,000 items (configurable) • Outlook 2013 • Contacts • Tasks • Calendar • Discussion List • Documents • Creates a PST file, synchronises when on online.
- 15. Using Search to find list or library items • It own indexing mechanisms, not subject to the List View Threshold • Results can be scoped narrow or widen as needed • Search is really fast when querying the list/library • Search on list/library attributes and use refiners to limit results
- 16. Metadata Navigation Feature and using indexed and fallback queries • Metadata Navigation and Filtering can assist with large lists, via query optimisation and best selection of queries. • If there’s no best query, Metadata Nav & Filtering will construct and execute a fallback query – limited results returned. • Intended to return some useful results when original query is blocked due to large list throttling. • Not 100% reliable
- 17. Using External Data • Q. Should it be in SharePoint in the first place? • Using BCS – provides standardized interfaces to external data sources • External List usually relates to transactional data – LoB, ERP. • Not bound be list throttling limits, it has it’s own configuration.
- 18. Demos • Create a simple or compound index • Create a filtered view based on indexed column • Metadata navigation • Central Admin - Resource Throttling • Qualitem Solution • Large List administration • Content Rollup support for large list • Paging and Search
- 19. Don’ts • Lookup columns, don’t exceed 12 per list • What classifies as a Lookup column? • Name (linked to document/edit with menu) • Edit (link to edit item) • Type (icon linked to document) • Standard lookup columns • Managed metadata (single and multiple-value) • People and group columns (single and multiple-value people and group columns • Created by , Modified by ( people ) • Workflow Status
- 20. If you get stuck! • Buy our tools to help you manage your lists – Qualitem • Increase the List View Threshold temporarily via CA • Administrators can run queries on large lists • Enable a daily time window for large queries in CA • Call me – Rick Rosato – 04 1111 7270 (rick@qualitem.com)
Editor's Notes
- #6: Point out just a few of these
- #7: List or library data in a site collection is stored in a SQL Server database table, which uses queries, indexes and locks to maintain overall performance, sharing, and accuracy. Filtered views with column indexes (and other operations) create database queries that identify a subset of columns and rows and return this subset to your computer. Thresholds and limits help throttle operations and balance resources for many simultaneous users. Privileged developers can use object model overrides to temporarily increase thresholds and limits for custom applications. Administrators can specify dedicated time windows for all users to do unlimited operations during off-peak hours. Information workers can use appropriate views, styles, and page limits to speed up the display of data on the page.
- #12: Demo - Items due today – where Due Date is equal to [Today].