Simple PowerShell script to work with SPListItem fields and data

Using PowerShell to find out about data in SharePoint is par for the course, but making the best use of PowerShell's various cmdlets is something that takes time.

A very powerful cmdlet that I have not made enough use of over the years is Out-GridView.  It gives you a filterable grid of information that lets you control column widths and easily find information.

Example script:

cls;
trap{break;}

$siteUrl = "https://dev01-web.#####.com/sites/its1";
$listTitle = "Pages";

$web = Get-SPWeb $siteUrl;
$list = $web.Lists[$listTitle];
$listItem = $list.Items[0];
$listItem.Fields | Select-Object -Property Title, InternalName, @{Name="Value";Expression={$listItem[$_.Id]}} | Out-GridView

The result is a really useful popup window:

- gb

SharePoint 2013 App Part Communication Framework

[UPDATE 09/12/2016]
This approach was quickly superseded by an integrated rendering framework which I will blog about ASAP.


Overview
One of the limitations of the app model and the use of app parts is that iframes are used to render content.  There are workarounds to blend the look and feel into the SharePoint site (host web css, postmessage resize, etc.) but ultimately having a JavaScript model that runs inside the same page is a much more practical approach, especially when trying to create a responsive design.  When developing a product-based app part, the limitations imposed by the iframe approach make sense security-wise but when you are create a company intranet using the app model approach, this iframe approach becomes an unnecessary limitation.

The app part communication framework has been created to overcome the restrictions that the use of iframes in app parts imposes and makes use of the benefits of having control over the master page when developing in SharePoint Online for a company intranet.

Its current state is the result of a couple weeks of on and off contemplation and a few hours of development - thus it is still in its early stages and is going to be expanded upon (and used in a production environment) and so I will revisit this post and update it accordingly.

The latest SharePoint announcements regarding upcoming changes later this year/next year should overcome the issue I am trying to solve here, so the timing of this is rather ironic, but I still feel that this framework is worth posting about, not least because I can point the development and support team to it.

Updated

2016-05-10: Added support for auto-loading app part JavaScript

2016-05-11: Added to codeplex.

Background
As solution architect on a new project (and solution architect on the previous project with the same customer) I am trying to find a way to create a practical coding approach to using app parts in a responsive design-based content management system.  Our previous portal (not responsive design) implementation contained numerous heavy-handed farm solution-based customisations and all of these need to be part of the new template-based portal which must ideally fit within the app model so that if/when we move to SharePoint Online, there is minimal effort required.  Thus my intent is to completely avoid the use of farm solutions.

The new project is aimed at providing responsive design-based reusable SharePoint site template(s) that can offer the same easy-to-use web parts (tiles, carousels, custom media, etc.) that are in the existing portal - but of course, these need to be app parts.  In creating these parts, our UI designer is currently creating these inline rather than in app parts and so I really wanted to provide a way to have the flexibility of an editor choosing to add parts to the page while editing but have the rendering happen in the main page, outside the iframe.  This meant that there would need to be a way for the app part's iframe to communicate about the app part to the main page and allow the main page to do the rendering.

There is already a script used by the SP app part framework for communicating back and forth and it makes use of the rather brilliant postMessage method.  This method can be called from an iframe which in turn triggers an event in the parent page.  In the parent page you can choose to receive these messages by registering to handle the onmessage event from which you will receive the data sent by the iframe.  The same works in reverse too - the parent window can send messages to the iframe via the postMessage method.  All of this means that if we have control of the master page then we can create a framework for the app parts "chatting" to the master page (or one another if desired - though that is not the intent of this framework, it would take only small modifications to allow that to happen).  By creating a framework for this "chatting" to occur, we can build a client-side JavaScript-based framework for rendering app parts in the context of the SharePoint page instead of the iframe and that will allow our responsive design framework to easily control the rendered nodes.

Solution
Firstly, this is in the very early stages of development but please feel free to suggest any changes/improvements in the approach taken or any of the coding methodology.

I will add diagrams at a later point but for now I am writing this up quickly so it can be used by the team.

The basic structure is this:

• Files
• APCF.js - resides in provider hosted website
• APCF[app part type].js e.g. APCFAppPartExample1.js - resides in the provider hosted website
• Provider-hosted app part - contains inline script that calls postMessage
• JS Objects
• The APCF object is an object initialised using literal notation and comprises a few methods for processing message and registration of app parts
• The APCFAppPartType object is used to describe an object type identifier - each app part registers itself as an app part type
• The APCFAppPartInstance is a base object from which each type of app part inherits a class (using the 'apply' method).  Each of these inheriting instance classes are the identified in the APCFAppPartType object
• This base object has a flow of execution upon registration: initialise, load, render
• Each of these is declared in the base object as a "_base" so initialise_base, load_base, render_base - and each of these methods first calls the respective methods (initialise, load, render) then calls the next base method
• This means that if an instance implements the initialise, load or render methods then they will be called in that order automatically

The basic flow is this:

• Main page loads jQuery
• Main page loads the APCF.js file which comprises:
• APCF declaration
• APCFAppType declaration
• APCFAppInstance declaration
• Registers onmessage event to call APCF.processMessage
• Each app part aspx loads jQuery and APCF.js and the relevant App Part Type JS file and register the APCF.processMessage to handle APCF events
• On load, app part uses postMessage to send a serialized object (currently manually created but should be wrapped)
• postMessage is picked up by APCF.processMessage running within the main page, deserializes the object and checks the messageType property
• A switch statement calls the relevant APCF method depending on the messageType
• The first call from the app part registers the App Type which contains the App Part Type registration information (including script URL)
• This causes the APCF to call the iframe back using postMessage to say the app type has been registered
• On handling the registration, the app part ASPX then registers the instance
• Registratioin of the instance causes the APCF to start rendering

http://spapcf.codeplex.com/

The code is available from the URL above.  Please feel free to comment.  I will add documentation to the codeplex site ASAP

SharePoint - Ghosted vs. Unghosted

I know this is a subject that has been covered many times but it's often hard to find an explanation that is aimed at the appropriate audience for a particular situation.  This explanation is for someone who does not necessarily know the ins and outs of SharePoint in detail but does understand web development and has an understanding of the basics concepts within SharePoint.  It is intended to purely describe what ghosted and unghosted mean and the underlying behaviour surrounding these terms.

What do ghosted and unghosted mean?
SharePoint has a myriad of files behind-the-scenes and it also has a lot of database content.  Many of the behind-the-scenes files are “blueprints” for content/data.  When you create site collections, sites, lists and libraries, etc. you are using these blueprints to create content in the SharePoint database(s), but some of this content will be simply a pointer to the blueprints.  These blueprints are cached by each web server and so if something remains unchanged from its blueprint version, the web server is able to use the cached version of the information, which is very efficient.  The blueprints come in the form of site definitions, list definitions, field definitions, module features, etc.

When content is changed within a SharePoint site (e.g. uploading a new version, using SharePoint Designer, using explorer view, etc.), if the content was previously linked to a blueprint file and thus cached on the web server, the new content is then written to the database and from that point onwards whenever that piece of content is requested it will be retrieved from the database.  When compared to the blueprint cached version, in performance terms, there will be an increase in time taken to retrieve the information, network bandwidth taken up passing the data to the web server and (usually minimal) increased storage space used within the database.

So, when a file is "ghosted" this means that it is cached - residing in memory on the web server - and as such there are performance gains.  When a file becomes "unghosted", it is written to the database and every time it is referenced it will be retrieved from the database.

Hope this helps, any comments greatly appreciated.

Using WSPBuilder with Visual Studio 2012

The context menu in WSPBuilder has always proved useful, allowing easy GAC deployment of code changes within a DLL and fast IIS debugging.  The latest version of WSPBuilder is aimed at Visual Studio 2010 and so if it is needed for Visual Studio 2012, the .addin file needs updating to tell Visual Studio that it is supported by the current version - 11.

If you execute "%ALLUSERSPROFILE%\Application Data\microsoft\MSEnvShared\Addins", you should see the WSPBuilder addin file listed.  Simply edit this in notepad (remember to 1) close visual studio and 2) run notepad as admin) and duplicate the two hostapplication nodes - "Microsoft Visual Studio" and "Microsoft Visual Studio Macros", amending the version to "11.0".  Once this is done, you will find that the add-in loads and the context menu will appear.  If you are only intending to use Visual Studio 2012 you could, instead, just update any of the nodes from their existing version to "11.0".

One caveat is that, at least in Windows Server 2012+SharePoint 2013, the "Recycle Services Timer" option does not work.  If I get the chance I will fix this and post an update - it's probably something simple.

SharePoint 2010 Visual Upgrade Issue - v4.master could not be found

A problem I came across when carrying out a SharePoint 2007 to 2010 upgrade was that I wasn't able to carry out a visual upgrade without v4.master being present, generating the following error:

**********************************************
Visual Upgrade failed. The default master page for this user interface could not be found at "/search/_catalogs/masterpage/v4.master". Add the requested master page at this path and try again.
**********************************************

This, of course, makes sense and an easy solution is to copy the file into the master page gallery.  The problem is that when there are hundreds of subwebs in the site collection this becomes a little tricky and by copying the file into the library, any changes to the server side v4.master will not carry through as the copied version will be stored in the database rather than reference the file system version.

The solution to the problem is to deploy a module feature which pushes a reference to the server side v4.master into the master page gallery for each site.  This can be done using the following module definition:

**********************************************
<Module Name="OTBMasterPages" List="116" Url="_catalogs/masterpage" RootWebOnly="FALSE" SetupPath="GLOBAL">
  <File Url="v4.master" Type="GhostableInLibrary" IgnoreIfAlreadyExists="TRUE">
    <Property Name="UIVersion" Value="4" />
    <Property Name="ContentTypeId" Value="0x010105" />
  </File>
  <File Url="minimal.master" Type="GhostableInLibrary" IgnoreIfAlreadyExists="TRUE">
    <Property Name="UIVersion" Value="4" />
    <Property Name="ContentTypeId" Value="0x010105" />
  </File>
</Module>
**********************************************

The module directly references the TEMPLATE/GLOBAL folder so there is no need to include the files in the feature definition.  Once this module has been deployed in a feature, the next step is to add the feature to each web in the site collection.  This can easily be done with a PowerShell script:

**********************************************
cls
trap{break;}

# VARIABLES
$url = "http://[site url]/";
$featureId = New-Object Guid("1ba455df-5ea7-4c60-a90f-20b6ec3728d3");

# INITIALISE
$site = New-Object Microsoft.SharePoint.SPSite($url);
$rootWeb = $site.RootWeb;

# METHODS
function ProcessWeb($web)
{
 # If the feature has already been activated then deactivate
 if($web.Features[$featureId] -ne $null) {
  "Deleting from " + $web.Url;
  $web.Features.Remove($featureId);
 }
 
 # Activate the feature and then deactivate it.  No need for the dependency.
 $feature = $web.Features.Add($featureId);
 $web.Features.Remove($featureId);
 
 "Completed " + $web.Url;
 
 # Process each subweb
 foreach($subWeb in $web.Webs) {
  ProcessWeb $subWeb;
 }
 
 $web.Dispose();
}

# DO WORK
ProcessWeb $rootWeb;

# CLEAN UP
$rootWeb.Dispose();
$site.Dispose();
*********************************************

The script opens up the site collection and recursively processes the tree of webs.  For each web, the feature is checked for activation and deactivated if it is currently activated.  The feature is then activated - thereby pushing the master page references into the web - and then deactivated.  The reason for the deactivation is that once the references exist, the feature is no longer required - and any unnecessary dependencies should always be removed.  I would suggest that the feature either be configured as hidden or deployed as part of a temporary solution installation that is removed when the script has been successfully run.

If there are any meeting workspaces within the site collection then references to the mwsdefaultv4.master will be required.  When I carried out this work there were only three meeting workspace sites so I took the easy way out and just manually copied the file into each site.  Had there been a lot then I would have created a new feature which referenced the mws master page and then within the PowerShell script checked the web to see if it was a meeting workspace and activated the feature as needed.

SharePoint 2010 / Visual Studio 2010 - Error Occurred In Deployment Step - Access Denied

Occasionally, while developing a SharePoint 2010 Solution in Visual Studio 2010 I would be faced with the following error when hitting "Deploy":


Error occurred in deployment step '[various]': Access to the path '[path to a hive file]' is denied.


An IIS reset would fix the problem but I felt that this had to be just a workaround for a more specific issue - the error only occurred at certain times so there must had to be a specific cause for the file(s) being locked.


Short answer
The debugging web application that you chose when creating the SP2010 project in VS2010 is the only web application that is reset (well, its application pool is recycled, to be specific) when deploying a solution.  If any other web applications that use a different application pool lock any of the files that you are working on within your project then they will hold on to those files, causing the access denied error.



Long answer
When creating a SharePoint 2010 project within Visual Studio 2010 you are required to enter a URL that will be used for testing purposes.  This URL is then used to determine a number of things including: which web application to deploy to if/when the solution contains Web Parts, etc.; where to automatically activate features; and, crucially, which application pool to reset before carrying out the deploy.


If you choose a debugging web application that you then don't use for testing and instead use a different web application then you are at risk of seeing exactly this issue.  Bearing in mind this will only happen if the debugging web application has a different application pool to the one you are using for testing, when you start to make use of server templates, for example a site definition, the application pool will open one of the files (e.g. default.aspx) and keep it locked.  When you choose to deploy through Visual Studio it will recycle the application pool for the debugging web application but if this application pool is not being used for your testing, then the application pool for your test site will hang on to those files and you will receive the error.


Solutions
1) Update the debugging URL in your project properties to the web application that you are using.  Of course, if you are using more than one web application for testing purposes then if possible just re-use the same application pool and if this isn't possible then solution 3 may help


2) Ensure that the debugging web application's application pool is the same as the one you are using for testing.  I tend to create an application pool that I then use for as many web applications as possible as this is easier to manage and less intensive on memory usage.


3) Use WSPBuilder and use the context menu options to recycle the application pools.


4) IIS Reset (heavy handed but has the same effect as solution 3)