Here is an issue that I came across that might be helpful for others. We were seeing these 2 errors recurring every hour in the application logs.
The synchronization runs every hour by default to ensure that the content database will be included in membership and profile information.
The most likely cause is a content database was moved, copied and used in same farm without being detached properly.
To avoid this situation always remember to run this command before moving content database:
stsadm -o preparetomove -contentDB database_server:database_name
This operation will allow the membership and profile synchronization to be included when you reattach the content database. If this is skipped it will result in static membership and profile information within the content database, preventing it from getting synchronized when it’s reattached.
//-----------------------------------------------------------------
Event Type: Error
Event Source: Office SharePoint Server
Event Category: User Profiles
Event ID: 5553
Date: 9/26/2007
Time: 4:01:02 PM
User: N/A
Computer: WEBSTER
Description:
failure trying to synch site a7b12d90-7d1d-4341-8df3-a1d92eba7a47 for ContentDB db7fb0d6-7d0d-4230-8de5-e7cc00ca0db7 WebApp 55de867d-5a9d-4ac4-83d6-f3ee9ee98df2. Exception message was Cannot insert duplicate key row in object 'dbo.UserMemberships' with unique index 'CX_UserMemberships_RecordId_MemberGroupId_SID'.
The statement has been terminated..
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
//----------------------------------------------------------------
Event Type: Error
Event Source: Office SharePoint Server
Event Category: Office Server General
Event ID: 7888
Date: 9/26/2007
Time: 4:01:02 PM
User: N/A
Computer: WEBSTER
Description:
A runtime exception was detected. Details follow.
Message: Cannot insert duplicate key row in object 'dbo.UserMemberships' with unique index 'CX_UserMemberships_RecordId_MemberGroupId_SID'.
The statement has been terminated.
Techinal Details:
System.Data.SqlClient.SqlException: Cannot insert duplicate key row in object 'dbo.UserMemberships' with unique index 'CX_UserMemberships_RecordId_MemberGroupId_SID'.
The statement has been terminated.
at System.Data.SqlClient.SqlConnection.OnError (SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError (SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning (TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader (SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds (CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader (CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery (DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.Office.Server.Data.SqlSession.ExecuteNonQuery (SqlCommand command)
at Microsoft.Office.Server.UserProfiles.WSSSynchSqlSession.SynchExecuteNonQuery (SqlCommand cmd, Boolean throwOnFail)
at Microsoft.Office.Server.UserProfiles.WSSSynchSqlSession.SynchExecuteNonQuery (SqlCommand cmd)
at Microsoft.Office.Server.UserProfiles.SiteSynchronizer.WriteChangeLogConsumed()
at Microsoft.Office.Server.UserProfiles.SiteSynchronizer.Synch()
at Microsoft.Office.Server.Diagnostics.FirstChanceHandler.ExceptionFilter (Boolean fRethrowException, TryBlock tryBlock, FilterBlock filter, CatchBlock catchBlock, FinallyBlock finallyBlock)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
//-----------------------------------------
Probable cause is due to backing up our current content DB and restoring it to a different application on the same farm.
What happened is the GUID for the content DB is the same causing a conflict during the sync process.
Following steps are a workaround the issue:
Get a list of the content databases GUIDs that are being affected.
Stsadm -o sync -listolddatabases 0
Now run a Select Query on the Config Databases “dbo.Objects” table and look for the GUID that matches the listed GUID.
Database: Sharepoint_Config, Table dbo.Objects
SELECT Id, ClassId, ParentId, Name, Status, Version, Properties
FROM Objects
WHERE (Id = '5bafe2af-48a7-4043-a36d-29160f3b88e2')
In our case this is just a development/testing site that will not be around for long so we just excluded it from the sync process.
Stsadm –o sync –ExcludeWebApps URL of webapplist
But for those that are not as fortunate as us you will want to do a full detach and rebuild of the content database. This will enable the Content Database to get new GUID and be included in the sync process.
Detach Content Database. This will not delete your database only break the association.
stsadm -o preparetomove -contentdb -ContentDB SQLserver:DB_NAME -site http://website
stsadm -o deletecontentdb -url http://website -databaseserver SQLserver -databasename DB_NAME
Reattach Content Database.
stsadm -o addcontentdb -url http://website -databasename DB_NAME -databaseserver SQLserver
Helpful "sysadm –o sync" options can be found here.
Friday, December 3, 2010
Event ID: 5553 & Event ID: 7888
Thursday, August 26, 2010
Convert a Sub-site to a Site Collection
REM Create a test web for exporting
stsadm -o createweb -url "http://intranet/testweb" -sitetemplate "SPSTOPIC#0"
REM Export the test web to the filesystem
stsadm -o export -url "http://intranet/testweb" -filename "c:\testweb" -includeusersecurity -versions 4 -nofilecompression -quiet
REM Create a managed path for the new top level site
stsadm -o addpath -url "http://intranet/testsite" -type explicitinclusion
REM Create an empty site with a default site template (note that if you don't specify a template you have to manually activate the required features)
stsadm -o createsite -url "http://intranet/testsite" -owneremail "someone@example.com" -ownerlogin "domain\username" -sitetemplate "SPSTOPIC#0"
REM Import the site
stsadm -o import -url "http://intranet/testsite" -filename "c:\testweb" -includeusersecurity -nofilecompression -quiet
Thursday, July 29, 2010
HOW TO: Implementing a Backup Strategy with OOB STSADM commands
STSADM Backup Syntax:
For site collection backup
stsadm -o backup
-url<URL name>
-filename<file name>
[-overwrite]
Things to look out for when using STSADM to perform backups
As your site grows, logically does your content databases, and if you are employing STSADM as the tool for backup, a bi-product of that will be “longer backup times”. If your environment will be taking longer to backup, your risk of someone updating a file/document/asset as you are performing your backup increases; as a result if a backup is in running and a file is open, that backup is at risk of failure. How do we combat that problem? We use yet another STSADM command to lock the Site in effect the database to read-only; we do this with the SiteLock command, there are two in particular, getsitelock and setsitelock see below for the full command syntax.
stsadm -o getsitelock
-url <URL of Site Collection>
stsadm -o setsitelock
-url < URL of Site Collection> -lock readonly
Automate your Backup Solution
Certainly you can navigate to the SharePoint 12-Hive and execute these commands as a single action; however, a more consistent action is to script the job with error handling and user feedback.
Script your Backup
@echo off
@echo——————————————————–
@echo This script will backup your site: [Your Site Name Here]
@echo——————————————————–
cd \Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN
@echo locking site collection before the backup process starts
@echo off
stsadm -o getsitelock -url http://YourSiteCollectionNameHere
stsadm -o setsitelock -url http://YourSiteCollectionNameHere -lock readonly
@echo site collection is now locked
@Echo Logic to create a new file name daily
@For /F "tokens=1,2,3 delims=/ " %%A in (‘Date /t’) do @(
Set Day=%%A
Set Month=%%B
Set Year=%%C
Set All=%%C%%B%%A
)
@echo off
stsadm -o backup -url http://YourSiteCollectionNameHere -filename \\ServerName\ShareName\Backup_ %All%.bak -overwrite
@echo backup completed
@echo off
stsadm -o setsitelock -url http://YourSiteCollectionNameHere -lock none
@echo Site lock removed
Save the above to a .bat file in a preferred location.
Automate your Backup
Create a new Windows Scheduled Task by going to Start–> Control Panel –> Scheduled Tasks –> Add Scheduled Task
Follow the wizard and schedule the .bat file created above to run at a frequency of your own choosing.
How to Build a SharePoint Development Machine
There are two choices: You can build a SharePoint server that contains all your developer tools too. Or you can build your SharePoint server in a virtual machine (VM or VPC), and keep all your developer tools local. Then you can treat your virtual machine just like a physical server, and surf or demo it just as you would a production environment. This way you avoid all the inconsistencies that come with running browser session on locked-down web servers, don’t fill your server’s GAC with quite as much junk, and really get the client experience. You can use this post as a starting point to build either.
This post describes the tools you will use. There are always others, so go ahead and tell me about yours in the comments section. I’ve left off CSS and XML editors because frankly the best aren’t free. Even though a few items listed here are commercial products (e.g. Visual Studio), all have evaluation versions that last long enough for you to learn the ropes and prepare for that first big SharePoint interview.
Client Hardware
The biggest controllable factors on the speed of your SharePoint development environment are RAM and hard drive speed. If you do a lot of development, you will also want a lot of storage space to hold all those virtual machines.
The configuration described here is intended for a good laptop, which today is limited to 4Gb RAM. If you have 4Gb, give the virtual machine 2Gb. If you have 3Gb, give the virtual machine either 1.5Gb or 2Gb. If you have 2Gb or less, buy more RAM or host your virtual server elsewhere. Seriously.
If you need Exchange Server for any reason (1 to 2Gb recommended), you’ve grown out of laptop labs and should build this on a Virtual Server (or VMWare) host if you want some hair left over when you’re done. To build medium or large farms in a virtual environment, Virtual Server running on a 64-bit Windows 2003 Server R2 machine will remove your memory limitations. A 64-bit Vista machine provides the same expandability with Virtual PC or VMWare.
There are several factors to hard drive speed: keeping your VM on a different drive than your OS, drive transfer rates, and drive speed. Put another way: you want a big fast external drive. You can get a pretty big drive for around $200CAD now (I like my Seagate), and an enclosure with Firewire or eSATA (external SATA) is about $50CAD. If you don’t have Firewire or eSATA on your laptop you’ll need an adapter for that as well (also $50). Why not USB 2.0? Firewire is 33% to 70% faster than USB depending on the operation. eSATA is about five times faster than USB 2.0. While many enclosures also have a USB port, the difference is well worth it. PCMCIA (CardBus) runs at 132Mbytes per sec so an adapter for any of these will treat you fine. Note that newer laptops may have an ExpressCard slot instead of a PCMCIA slot.
| Technology | Transfer Rate (Mbits per second) |
| USB 2.0 | 480 |
| Firewire 800 | 786 |
| eSATA | 1500 to 2400 |
Minimal Configurations
The assumption is that you know what you like, know what you want to do, and will download and install accordingly. That said, maybe you really do want to develop with Notepad so here goes.
On your development machine you need the .Net Framework version 3.0 and a development interface (IDE), whether it’s Visual Studio, SharePoint Designer, or a plaintext editor like Notepad. If you don’t have a physical server available, you will use either Virtual PC or VMWare to host SharePoint. Beyond these, everything is optional.
On your server, you need Windows Server, IIS, the .Net Framework version 3.0, and either WSS, MOSS Standard, or MOSS Enterprise.
Building the Client
.Net Framework 2.0. This is optional, but if you do any development beyond SharePoint you likely have clients on v2.0.
.Net Framework 3.5
WPF, WCF, and essential to SharePoint: Workflow. Note that MOSS 2007 works great with .NET 3.5, is supported by Microsoft, and there is no reason not to deploy it for all your servers.
[Update 2008-11-04] Do not install .NET 3.5 SP1 if you require Records Center, and until the cause is identified there is a risk that other features are also affected. .NET 3.5 itself remains okay, this issue is only reported with SP1.
Microsoft Office SharePoint Server 2007 SDK [Read it online]
The MOSS SDK includes the Enterprise Content Management (ECM) Starter Kit, a BDC definition editor, an Excel UDF sample, great workflow samples, and lots more.
Windows SharePoint Services (WSS) 3.0 Software Development Kit (SDK) [Read it online]
The MOSS SDK contains this so you don't need both. If you only do WSS development, then you only need the WSS SDK.
Red Gate .NET Reflector (was Lutz Roeder's Reflector)
One of the best .NET tools ever, and the way you'll browse object models and cut the class-assembly strings you'll need to paste into your manifests and web.config. “Reflector is a class browser, explorer, analyzer and documentation viewer for .NET. Reflector allows you to view, navigate, search, decompile and analyze .NET assemblies in C#, Visual Basic and IL.”
SharePoint Manager 2007
The best SharePoint management tool there is. Browse the complete object model, deactivate features, and even update object properties. You do need to be careful with this one, but wow it's powerful.
Office System Professional Plus Edition
Grab your disc, MSDN, or get the evaluation version. The Professional Plus edition includes Word, Excel, Outlook and PowerPoint plus: Access, Publisher and InfoPath. There is also an Enterprise edition which also includes OneNote and Groove.
SharePoint Designer (SPD)
Grab your disc, MSDN, or get the evaluation version.
Visual Studio 2005 or 2008
Grab your disc, MSDN, or get the evaluation version. I prefer either VS Professional or VSTS Developer Edition.
Visual Studio 2005 Extensions for .NET Framework 3.0 (Workflow Foundation)
Not required with VS 2008 where these features are baked-in. Design Windows Workflow Foundation solutions for SharePoint and other .NET applications.
Visual Studio 2005 Tools for the 2007 Microsoft Office System (VSTO)
VSTO is used to build: Office System (Word, Excel, etc.) Add-ins and Ribbon elements, Outlook forms, and InfoPath templates.
Other Visual Studio Extensions. Optional extensions are available unrelated to SharePoint, though you should know they exist if you do other .NET development.
This is where you will build all those beautiful webs! VMWare is a popular alternative.
SQL Server Express Edition
You may want SQL Express if you develop more than SharePoint and need a local data store. If not, it is built into the SharePoint standalone install so you do not need to download or install it locally.
Optional
Fiddler HTTP Debugger
Optional and strongly recommended. Inspect and manipulate HTTP traffic between your browser and the server, great for debugging forms and web services, determining whether Kerberos tokens are being generated, and more.
Internet Explorer Developer Toolbar
The IE Developer Toolbar is optional, but recommended. Inspect HTML source, style usage, IFRAME content, image locations, and more.
PowerShell (formerly Monad). [SharePoint Provider] [Script Repository]
PowerShell is optional, but recommended. Write scripts to automate administration and configuration tasks. Colin Byrne has posts to get you started: PowerShell and SharePoint, Build Me A Portal, and Upload a Directory in 4 Lines.
BDC Metadata Manager
Generate BDC XML from SQL data sources. The commercial version also builds definitions for web services. An alternative is baked into the MOSS SDK, but BDC Metaman will continue to be the gold standard.
Visual Studio 2005 Extensions for Windows SharePoint Services 3.0 (VSeWSS)
Not recommended. VSeWSS contains nothing to help you deploy features on anything but the machine running VS, does not support x64, and is required to open projects that it creates. So if you create a project today on a 32-bit machine, you will be unable to open it in a year when you're building on 64-bit. This is bad. The one good feature included is the SharePoint Solution Generator (SPSolGen) which converts lists and sites “Saved as Template” into list and site definitions. [Learn more here] [Registry hack to install VSeWSS to a machine that is not the server]
Required to create a private LAN between the client and server
a. Click Start, Control Panel, and open the Add Hardware wizard.
b. Click Next. Wait a moment while the wizard scans for hardware.
c. Select: Yes, I have already connected the hardware, click Next.
d. Scroll to the bottom of the list and select Add a new hardware device. Click Next.
e. Select: Install the hardware that I manually select from a list (Advanced). Click Next.
f. Select: Network adapters. Select: Microsoft. Select: Microsoft Loopback Adapter.
g. Click Next. Click Next again. Wait a moment. Click Finish.
Configure the Loopback Adapter
h. Click Start, Control Panel, and open the Network Connections utility.
i. Right-click the Microsoft Loopback Adapter connection and select Properties.
j. Double-click Internet Protocol (TCP/IP).
k. Now configure an address for this adapter.
Select: Use the following address
IP Address: 10.50.50.1
Subnet mask: 255.255.255.0
Default gateway:
You can ignore DNS. Click OK. Click OK again.
Building the Server: Articles
Bob Fox explains step-by-step how to install Windows 2003 to Virtual PC and VMWare (up to the point where you install the OS).
Bob Fox explains good habits on re-using your base image for several implementations, and why you might need to add servers to the farm (e.g. for Exchange).
How to Create a MOSS 2007 VPC Image. This step-by-step series with screenshots by Tony Zink is as complete as complete can be and starts with configuration of IIS. Note that he puts SPD and Office on the server, while the goal of this post is to keep your server clean of client apps.
Microsoft Guidance on Standalone and Farm configurations.
Kevin Hoffman explains how to build a Standalone SharePoint server. He installs Visual Studio to the server, you can skip that step and run it locally on your development machine.
Jonathan Bradshaw explains how to build a Small Farm configuration with SQL and AD on a separate machine. His “OfficeDev” machine looks the most like our local development machine. One day someone will write an article with configuration steps, until then this is a good start.
Building the Server: Step-by-Step
Step 1: Install the Operating System
Windows 2003 Server R2. Grab your disc, MSDN, or get the evaluation version. Bring the machine up-to-date with Windows Update. Note that this process can last several hours.
Step 2: Configure Networking
R. Aaron Zupancic describes how to configure an IP address and host file entry so you can surf your virtual SharePoint sites from your development machine.
Simon Guest describes how to get all the machines in a virtual farm to talk to each other. In this configuration, while your virtual machines will see each other, your local machine will not be able to surf or debug the virtual machines.
Note that if you can join your virtual machine to your company’s domain and create domain accounts there (but different accounts than you will use for testing or production), it will simplify configuration and debugging, and you won’t need to make this or any machine in your development farm an Active Directory domain controller (DC). The drawback is that you need to be connected to the domain to get any work done.
Step 3: Install the .NET Framework
.Net Framework 3.0
WPF, WCF, and essential to SharePoint: Workflow.
Once activated and configured, Shut Down and back up your base server image. In the next steps you’ll prepare this machine for a specific purpose.
Step 4: Install Active Directory and DNS.
If this is going to be a Standalone configuration, you need a domain controller (DC). The Configure Your Server Wizard (Start, Administrative Tools) is a great way to assign roles to a server. Start with Domain Controller (Active Directory) and DNS.
If this is going to be a Small Farm, copy your base server image from the last step, and start building a second machine to host Active Directory and SQL Server. Note that after you install SharePoint you will not be able to make the machine a domain controller (unless you’re prepared for a world of hurt).
Step 5: Install Internet Information Server (IIS)
Once again, the Configure Your Server Wizard (Start, Administrative Tools) is an easy way to add the Application Server role to your VM.
Once IIS is installed, you will need to Allow ASP.NET 2.0.* in the Web Server Extensions section of the IIS Manager. If this option is not available, you may need to run the ASP.NET 2.0 setup and Repair your installation, which also sets the Allow flag.
Step 6: Install SQL Server
SQL Server Developer Edition [Google]
This is optional. If you don’t install it, Express Edition will be installed when you install SharePoint. However, Developer Edition includes the excellent SQL Server Management Studio, so if you have MSDN you may prefer it. For a production environment you will need to license SQL Server Standard or Enterprise Edition.
Step 7: Install SharePoint (WSS or MOSS)
Windows SharePoint Services (WSS) 3.0
WSS is a free extension to Windows Server. If your installation didn’t include WSS as an option, you can download now. If installing MOSS, then you do not need to separately install WSS. Also remember that you can upgrade a server, but you cannot downgrade from MOSS back to WSS. If you’re evaluating SharePoint you may want to start with WSS.
If you installed SQL Server Developer Edition, choose Advanced Installation to select your database.
Install the WSS Service Pack.
Microsoft Office SharePoint Server (MOSS) 2007
Grab your disc or MSDN, or get the evaluation version. Standard and Enterprise editions exist, so again keep in mind that you can upgrade but not downgrade. Enterprise includes Forms Server and the BDC.
Install the Office SharePoint Server Service Pack. [SP1]
Shut down and back up your virtual machine image(s).
Step 8: Configure SharePoint
Create a domain group for SharePoint Administrators, assign rights to it rather than any single account. The exception will be configuring SQL Server, where rights buried inside groups aren't always resolved and you need to assign permissions per-account.
Create an account to use as your SharePoint Service account (e.g. SPService). Configure this to be your Application Pool identity (you'll need to search what rights are required until I complete this section).
Create a content crawler account (you'll need to search what rights are required until I complete this section).
Create a developer account with the same name as the account you use on your development machine (and it better not be Administrator). Add your developer account to the Administrators group (Global Administrators if this is a DC, local Administrators if not) and Debugger Users group.
Create a typical end-user account for testing. Make it a member of Domain Users.
Step 9: Configure the server for Development
If you must develop on the server, install Visual Studio now. On one side, developing locally is simpler than developing remotely. On the other side, it’s so handy to keep all your code in one place and you won’t ever wonder where the most recent version of x is residing. If you really can’t get the hang of remote debugging, you can always install VS later.
Open up File Explorer on the server and Add to Favorites your frequently-used paths including the 12 hive and the default web root.
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\
C:\inetpub\wwwroot\
Backup the 12 hive into a zip file marked as read-only, and named “[Drive]:\Backups\12Hive-Out-of-Box-DoNotUpdate.zip”. You get the idea. When you accidentally edit a default ONET.XML file with WordPad and kill your site you will be glad you did.
And finally, make a text file on your desktop with Joubin’s SharePoint paths.
Step 10: Configure remote debugging.
Once SharePoint is installed, share the folder with SharePoint’s assemblies (*.DLL). You may also want to copy the assemblies to your development machine:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI
Share the server’s web root and enable Read/Write.
C:\inetpub\wwwroot\
Install the Visual Studio 2005 Debugging Components to the server. You want Full Remote Debugging, not Native Remote Debugging.
Follow Ross Dargan’s instructions on configuring remote debugging. You’ve just completed his step 1 and if you created your developer account as specified then you’ve also done 2 and 3.
Take a look at Andrew Noon’s document on Remote Debugging SharePoint Web Parts. He has additional steps for enabling error messages in web.config and screen shots that show what debugging looks like in action.
Read: Setup Remote Debugging
Friday, July 23, 2010
Downloading Content From SharePoint (Let me count the ways)
In this post I will be doing comparisons for downloading content from SharePoint. Once again there are many ways to get content from SharePoint. When I say content I am focusing on files and metadata. There is an additional way to get a list item using the Lists web service and the GetListItems method; however, this only returns metadata.
| Method | Complexity | Scalability | Metadata | Versions |
| Copy.asmx | 2 | 4 | yes | no |
| WebDav | 5 | 5 | yes* | yes** |
| RPC | 10 | 10 | yes | yes |
*must be used in conjunction with Lists web service
**must be used in conjunction with Versions web service
Copy Web Service
The copy web service is the easiest to use to get content from SharePoint because it is familiar web service programming and it allows the content and the metadata to be retrieved in one method call. The scalability is not as good as the other methods because it uses the more verbose soap protocol. Finally, one of the biggest disadvantages using the copy web service is the fact that it does not support returning versions of a document. The copy web service was made for migrating content between document libraries and sites within the same site collection not necessarily versions. I did try using the copy web service to retrieve an older version of a document by setting the url argument to the GetItem method using the older version’s url.
The version url looks like this:
http://servername/site/_vti_history/512/documentlibrary/filename.tiff
Unfortunately, when sending in a version url the GetItem method executes successfully but both the metadata and binary are empty.
public static void DownloadDocument()
{
copyservice.Copy c = new copyservice.Copy();
c.Url = "http://basesmcdev2/sites/tester1/_vti_bin/copy.asmx%22;
c.UseDefaultCredentials = true;
byte[] myBinary = new byte[] { };
copyservice.FieldInformation information = new copyservice.FieldInformation();
copyservice.FieldInformation[] info = { information };
string docUrl = "http://basesmcdev2/sites/tester1/Shared%20Documents/+aaa/mama.bmp%22;
uint result = c.GetItem(docUrl, out info, out myBinary);
if(File.Exists("c:\\newfile.bmp")) File.Delete("c:\\newfile.bmp");
using (Stream s = File.Create("c:\\newfile.bmp"))
{
s.Write(myBinary, 0, myBinary.Length);
s.Flush();
}
}
WebDav
Using webdav to download a document is fairly simple if you have the full url of the document. However, if you want to include metadata you must call the GetListItem of the list web service. This becomes tricky when the only piece of information you have is the url. You can use the list web service but you will need to know the name of the list and the id of the item in order to retrieve the metadata. In the end, if you are going to call a SharePoint web service then just call the copy web service and get both the metadata and file in one call.
public static void GetFileWithMetaData(string url)
{
WebClient wc = new WebClient();
wc.UseDefaultCredentials = true;
byte[] response = wc.DownloadData(url);
string returnStr = Encoding.UTF8.GetString(response);
if (File.Exists("c:\\default.txt")) File.Delete("c:\\default.txt");
using (Stream s = File.Create("c:\\detault.txt"))
{
s.Write(response, 0, response.Length);
s.Flush();
}
}
So how does WebDav support versions? You can give the DownLoadData method the url of the version mentioned previously and it will return the binary for that version. So, now the question is how to you obtain the url’s of previous versions? Below is a code snippet that uses the Versions web service to retrieve a particular version’s url for a document for a given file url and version number.
public static string GetVersionUrl(string url, double version)
{
double versionNumeric = 0;
string versionNumber = string.Empty;
string versionUrl = string.Empty;
versionservice.Versions vs = new versionservice.Versions();
vs.Url = "http://basesmcdev2/sites/tester1/_vti_bin/versions.asmx%22;
vs.UseDefaultCredentials = true;
XmlNode versionsNode = vs.GetVersions(url);
if (versionsNode != null)
{
using (StringReader sr = new StringReader(versionsNode.OuterXml))
{
using (XmlTextReader xtr = new XmlTextReader(sr))
{
XElement versionInfo = XElement.Load(xtr);
var versionResults = from r in versionInfo.Elements() where r.Name.LocalName == "result" select r;
foreach (XElement versionElement in versionResults)
{
var versionAttr = (from a in versionElement.Attributes() where a.Name == "version" select a);
versionNumber = versionAttr.First().Value;
versionUrl = versionElement.Attributes("url").First().Value.ToString();
//current version is in the form of "@1.0" and will not pass TryParse
if (versionAttr.Count() > 0 && double.TryParse(versionNumber, out versionNumeric))
if (version == versionNumeric) return versionUrl;
else
if (version == Convert.ToDouble(versionNumber.Substring(1))) return versionUrl;
}
}
}
}
return string.Empty;
}
FrontPage RPC (Remote Procedure Calls)
Most developers are not familiar with frontpage remote procedure calls. However, they are the most efficient and yet the most complex to code against. You could create your own wrapper classes to simplify coding. You must understand the command structure and be able to parse the return html correctly. In the case of downloading a document it is especially complex. You must come up with a way to parse out the returned metadata within the html. Below is an example of a successful return of a “get document” rpc call. Anything below the closing </html> tag is the file.
<html><head><title>vermeer RPC packet</title></head><body><p>method=get document:12.0.0.4518<p>message=successfully retrieved document 'tester4/Code128.tif' from 'tester4/Code128.tif'<p>document=<ul><li>document_name=tester4/Code128.tif<li>meta_info=<ul><li>vti_rtag<li>SWrt:FED298A3-6030-40DA-A984-D0A04A673741@00000000013<li>vti_etag<li>SW"{FED298A3-6030-40DA-A984-D0A04A673741},13"<li>vti_parserversion<li>SR12.0.0.6318<li>vti_modifiedby<li>SRBASESMCDEV2\steve.curran<li>vti_filesize<li>IR3387<li>vti_timecreated<li>TR22 Oct 2008 19:45:25 -0000<li>ContentType<li>SWDocument<li>ContentTypeId<li>SW0x01010087C42E0D80709D4CB61D6558C94571E4<li>vti_title<li>SW<li>statechoice<li>SWstateone<li>vti_lastheight<li>IX2200<li>vti_timelastmodified<li>TR13 Apr 2009 22:04:31 -0000<li>vti_nexttolasttimemodified<li>TR13 Apr 2009 22:02:20 -0000<li>vti_candeleteversion<li>BRtrue<li>vti_canmaybeedit<li>BXtrue<li>vti_backlinkinfo<li>VXLists/threeStateTasks/3_.000 Lists/threeStateTasks/4_.000<li>myreq<li>SWnbnbvnbv<li>vti_author<li>SRBASESMCDEV2\steve.curran<li>vti_lastwidth<li>IX1696<li>vti_sourcecontrolversion<li>SRV1.0<li>vti_sourcecontrolcookie<li>SRfp_internal<li>vti_level<li>IR1</ul></ul></body></html>
Finally, below is an example on how to call the “get document” rpc method. You must work with the returned byte array and copy the section which represents the file to another byte array. This method supports retrieving older versions. Just pass in the version number, zero would represent the current version. The fileUrl argument represents document library name plus the file name. For example, “Shared Documents/FileName.tiff”.
public static void DownloadDocumentRPC(string fileUrl, int version)
{
string method = "get document: 12.0.0.4518";
string serviceName = "http://basesmcdev2/sites/tester1/_vti_bin/_vti_aut/author.dll";
string verstr = version > 0 ? "V" + version.ToString() : string.Empty;
string document = fileUrl;
byte[] data;
string returnStr = string.Empty;
byte[] fileBytes = null;
string fpRPCCallStr = "method={0}&service_name={1}&document_name={2}&doc_version={3}&get_option={4}&timeout=0";
method = HttpUtility.UrlEncode(method);
fpRPCCallStr = String.Format(fpRPCCallStr, method, serviceName, document, verstr, "none");
try
{
//add line feed character to delimit end of command
byte[] fpRPCCall = System.Text.Encoding.UTF8.GetBytes(fpRPCCallStr + "\n");
data = new byte[fpRPCCall.Length];
fpRPCCall.CopyTo(data, 0);
HttpWebRequest wReq = WebRequest.Create(serviceName) as HttpWebRequest;
wReq.Credentials = System.Net.CredentialCache.DefaultCredentials;
wReq.Method = "POST";
wReq.ContentType = "application/x-vermeer-urlencoded";
wReq.Headers.Add("X-Vermeer-Content-Type", "application/x-vermeer-urlencoded");
wReq.ContentLength = fpRPCCall.Length;
using (Stream requestStream = wReq.GetRequestStream())
{
requestStream.Write(fpRPCCall, 0, fpRPCCall.Length);
int chunkSize = 2097152;
//Now get the response from the server
WebResponse response = wReq.GetResponse();
int lastBytesRead, totalBytesRead;
long contentLength = response.ContentLength;
bool noLength = false;
if (contentLength == -1)
{
noLength = true;
contentLength = chunkSize;
}
byte[] returnBuffer = new byte[(int)contentLength];
using (Stream responseStream = response.GetResponseStream())
{
totalBytesRead = 0;
do
{
lastBytesRead =
responseStream.Read(returnBuffer, totalBytesRead, ((int)contentLength) - totalBytesRead);
totalBytesRead += lastBytesRead;
if (noLength && (totalBytesRead == contentLength))
{
contentLength += chunkSize;
byte[] buffer2 = new byte[(int)contentLength];
Buffer.BlockCopy(returnBuffer, 0, buffer2, 0, totalBytesRead);
returnBuffer = buffer2;
}
}
while (lastBytesRead != 0);
}
if (noLength)
{
byte[] buffer3 = new byte[totalBytesRead];
Buffer.BlockCopy(returnBuffer, 0, buffer3, 0, totalBytesRead);
returnBuffer = buffer3;
}
returnStr = Encoding.UTF8.GetString(returnBuffer);
//get begining of file bytes
int startpos = returnStr.IndexOf("</html>") + 8;
using (MemoryStream ms =
new MemoryStream(returnBuffer, startpos, returnBuffer.Length - startpos))
fileBytes = ms.ToArray();
if (File.Exists("c:\\newfile.bmp")) File.Delete("c:\\newfile.bmp");
using (Stream s = File.Create("c:\\newfile.bmp"))
{
s.Write(fileBytes, 0, fileBytes.Length);
s.Flush();
}
}
}
catch (Exception ex)
{
//error handling
}
}
Once again I hope this comparison of the different methods of downloading content from SharePoint will help you plan your next application’s SharePoint integration. Knowing the different methods will prevent you from having write your own web service. Ultimately, taking advantage of SharePoint “out of the box” tools will make your applications easier to install, configure and maintain.
Creating a custom webservice
Creating a Custom Web Service
1. The first step is to create an ASP.NET web service project in Visual Studio 2005. If you don't find a web service project template in visual studio, that means that you are still running an old version of Visual Studio 2005, the one without the service pack. You will have to download and install the Visual Studio 2005 Service Pack 1 Beta from the Microsoft. Here is the link:
It's a 371.9 MB download and let me tell you in advance that the installation is very slow and takes a lot of time.
On the File menu, click New Project.
2. In the Project Types box, select Visual C#.
3. In the Templates box, select ASP.NET Web Service Application.
4. In the Name box, type UploadService. In the Location box, type the following path:
C:\WebService
You can also click the browse button to browse the folders and select the destination folder. This is the path where the project will be stored.
In the Solution Name box, type UploadService and check Create directory for solution checkbox.
5. Click OK.
6. In the Solution Explorer, right-click Service1.asmx and rename the file Files.asmx and then right click Files.asmx and click View Code.
7. Add a reference to the assembly for Microsoft Office SharePoint Server 2007 (Microsoft.SharePoint.dll). This assembly is located in the following directory:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI
Please note that if you are developing on a machine that does not have SharePoint installed then you can copy the required files from the SharePoint machine to your development machine.
8. Make sure following using directives are included at the top:
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using System.Net;
9. Change your class name from Service1 to Files.
public class Files : System.Web.Services.WebService
10. Comment out the web method definition. The commented out web method code will look like as follows:
//[WebMethod]
//public string HelloWorld()
//{
//return "Hello World";
//}
11. Add following code (web method) in the class:
[WebMethod]
public string UploadDocument(string fileName, byte[] fileContents, string pathFolder)
{
if (fileContents == null)
{
return "Null Attachment";
}
try
{
int iStartIndex = pathFolder.LastIndexOf("/");
string sitePath = pathFolder.Remove(iStartIndex);
string folderName = pathFolder.Substring(iStartIndex + 1);
SPSite site = new SPSite(sitePath);
SPWeb web = site.OpenWeb();
SPFolder folder = web.GetFolder(folderName);
string fileURL = fileName;
folder.Files.Add(fileURL, fileContents);
if (folder.Files[fileURL].CheckedOutBy.Name != "")
{
folder.Files[fileURL].CheckIn("File Checked In");
}
return "File added successfully!";
}
catch (System.Exception ex)
{
return "Error: " + ex.Source + " - " + ex.Message;
}
}
12. Open Files.asmx markup page. In Solution Explorer, right-click Files.asmx and select View Markup. You will notice that the markup page has following line:
<%@ WebService Language="C#" CodeBehind="Service1.asmx.cs" Class="UploadService.Service1" %>
Change it to the following line:
<%@ WebService Language="C#" Class="UploadService.Files" %>
This was the easy way. You can also strong name your assembly using the command line utility called as sn.exe. Use following steps if you want to strong name your assembly manually:
1. Strong naming utility (sn.exe) can be found in the following folder:
C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin
Copy the sn.exe utility to the folder where your assembly is located and run the following command to strong name your assembly:
sn.exe -k key.snk
Resulting key will be written to the key.snk file.
2. Go to your project's properties (right-click project name in the Solution Explorer and select Properties from the menu) and select Signing from the menu that appears on the left. This will open a form.
3. Check Sign the assembly checkbox and choose the key file from the drop down (Click the Browse... button in the drop down to locate the key.snk file that you generated in the previous step).
4. Re-compile your assembly.
15. Compile the web service project.
16. As we are developing this web service on a machine that does not host SharePoint, therefore, we need to create a virtual directory in IIS. Click Start, point to Administrative Tools (You may have to go to the Control Panel first to select Administrative Tools), and then click Internet Information Services (IIS) Manager.
Note: In case you are developing the service on a SharePoint server, then skip steps 17-20 and use following instructions and then start with step 22:
1. Copy the Files.asmx to the following folder:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS
2. Run disco.exe at the command prompt from the LAYOUTS directory to generate .disco and .wsdl files.
17. Expand the branch for the server computer to which you want to add a virtual directory. Under the server computer branch, expand the Web Sites folder, and right-click the Default Web Site and select New and then Virtual Directory.... If you don't want to use Default Web Site, you can create a new web site.
18. Click Next and enter an alias in the text box, for example, for this service you can enter UploadService.
19. Click Next and then click Browse... button to browse to the project folder (containing the .asmx file) and click Next.
20. Click Next again and then click Finish.
Generating and Modifying Static Discovery and WSDL Files
21. Use Disco.exe to generate .disco and .wsdl files. This command line utility is located in the following directory:
C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin
Open command prompt and type the following line and press Enter:
Disco http://localhost/uploadservice/Files.asmx
Make sure you have entered the correct path in the command above otherwise you will get an error. If you created a virtual directory on a port other than the port 80, then you must mention the port number in the path (For example, http://localhost:8080/uploadservice/Files.asmx). This will generate the .disco and .wsdl files.
<%@ Page Language="C#" Inherits="System.Web.UI.Page"%>
<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<%@ Import Namespace="Microsoft.SharePoint.Utilities" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<% Response.ContentType = "text/xml"; %>
23. In the .disco file, modify the contract reference and SOAP address tags to be like the following example, which replaces literal paths with code generated paths through use of the Microsoft.SharePoint.Utilities.SPEncode class, and which replaces the method name that is specified in the binding attribute:
<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request) + "?wsdl"),Response.Output=""); %>
docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q1="http://tempuri.org" binding="q1:FilesSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q2="http://tempuri.org" binding="q2:FilesSoap12" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
24. In the .wsdl file, make the following, similar substitution for the SOAP address that is specified:
<soap:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />
Make the following substitution for the SOAP12 address:
<soap12:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />
25. Rename both files in the respective formats Filedisco.aspx and Fileswsdl.aspx so that your service is discoverable through SharePoint.
Deploying web service on the SharePoint server
Copy the Web service files to the _vti_bin virtual directory
26. Copy the web service files to the _vti_bin directory of the SharePoint server. Web service files that are to be copied are as following:
Files.asmx
Filesdisco.aspx
Fileswsdl.aspx
The _vti_bin virtual directory maps physically to the Local_Drive:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI directory, which contains the default Web service files used in Windows SharePoint Services.
To include the Web service in the list of web services on the server
27. In Notepad, open the spsdisco.aspx file. spsdisco.aspx is located in the following directory (on the SharePoint server):
Local_Drive:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI
28. Add the following lines to the end of the file within the discovery element and save the file:
<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx?wsdl"), Response.Output); %> docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx"), Response.Output); %> xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx"), Response.Output); %> xmlns:q1="http://schemas.microsoft.com/sharepoint/soap/directory/" binding="q1:FilesSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
Copying the assembly to the correct Bin folder
29. I have seen blogs and forums where people have recommended to copy the assembly to the following folder:
Local_Drive:\program files\common files\microsoft shared\web server extensions\12\isapi\
Some people say it should be copied to the following folder:
Local_Drive:\Inetpub\wwwroot\bin
bin folder is not there by default and has to be created but safest place to copy the assembly is the bin folder of the virtual directory of the web application where you intend to use the web service. Following path contains the virtual directories of web applications that you have created on your SharePoint server.
Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories
For example, if you want to use the web service in web application at port 81, then you should copy the assembly to the following folder:
Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories\81\bin
Similarly, if you want to use the web service in web application at port 17316, then you should copy the assembly to the following folder:
Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories\17316\bin
I am not saying that you can not copy the assembly to the _vti_bin folder or wwwroot folder, of course you can but I have often seen people struggling with the service deployment. People often ask me where they should copy the assembly and what is the best place to put the assembly in. I tested my web service by putting the assembly file in all the recommended places and I found that bin folder of the virtual directory of the web application is the safest place where your web service is guaranteed to work.
Copy the assembly to the bin folder of the virtual directory of the web application of your choice. You will have to create the bin folder yourself. It is not there by default.
Adding web service to the GAC
30. You must add your assembly in the GAC on your SharePoint server. This is necessary otherwise you will get permissions error on the server. There are three ways to avoid this permissions error. One way is to add the assembly in the GAC. Second way is to change the trust level to medium in the web.config file and the third is to create a custom trust policy of your own. There is an article on Microsoft site that has all the details of Code Access Security (CAS) and it also shows how to create a custom policy file. Here is the link to that article if you are interested in exploring this topic:
http://msdn2.microsoft.com/en-us/library/ms916855.aspx
Unfortunately, this is an old version and works only with SharePoint 2003. I am not sure if Microsoft has already released an updated version of this article or not. I intend to write an updated version of this article (at least the custom policy file part) for Office SharePoint Server 2007 and Windows SharePoint Services 3.0.
1. Go to control panel, click Administrative Tools and select Microsoft .NET Framework 2.0 Configuration.
2. Click Manage the Assembly Cache and then select Add an Assembly to the Assembly Cache. Locate the assembly and click Open.
Creating a Windows Application to Consume the Web Service
31. Create a new C# Windows Application in Visual Studio 2005.
32. In Solution Explorer, right-click References, and then click Add Web Reference.
http://localhost/_vti_bin/Files.asmx?wsdl
If you installed the service on another port, for example, port 17316, then the url would be:
http://localhost:17316/_vti_bin/Files.asmx?wsdl
You can also browse all web services available on the server. Files.asmx will be listed in the web services available on the server. There will be multiple entries, each entry representing the service available for a different web application. Therefore, if you want to upload documents to a site on port 80, then you should select Files.asmx for port 80.
34. Include following using directives at the top:
using System.Net;
using System.IO;
35. Add following code to your application:
try
{
localhost.Files oUploader = new localhost.Files();
oUploader.PreAuthenticate = true;
oUploader.Credentials = CredentialCache.DefaultCredentials;
string strPath = @"C:\test.doc";
string strFile = strPath.Substring(strPath.LastIndexOf("\\") + 1);
string strDestination = "http://sp:17316/Docs";
FileStream fStream = new FileStream(strPath, System.IO.FileMode.Open);
byte[] binFile = new byte[(int)fStream.Length];
fStream.Read(binFile, 0, (int)fStream.Length);
fStream.Close();
string str = oUploader.UploadDocument(strFile, binFile, strDestination);
MessageBox.Show(str);
}
catch (Exception ex)
{
MessageBox.Show(ex.Source + " - " + ex.Message + " - " + ex.InnerException + " - " + ex.StackTrace);
}
localhost is the name of the web reference that you added in previous step. strPath contains the filename that is to be uploaded. This is just a sample to show you how the service works, that's why i have hard coded the filename but in a real life situation you may want to add a text box and a browse button to select files from your hard disk. strDestination contains the destination path. This should be the path representing the document library where file is to be uploaded.
36. Finally, before testing the service, make sure the current user has privileges to upload documents in the destination document library. The user should have "Contributor" rights in the document library otherwise, you will get "401: Unauthorized" error. Run the application to test the service.
Tuesday, July 20, 2010
Check in multiple documents
You can simply just choose:
- Take ownership of documents you want to check in
- Site Actions
- Manage Content and Structure
- Find the folder where your files are
- Select all files (checkbox for checking all or single checkbox next to each file name)
- Choose Actions --> Check In
- Make a comment on the check in
- Done