Wednesday, February 06, 2008

RESTful file uploads with HttpWebRequest and IHttpHandler

I've currently got a requirement to transfer files over a web service. I would normally have used MTOM, but after reading Richardson & Ruby's excellent RESTful Web Services and especially their description of the Amazon S3 service, I decided to see how easy it would be to write a simple file upload service using a custom HttpHandler on the server and a raw HttpWebRequest on the client. It turned out to be extremely simple.

First implement your custom HttpHandler:

public class FileUploadHandler : IHttpHandler
{
    const string documentDirectory = @"C:\UploadedDocuments";

    public bool IsReusable
    {
        get { return false; }
    }

    public void ProcessRequest(HttpContext context)
    {
        string filePath = Path.Combine(documentDirectory, "UploadedFile.pdf");
        SaveRequestBodyAsFile(context.Request, filePath);
        context.Response.Write("Document uploaded!");
    }

    private static void SaveRequestBodyAsFile(HttpRequest request, string filePath)
    {
        using (FileStream fileStream = File.Open(filePath, FileMode.Create, FileAccess.Write))
        using (Stream requestStream = request.InputStream)
        {
            int bufferSize = 1024;
            byte[] buffer = new byte[bufferSize];
            int byteCount = 0;
            while ((byteCount = requestStream.Read(buffer, 0, bufferSize)) > 0)
            {
                fileStream.Write(buffer, 0, byteCount);
            }
        }
    }
}

As you can see, it's simply a question of implementing IHttpHandler. The guts of the operation is in the ProcessRequest message, and all we do is save the input stream straight to disk. The SaveRequestBodyAsFile method just does a standard stream-to-stream read/write.

To get your handler to actual handle a request, you have to configure it in the handers section of the Web.config file, for examle:

<httpHandlers>
 <add verb="*" path="DocumentUploadService.upl" validate="false" type="TestUploadService.FileUploadHandler, TestUploadService"/>
</httpHandlers>

Here I've configured every request for 'DocumentUploadService.upl' to be handled by the FileUploadHandler. There are a couple of more things to do, first, if you don't want to configure IIS to ignore requests for non-existent resources you can put an empty file called 'DocumentUploadService.upl' in the root of your web site. You also need to configure IIS so that requests for .upl files (or whatever extension you choose) are routed to the ASP.NET engine. I usually just copy the settings for .aspx files.

On the client side you can execute a raw HTTP request by using the HttpWebRequest class. Here's my client code:

public void DocumentUploadSpike()
{
    string filePath = @"C:\Users\mike\Documents\somebig.pdf";

    string url = "http://localhost:51249/DocumentUploadService.upl";
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.Accept = "text/xml";
    request.Method = "PUT";

    using(FileStream fileStream = File.OpenRead(filePath))
    using (Stream requestStream = request.GetRequestStream())
    {
        int bufferSize = 1024;
        byte[] buffer = new byte[bufferSize];
        int byteCount = 0;
        while ((byteCount = fileStream.Read(buffer, 0, bufferSize)) > 0)
        {
            requestStream.Write(buffer, 0, byteCount);
        }
    }

    string result;

    using (WebResponse response = request.GetResponse())
    using (StreamReader reader = new StreamReader(response.GetResponseStream()))
    {
        result = reader.ReadToEnd();
    }

    Console.WriteLine(result);
}

Here too we simply stream a file straight into the request stream and then call GetResponse on the HttpWebRequest. The last bit just writes the response text to the console. Note that I'm using the HTTP method PUT rather than POST, that's because we're effectively adding a resource. The resource location I'm adding should be part of the URL. For example, rather than:

http://localhost:51249/DocumentUploadService.upl

it should really look more like:

http://localhost:51249/mikehadlow/documents/2345

Indicating that user mikehadlow is saving a file to location 2345. To make it work would simply be a case of implementing some kind of routing (the MVC Framework would be ideal for this).

17 comments:

MAG said...

All sounds good, but ... I am doing something similar and the httpRequest.InputStream property actually buffers the whole body, so it isn't so good for large files.

Mark

Kannan.S said...

Dear Sir,

I have a Clarification where can i write the FileUploadHandler code whether it is a class file named as FileUpload Handler or a aspx file or a webservive?

Thank you

Mike Hadlow said...

Hi Kannan,

It's an Http Handler. Here's the documentation for them: http://msdn.microsoft.com/en-us/library/5c67a8bd(VS.71).aspx

Purnachandra said...

Hi, when writing the file back to the client, how do we separate the contents of the webpage with the file data. I mean, if the client webpage downloading the file has some UI controls on it, then how will the file contents be separated from mixing???

Mike Hadlow said...

Hi Purnachandra,

I think you've totally got the wrong end of the stick here. This is about transferring files over HTTP between two .NET applications. There's no web page involved.

Mike

Anonymous said...

Hi,

If I were to add this to an existing website, is there any way I can secure/authenticate this with a digital certificate (not just the ssl certificate that I'm already using with my site) and not require my site pages to have to authenticate this way at the same time?

Thanks!

Mike Hadlow said...

Hi Anonymous,

If you've already got HTTPS set up on your web site, why not pass the credentials as HTTP headers?

Anonymous said...

Hi again,

Well I've been tasked with writing something that will accept a file sent in an HTTP multi-part MIME message (and respond to it with an http status acceptance code) over https, that has to be authenticated with username/password, and a digital certificate is required to establish communication (and then supposed to make this "root public certificate" available somehow so it can be used for POSTing). I was thinking I could use an HttpHandler to do this, specify some made up path for it (kind of how like you've done here) so it's only used for someone to post the files, and I'll retrieve the username/password from the query string to authenticate against my user database (unless you think putting it in the header is better). Is a root public certificate, an X.509 certificate? Is that the same thing as an SSL certificate? ...because this handler will exist as part of my website, which is already set up with an SSL certificate in IIS. So then would I still need this root public certificate? What do you think? I've been posting in forums and no one seems to be able to help. I've never had to do anything like this before, and am not very familiar with httphandlers, web services, etc. Thank you for your advice!

Anonymous said...

Hi,

How do i implement the same in Java based on RESTful way

Nayyar Abbas said...

Thanks! it solved my issue.

Raja said...

Hi Mike,
I have a problem in file uploading to Rest API, this is an multipart message.When we upload wav files(around 5mb,256kbps bitrate), we get good response from Rest API. But when we upload mp3 files(any kb to mb size, 32 kbps bitrate) we get Null Reference error at Streamreader in HttpWebResponse block. Is there anything to be done on low bitrate file uploading or anything to be changed in ContentType? or byte[]? Please help me in this.

Fordy said...

Quite simply one of the best .Net file upload solutions out there. I changed my web.config so it works for large files
e.g. added

Copy and pasted it into my MVC project and it just worked. Simple, clean code that was a huge help. Great work.

Fordy said...

Seems that my web.config changes don't appear right in the above post. So for reference I just added:

executionTimeout="3600" maxRequestLength="1048576"

to the httpRuntime tag in the web.config. This allows for 1 Gig files to upload up to 3600 seconds (1 hour).

Anonymous said...

Can you post solution file for this code? that would be great!

Raja said...

Hi, Here I have a request to do a program in c#, that is posting mp3file to a webservice without as MIME multipart message, and that webservice has few paramters also to call some methods of it.Can you please help us explaining with a sample code?

Anonymous said...

Hi, i've implemented your codes (where possible) in my web api, but i've encountered an error when sending from console app (with your codes as well) error is: Method not allowed.

i think it is because i'm sending mime type application/octet-stream. how do i set the api to accept any type of contents?

Anonymous said...

"i think it is because i'm sending mime type application/octet-stream. how do i set the api to accept any type of contents?"

Hi again. I've managed to fix the problems in my project:

- i needed to add authorization header
request.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + token);
- and when deploying to IIS...i needed the destination folder to be on Full trust permissions in file explorer for user: IIS_IUSR so that it can create the file on server, otherwise it gives error 500

thanks for the wonderful code :)