Skip to main content

Wpf Over Websockets (WoW) – How to Pump WPF windows & animations to browsers over HTML5 Websockets And Canvas

imageOver the weekend, I just put together a hack for pumping WPF windows over Websockets and rendering it using HTML5 canvas – also with basic mouse input support. For each user connecting to the server, a new window instance is created, and screen will be pumped over web sockets to the browser as base64 encoded images. These images are painted on an HTML5 canvas. The entire source code is here at codeplex –

What does this mean?

Once HTML5 specification is final - it is almost certain that it’ll be the most widely adopted platform for delivering applications to end users. With Websockets in HTML5, we are going beyond the unidirectional, stateless http to a full duplex, high speed connection infrastructure between the server and the client. This will definitely bring up lot of interesting possibilities, including exciting and feature rich user experiences. WPF is already a widely adopted, feature rich desktop application development platform, and this hack is a quick POC to bring the WPF experience over the web to the end user using HTML5 web sockets and canvas.

You can see that I’m accessing my WPF application over multiple browsers in the below video. Just to clarify, it is rendered in the browser purely using HTML5 canvas – No Silverlight or Flash Smile


Rendering the screen in the browser

HTML5 draft specification includes Websockets - 

WebSocket is a technology providing for bi-directional, full-duplex communications channels, over a single Transmission Control Protocol (TCP) socket. It is designed to be implemented in web browsers and web servers but it can be used by any client or server application. The WebSocket API is being standardized by the W3C and the WebSocket protocol is being standardized by the IETF.

Here is a pretty minimized implementation of the page that opens a web socket, retrieve the image data, and draws the same on a canvas

var socket = new WebSocket("ws://localhost:8181/main");
socket.onopen = function () {


socket.onmessage = function (msg) {
	var canvas = document.getElementById("canvas");
	var ctx = canvas.getContext("2d");
	var img = new Image;
	img.onload = function () {
		ctx.clearRect(0, 0, canvas.width, canvas.height)
		ctx.drawImage(img, 0, 0); // Or at whatever offset you like
	img.src = "data:image/gif;base64," +; ;

The server

I was planning to implement a quick socket server based on draft specifications – but I found that there are already a couple of .NET based implementations out there. One among them is Nugget - – I’m having a thin layer on top of Nugget for creating a window instance when ever a client socket connects to the server.
	var srv = new WebSocketServer(port, origin, "ws://" + server + ":" + port);
	srv.RegisterHandler<ElementSocket<T>>("/" + name);
	return srv;

Most of the logic required for creating window instances, and pumping them back to the connected client socket goes in my ElementSocket implementation.

Capturing User Inputs

Presently, the support for input devices (mouse, keyboard) etc is minimal. Only Mousebutton down and Mouse button up events are supported. To do this, we are capturing the mouse inputs using JQuery, and then pumping it back to the server.
            $("#canvas").mousedown(function (e) {
                var point = getXY(e);
                socket.send("mousedown;" + point.X + ";" + point.Y);

            $("#canvas").mouseup(function (e) {
                var point = getXY(e);
                socket.send("mouseup;" + point.X + ";" + point.Y);
On the server side, the incoming messages are used to simulate the events on the WPF window instance that corresponds to the socket, like
                var pnt=this.TranslatePoint(new Point(x, y), this);
                var res=VisualTreeHelper.HitTest(this, pnt);
                if (res != null && res.VisualHit != null && res.VisualHit is UIElement)
                    SimulateEvent(res, action);


There are a number of challenges – Ideally we should push only the region copies once it is modified. As of now, I’m pushing the entire screen image (bad) as that’s pretty easy (This is a weekend hack, rememberWinking smile). Now, another challenge is proper user input simulation on the instance windows – I still need to find a good way to trigger user inputs to the specific windows, with out affecting the actual input devices. For example, when one user moves the mouse, the move message should be triggered to the corresponding server side window instance, with out affecting the actual mouse cursor. May need to have a look at the WM_ messages, not sure to what extend it can be applied on WPF top level windows.

Related Reads & Hacks

As I mentioned, the entire source code of this hack is there at – Enjoy.

Popular posts from this blog

MVVM - Binding Multiple Radio Buttons To a single Enum Property in WPF

I had a property in my View Model, of an Enum type, and wanted to bind multiple radio buttons to this.

Firstly, I wrote a simple Enum to Bool converter, like this.

public class EnumToBoolConverter : IValueConverter { #region IValueConverter Members public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture) { if (parameter.Equals(value)) return true; else return false; } public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture) { return parameter; } #endregion }

And my enumeration is like

public enum CompanyTypes { Type1Comp, Type2Comp, Type3Comp } Now, in my XAML, I provided the enumeration as the ConverterParameter, of the Converter we wrote earlier, like

Creating a quick Todo listing app on Windows using IIS7, Node.js and Mongodb

As I mentioned in my last post, more and more organizations are leaning towards Web Oriented Architecture (WOA) which are highly scalable. If you were exploring cool, scalable options to build highly performing web applications, you know what Node.js is for.After following the recent post from Scott Hanselman, I was up and running quickly with Node.js. In this post, I’ll explain step by step how I’ve setup Node.js and Mongodb to create a simple Todo listing application.Setting up Node.jsThis is what I’ve done.1 – Goto, scroll down and download node.exe for Windows, and place it in your c:\node folder2 – Goto IIS Node project in Git at, download the correct ‘retail’ link of IIS Node zip file (I downloaded the already built retail package, otherwise you can download and build from the source).3 – Extract the zip file some where, and run the install.bat or install_iisexpress.bat depending on your IIS Version. If you don’t have IIS in…

Top 7 Coding Standards & Guideline Documents For C#/.NET Developers

Some time back, I collated a list of 7 Must Read, Free EBooks for .NET Developers, and a lot of people found it useful. So, I thought about putting together a list of Coding Standard guidelines/checklists for .NET /C# developers as well.As you may already know, it is easy to come up with a document - the key is in implementing these standards in your organization, through methods like internal trainings, Peer Reviews, Check in policies, Automated code review tools etc. You can have a look at FxCop and/or StyleCop for automating the review process to some extent, and can customize the rules based on your requirements.Anyway, here is a list of some good Coding Standard Documents. They are useful not just from a review perspective - going through these documents can definitely help you and me to iron out few hidden glitches we might have in the programming portion of our brain. So, here we go, the listing is not in any specific order.1 – IDesign C# Coding StandardsIDesign C# coding stand…