Skip to main content

XGenPlus - A flexible tool to generate typed XML serializers for your .Net applications

Introduction

If you have ever used XML Serialization heavily in your projects, chances are that you have pulled your hair a couple of times before getting things right.

Memory Leaks

When you use XmlSeializer to serialize or deserialize an object, XmlSerializer will create a dynamic assembly on the fly containing the serialization code, specific to the type of that object. For instance, when you do something like

XmlSerializer ser=new XmlSerializer(typeof(Customer))

Now, behind the scenes,

  • The XmlSerializer constructor will reflect the customer type you are passing to the XmlSerializer constructor, and generate the code for a typed serializer for the same.
  • The code for the typed serializer for Customer type is compiled by calling the compiler services at run time
  • The cached assembly which contains the typed serializer for Customer type is loaded to the application domain, and is cached for future uses.

How ever, there is a known problem with XmlSerializer - few XMLSerializer constructors (other than the simple constructors) will regenerate the typed serializer assembly each time, instead of getting it back from the cache. Only these two constructors will get the serializer back from cache when subsequent calls are made.

•System.Xml.Serialization.XmlSerializer(Type)
•System.Xml.Serialization.XmlSerializer(Type,String)


In other words, XmlSerializer is not using the cache mechanism in all constructors For eg, assume that you are invoking XmlSerializer in a web application. If you use any of those overloaded, 'feature rich' constructors of XmlSerializer, you are going to run out of memory.

XmlSerializer serializer = new XmlSerializer(typeof(Customer), new XmlRootAttribute(""));


For each call, a new Serializer is created, and you may soon run out of memory if you have this piece of code in a web application which is expected to scale up to a fair amount of users. In Microsoft's own words, "If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, resulting in a memory leak and poor performance" (as cited in MSDN documentation for XmlSerializer - http://msdn2.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx). We still don't know whether this is 'by design' or this is a bug with XmlSerializer.

Poor Startup Performance

Even if you are planning to go with the simple constructors, still, this results in run time generation of typed serializers, at least for the first time when XmlSerializer is initialized with a specific type. The solution? Generate the typed serializers before you compile your application - so that you don't have to worry about run time typed serializer generation and the memory leaks.

Getting it right

When we got into the problems of untyped Xml Serialization, I frankly never thought we'll end up developing a utility for creating typed serializers. We were dealing with performance enhancements of a project which involved tones of xml serialization and de serialization. Initially, we decided to use Microsoft Sgen to create typed serializers, but we ran across a couple of problems.

  • You can't generate typed serializers for a selected set of types
  • After generating typed serializer assemblies, you are expected to create a strong reference to the same from your project
  • Failed to handle few scenarios

Another tool we came across was Mvp.Xml.Xgen, which may help you to create typed serializers at design time. You can configure it so that you can run it as a custom task from with in Visual Studio. How ever,

  • We never wanted the serializers as part of the main assembly.
  • We required more flexibility for selecting types to generate serializers
  • We wanted a loosely coupled way to invoke serializers, with out making any major changes to the existing code.

Introduction to XGenPlus

the result, we rolled out a small tool, XGenPlus, mainly combining the nice features of SGen and Mvp.Xml.Xgen. Here are few features

  • Provides a set of command line options for createing typed serializer libraries for all types or selected types in an assembly.
  • Allows programmers to create a typed serializer with out actually referreing the typed serializer library directly. For this, XGenPlus.SerializerLib can be used.
  • An MSBuild task is available to integrate XGenPlus in your Build script
  • You can run XGenPlus using a configuration file.

Using XGenPlus from command line

Here is a brief overview of what it can do

Usage: XGenPlus /assembly:assemblyname [/exclude:namespace1,namespace2] [/include:namespace1,namespace2] [/reference:assembly1,assembly2] [/copyto:path] [/nocompile] [/nogenerate] [/getconfig:filename] [/putconfig:filename] [/serializeall] [/from:namespace]


• /assembly:assemblyname - To specify the assembly
• /exclude:namespace1,namespace2 – Exclude types in the specified namespaces
• /include:namespace1,namespace2 - Include types in the the specified namespaces
• /reference:assembly1,assembly2 - Specify reference assemblies
• /nocompile - Won't compile the source files generated
• /nogen - Won't generate any source files
• /getconfig:filename - Run the application using the configuration specified in the filename
• /putconfig:filename - Write the current configuration (passed over command line) to the filename
• /copyto:path - Copy the generated assembly to the path
• /serializeall - Generate serializers for all types, not only for classes with System.Serializable attribute applied
• /from:namespace - Generate serializers for types in namespaces starting from the specified namespace. Generated assembly name will be the namespace.dll

Usage Examples

Here are few examples.

Case 1: The following command will generate serializers for types starting with the namespaces SomeName.Objects.DTO. The generated assembly name will be SomeName.Objects.DTO.dll

xgenplus /a:SomeName.Objects.DTO.dll /r:SomeName.Objects.Messages.dll
/i:SomeName.Objects.DTO.CalcUI

Case 2: The following command will generate serializers for types starting with the namespaces SomeName.Objects.DTO and exclude everything else. This will also generate a configuration file with the name SomeName.Objects.config

xgenplus /a:SomeName.Objects.DTO.dll /r:SomeName.Objects.Messages.dll
/i:SomeName.Objects.DTO. CalcUI /putconfig:SomeName.Objects.config

Case 3: The following command has the same effect as above command once the config file is in place. The parameters will be taken from the config file.

xgenplus /getconfig:SomeName.Objects.config

Case 4: The following command will just generate a configuration file, from the command line parameters.

xgenplus /a:SomeName.Objects.DTO.dll /r:SomeName.Objects.Messages.dll
/e:SomeName.Objects.DTO.State /putconfig:SomeName.Objects.config /nogen /nocomp

Case 5: By default, serializers will be generated only for types marked with System.Serializable attribute. You may use the /serializeall switch to generate serializers for all classes

xgenplus /a:SomeName.Objects.DTO.dll /r:SomeName.Objects.Messages.dll
/i:SomeName.Objects.DTO.CalcUI /serializeall

Case 6: The following command will generate a serializer dll, named SomeName.Objects.DTO.CalcUI.dll

xgenplus /a:SomeName.Objects.DTO.dll /r:SomeName.Objects.Messages.dll
/f:SomeName.Objects.DTO.CalcUI

Using XGenPlus as a MSBuild task

Using XGenPlus as a task with Ms Build is rather easy. XGenPlusTask class in the XGenPlus.exe can be used directly from the build configuration. Modify your project file with the following information.

Make sure that UsingTask declaration references the correct path where XGenPlus resides.

<UsingTask TaskName="XGenPlusTask" AssemblyFile="YourPath\XGenPlus.exe" />

And then define the task in the appropriate section (In this case, BeforeBuild).

<Target Name="BeforeBuild"> <XGenPlusTask AssemblyName="bin\debug\SomeName.Objects.DTO.dll" NoGenerate="false" NoCompile="false" IncludeList="SomeName.Objects.dto.calcui;SomeName.Objects.dto.cobrowse" ReferenceList="bin\debug\SomeName.Objects.messages.dll;System.Data.dll" /> </Target>

How to use the typed serializers

Step 1 - Use XGenPlus to generate serializer assemblies for your objects, and place the generated *.Serializer.dll file(s) in your applications bin folder.

Step 2 - Once you have your serializer libraries in the bin folder, the following call to the FactoryProxy class in XGenPlus.SerializerLib will return a typed serializer for your type (Make sure that you have a reference to XGenPlus.SerializerLib in your project)

XmlSerializer ser = FactoryProxy.GetSerializer(typeof(yourtype));

In the back ground, the XGenPlus.SerializerLib will do everything else.

Inside XGenPlus

Great, isn't it? Now let us have a very brief look inside the XGenPlus project. What XGenPlus does when you invoke it is, pass the types to an instance of XmlSerializer, and then steal away the code XmlSerializer generates :). Pretty simple, isn't it? How ever, a couple of other stuff is also required to make things work the way we need it.

XGenPlus

The Runner class has a static method, InvokeRunnerInOwnAppDomain which actually creates an instance of the Runner class using reflection - and then iterate the types one by one to invoke the XmlSerializer by passing that type - To 'steal' the code generated by XmlSerializer.

Please note that, to steal the code of typed serializers generated by XmlSerializer, we should do something naughty. We should modify the config file to add a switch, XmlSerialization.Compilation, to tell XmlSerializer that it should leave behind the temporary files it generated, during the process of generating a typed serializer!!.

<code><code><configuration> <system.diagnostics>
<switches>
<add name='XmlSerialization.Compilation' value='4'/>
</switches>
</system.diagnostics>
</configuration>



InvokeRunnerInOwnAppDomain is actually invoked either from the Program class (If you are invoking XGenPlus from command line), or from the XGenPlusTask class. (If you are a invoking XGenPlus as an MSBuild task). The GenerateAndCompile method in the Runner class actually does the ground work (creating folders, load reference libraries to the application domain etc) and invoke GenerateCode method in the XmlSerializerGenerator class.

Other than just stealing away the code created by XmlSerializer, we are also generating code for a factory class in the generated serializer library. The code that we've stolen from XmlSerializer and the code we generated for factory class, is compiled together to form the serializer library.

XGenPlus.SerializerLib

All the factories we generated are implementing the ISerializerFactory interface in XGenPlus.SerializerLib library. Also XGenPlus.SerializerLib library provides a convenient way for you to create serializers with out actually refering directly to the serializer assemblies you generated with XGenPlus. (You may have multiple serializer assemblies in your project, isn't it?)

Here, the catch is the FactoryCache class - which has a static constructor, that loads all factories from the *.Serializer.dll files in your application's execution path. Optionally, you can use the SerializerDllPath setting in your config file, to specify the location of your generated serializer assemblies. And finally, GetFactory method in XGenPlus.SerializerLib.FactoryProxy will find the appropriate factory based on your type name, invoke the corresponding typed serializer, and may return the same to you.

Please feel free to download the source code and binaries attached with this article.

XGenPlus project is maintained in CodePlex and is distributed under GPL. Please checkout www.codeplex.com/xgenplus for updates.

Comments

Popular posts from this blog

Top 7 Coding Standards & Guideline Documents For C#/.NET Developers

Some time back, I collated a list of 7 Must Read, Free EBooks for .NET Developers, and a lot of people found it useful. So, I thought about putting together a list of Coding Standard guidelines/checklists for .NET /C# developers as well.As you may already know, it is easy to come up with a document - the key is in implementing these standards in your organization, through methods like internal trainings, Peer Reviews, Check in policies, Automated code review tools etc. You can have a look at FxCop and/or StyleCop for automating the review process to some extent, and can customize the rules based on your requirements.Anyway, here is a list of some good Coding Standard Documents. They are useful not just from a review perspective - going through these documents can definitely help you and me to iron out few hidden glitches we might have in the programming portion of our brain. So, here we go, the listing is not in any specific order.1 – IDesign C# Coding StandardsIDesign C# coding stand…

Hack Raspberry Pi – How To Build Apps In C#, WinForms and ASP.NET Using Mono In Pi

Recently I was doing a bit of R&D related to finding a viable, low cost platform for client nodes. Obviously, I came across Raspberry Pi, and found the same extremely interesting. Now, the missing piece of the puzzle was how to get going using C# and .NET in the Pi. C# is a great language, and there are a lot of C# developers out there in the wild who are interested in the Pi.In this article, I’ll just document my findings so far, and will explain how develop using C# leveraging Mono in a Raspberry Pi. Also, we’ll see how to write few minimal Windows Forms & ASP.NET applications in the Pie as well.Step 1: What is Raspberry Pi?Raspberry Pi is an ARM/Linux box for just ~ $30. It was introduced with a vision to teach basic computer science in schools. How ever, it got a lot of attention from hackers all around the world, as it is an awesome low cost platform to hack and experiment cool ideas as Pi is almost a full fledged computer.  More About R-Pi From Wikipedia.The Raspberry Pi

5 Awesome Learning Resources For Programmers (To help you and your kids to grow the geek neurons)

Happy New Year, this is my first post in 2012. I’ll be sharing few awesome learning resources I’ve bookmarked, and will be pointing out some specific computer/programming related courses I've found interesting from these resources.Also, thought about saving this blog post for my kids as well - instead of investing in these Child education schemes (though they are too small as of today, 2 years and 60 days respectively ). Anyway, personally my new year resolution is to see as much videos from this course collections (assuming I can find some free time in between my regular job && changing my babies diapers).1 – Khan AcademyAs I mentioned some time back, you and your kids are missing some thing huge if you havn’t heard about Khan Academy.  It is an awesome learning resource, especially if you want to re-visit your basics in Math, Science etc.With a library of over 2,600 videos covering everything from arithmetic to physics, finance, and history and 268 practice exercises, th…