MEF for Windows Phone 7

by Administrator 25. June 2010 03:52

Given the momentum behind MEF at Microsoft and the fact that MEF has shipped in the box with Silverlight 4, you may find it somewhat disappointing that MEF is not available for Windows Phone 7.  Due to low level differences in the CLR for Windows Phone 7, you can’t just grab MEF from Codeplex either – read on to see the solution.

{Edit: I later posted the binaries http://www.damonpayne.com/post/2011/03/01/MEF-for-Windows-Phone-7.aspx }

The Goal

Given the Marketplace restrictions, we know that our application code must go through a QA process and be signed in order to be deployable to the Windows Phone 7 (henceforth The Phone, or WP7). It’s not surprising then that the popular Silverlight Dynamic Recomposition techniques using MEF and downloaded XAP files won’t work.  However, it’s still incredibly useful to be able to use DeploymentCatalog and friends to wire together components in an application.  Getting this scenario working on The Phone is how I’m defining success for now.

Getting MEF to Build and Run

Prior to .NET 4 and Silverlight 4 shipping MEF has been available in preview form on Codeplex.  I was surprised that, being a daily reader of Silverlightshow.net, I hadn’t already seen an article submitted by someone who had built MEF for Windows Phone 7.  Now I know why.

Trial 1 – Building MEF Preview 9 as a Silverlight 3 Project

Many Silverlight 3 assemblies will turn out to just work on The Phone.  It seems reasonable that the first thing to try is to get MEF Preview 9 from CodePlex.  You can build this as a Silverlight 3 project and then reference the binary output from a WP7 project.  Trying to use DeploymentCatalog immediately blows up at runtime though.  That would have been too easy.

Trial 2 – Referencing MEF Preview 9 as a Silverlight 3 Project with Source

Rather than just building and referencing System.ComponentModel.Composition and System.ComponentModel.Composition.Initialization as assemblies, I thought I’d add the source projects to my test solution and step through to see what was going on.  It could always be something simple and obvious.  I was getting some interesting MissingMethodException and MethodAccessException behavior so as feared this was likely going to involve source code modifications to get it working, if it was possible at all.  I moved on to the next logical choice.

Trial 3 – Building MEF as Windows Phone 7 Class Libraries

At this point I assumed some low-level binary incompatibility or missing Type was the issue so I decided to create two new WPF class library projects and add the respective MEF source files to these.  Building MEF using Windows Phone 7 Class Library as the project type would immediately show me any issues with base class library parity.   At this point I was confronted by three major issues I was previously unaware of.

No System.Reflection.Emit

I was always extremely happy that SRE was included in Silverlight, so I was a bit surprised to find it missing from The Phone.  Sure, we never had it in Compact Framework development but The Phone is a much more heterogeneous platform compared to the giant universe of Windows CE/Windows Mobile hardware.  This spells doom for some of my more advanced Silverlight development scenarios, and for certain MEF features as well.  MEF uses SRE to create dynamic implementations of interfaces for Metadata – very handy for Lazy Imports. 

I have a reasonably easy workaround for the Import Metadata issue, but eager to get my main success scenario working I commented this feature out for now and moved on.

No IQueryable

The MEF code uses a fair bit of IQueryable internally.  Not being familiar enough with LINQ Expression hacking I can’t fathom why this was left out.  Performance implications?  Required Reflection Emit?  I don’t know.  Nonetheless, this code had to be changed in places like TypeCatalog, AggregateCatalog, and the ComposablePartCatalog base class.  Until I at least got this to compile I wouldn’t know if IEnumerable would work as a surrogate for IQueryable or not.

{No IQueryable http://social.msdn.microsoft.com/Forums/en-US/windowsphone7series/thread/79858a29-4db0-460d-9a75-3630211a28fb/ }

No Dynamic Assembly Loading.  Period.

Silverlight XAP files are packaged with a Deployment Manifest that makes it easy to enumerate the assemblies included in a given XAP file.  In the Full versions of Silverlight you can use these AssemblyParts to load the assemblies and do useful things like add them to an AssemblyCatalog for MEF.

In their Zeal to prevent you from somehow loading code onto The Phone that had not gone through The Marketplace, you can’t load assemblies by name using this technique or any other technique.  Unfortunately this even includes assemblies that are hard-referenced by your WP7 Application.  What the harm would be in allowing developers to get a handle to all the assemblies that are hard-referenced is beyond me but this one is a showstopper.  Or is it?

We Can Continue, but Should We?

Suppose we can keep going.  Is MEF worthless in the Windows Phone 7 scenario given these limitations or is there still value to be had?  MEF is about Composition, Isolation, Separation, and Extensibility.  MEF is about programming against abstractions and making it easy for third parties who are unknown at compile time to extend your applications.  While some of these benefits cannot be realized given the WP7 limitations there are still meaningful benefits.

  1. Suppose you are developing frameworks and using MEF as your mechanism of wiring up Abstractions to their Implementations.  If you want to share any code with The Phone, it would suck to not at least be able to use MEF to map “IWidget to DefaultWidget”.  Programming against abstractions is always good.
  2. You may simply prefer the Import/Export programming model to alternatives like Factories and Service Locators.  Doing these things manually can take a lot of code.
  3. You may be used to ImportMany semantics, which is not a common feature in IoC containers that I know of.
  4. Microsoft may loosen up the WP7 deployment story in the future and you want to be ready.
  5. You love MEF and you have a Man-crush on  Glenn Block and you want to achieve Universal MEFness

The Current Workaround

For my part, I am working on frameworks that I’d like to recompile for Windows Phone 7, and I’ve become very accustomed to the great programming model MEF enables.  I’m NOT going to manually register every single Type I use to get around the DeploymentCatalog issues though.  Given that we can’t load assemblies by name, is there any compromise we can make?  Yes there is.

Since we can only hard-reference assemblies for Windows Phone applications, we can cheat and get a handle to each assembly we want to participate in Composition by using any Type from each assembly.  By then modifying DeploymentCatalog to add an additional constructor we can at least run some sample code and see what’s working and what’s not.  I went ahead and created a Mobile version of the Metro BBQ example I’ve been using lately.  I have an entry point application and two module assemblies and I’d like to to use MEF to glue them all together.

projects

In the entry point application there is a simple UI that displays Gas Grills and Charcoal Grills, and I’d like to create a simple ViewModel that gets populated with IGrill implementations via MEF Catalogs.

namespace MetroBBQ.Mobile
{
    public class ShellViewModel : INotifyPropertyChanged
    {
        private IEnumerable<IGrill> _AllGrills;

        [ImportMany]
        public IEnumerable<IGrill> AllGrills
        {
            get { return _AllGrills; }
            set
            {
                _AllGrills = value;
                OnPropertyChanged("AllGrills");
            }
        }              

        public event PropertyChangedEventHandler PropertyChanged;

        protected void OnPropertyChanged(string propName)
        {
            if (null != PropertyChanged)
            {
                PropertyChanged(this, new PropertyChangedEventArgs(propName));
            }
        }
        
    }
}

Now using some bootstrapper types from each assembly along with my DeploymentCatalog modifications, I create a CompositionContainer, and give it all a try.

private void PhoneApplicationPage_Loaded(object sender, RoutedEventArgs e)
{
    var shimTypes = new List<Type> { typeof(MetroBBQ.Mobile.GasGrills.Bootstrapper), typeof(MetroBBQ.Mobile.CharcoalGrills.Bootstrapper), GetType() } ;
    var deploymentCatalog = new DeploymentCatalog(shimTypes);
    var container = new CompositionContainer(deploymentCatalog);
    var vm = new ShellViewModel();
    container.ComposeParts(vm);
    DataContext = vm;
}

Sure enough, I’m now able to Compose Parts just like my MEF experience on other platforms.  There are a LOT of other scenarios to test but I’m encouraged by this initial success.  My ViewModel gets populated with IGrill implementations from two other assemblies in my Windows Phone 7 application.

GrillsScreen

 

What do you think, dear reader?  Is it worth doing more testing or shall I add this to the collection of forgotten novelties on my shelf?  Would you like the source code?  Is this worthy of MEFContrib? Leave me a comment!  For my part, I’m happy to be able to continue my MEF experience on this new platform.

Tags:

Great Features for MVVM Friendly Objects Part 1 - Introducing Property Change Behaviors

by Administrator 20. June 2010 20:23

In the next article in this series, I introduce the idea of Property Change Behaviors.  This concept is used to provide a means of easily attaching useful features to data objects.  Any number of features can be added without requiring a complex inheritance structures or keeping you from configuring the features on a per-property or per-instance basis.

This article is part of a series about useful MVVM friendly features for your data objects and favoring composition over inheritance.  You can read the other parts:

Can we all agree to call Silverlight/WPF development “XamlWorld” ?

Introducing Property Change Behaviors

Recall that we opened the first article with a list of actions we might want to take when a property on a XAMLWorld data object changes:

  1. Firing change notification
  2. Handling property-level validation
  3. Tracking when an object has changed (dirty/change tracking)
  4. Update dependent or calculated properties
  5. Providing Undo support

It would be ideal if we could provide a mechanism to do all of these things, and whatever else we think of in the future, in a generic fashion.  Since XAML technologies provide fantastic data binding support centered around the notion of CLR Properties, connecting our behaviors to properties is a very good approach.  If you want to use anything that’s built into XamlWorld, you need to make peace with the notion that The Property is the coin of the realm. 

We can think of Property Changed Behaviors as a sort of “visitor” or “Chain of Responsibility”.  Here is the interface for Property Change Behaviors:

public interface IPropertyChangedBehavior
{
    /// <summary>
    /// Do something useful when a property changes
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="owningInstance">Object instance whose property has changed</param>
    /// <param name="oldVal">current property value</param>
    /// <param name="newVal">new property value</param>
    /// <param name="propertyName"></param>
    /// <returns>Returns true of more behaviors can keep processing</returns>
    bool PropertyChanged<T>(object owningInstance, T oldVal, T newVal, string propertyName);
}

There are more complex scenarios for IPropertyChangedBehavior, but for this series of articles we’ll stick with the first draft.  This interface allows us to do interesting things when Property values change.

Remember what our BindableType property setter looked like previously?  Recall that we did this to avoid re-creating the basic INotifyPropertyChanged implementation over and over:

protected void OnPropertyChanged(string propName)
{
    if (null != PropertyChanged && !SuspendChangeNotification)
    {
        PropertyChanged(this, new PropertyChangedEventArgs(propName));
    }
}

In order to move forward I’ve changed the semantics somewhat.  The code snippet for setting properties in this style is provided at the end of this article.  Keeping with the Metro BBQ example from the previous article, here’s what the properties look like on my BBQGrill class.

Materials _KettleMaterial;

public Materials KettleMaterial
{
    get { return _KettleMaterial; }
    set { Set<Materials>(ref _KettleMaterial, value, "KettleMaterial"); }
}

This allows me to use semantics like the following on the BindableType base class.  Yes, we’re still extending BindableType for now, one step at a time.  We will eventually refactor to avoid the need to extend BindableType.

protected void Set<T>(ref T local, T newVal, string name)
{            
    T localCopy = local;
    local = newVal;

    if (null != _changeBehaviors)
    {                
        foreach (var behavior in _changeBehaviors)
        {
            bool @continue = behavior.PropertyChanged<T>(this, localCopy, newVal, name);
            if (!@continue) { break; }
        }
    }
    else//no behaviors, just carry on with the old way
    {
        OnPropertyChanged(name);
    }                
}

Now things are starting to get interesting.  BindableType has an instance variable to contain the IPropertyChangedBehaviors we want to use.

protected IEnumerable<IPropertyChangedBehavior> _changeBehaviors;

Each object now has a configurable list of interesting actions that we can go through any time a property value changes.  Based on the fact that we’d like to enable Binding updates, the choice for the first IPropertyChangedBehavior implementation is obvious:

public class NotifyPropertyBehavior : IPropertyChangedBehavior
{
    public NotifyPropertyBehavior(INotifyPropertyChanged target, Action<string> changedCallback)
    {
        Requires.NotNull(target, "target cannot be null");
        Requires.NotNull(changedCallback, "changedCallback cannot be null");
        Target = target;
        ChangeCallback = changedCallback;
    }

    public INotifyPropertyChanged Target { get; set; }

    /// <summary>
    /// Needed since only the owning instance can fire the event
    /// </summary>
    public Action<string> ChangeCallback { get; set; }


    /// <summary>
    /// Fire change notification only if the new value != old
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="owningInstance"></param>
    /// <param name="oldVal"></param>
    /// <param name="newVal"></param>
    /// <param name="propertyName"></param>
    /// <returns></returns>
    public bool PropertyChanged<T>(object owningInstance, T oldVal, T newVal, string propertyName)
    {
        if (!Object.Equals(oldVal, newVal))
        {
            ChangeCallback(propertyName);
            return true;
        }
        else
        {
            return false;
        }
    }
}

… and in the constructor of BindableType, we create the following defaults used in the Setter, above:

_changeBehaviors = new List<IPropertyChangedBehavior>

{ new NotifyPropertyBehavior(this, s=> OnPropertyChanged(s) ) };

Now, we are in a situation where our data objects can have any number of IPropertyChangedBehaviors attached.  For now, the NotifyPropertyBehavior has merely replaced how we fire change notification, but the behaviors shown in the following articles will implement much more interesting functionality.

PropertyChangedBehaviorCD

Conclusion

So, we’ve created some basic functionality for iterating through an ordered list of interesting actions whenever a property value changes.  Based on the logic inside BindableType, if the property values are equal we would stop processing subsequent Property Change Behaviors for this instance by returning false from NotifyPropertyBehavior.

In the next article we’ll start adding more useful and interesting features using the IPropertyChangedBehavior approach.

Download the bprop code snippet.

Tags:

Great Features for MVVM Friendly Objects Part 0: Favor Composition Over Inheritance

by Administrator 17. June 2010 02:18

Even if you’re using a framework like RIA Services, one of the first design decisions you’ll run into when building a new Silverlight application is how to represent your data on the client.  Above and beyond implementing change notification (INotifyPropertyChanged for bindings) there may be several other features you need such as Undo support, dirty tracking, dependent properties, and validation.  I’m going to show you how to do support these features in a plug-and-play style.

This article is the First Part in a series about useful MVVM friendly features for your data objects and favoring composition over inheritance.

{The future articles will be linked to here as well}

A Base Class and a Mess?

We need to start representing our data types in Silverlight.  If you’ve written a few Silverlight applications, you’ve probably implemented a base class to avoid retyping the INotifyPropertyChanged basics.  In my case, I also write some code snippets to be able to create these properties quickly.

namespace HandWaver.PresentationModel
{
    /// <summary>
    /// A handy base class for classes that want to participate in data binding
    /// </summary>
    public class BindableType : INotifyPropertyChanged
    {
        public BindableType()
        {
            SuspendChangeNotification = false;
            _changeBehaviors = new List<IPropertyChangedBehavior> { new NotifyPropertyBehavior(this, s=> OnPropertyChanged(s) ) };
        }

        public event PropertyChangedEventHandler PropertyChanged;

        /// <summary>
        /// 
        /// </summary>
        /// <param name="propName"></param>
        protected void OnPropertyChanged(string propName)
        {
            if (null != PropertyChanged && !SuspendChangeNotification)
            {
                PropertyChanged(this, new PropertyChangedEventArgs(propName));
            }
        }

        /// <summary>
        /// Allow turning of change notification
        /// </summary>
        public bool SuspendChangeNotification { get; set; }

Now, depending on your application requirements you may also want to support some of the following features:

  • Validation: You’d like to hook into the Silverlight validation framework to display data entry errors to the end user.
  • Dirty Checking: Maybe you’d like to only enable the Save button when something actually changes on your Model objects.  Maybe you don’t want to bother calling your “Save” WCF service if nothing has actually happened.  Has a given Instance changed or is it sill in it’s original state?
  • Dependent (Reactive or Calculated) properties: Suppose when one property of an instance changes, you need to update another property.  For example, you may have a read-only property which returns the sum of several other properties.  
  • Undo: You may have an Undo requirement, and adding functionality during property change is a good place to implement this. Having excellent multi-level undo support isn’t as hard as you might think!

Given the fact that I want to reuse my INotifyPropertyChanged code, I may wind up with an inheritance hierarchy for each of these features:

click for large version

The new UML modeling features in Visual Studio 2010 Ultimate are awesome!

Take a look at the two notes at the bottom of this class diagram.  We encounter problems when we need to start mixing and matching this functionality.  If I need both a DirtyAware type and a Validating type, I’ll wind up creating a ValidatingDirtyAware base class.  This quickly leads to explosion of the number of classes.  It also means the code I implement for each feature might get copied around, making it hard to maintain.  We shouldn’t need to create a new base class for each new feature we think of. 

Introducing Metro BBQ

While I have several “real” Silverlight applications I work on, I like to use fun samples.  I have also adopted a “No battleship gray” policy lately in order to force myself to work on design and UX more.  Still, this takes time so for this series of articles I will use one of the new Silverlight 4 Application Themes.  This is a Silverlight application dealing with BBQ grills, BBQ-ing, smoking, etc.  As I am using the Metro theme, I am calling this Metro BBQ. 

The part of Metro BBQ we’re interested in is a screen where users can tell us about their ideal BBQ Grill configuration.

metrobbq0

This simple form is missing some functional and design features :

  • We need to be able to handle Validation without relying on a base class
  • We’d like to keep the Submit and Reset buttons disabled until the user makes changes (dirty checking)
  • We need to update the “Metro BBQ Grill Grade” as the user changes values in this form (dependent properties)
  • We’d like to provide an Undo button to provide undo for the user’s input

A Distinct Responsibility

The first responsibility we can break out is to offload the bulk of the Validation Error implementation work from BindableType into it's own class.  ValidationErrorManager does most of the work needed for an INotifyDataErrorInfo implementation.

using System;
using System.Linq;
using System.ComponentModel;
using System.Collections.Generic;
using HandWaver.Core;

namespace HandWaver.PresentationModel
{
    /// <summary>
    /// 
    /// </summary>
    public class ValidationErrorManager
    {
        public ValidationErrorManager(INotifyDataErrorInfo client, Action<string> errorsChangedAction)
        {
            Requires.NotNull(client, "Must specify target client");
            Requires.NotNull(errorsChangedAction, "errorsChangedAction: callback must be specified");
            _client = client;            
            ErrorsChanged = errorsChangedAction;
        }

        INotifyDataErrorInfo _client;
        Dictionary<string, List<string>> _errors;
        
        /// <summary>
        /// 
        /// </summary>
        public Action<string> ErrorsChanged { get; set; }

        /// <summary>
        /// Clear errors for the given property and fire ErrorsChanged for that property
        /// </summary>
        /// <param name="propName"></param>
        public void ClearErrors(string propName)
        {
            if (null != _errors && _errors.ContainsKey(propName))
            {
                _errors[propName].Clear();
            }
            ErrorsChanged(propName);
        }

        /// <summary>
        /// Add an error for the given property and fire ErrorsChanged for that property
        /// </summary>
        /// <param name="propName"></param>
        /// <param name="error"></param>
        public void AddError(string propName, string error)
        {
            EnsureErrorContainer(propName);
            _errors[propName].Add(error);
            ErrorsChanged(propName);
        }

        /// <summary>
        /// 
        /// </summary>
        protected void EnsureErrorContainer()
        {
            if (null == _errors) { _errors = new Dictionary<string, List<string>>(); }
        }

        /// <summary>
        /// 
        /// </summary>
        /// <param name="propName"></param>
        protected void EnsureErrorContainer(string propName)
        {
            EnsureErrorContainer();
            if (!_errors.ContainsKey(propName))
            {
                _errors[propName] = new List<string>();
            }
        }

        /// <summary>
        /// 
        /// </summary>
        /// <param name="propertyName"></param>
        /// <returns></returns>
        public System.Collections.IEnumerable GetErrors(string propertyName)
        {
            if (null == _errors
                || !_errors.ContainsKey(propertyName))
            {
                return null;
            }
            return _errors[propertyName];
        }

        /// <summary>
        /// 
        /// </summary>
        public bool HasErrors
        {
            get
            {
                if (null == _errors) { return false; }
                int propsWithErrCount = _errors.Values.Where(l => l.Count > 0).Count();
                return (propsWithErrCount > 0);
            }
        }

    }
}

With comments ValidationErrorManager is over 100 lines of code, completely worth moving into it’s own class.  The BBQGrill class can use this functionality thusly:

public class BBQGrill : BindableType, INotifyDataErrorInfo
{
    public BBQGrill()
    {
        FireType = FireTypes.Charcoal;
        Smoker = false;
        KettleMaterial = Materials.Aluminum;

        _errMgr = new ValidationErrorManager(this, s => OnErrorsChanged(s));
    }

    ValidationErrorManager _errMgr;

    double _GrillingArea;

    /// <summary>
    /// Grilling area in square inches
    /// </summary>
    public double GrillingArea
    {
        get { return _GrillingArea; }
        set
        {
            if (value < 150.0)
            {
                _errMgr.AddError("GrillingArea", "You need a bigger grill!");
            }
            else
            {
                _errMgr.ClearErrors("GrillingArea");
            }
            Set<double>(ref _GrillingArea, value, "GrillingArea");
        }
    }

We’re still extending BindableType for now, but we’ve made it very easy to add a new feature (property-level validation) to our data types.  At this point we have added this feature without adding a new layer to the inheritance hierarchy.  The class diagram for BBQGrill looks like this now:

validationerrormgrcd

Conclusion

This may not look like much at this point, but we’re taking the first step towards a powerful and flexible way of thinking.  In the next part, we’ll demonstrate a mechanism that’s useful for mixing and matching even more powerful features for data types,starting with change tracking.

Tags:

Gone Fishing

by Administrator 6. June 2010 20:35

I did most of my growing up in The Show Me State which for all intents and purposes is much more rustic and old fashioned than Wisconsin.  My technical readers may be shocked to learn that had my family not moved to Wisconsin when I was 15 I would no doubt be gainfully employed in the Lumber Industry of the Ozarks.  At any rate, I used to be seriously into fishing – this was perhaps the first incarnation of my Obsessive Research-Driven Hobby Syndrome (ORDHS).  I would pour over each new issue of the Bass Pro Shops catalog mixing and matching the combination of fishing gear that would create the Ultimate Rig.  I would later start to do this with computer parts, cars, and cooking ingredients.

Teenage life in Wisconsin took me away from the hobby entirely.  I haven’t been freshwater fishing in nearly 20 years.  I say freshwater because my father has enticed me a few times to hit the Gulf of Mexico and it’s been fun.  How could I have enjoyed Beer and Fishing separately for so long without putting the two together?  The same instigator took my daughter fishing two summers ago and let me tell you she still sits in the back yard with her Spongebob Squarepants rod and reel practicing.  There’s something about fishing that small kids tend to love.  The same little girl who reminds me not to unnecessarily step on ants has no qualms about demanding we catch and subsequently cook and eat wild fish.  That combined with my son having just turned two years old has left me feeling that I’m shirking some of my fatherly responsibilities.

Yesterday I purchased a Wisconsin fishing license and some ultralight spinning gear.  As soon as I figure out when and where, I’m going fishing!

Tags:

Chicago Silverlight Users Group MVVM Code and Slides

by Administrator 4. June 2010 01:58

Thanks to everyone who came out last night, there were a lot of good question.  I was pleased to claim victory over Chicago traffic.

Yes, we did go to Fogo de Chao afterwards.  I’ve been invited back to hit some more advanced MVVM topics or perhaps a “MEF and Silverlight” show.

As promised, here’s the material from my MVVM talk last night. 

Power Point

Green Screen code

Tags:

Presenting on MVVM at the Chicago Silverlight Users Group

by Administrator 24. May 2010 04:48

Next week I’ll be speaking about the MVVM design patter in Silverlight (and WPF) in Chicago.

The Model View ViewModel (MVVM) is one of the most quickly and widely adopted design patterns in recent memory.  The models and conventions built into Silverlight and WPF lend themselves to this pattern extremely well. Join Damon Payne as he discuss a history of Presentation Model patterns, MVVM in Silverlight and WPF, and how to achieve Blendability at design time with MVVM.  He'll also spend some time demonstrating how MVVM helps you cope with the asynchronous constructs in Silverlight.

You can register here:

http://chicagosilverlight.eventbrite.com/?ref=eivte&invite=MzU0OTM0L2RhbW9uQGRhbW9ucGF5bmUuY29tLzE%3D%0A&utm_source=eb_email&utm_medium=email&utm_campaign=invite

The Illinois Technology Association
200 S. Wacker Drive, 15th Floor
Chicago, IL 60606

One of my favorite topics in one of my favorite cities, this should be fun.  If anyone from the Metro Milwaukee area would like to come down and perhaps do some Fogo de Chao afterwards please let me know!  Maybe some Chicagoland carnivores will join us?

Tags:

DataContract based Binary Serialization for Silverlight

by Administrator 24. May 2010 04:36

This is an article about how I set out to write a Binary Serializer for Silverlight and what I learned along the way.   I have some very large Reference Data entity sets in isolated storage and loading these sets so I can run LINQ queries against them has been extremely painful – I don’t want to have to wait 30 seconds before my feature is usable.  My first analysis lead me to the conclusion that disk IO was my main performance bottleneck and that drastically reducing the file sizes by using Binary Serialization would be just what the doctor ordered.

[Note: if you’ve seen me speak recently, yes I’ve been talking about posting this for quite a while]

Design goals:

  1. Uses familiar DataContract/DataMember/KnownType semantics – use something that my types are already decorated with and that developers are already familiar with.
  2. No fixed buffer size for strings – I have seen some other Silverlight/Binary Serialization schemes that use fixed buffer sizes for strings.  This is undesirable.
  3. FAST – I wanted this to be faster than the built in XML + DataContractSerializer mechanism.
  4. Theoretically Survive assembly rebuilds – Most binary serialization schemes (think ASP.NET SQL server session state) do not tolerate different assembly versions but rather require an exact match.  Since I’m using [DataContract] semantics I hoped to allow whatever properties still match to survive across builds if possible.
  5. Handle lists and complex object graphs
  6. Low disk usage: binary files should be far smaller than equivalent XML files

Building the Serializer

There’s no built in binary serialization for Silverlight or I wouldn’t be writing this article.  There are, however, BinaryWriter and BinaryReader classes.  This saves one from needing to write custom logic for all primitive types and in particular being careful to encode strings with additional length attributes.  This leaves us with fewer problems:

  1. Logic to recursively serialize object graphs
  2. Handling null objects
  3. Handling collection types
  4. Persisting data about what types and what properties of those types were serialized.  This is key for meeting the durability design goal (#4)

The source code will be provided so we’ll only go over some highlights here.  The first thing to do was to build an object with several different property types to test with.  Instances will populate themselves with random values.  While this can throw off apples-to-apples comparisons the differences should be significant enough to ignore this.

[DataContract]
public class RefDataRow
{
    /// <summary>
    /// Set some random values
    /// </summary>
    public RefDataRow()
    {
        Random r = new Random();
        Id = Guid.NewGuid();
        Field0 = r.Next(100000);
        Field1 = r.Next(100000);
        Field2 = r.Next(100000);
        Field3 = r.Next(100000);
        Field4 = r.Next(100000);
        Field5 = r.Next(100000);
        Field6 = r.Next(100000);
        Field7 = r.Next(100000);
        Field8 = r.Next(100000);
        Field9 = r.Next(100000);
        Value = r.NextDouble();
        int dLen = r.Next(50);
        var sb = new StringBuilder();
        for (int i = 0; i < dLen; ++i)
        {
            sb.Append((char)r.Next(26) + 65);
        }
        Description = sb.ToString();
    }

    [DataMember]
    public Guid Id { get; set; }

    [DataMember]
    public int Field0 { get; set; }
    [DataMember]
    public int Field1 { get; set; }
    [DataMember]
    public int Field2 { get; set; }
    [DataMember]
    public int Field3 { get; set; }
    [DataMember]
    public int Field4 { get; set; }
    [DataMember]
    public int Field5 { get; set; }
    [DataMember]
    public int Field6 { get; set; }
    [DataMember]
    public int Field7 { get; set; }
    [DataMember]
    public int Field8 { get; set; }
    [DataMember]
    public int Field9 { get; set; }

    [DataMember]
    public double Value { get; set; }

    [DataMember]
    public string Description { get; set; }

We add some more code to this class later, but for now this will do.  So far we’ve done nothing but decorate this class with DataContract/DataMember attributes.  There’s a mix of various data types in here, with a lot of integers in the middle.

Test UI

A user interface to run tests and show results will help.  I’ve come up with the following options which can be ran in order if we wish to perform all tests.  The last two buttons can be ignored for now.

binser0

Running Some Tests

My Generate Test Data command creates 300,000 randomly instantiated instances of my RefDataRow class.

I am using some new framework libraries I’m working on here, but in essence there’s a ViewModel with a command bound to each button.  You can see that using the BinarySerializer looks very similar to DataContract serializer:

DataContractSerCmd = new TimedCommand<string>(s =>
{                
    using (var fs = file.CreateFile(XmlDataFileName))
    {
        var dcs = new DataContractSerializer(typeof(List<RefDataRow>));
        dcs.WriteObject(fs, TestData);
    }
    ReportSizes();

}, timer, "Data Contract Serialization");


BinarySerCmd = new TimedCommand<string>(s =>
{
    using (var fs = file.CreateFile(BinDataFileName))
    {
        var bs = new BinarySerializer(typeof(List<RefDataRow>));
        bs.Serialize(TestData, fs);
    }

    ReportSizes();

},timer, "Binary DataContract Serialization");

DataContractDeSerCmd = new TimedCommand<string>(s =>
{
    using (var fs = file.OpenFile(XmlDataFileName, FileMode.Open))
    {
        var dcs = new DataContractSerializer(typeof(List<RefDataRow>));
        var obj = dcs.ReadObject(fs);
    }

}, timer, "DataContract Deserialize");

BinaryDeSerCmd = new TimedCommand<string>(s =>
{
    using (var fs = file.OpenFile(BinDataFileName, FileMode.Open))
    {
        var bs = new BinarySerializer(typeof(List<RefDataRow>));
        var obj = bs.DeSerialize<List<RefDataRow>>(fs);
    }

}, timer, "Binary Deserialize");

So, we have four DelegateCommand<T> style commands wrapped in a low-resolution timer.  Since I’m randomly generating test data each time the results will be slightly different, but I found the run below to represent an average case:

binser1

 

Thinking About the Results

So, I’ve met my design goals, but my performance goals are way off the mark.  What’s going on?

Disk Space Usage

Without paying the angle-bracket tax, and by using binary serialization, the binary file is less than 1/3 the size of the XML file.  The one saving grace I can think of, in terms of my performance goals, is that I’m doing these tests on a Solid State Drive.  With incredibly high sequential write speeds, any performance gains that might come from writing a much smaller file to disk are certainly minimized.

Performance

The out of the box DataContractSerializer is faster, and on deserialization it completely eats my lunch.  What happened?  I have a theory.

What happens when you try to out-do Microsoft engineers?  Sometimes you lose.  Reflection is an awesome and powerful CLR feature and has always been known as a performance no-no.  While I read many times how much faster Reflection (and App Domains) became in the 2.0 runtime it’s still really slow to invoke methods via reflection.  My code is calling all getters and setters here using PropertyInfo.  The built in DataContractSerializer is almost certainly using Reflection Emit to generate “real” classes to get and set values and that is going to be a lot faster than what I’m doing here.  While I’ve written a lot of SRE code in the past couple of years, I want to test my theory before I take the time to do that here.

Another Try

In order to prove that Reflection is my downfall, I create a new .NET interface:

public interface IBinarySerializable
{
    void WritePrimitiveValue(string propName, BinaryWriter bw);
    void ReadPrimitiveValue(string propName, BinaryReader br);
}

The goal here is to have my RefDataRow call setters on itself, looking up a lambda expression in a Dictionary by property name.  Some of the Writer code might look like so…

_actions= new Dictionary<string,Action<BinaryWriter>>();
_actions.Add("Id", bw => bw.Write(Id.ToByteArray()));
_actions.Add("Value", bw => bw.Write(Value));
_actions.Add("Description", bw => bw.Write(Description));
_actions.Add("Field0", bw => bw.Write(Field0));

… with the corresponding reader code looking like this:

_readActions = new Dictionary<string, Action<BinaryReader>>();
_readActions.Add("Id", br => Id = new Guid(br.ReadBytes(16) ) );
_readActions.Add("Value", br => Value = br.ReadDouble());
_readActions.Add("Description", br => Description = br.ReadString());
_readActions.Add("Field0", br => Field0 = br.ReadInt32() );

I am then giving the BinarySerializer a hint, telling it to check for DataContract objects implementing IBinarySerializable and using real method calls. Even though these are Virtual method calls, they’re going to be a lot faster than Reflection.  What’s the result?

binser2

I finally take a strong lead in serialization, and deserialization has increased by 100%, but that aspect is still slower than the built in DataContractSerializer. 

Conclusion

This needs a lot more testing before it’s production ready for complex object graphs but it does work and it’s easy to use.  If I can do some more research and get the read&deserialize speed way down, I’ll go ahead and do the System.Reflection.Emit work necessary to make the speed gains from IBinarySerializable automatic without having to implement this interface.  The source code will be provided as part of a new application framework I’m working on.

Since reading is my issue here, I have to wonder about implementing a XamlWriter in Silverlight, so that I could use XamlReader on the reading end.  XamlReader is extremely fast as it uses a lower level engine than my object models can get at. 

On the flip side, if you are dealing with less than 300,000 complex objects, this may be just what you wanted. Stay tuned for more research on this topic.

Tags:

Speaking at Wisconsin .NET User&rsquo;s Group

by Administrator 6. May 2010 23:03

Someone had to cancel their talk at the Wisconsin .NET User’s Group so guess what: you’re stuck with me.  Although this is only a few days away the topic is still up for grabs.  Check out the twitter poll:

http://twtpoll.com/olezfc

I’ll either be going over What’s new in Silverlight 4 or Silverlight for Windows Phone 7 development, or maybe a little of both.  Both are really fun topics and I hope to see you there.  I’ll update this post soon with the results of the poll.  You can also leave comments here.

[Update: Looks like Windows Phone 7 Development won the poll!  This will be a fun talk, and don’t worry we can talk about Silverlight 4 during Q and A]

Tags:

Code (and slides) from Chicago Code Camp

by Administrator 6. May 2010 15:49

I had a great time at the Chicago Code Camp this year, although I was really only able to dash in, give my talk, and dash out.  What’s more, I was competing with some famous people, so I was pleased that my room was packed and that there were so many thoughtful questions.  One gentleman even told me he drove three hours to this event just to see me.  Wow!

Here are the materials from my presentation:

Slides

Code

Since the topic of run time composition came up more than once, see here for a use case and sample code.

I hope to be invited back next year!

Tags:

Chicago Code Camp 2

by Administrator 23. April 2010 01:19

[Update: the schedule has been posted]

I’m pleased to announce that the latest version of my “What can I do with MEF?” talk has been accepted for Chicago Code Camp 2 on May first.  While I actually have a release at work that day they’ve been kind enough to let me go towards the end of the day.

In this talk I’ll go over MEF soup-to-nuts with an emphasis on isolation and Silverlight applications.  In staying true to the code camp manifesto there will be only a couple of slides and a lot of code.  I hope to see a lot of familiar faces from the metro-Milwaukee area.

If (for some reason) coming to see me won’t get you into a Chicagoland code camp on a Saturday, there’s also the .NET Rocks! Visual Studio 2010 Road Trip.  Come meet Carl and Richard and do a review of the event in order to get a free special edition .NET Rocks! mug.

Tags:

About the author

Damon Payne is a Microsoft MVP specializing in Smart Client solution architecture. 

INETA Community Speakers Program

Month List

Page List

flickr photostream