CF Performance Part 2

by Damon 30. April 2005 05:00
For round two of my performance testing I made an application that makes better use of CPU cycles and transactions and gives SQL Mobile a chance to show its true performance colors. The test platform was the same as before: CE.NET 4.2 device with 400mhz xScale and 128m RAM. The data used was also the same as Test 1: 5,000 inserts, then 10,000 inserts into a different table, then 1000 Updates and 100 selects against the tables.
If you don't care to read all the details, here's the executive summary of what you can expect from SQL Mobile:
  • Inserts for moderately sized tables are 10-12 times faster on SQL Mobile
  • Updates are as much as 100% faster on average
  • Table-scan Selects take less than half the time on SQL Mobile



I made a guess last time that an increased cost of getting connections and starting transactions (SQL Mobile has full ACID transactions whereas SqlCE is easily corruptible) might make SQL Mobile appear slower for "one off" operations. This time I simply changed the test to re-use an open connection and SqlCeCommand.Prepare(). I am truly pleased by the improvement in Insert performance in particular. For Inserts and Updates in Mobile applications where large amounts of data are being manipulated, grouping records into transactions would improve times even more. To Microsoft: After years of complaining about SqlCE, the only suitable adjective I can find for SQL Mobile is impressive.

An exhaustive SQL Mobile vs. SqlCE 2 test would be nice but also take more time than I have right now. I have not done any stored procedure performance tests, Transaction tests, nor any tests of table joins, nor any selects against Indexes, but this should be a rough indication of performance. In my next article, I will do some general performance testing on the Compact Framework V2 and one more SQL Mobile test. I intend to come up with a benchmark for:
  • Windows forms drawing performance
  • File IO
  • XML Web service performance
  • SqlCeResultSet performance

As always, I welcome any comments or questions.

Tags:

CF 2 Performance

by Administrator 28. April 2005 07:00

(Update: I have been noticing a lot of google traffic, please check out the follow up article for a better picture.

In all the mobile applications I work on, the performance of SQL CE 2 is a big source of pain. I've been hearing for some time that the replacement (SQL Mobile) is going to pay more attention to performance. Specifically, query performance on SQL CE 2 isn't that bad, but INSERT performance is abysmal and UPDATE performance not far behind. Yesterday I had a minute to look into some performance testing with Beta 2.

The first thing I discovered was that you cannot just connect a device to visual studio to quickly deploy. VS 2005 reports "The current version of active sync is not supported. Download the latest from www.microsoft.com." A quick groups search indicated that a Deploy from visual studio will require Active Sync 4, not yet released. So, I dug through the Visual Studio 8 directory to find the 3 cab files I needed to install Compact Framework 2 and Sql Mobile. I installed these, noticing that it appears you can run CF 1 and SQLCE2 along side CF 2 and Sql Mobile on the device. I was a little worried that I didn't have a way of knowing which version of the CF was loaded when I ran my command line tests. Right now I still need CF 1 on the device, but I may uninstall CF 1 later and redo one of my performance tests later to be sure I wasn't running uder CF 1 and just loading the Sql Mobile assemblies. I created a Smart Device CAB project and installed my performance test. On my device. I did a visual studio deploy from VS 2003 to a different directory to obtain CF 1 performace numbers.

My fist testing involved doing 4 different database operations, stolen from one of my mobile apps involving car data.

  1. Insert 5000 records in a simple table representing colors.
  2. Insert 10,000 records in a table involving VINs.
  3. Do an Update of 1,000 randomly chosen color records.
  4. Do a like% query 100 times against the table the VIN table.
In this test, the database connections are opened inside the loops, and the commands/command paramters are created inside the loops. No transactions or prepared commands can be used under this method. Obviously this is a horrible practice, but the goal was to exagerate the cost of opening connections to the different databases and such. Here are the numbers:

These tests were done on a Windows CE.Net 4.2 Device, 400mhz xScale processor, with 128 megs of ram. The data store is on a CF card. I should note that I had thought there would be an xScale optimized build of the compact framework, but I found none so I installed the ARMV4 build.

The first observation I made is that for pure DB operations, Release builds were slightly slower on average than debug builds. This was alarming, previous tests I've done have shown that a Release build is 5-7% faster over all for a Winforms app doing web services etc. The second, very disheartening observation, is that Inserts and Updates under Compact Framework 2 are twice as slow as the same code running against CF 1.0 Service Pack 3. The selects are twice as fast on the new platform, but selects weren't my complaint to begin with.

Note that I did not time every piece of each operation yet, and obviously this code is purposefully slow. Since Sql Mobile supports multiple concurrent connections, it may be that the longer times are related to an increased cost of getting a connection and the implicit transactions. The next test will involve preparing commands, re-using the same connection object, and testing UPDATE statements against a large (100,000 - 1,000,000 records) Sql Mobile table. Stay tuned.

Tags:

Almost Famous

by Damon 25. April 2005 05:00
I've a decent amount of .NET compact framework programming and I've started to consider Mobile one of my specializations.

This article mentions a very cool mobile solution I was the sole .NET architect/developer for. I had no idea they were going to do a newspaper article on this; funny that the article doesn't mention how the solution was developed... maybe I should ransom the source code for another article talking about the dedicated problem solver who made it all possible? (Carspot, I don't mean that, no need to call the lawyers)

I'm wondering if I can work this into some form of shameless self-marketing. Right now I am working on another "big nation wide, challenging, hardware issues we don't know if this will work but if it does its going to make money" Mobile project. Maybe this client will do an article as well? Perhaps I can work it into my contracts that clients will do a press release as soon as I'm brought onto a project? ;) While I don't mean to make too big a deal out of my 10 seconds of implied fame, this does give me cause to reflect on two career points:
  1. I'd really much rather be known as a hand waving architect, so I'm not sure how much I really want to promote myself as The Mobile Guy™. Still, I wish I had time for working on 4 projects at once and hanging out on the newsgroups and writing articles to try to get a Compact Framework MVP.
  2. As Travis pointed out to me, its becoming horribly obvious that people pay me to write products, which they then make a great deal of money from. I really should gain some knowledge in a particular business space and release my own products. No matter how hard I work and how high my rate gets, time only flows in one direction so there's only so much I can do. Taking advantage of the Business of Software is getting closer to the top of my to do list. The problem is good old fashioned American Instant Gratification Syndrome. (AIGS) I am gauranteed to get $X.00 for working X hours at X rate. I am building a new home right now, an endeavor which makes liquid cash handy if not necessary. After that, time to conduct myself more in line with my long term goals.

Tags:

VB.NET Part 1

by Damon 22. April 2005 05:00
I saw this over on Casey Chestnut's site and it set me off. I have long promised a VB.NET rant to some people in the Milwaukee community (you know who you are). I shouldn't care (and don't have time to care) about whatever antedeluvian technology every luddite on the internet (luddites on the Internet how's that for irony?) is carrying the torch for. I would like to point out some things that seem to me to fly in the face of the "equality" of VB which is shouted from the mountaintops.

This will not be an attack on the VB.NET community, I know a lot of great developers who perfer the semantics of VB.NET. To be continued...

Tags:

CLR Performance

by Damon 13. April 2005 05:00
I saw Jacob Cynamon speak at the local .NET user's group last night. He presents well and has a cool name. His topic was general performace tuning for the .net framework. Having gone through an "oh no perforance sucks" experience on a large ASP.NET project and an almost as large compact framework project already this year, this topic was of some interest to me; I wanted to see if I'd missed anything major.

The presentation was good, but it did look like I've hit the major points in my own optimization efforts. I wanted to add my own points to two of Jacob's: the cost of boxing and the cost of throwing exceptions.

Everyone talks about how expensive boxing is as if it's like downloading a 5 meg file over a 14400baud modem. Until last night I had not thought about exactly why boxing is so expensive or how expensive it is compared to other operations. Jacob's initial example showed value type assignment, compared to value type to object and object to value type. The boxing operation took about 13 times as long as assigning a value type. The first impression might be "See, boxing is the devil, we told you so". What could make it so expensive an operation? If I were implementing it I would create a container object to hold the value type and return it when asked. The cost would just be the creation of an object on the heap and maybe a tiny bit of relfection.

After the presentation was over I asked if he could throw a fourth comparison in there: object o = new object();. Sure enough, just calling "new()" took 99% as long as the boxing operation. What's my point? Yes, boxing is expensive compared to just using value types. Run CLR profiler once and see how many objects are created and garbage collected during a mundane operation we take for granted in .NET, like calling a web service. Boxing will be pretty low on my list of things to look for next time I need to squeeze performance out of an application.

The next item I wanted to add to was throwing exceptions. I read a statistic on MSDN somewhere (I can't find the link now) that claimed throwing an exception slowed down the executing thread by a factor of six, or something like that. This makes some sense, in terms of the op codes that would need to be executed to suddenly break execution a lot of things would need to happen: doing a "goto" and running Dispose methods for using and try/finally blocks can't be cheap.

My thought regarding exceptions was similar to my thought on boxing. You should only use exceptions for exceptional situations, not to control program flow. This is not a new tip, I read it first in one of Bjarne Stroustrup's books on the language he invented. Many developers may not be aware of how many exceptions the .NET class libraries are throwing and subsequently swallowing. You'll never see these exceptions unless you run your app in the debugger, and set all CLR exceptions to break into the debugger whether or not they are handled. You might be surprised how many times security violation exceptions are thrown while trying to read resource files, or to know that VB methods like CanParse or IsDate function by just trying to Parse and catching the FormatException if the operation fails. Depending on what functions your app is calling you may be suffering from hundreds or thousands of swallowed exceptions, especially at the time your app starts up. Matt Terski gave another good example in that every time you call Response.Redirect("foo") in ASP.NET a ThreadAbortedException is thrown. Check those First Chance excpetions!

Tags:

Smartphone Impressions

by Damon 11. April 2005 05:00
I've had my Smartphone for a few days and had a chance to mess around with the phone and the WindowsMobile 2003 Smartphone SDK.

I love this thing. Microsoft did a fantastic job on usability for the phone. I can access everything I need with my thumb and no need for typing. There are many little nice thoughts that show the attention to detail that went into this:
  • You can dial or text-message directly from your outlook contacts
  • During a call, the screen shows the person you're talking to. There are buttons on the call screen to pull up your calendar and contact info; if you're on the phone and somoene wants an appointment, you're right there.
  • Wireless sync is easy to set up
  • Performance is much better than I expected

I've actually used the data connection for "real world" things already. For example, I was out with friends Friday and we needed directions to a bar so having Pocket IE pull up Yahoo! yellow pages was useful. The speed of the data connection on the Cingular network is my only complaint so far.

The Smartphone SDK setup was as straightforward as it could be. Obviously there are some challenges to making an app usable on the phone, and I will post later about the multi-screen UI framework I came up with. The first time I ran my test app I got a security warning each time a new assembly was loaded. I need to look into what I need to do to get rid of this, maybe just signing will take care of it. I will say also that I have managed to "crash" the phone once after the compact framework installation. It doesn't seem like managed code should be able to do anything that would force you to reboot your phone to get sound back. Then again, I'm very good at breaking things.

As soon as I think of something useful I will try a connected app with some web services. Perhaps TechEd will give me some good ideas.

Tags:

Smartphone

by Damon 6. April 2005 05:00
My phone has been slowly dying, giving me an excuse to buy a WindowsMobile 2003 smartphone, the Motorola Mpx220. While I can't immediately think of exactly why I need to run connected .NET apps on my phone or what those apps may be, I downloaded the SDK already and wrote a simple app so I'm ready to go. Hopefully I don't regret getting a GSM phone and all that.

I will get back with my Smartphone observations after my number is ported and I've had it for a few days. I will get back to TRAP soon as well, but I am in the process of building a house right now which takes some time. That and actual paying customers have been very distracting lately.

Tags:

About the author

Damon Payne is a Microsoft MVP specializing in Smart Client solution architecture. 

INETA Community Speakers Program

Month List

Page List

flickr photostream