.NET Architect, gourmet cook, and all-around good guy Matt Terski confided in me once that the great hurdle in getting peolpe to understand SOA was not teaching them transport protocols and such but getting them to Think Asynchronously. I have posted before about how the Cage Builder culture in some organizations tells developers to "Never use threads". This is a barrier to responsive apps, great performance, and adoption of SOA.
For my current client I built another custom data synchronization process for a mobile device. We are not using Merge Replication for any number of reasons, mostly because of the constraints it puts on your database schema. This sync process runs over a Sprint Airlink modem mounted on a truck; the speed of the modem is entirely dependent upon the quality of the connection it has at any given time. Since the trucks could be in the middle of nowhere or surrounded by power lines I'm lucky to get 4k/s on some days. A normal sync was taking about 3 and a half minutes and I was asked to optimize it. We are moving to SQL Mobile in Q2 2006 but that doesn't help us today so I sat down to see what I could do to make this faster.
Long story short, I have it to just under one minute now. They are pleased.
Most of the Performance talk you see going around deals with things like "avoid boxing", "use StringBuilder", "stored procedure", etc. Those are the last things I look at, the first things are improving the efficiency of data stores and Asynchronous processing.
Improving the efficiency of your database is a huge topic and one that I am only now becoming more familiar with. My current sync process is largely based on DateLastChanged fields on various tables, and some of those are large for SQL CE, say 20,000 rows. Adding an Index on the DateLastChanged fields of some large tables changed the operation from Table Scan to Bookmark seek, and took many seconds off of my times. There is no execution plan viewer on SQL CE 2 to prove that statement, but the proof is in the performance.
The larger part of my 300% improvement was "Processing Item A while waiting on Item B to be done". I am going to write three articles as I get time about some very simple Practices I use to keep me honest and reap the benefits of Thinking Asynchronously. My 3-part intro to threading will consist of: