1. Reduce dependencies
   Isolate them if you can't remove them.
2. Log Don't Explode
3. Don't Handle Errors if You Don't Know How
   Let them raise so they can get identified and fixed

Or ORM's Are Not the Whole Answer
OR Some ORM's Are A Better Answer

One point I feel I need to make.  So many people seem to think that at some point an ORM will appear that will just magically work.  It will map all your entities to objects and "just work."

So an endless parade of ORM's continue to lurch forth from the hinterlands. 

I'm amazed that people like Bruce Eckel think that an ORM powerful enough will just solve the problem.

The thinking is usually something like "we can shield the user from that ugly, mean SQL with some nice procedural and markup tricks." 

I've come to believe that anytime you try to hide/wrap/abstract a powerful technology/language (SQL, JavaScript, HTML, etc) you end up with at best a leaky abstraction and at worst a complete mess.

SQL is concise and powerful.  It doesn't need to be hidden.  It's not that hard to understand.  It's at least easier to understand than most of the ORM frameworks.  It's quite ridiculous what kind of code acrobatics you must perform to add parameters to a query. 

One of the biggest problems with an ORM is predicting what kind of code they will generate/execute.  The more considerate frameworks include a logging function of some sort.  This is nice but when I go to look at some code and I find a bunch of ORM garbage code, I find myself wishing for the SQL.

I used to hate embedded SQL, then I realized there really is nothing wrong with it, and it fact it's preferable as I found out what a web page is doing all in one file, without analyzing a (now unsupported) ORM or trying to find a stored procedure in the database.

The stored procedure approach is not that bad, I just find myself shying away from them as code re-use is a myth.  Yes, now I've become a true OOP Heretic.  Code re-use, especially between projects, is a rarity.  And this is the major problem with ORM's, they tend to emphasize the one object for every situation paradigm.

If I have an employee table, naturally I will have an employee object, if I'm an OOP ORM Myrmidon.  So I try to use this Employee object for my employee edit screen, my employee absence report, my employee salary report, my company org chart, etc, etc.  However, I soon find my nice clean ERD and UML diagrams become corrupted by the business needs.  We have to join in extra tables.  Pretty soon we are writing all manner of custom logic in our programming language of choice.  We have to use a little somthing like join_to(tables.employee_benefits).get_by(table.columns.user_name='a%').limit(10).sort_by(table.columns.user_name). 

We are wasting resources by bringing back unneeded columns.  Because the One True Employee contains fields for every situation, we populate the whole object for every request.  We can resort to lazy loading and other performance optimizations for expensive entities. 

The Better Solution

The answer is that there is no answer.  However, I think the frameworks like CoolStorage.NET and the db library in Web.Py have a nice ability to load data from queries into objects.  This takes away about 80 percent of the pain of Database/Object interaction and let's us use queries specific to the Use Case at hand.

I think the best we can do is to use the best tool for the job, in the case of talking to databases that tool is SQL.  We should use SQL and take advantage of automatic mapping. 

Learning SQL one time is much better than learning a new ORM every year.  I feel the same for HTML, JavaScript, CSS and the rest of fundamental technologies of our trade. 

Talk about a word until it makes no sense.  Why would you do this?  This is much of what philosophy seems to consist of, attempting to reach the absolute definition of a word with zero ambiguity.  While a neat intellectual exercise, there exists a threshold where this ceases to be useful for most humans.  Typically most words can be defined in a short phrase.  This suffices to give us a mental picture of what a word is meant to convey.  In fact passing this mental threshold might be somewhat harmful.  There be dragons there. 

We can draw a parallel to programming or to program design.  Consider a database schema or a UML diagram of a painstakingly designed class hierarchy.  Each quantum of information neatly defined in terms of other types, just as a word¹ is defined by other words.  Each word defined to the point of lowest ambiguity.  At least this is the intention of the designer if one follows the typical methodologies for such things.

However, is there a need to leave ambiguity in the system?  Can we equate ambiguity to abstraction?  On some level abstraction and ambiguity are linked. 

Consider the need the most organisms have for ambiguity in their perception of their environment.  Humans do night see all spectrums of light.  We do not hear all frequencies of sound.  The human eye cannot distinguish every possible shade of color.  If you say a restroom is for woman, your brain can picture the type of person that may enter that room.  The general class of woman is sufficient for the needs of blocking males from entering the females restroom.  Our brains have the ability to remove unneeded information so that we can concentrate on what is important.  

This is the crux of the issue.  This is abstraction.  Instead of dealing with the billions of women on the planet, we can deal with the one class of women.  We abstract away the complexity and gain ambiguity.  If we say that all women have long hair, we over specify the general case of women and lose the power of abstraction. 

What information is important in our system and how much do we really have to specify it.  Is it enough that we know that something is a series of characters forming a string?  Is a person’s name a series of fields like prefix, first name, middle name, last name, and another series of suffixes?  Or is it simply one string of characters?  

Let’s examine the first case.  In a large organization there is often a design or architecture team.  This team is typically obligated to define an organizations data.  The team most likely will tend to choose the multi field very explicit representation of a person’s name.  After all, what is there job but to minimize ambiguity and maximize specificity? 

In the second case a lone hacker with a deadline may decide that one long database field is enough to store the name.  This is most expedient and it gets the job done. 

Why does the lone hacker’s system survive longer and cause fewer problems?  Consider the first case again.  Most likely we will have to define a suffix table where every known suffix (Jr, Sr, MD, CPA) is defined.  After all a user must now choose this from a dropdown.  Also the prefix must be entered.  We will need another table to define these.  What happens with a person who has two middle names?  Do we define another column?

It appears that even in the simplest data point definition we have run head first into mushrooming complexity.  How long do you think the design meetings will take JUST to determine how to define a name?  I’ve been in on this and it can take a surprisingly long.

The lone hacker has written most of his system by the time the committee has determine the proper representation of each data point in the system. 

The fact is that ambiguity or rather the ability to handle ambiguity is a feature of a system not a flaw.  Many times the ability to handle ambiguous data from a user or external system saves a monumental amount of redesign and rework.  An example would be a company that decides to open a foreign branch and now must handle names in a different format.  Do we need to redesign our system or does it just work?

A caveat here is the interfacing to other (broken) systems that require the data to be split into small pieces and fed in.  A partners system requires first and last names.  Is this a problem?

It can be.  One solution is to parse the name field.  Ninety percent of the names will probably be first and last name only.  This is because most users are lazy and only want to enter the minimum amount of data.  Some self absorbed types are going to enter every credential in the list including their Boy Scout merit badges.  Now we may have to kick back this data to a human to look and correct.  Last I checked data entry was cheap, programming was expensive.  A great solution?  Maybe, maybe not.  How much money did your design team burn by arguing over what should be in the prefix and suffix table? 

In the age of search, ambiguity becomes more useful.  If we can search on any field in the database than I no longer have to use last name to organize a list of people.  Searching for them may work much better.  Or it may not and you find that you want a separate first and last name field.  

Of course if your requirements call for separate fields because they must be fed to another legacy system, then by all means specify two fields.  You have a valid reason to do so this increases the coherency of the system.   My point is about over specification in the absence of requirements.  

In the absence of requirements, don’t over specify.  Creating unneeded complexity in a system does not benefit anyone.  Ambiguity, as a facet of abstraction, provides the designer a way to deal with complexity or at least to mitigate it.

 

1-In fact languages like Forth or Factor call the smallest unit of execution a “word.”  The word in such a language is equivalent to a function in the more “mainstream” languages.

One argument script pimps use frequently is that it doesn’t matter what tools a contractor uses to build a house as long as the finished product is the same.

The problem with this argument is that the premise and the analogy are flawed.  Comparing the language to the tools is not quite accurate.  The language is more aptly compared to the building materials used instead of the tools.  The tools would instead include things like the IDE.  With the analogy corrected we now see that the choice of building material is very important.  Using 2x4's instead 2x6's will have a big impact on insulation properties (thicker wall cavities) and cost. 

Is this important?  Absolutely!  

Unfortunately, too many scripting language fans (script pimps) make the same flawed analogy in order to justify the use of their pet language instead of a higher performance language like C++ (or C#, Java, etc).  Scripting languages like Ruby are slower then compiled languages.  Also their nature precludes them from having good IDE support.  This is because much of the meaning (context) can’t be derived until runtime so things like IntelliSense become difficult if not impossible to implement. 

Following the scripter’s analysis we are supposed to assume that scripting languages allow faster construction which I would say is another falsehood.  Yes, some contractors might be able to cut wood just as fast with a handsaw but most want to use as much automation as possible.  Hence, table saws, compound miter saws with laser guides, etc.  These automation tools are closer to Visual Studio than VIM. 

In my experience with Ruby (among other languages) I found that the resulting programs were usually slower, which a scripting language will be by its nature.  I also found that the emperor wears no clothes!  Shockingly, developing in a scripting language is slower because of the lack of automation tools.  Example, try to find all references to variable in a Ruby on Rails project.  Now try the same thing in a C# project using Visual Studio.  In Visual Studio it’s a right click and “Find All References”.  In VIM you could search for the references but you don’t have any support for context.  You don’t know that is the only reference to that variable.  You are using a handsaw instead of the best tools available.

I'm sorry but reality demands that 1 = 1 and slower is slower and speed is important.

I think when Microsoft introduced .NET and the declared that you where free to write in any language most people focused on bringing old crusty languages like COBOL to the modern .NET framework.  I'm not sure how successful this has been.  I'm not being critical, I simply do not know because I use C#, arguably the most mainstream of the .NET language families.  For the past few years I have flirted with languages like Ruby and Lisp and found some of the features of these languages to be intriguing.  C# is fairly powerful and is still being extended by Hejlsberg and company and is even starting to stretch into some Lispy areas with features like LINQ.  What I miss most in almost every language is the Lisp style macro.  These things are powerful, if you don't know what I'm talking about you need to check out Peter Seibel's Practical Common Lisp book.  Once you implement the MP3 database in Chapter 3 you should have a good idea what macros can do for you and your coding, especially if you write a lot of repetitive code (hint: database access, DAL, etc). 

The problem is that even thought I enjoy Ruby and Lisp the implementations for these languages are hit and miss.  The open-source implementations all feel like second class citizens on Windows XP.  Using Emacs as a text editor for Lisp when you are used to Visual Studio 2005 is a little disappointing.  Using GVIM to edit Ruby on Rails files leaves me with a similar sense of dissatisfaction.  These are great languages without great tools and with hit and miss implementation issues.  I have to say however that I am fairly comfortable with some of the commercial Lisp Implementations with LispWorks being the best all around when you consider licensing. 

Back to my point, I did not set out to author a diatribe on Lisp and Ruby, and I will leave that to the Flamer and the Flaming Flamed.  My intent is to write about two rather new languages that give me a lot of hope for the direction and potential of CLR languages: Boo and Nermerle.  Both of these languages support cool syntactic macros and add their own little twist to CLR programming.  Boo is about agility and keeping the code lean and clean and uses macros to support this concept.  Nemerle is a hybrid functional language very similar to C# but with the cool functional stuff like Pattern Matching and also C# 3.0 features.  What is interesting is the C# 3.0 features are implemented in the C# compiler and does not require support from the CLR.   CAUTION GENERALIZATION AHEAD!  Lisp macros are simply code that runs at compile time instead of run time.  So we are doing things in C# that sounds like an area where macros could make a huge impact.  In fact this is how Boo and Nemerle implement many of their advanced features, they are simply using macros.  So Greenspun's 10th Rule plays out again this time inside the CLR.  In fact if we design a language with Lispy-ness in mind we know that we need very few constructs to develop a very powerful language and I think Nemerle especially takes advantage of this fact.

These languages are not designed by Microsoft.  They are independent languages targeting the CLR.  So yes even though we are coming back around to Lisp, I for one am not disappointed but rather quite pleased.  If I can get the power of Lisp on a framework like .NET then I will be a happy camper.  I'm waiting for the tools around these languages to mature a bit and at this point I'm leading more towards Nemerle because it seems to be farther along and very well documented.  Once a plug-in for Visual Studio 2005 becomes mature I will definitely spend more time playing with this Nautilus machine for the mind. 

So all I want is a great language and great tools and I will be happy.  Is that really so bad?

So I had a project where I need to have pretty good uptime (.99999) or something like that.  My first thought was to write a Window's Service to house my code.  I had built the prototype as a Console Application and it wouldn't be that hard to move the logic into a Service.  I was planning on using the customizable failure handlers in Windows (right click any service in the services manager MMC and then click on the Recovery Tab to see what I am referring to) to restart the service and possible do some other stuff.  Window's services also have built in support for writing to the event log.

However, I had already added log4net logging and used the email logging to send notifications on failures, all I really needed was the restart-ability that services provide.  Then I had a flash back to my Tech Services days when I used to supported many automated batch type applications that were managed with the Window's Task Scheduler.  I decided to rely on one not so known feature which I will describe in a moment.   Most tasks are scheduled once a day or week,  month, etc.  I scheduled my tasks to run starting now and then repeat every one minute ad infinitum (forever, basically).  Here's the magic: the Window's task schedule won't start a task that is already running!  So now if my task fails it will restart in <= a minute without any of the pains of a Window's Service (installation and configuration, etc).

One day I will find a reason to write a Window's Service but every time I think I need to, I find some way around it.

Here's a link to a presentation about some seemingly disjointed topics that blend into an architectural discussion about simplicity. 

Highlights:
We don't need middleware (There is no such thing as transport independance).
URL's are important
Small disconnected functionality can be combined with unexpted results (Google Maps + Craig's List)


http://intertwingly.net/slides/2005/rs/

ORM (Object Relation Mapping) is the new automatic memory management.  You wouldn’t want to go back to having manage pointers and allocate and de allocate memory.  This is how we advance.   The less unnecessary complexity the better.  I think most languages and platforms will eventually have this built in.  Rails provides Active Record which is based on a Martin Fowler pattern of the same name.  Java has hibernate, cocoon and about a hundred other frameworks most of which have/are being replicated into the .Net space.

So if you have a decent database design you can really take advantage of the ORM to make your life so much easier.  Of course queries can be optimized etc if you need them to be but all of this logic can be put into the Ruby (Java, .NET) language functions used to query the database.  Things like order by, limit, sorting can also be specified. 

The nice part is that the schema needs to make sense to be used.  The id field is always called “id”.  Foreign keys are always named with the tablename_id so it’s consistent.  That’s the advantage of Rails, there’s so much organization built in and you always know where pieces of code are going to be, ie models, views, controllers, libraries, etc.  

How many times have you opened someone’s project and had to hunt around trying to find what you are looking for.  

Why do we have 2 x 4’s?  Wouldn’t it be great if each carpenter could choose the dimensions of lumber to use for each project?  Why limit the choice? 

The answer is obvious or should be and the same thing goes for software development.  Microsoft has done a great disservice for developers by not mandating a coding standard which includes indentation and style along with project organization.  

This is the only way we can continue to handle more complexity; we must delegate some of it to the computer and use standards. 

As usual Joe is right on target with this article.  He explains why MS Project sucks for software projects (it's meant to manage construction of buildings not software) and why Excel is the most complicated tool you need to create a good software schedule.  I believe the SCRUM process also uses spreadsheets.  I had a colleague who was big on the SCRUM process and it seemed very promising in that it let the developer plan the work without a dog and pony show.

http://www.joelonsoftware.com/articles/fog0000000245.html

I've often wondered if you could prove that the speadsheet is one of the most valuable pieces of technology ever, perhaps behind the hand calculator, email and the word processor.

Once again, Joel is a great resource.

In this article he links to the project specs for Aardvark which is a remote help system that you can use to help your aunt get her pc working again (if she can connect to the internet).  The product itself looks nice but I'm more interested in the spec itself at this point because of the best practice gems it contains.

Example 1:  Use CSS for formatting but tables for layout because CSS layout support is so buggy in modern browsers.  This is especially true if you have to support multiple browsers.  Trying to lay this out with div's can be an extreme exercise in patience. 

There are other examples such as making the user experience as simple and easy as possible.  Don't make the user answer unnecessary questions that they don't know the answer to anyway. 

It's worth a read, especially if your looking for modern web best practices as many sites out there are still publlishing information from 1999-2000.

One of the best sites for clean, clean and yes cutting-edge CSS is Stu Nicholls' Cutting Edge CSS.

I've integrated many of his examples into my projects including 3 column layout with DIV's and a nice 3-D tabular effect for menus .

I always try to check this site to see if there any updates because most of Stu's examples make it easier to design standards compliant pages with nice functionality.

I have completed the migration of CHT to use a 3 column dynamic layout using DIV's.  I haven't found many examples of this especially with ASP.NET.  I will eventually expand this into a full tutorial.

Advantages of DIVs vs TABLEs
You can find this someplace else but I want to emphasize the reason i switched to div's was for troubleshooting and ease of maintenance not standards compliance (but that's another upside).  I find that I can read through my HTML much more quickly and the div actually provides a semantic meaning to the content versus a bunch of nested garbled tables. 

Traps
The hardest part was making this dynamic.  For instance what happens when you only want a two column layout or a one column layout? 

Implementation
The key is using a panel control (actually 3 of them or more).  The ASP.NET panel control is actually rendered as a div (in most browsers, you may have to use the browser caps hack to get the correct rendering in all browsers but this is pretty easy).  Once you are working with divs then the normal CSS rules apply.  The secret is applying styles dynamically so that the divs can be resized based on whether they contain content or not.  This is actually straight forward.

I promise I will post the sample code soon.

In reference to this previous post, which was inspired by a java patterns book, Holub on Patterns: Learning Design Patterns by Looking at Code, I was again looking for the grail. 

Holub uses a builder pattern approach to avoid exposing each private field through a getter and setter.  This approach is interesting but it quickly becomes complicated in the implementation as a series of interfaces and implementations become necessary and this leaves the casual observer wondering if the getter/setter method might be much simpler to understand.

Holub's assertion is that accessors are used mostly for visual designer created UI's.  This may be true to a large extent but the alternative seems pretty convoluted.  Holub also argues that accessors are very procedural which may also be true and that object oriented progroms become procedural at the margins where they must communicate with procedural systems like UI and database access.  However, in the OO middle the program should be objects which consist of an almost service stack like service object which provides methods in which to accomplish tasks (isn't this procedural?).

Conclusion:  I have headed the warning of not to go crazy with accessors and mutators but the alternatives don't have a good smell. 

An article on OO design and CASE tools versus CRC cards and hand drawn diagrams.

I would have to agree that many times the tool causes such a disconnect from the model that it is more trouble then it is worth.  There is some fundamental mind-body link that causes the mind to be stimulated by drawing and writing. 

This is why the CRC cards, a white board and a digital camera are essentials for the SDK.

Referring to a previous post, I have been looking at the ways a business object can be represented or accessed through various tiers.  Of the ways this may be done is using the builder pattern. 

The builder pattern leaves the the representation of an object open-ended meaning that an object can have many different representations.  However, we do not need to know all of the representation at the time of construction.  Additional representations of the object can be added later. 

I have seen this pattern used extensively in java and I am actively searching for examples in C#.  Java seems to be a very pattern rich language and often strikes me as an academic language.  More on this in another post.