Or ORM's Are Not the Whole Answer
OR Some ORM's Are A Better Answer

One point I feel I need to make.  So many people seem to think that at some point an ORM will appear that will just magically work.  It will map all your entities to objects and "just work."

So an endless parade of ORM's continue to lurch forth from the hinterlands. 

I'm amazed that people like Bruce Eckel think that an ORM powerful enough will just solve the problem.

The thinking is usually something like "we can shield the user from that ugly, mean SQL with some nice procedural and markup tricks." 

I've come to believe that anytime you try to hide/wrap/abstract a powerful technology/language (SQL, JavaScript, HTML, etc) you end up with at best a leaky abstraction and at worst a complete mess.

SQL is concise and powerful.  It doesn't need to be hidden.  It's not that hard to understand.  It's at least easier to understand than most of the ORM frameworks.  It's quite ridiculous what kind of code acrobatics you must perform to add parameters to a query. 

One of the biggest problems with an ORM is predicting what kind of code they will generate/execute.  The more considerate frameworks include a logging function of some sort.  This is nice but when I go to look at some code and I find a bunch of ORM garbage code, I find myself wishing for the SQL.

I used to hate embedded SQL, then I realized there really is nothing wrong with it, and it fact it's preferable as I found out what a web page is doing all in one file, without analyzing a (now unsupported) ORM or trying to find a stored procedure in the database.

The stored procedure approach is not that bad, I just find myself shying away from them as code re-use is a myth.  Yes, now I've become a true OOP Heretic.  Code re-use, especially between projects, is a rarity.  And this is the major problem with ORM's, they tend to emphasize the one object for every situation paradigm.

If I have an employee table, naturally I will have an employee object, if I'm an OOP ORM Myrmidon.  So I try to use this Employee object for my employee edit screen, my employee absence report, my employee salary report, my company org chart, etc, etc.  However, I soon find my nice clean ERD and UML diagrams become corrupted by the business needs.  We have to join in extra tables.  Pretty soon we are writing all manner of custom logic in our programming language of choice.  We have to use a little somthing like join_to(tables.employee_benefits).get_by(table.columns.user_name='a%').limit(10).sort_by(table.columns.user_name). 

We are wasting resources by bringing back unneeded columns.  Because the One True Employee contains fields for every situation, we populate the whole object for every request.  We can resort to lazy loading and other performance optimizations for expensive entities. 

The Better Solution

The answer is that there is no answer.  However, I think the frameworks like CoolStorage.NET and the db library in Web.Py have a nice ability to load data from queries into objects.  This takes away about 80 percent of the pain of Database/Object interaction and let's us use queries specific to the Use Case at hand.

I think the best we can do is to use the best tool for the job, in the case of talking to databases that tool is SQL.  We should use SQL and take advantage of automatic mapping. 

Learning SQL one time is much better than learning a new ORM every year.  I feel the same for HTML, JavaScript, CSS and the rest of fundamental technologies of our trade. 

http://www.aspfaqs.com/webtech/042606-1.shtml

One of the best (not to mention easiest) paging methods I've seen.  In fact it requires no temp tables or table variables. 

Paging is something that is so common yet seems to hard to get right and keep performant that it pays to have a nice easy template to start from.  I've posted this mainly for my own benefit so I have a permanant bookmark to this code.

The example uses a stored procedure, but its important to understand that a stored procedure is not required to make this work.

Here's the code:

CREATE PROCEDURE [dbo].[usp_PageResults_NAI]

(
@startRowIndex int,
@maximumRows int
)
AS

DECLARE @first_id int, @startRow int
    
-- A check can be added to make sure @startRowIndex isn't > count(1)
-- from employees before doing any actual work unless it is guaranteed
-- the caller won't do that

-- Get the first employeeID for our page of records
SET ROWCOUNT @startRowIndex
SELECT @first_id = employeeID FROM employees ORDER BY employeeid

-- Now, set the row count to MaximumRows and get
-- all records >= @first_id
SET ROWCOUNT @maximumRows

SELECT e.*, d.name as DepartmentName
FROM employees e
INNER JOIN Departments D ON
e.DepartmentID = d.DepartmentID
WHERE employeeid >= @first_id
ORDER BY e.EmployeeID

SET ROWCOUNT 0

GO

In my previous post about what people use computers for, I stated that computers are for saving state.  Duh.  I felt I needed to make that clear to myself as I explored functional programming languages like Erlang, F# and Scheme.  State seems to be a dirty word relegated to second class citizenry in a functional world.

I think that functional language have an uphill battle for mainstream acceptance.  A bit bolder, I think they are a small niche and always will be a small niche.  In fairness, I think most of literature on functional languages portray them as a special purpose languages.  As the saying goes, the right tool for the job.  Microsoft is looking for F# developers to write driver validation software. 

My goal is to find languages that are applicable to general purpose problems.  I'm not a mathematician and I don't need so solve theorems.  Most of the programming jobs I've completed involved storing and manipulating some shared stated, a database.

Yet another realization courtesy of a man seeking to explain the failure of the best language on earth. 

Local state in programming is just like scratch paper for mathematics.  Some rare gifted individuals don't need it and can do the calculus in their head.  For the rest of us scratch paper and number two pencils are the only alternative.

While debugging, I can inspect the state of the system, either with the debugging facilities or with well placed print statements. 

So local state helps us understand our systems.  Global state, again databases, are the reason our systems exist.  Some in the Erlang camp want to wish this away.  However, I think Erlang advocates understand that Erlang was built for telecom switches.  Erlang's usefulness in general purpose programming is yet to be determined.  If you're building a system that manipulates global shared state (most of the web applications in existence), choosing a language that waves its hands at shared state in the name of concurrency might not be your best choice.

Now we come to the revelation stage of our journey.  What can we do in mainstream languages to deal with global  shared state (databases) effectively?  I've found something in my own development work that is effective for me.  It's not a new language, but in fact, works in many languages, including C# and Python.

My solution may not be a silver bullet, but then I might not be hunting werewolves.  I can't claim to have invented it.  For from it.  All I can claim is that I've seen this pattern work across languages.  I can also tell you that it might be a wooden stake, useful if you are hunting vampires.

 

For a (brief) while, I was learning F#.  It proved an interesting excursion.  If you into mathematical correctness F# and OCaml maybe just the languages for you.  Myself?  Well I'm looking for something with a little more to offer in the functionality department.  I want a language where I can do most anything.  I'm willing to sacrifice mathematical niceties.

Python seems to be that language.  Python is one of the languages I knew I would love when I tried it, so I've held off diving into it until I had gone through almost every other one.  I've journeyed through Lisp, Scheme, C++, C#, Java, Ruby, F#/OCaml and dabbled in a few less known like Factor and Cat. 

Python Advantages:

• Available and almost every platform (every platform I'm interested in)
• Terse clear syntax
• Whitespace (yes, I do enjoy the whitespace!)
• Easy to learn, easy to teach (I taught C++ to undergrads during my time in Hell)
• More complete than Ruby
• Better documented than Ruby
• Good riddance to the Edit/Compile/Wait/Debug loop (hello Edit/Test, Edit/Test, Edit/Test...)
• Python is available on the CLR as Iron Python and on Java as Jython.

Disadvantages

• It's not C# (heh)
• It's not Java (heh, heh)
• Performance (slower than the above)

I believe that Gabriel is correct and that worse is better.  Worse as in Python is not Lisp.  Even Dr Norvig, chief brain in a jar at Google, thinks Python is good for something.

Note that Python is one of the four languages in use at Google.  Google has, in fact, hired the Frost Giant.

Python is used in game scripting.  My favorite game series of all time uses Python as the scripting engine.

I've also found one of the best Web Frameworks to date to use with Python.  I'll detail that it a future post. 

Right now Project X is moving forward on rockets fueled by Python.  This is the most complete proof of concept I've completed in an alternate language (not C#) and I'm LOVING it.

 

This is an interesting article from the 1920's written by a proprietor of a grocery supply business.  What the author considers brilliant may not fit the definition that many of us have.  I think a better term would be "schemer" or "dreamer" or something similar.  Anyway my main interest in the article is based on my assertion that many in the software profession are "brilliant." 

I myself find it hard to finish projects but easy to start them.  If you believe in personality tests than I am an INTP.  This means I have a tendency to live in my head.  I need to constantly fight to finish things and to stay interested in projects.  I'm always exploring new languages and technologies.  This is both a boon and a burden in the technology field.  I'm usually up to date on the latest technology but I have to push myself to finish projects that become a little too mundane.  I also read many articles from programming.reddit.com and similar sources.  If I'm not careful I can consume a whole night or day reading such articles.  The plus side is that I'm a sponge for information.  The negative is that I may have a hard time applying it as I'm always trying something new.

I believe that we must understand ourselves to be successful.  I am aware of my weaknesses and by my awareness I am able to realize what I must work on to be successful.  I must force myself to complete current projects and to apply my research.  When I am able to do this I can produce results that are amazing.  However, I always have the potential to waste days and nights reading.  However, it may be debatable whether reading is really ever a waste.  I watch very little television as I prefer to read and program instead.

In my opinion, this article can be summed up as "find hard workers."  Not brilliant by any means but a good read none the less.

Article:  Why I Never Hire Brilliant Men

Nullables always felt not quite done.

Here is a technique to add an extra minute to the microwave timer and get a more fully compatible conversion function:

http://blog.pumacode.org/2006/05/18/using-convert-changetype-on-nullable-types/

Nullable types are awesome for dealing with databases.  Its a fact of life that most databases are going to use null and you need to deal with it in your code.  However what does null map to for a value type? 

The answer is unknown.  If C# were truly object oriented then wouldn't value types be reference types instead.  I'm guessing there are some big performance implications to this.  So we have nullable types.  A solution that is almost ready for prime time. 

Side Note:  If you truly wanted a consitent web front end for a database then null would map all the way through to things like dropdownlists where they would map to the default "please choose" option.  This would be an awesome rails like feature.  I'll have to do that when I get around to building my own web framework.

That's not likely to happen anytime soon.  I actually built server controls that did the above.  However, the overhead of maintaing server controls and making them behave just like regular built in server controls is just too much of a burden at this point.  I'll have to revisit this at some point.

The one thing that irks me about compiled code is the compilation tax.  It takes a while to re compile everything even if there is only a small change in a page or class.

Here are some tips to reduce those build times.

http://weblogs.asp.net/bradleyb/archive/2005/12/06/432441.aspx

I have to say that it certainly seems faster after I've used these tips.

One improvement I have made is to not debug using Visual Studio.  I simply start app and stop it right away.  I then just browse to my web site with another browser instance and test that way.  The development web server will just sit there running and serving up content. 

I find this way that I use logging much more than debugging.  Of course, there are some problems where debugging just can't be beat but those instances are few and far between. 

This method works really once especially if you have seperate assemblies for each page/folder.

I find that I only build the whole site if I make signifigant structural changes to the site like renaming classes. 

In the end, faster edit, test, debug cycles are what we are after.  We want the advantages of a dynamic language without the penalites. 

Michael Feathers asks if syntax is important.

I think the answer is maybe.  Certainly a language with a familiar syntax will allow you to pick it up faster if you have already learned a language with similar syntax.  For example, I found Java very easy to understand since I had a background in C#.  

Of course, there are the countless winers who complain about lisp's syntax or lack thereof. 

One tricky point is syntax that is legal in one language but not so in another.  I constantly found myself typing foreach in java just to be kicked in the head by the compiler.

I think syntax really matters for the first month or so of learning a new language.  Once you get beyond that you are able to "think" in the new language and your ideas start showing up in your new language.

Technically its more like a few hours spread over a week but so for so good.  I've come to begin to understand functional concepts.  Currying and partial application now seem to be rather easy to understand but hard to explain to someone else. 

However, what I really love is not necessarily the functional aspects, rather the terseness of the language.  I especially like type inference.  I'm not sure why we couldn't have had that sooner in C#.  In languages like C# you expect an IDE to be present so it works really well in situations where you have a toolset to back you up.  It also works really well in scripting languages (Ruby, Python, etc).  However, with F# (and OCaml) you get the benefits of strong static typing with the benefits of less typing.  I really like it.

It seems that many concepts are just simpler in F# rather than C#.  I also see cases where it is easier to define a smaller single use type (like when you query a small set of data from a database) than to declare a full blown class.  I think many times I've found it a pain to create a whole class when I really don't need to reuse it.

Again, these are just first impressions.  Things may change as I get farther into the tutorials and learning processes.  I find F#/OCaml very easy to learn.  As I was working on Scheme/Lisp it syntax always felt alien.  I was learning a lot but it didn't fell quite right.  F# just seems to feel good.  Probably because it is close to C# in some ways.

I really need to get into a small project with it to see if I really like it.  Once I have a real world application under my belt I can usually tell if I'm going to keep a language in my toolbelt or put it back on the shelf for another time.

I also think I have a solution for the ASP.NET development.  I'm going to use ASP.NET without ASP.NET...  (I'll explain more in the future, but if you read some of my previous posts, you can probably figure out what I'm going to use.)