The Perils Of Opinionated Software (like Rails)
This is a guest opinion piece by Xavier Shay.
Ruby on Rails, by its own admission, is an opinionated framework. From day one it has positioned itself as the Kryptonite to enterprise software, providing an easy to use, rapid development framework. The infamous 15 minute blog video set a tone which has continued through to this day: if it's not quick to build and in Ruby, we're not interested.
Rails takes an extreme viewpoint, the polar opposite of the XML heavy, configuration tangled mess of traditional enterprise software. This viewpoint is necessary---without it we have no alternative---but that doesn't mean it is the best way to build applications. In their book "First, Break All The Rules", Marcus Buckingham and Curt Coffman explain that the the common method of selecting top managers by looking for the opposite of bad managers is wrong. Research showed that this strategy tended to select good managers, but not the best managers. They found that the best and worst managers actually share a lot of the same characteristics! If you want good managers, then study good managers, not bad ones. The same principle applies to software: you won't figure out how to build the best software by studying the worst.
Studying the bad is exactly the approach the Rails team has taken, though. They have studied enterprise software, and built the opposite. In the process, they have unquestionably made some breakthroughs in how to write quality software fast (that's why we all use it!), but in what ways have they thrown the baby out with the bathwater? What attributes of the worst enterprise software are also found in the best software?
Relational databases have been around for decades, and are solid pieces of infrastructure. They can do some really cool stuff, but Rails considers them no more than a necessary evil. The less the database does, the better. I appreciate the philosophy: if we can move logic from the database in to our ruby application code, that's a win, right? In practice however, this doesn't work. It physically cannot work. In addition, it's actually slower (in terms of development time) and _less_ maintainable to do certain things in ruby rather than the database!
Relational databases know how to manage relational data. That's what they do, it's in their name. Referential integrity ensures that a child row always has a valid reference to a parent row. If a child has parent_id of 1, then a parent with an id of 1 is guaranteed to exist. Referential integrity has been a solved problem for decades using foreign keys. It is a proven fundamental property of solid applications, yet Rails has no best practice for using them. Rails has tried to reinvent the wheel and do it in ruby (with the has_many :dependent option), but it's half baked and cannot cover even simple edge cases. Try this thought experiment: one of your application servers adds a child row to a parent, while another one deletes that same parent at the same time. How you can prevent the child row from being orphaned? You can't, not without using your database.
Even something as simple as not null constraints, a fundamental principle of data design, are thrown away in the extreme polarized Rails opinion. You can just use validates_presence_of! Once again nice in theory, but I have yet to see a large evolving Rails application that hadn't had at least one null in its database where it shouldn't have been (subsequently causing errors). Even if I had, it's the cheapest safety net you can buy: all it "costs" you is a :null => false declaration in your migration and you never have to think about it again. No one in your team, not even the new rookie graduate, ever has to think about it again or run the risk of accidentally bypassing validations.
I'm not faulting the Rails team here. They have taken a hard line position and stuck with it, which is important for our industry. As software professionals though, we have a very different mandate than the Rails team. They are defining and pushing the boundaries of how we build software, where as our job is to find the best compromise of existing technologies to provide business value. This sweet spot will inevitably lie somewhere between extreme opinions.
This is a guest post by Xavier Shay. He is touring his "Database Is Your Friend" training course through the US and UK in July, August and September, in which he teaches you how to find the sweet spot of database usage for building rock solid Rails applications without compromising your velocity. The tour is more than likely coming to your city, so check www.dbisyourfriend.com for more details.
July 6th, 2010 at 2:41 am
If it were 2006 maybe this article would be relevant. Rails took a bold first step against heavily customizable enterprisey software that were great for vendors and their certfied herd. But since then, if you need something done fast with minimum quality we can follow Rails opinionated stack, otherwise it is not that difficult to override it (though annoying at times, I agree). And they learned their lesson and evolved forward with Rails 3 being ORM agnostic and friendly to other libraries such as DataMapper. We could also argue that not everybody feels that this is bad thing otherwise someone would have stepped in to implement what is missing (the whole point of it being open source). And with so many projects going the NoSql path I think this is non-issue and, therefore, unimportant. I know this is a boring argument, but there is this notion that "someone" owes "something" to me just because I miss it: this is the kind of thing that would make sense if I were paying that "someone" - as is the case with Microsoft, IBM, Oracle, but makes no sense in a voluntary based open source field.
July 6th, 2010 at 2:42 am
This is unfair. You've taken a couple of Rails bad points and took them as representing the whole framework.
Rails does NOT consider the database a necessary evil, it embraces it. Compare Rails (ActiveRecord, actually) to other ORM frameworks: AR actually considers the database as the authoritative source of information and doesn't try to shield the programmer from SQL or anything else — now try JPA and you'll a what a framework that's all about making the database disappear looks like.
Model validations aren't supposed to replace database constraints, they should be used in addition to them — they're just a way to communicate your constraints in a more performant and user friendly way. It's just cheaper to validate (and report) simpler constraints in ruby code than require a roundtrip to the database.
This is Rails being pragmatic and offering you the faster way to build a quality, user friendly application. None of this means you should forgo relational constraints or any other feature your database offers to ensure your data integrity.
July 6th, 2010 at 3:33 am
Xavier
I've been using Rails quietly and passionately since 2005 in govt enterprise, but always with database constraints to ensure valid data. Works well for me. So I agree with you on the importance of such constraints. Your point about the importance of these constraints is relevant to anyone using an RDBMS.
I'm not with your on Rails is the "opposite" of enterprisey bad. Java enterprise developers have been rejecting database constraints and rewriting their own buggy, flawed versions of them since 1995. Part of the write once, run anywhere (on any database) Java meme that is has addled application design in the last 15 years or so.
In fact most application programmers in the post-1995 era regard databases as uncool and old-fashioned - a mere data dump. It's a costly view widely shared by Java and Rails programmers alike.
Rails is great because it lets me put in my constraints anyway, along with all the other goodies.
July 6th, 2010 at 4:10 am
This "opinion" piece is utter crap. In addition to that the previous two commenters have said, I have this to add:
1. Has the author never used the :references keyword in a migration, which would actually set up the foreign key associations he longs for?
2. What about if I want to use MongoDB or Redis or Riak? How does this apply to that? (Hint: it doesn't)
3. Why is there a difference between a "Rails team" and "software professionals"? Are Railsists inherently not professional software developers? That's news to me, since I've been making money using Rails for almost a year now.
This is a very, very poor blog post, undeserving of Rails inside.
July 6th, 2010 at 5:29 am
This post is an insight, not a criticism.
Akita, I don't believe Rails should do anything different or owes us anything. This is a reminder that we shouldn't follow opinionated software blindly (which my reading of your comment suggests you agree with). This opinion is still relevant because there are still a large many rails applications that don't do things like use foreign keys. (People who do use foreign keys never believe this.)
Marcus, this post isn't a criticism of Rails. I use the same type of language Rails uses to describe itself. I use Rails, I love Rails, but we need to know when to compromise and stray from The Rails Way. This opinion could apply to *any* framework.
Greg, Rails being the opposite of enterprise isn't my opinion, it's my impression of how Rails sells itself (perhaps more so in the early days).
Srdjan,
1. I just ran up a migration with rails 3 using :references, it didn't create a foreign key nor an index. What version are you using?
2. Not sure how this is relevant. Like any technology, NoSQL is great for some problems, less so for others.
3. From my last paragraph "[rails team] are defining and pushing the boundaries of how we build software, where as [software professionals] job is to find the best compromise of existing technologies to provide business value." I don't believe this to be a controversial statement, nor a criticism of either camp.
July 6th, 2010 at 6:15 am
@Srdjan - some respect would be appropriate especially when your first point is wrong. The Rails API docs indicate that references simply creates an integer column with '_id' suffixed and does not create constraints.
I have always used foreign key constraints in my Rails apps, via the RedHillOnRails Core plugin. But I am beginning to see less utility to them as I don't believe the way we use them is particularly effective. With ORMs now they simply guarantee that a particular integer is present in another table, this doesn't appear to have much value to me. The only real use I see is the one Xavier mentioned, when deleting records they will maintain data integrity.
July 6th, 2010 at 6:22 am
There's plenty of support for db contraints in the Rails ecosystem and getting a rails app to do the exact contraints that he describes, in the manner that he describes, through the db, is... trivial. Conflating, the core team's exclusion of constraint support in Rails as doing something the opposite way for the sake of being contrarian is inaccurate. They didn't study the bad and then do the opposite as a way of making Rails good. An important point that the author has left out is that there are plenty of production rails apps that run without db contraints and don't run into any problems with data integrity despite the perceived shortcomings he described (yes, they run just fine without foreign keys!). The sweet spot lies in however way you want to roll and the flexibility of Ruby gives you the power of redefining your tools into the form of your liking. One of the things an article like this doesn't cover is how many of us in the ruby world disagree with one another about how to do things and how we've solved these disagreements by just reshaping the framework through plugins, gems, and some ruby metamagic. If you want foreign keys, then go for it, don't let the core team tell you what to do. What do they know?
July 6th, 2010 at 6:37 am
Interesting viewpoint.
Remember that Rails started with MySQL and, to a lesser degree, SQLite as their databases. Neither MySQL nor SQLite have an imperative programming language built into their implementations of SQL (as opposed to PostgreSQL, Oracle, IBM DB2, and MS-SQL Server). As a result, one needed software in the Ruby/ActiveRecord layer to enforce data constraints that could be done with stored procedures in other PL/xxx languages with PostgreSQL, Oracle, etc.
I have observed Rails for a number of years, and it looks like initially, the database was just a way for Rails to save Ruby objects in an enterprise-like way. I used to smile with every point release of Rails because they would end up putting in more and more SQL-like features. Rails 3 almost takes the cake with their AREL (Active Record Relational??) objects.
If this march of using SQL databases more and more sensibly continues, I think that a future release of Rails will include the ability to put foreign key constraints and other check constraints straight into the database migrations so that these constraints don't have to be enforced at the Ruby/ActiveRecord layer via the validates_XXX functions.
The persuasiveness of this article is weak. It starts with some general thoughts of discerning between good and evil, and then picks a part of Rails that started off relatively weak. After picking it apart the author tries to infer that therefore the Rails team is not comprised of software professionals who are motivated to deliver business value. This flies in the face of the fact that a lot of contributors to the Rails code base are also software developers at successful and growing software businesses that use Rails as a core technology.
Although I completely agree with the author that Rails has done a poor job of using relational databases effectively, I think that the fact that the Rails team has implemented more and more SQL-like functionality in ActiveRecord is a mark of a team trying to deliver more business value by using the relational databases more effectively.
Rails delivered business value and productivity and high quality software by using "advanced" features of the Ruby language to implement the proven Model-View-Controller architecture, and proven methods for generating views (e.g. ERB or HAML which are very similar to JSP, ASP and PHP). More value was delivered by aggressively factoring out common code into libraries or code generators. As a case in point, you don't see too many Rails developers switching back to Java or .Net because they felt it made them more productive.
If Rails has a major fault, it is that they have not come up with a Rails-y way to build Rails applications based on an existing database schema and an existing mountain of data...
July 6th, 2010 at 7:58 am
These Object-relational impedance mismatch issues are not limited to Rails; any ORM will have the same problem. I don't think there will ever be a good solution to this problem unless we either, move to a different database type or programming paradigm.
July 6th, 2010 at 8:22 am
I have been wondering about this for a while: How does AR respond to a 'true' relational database, with foreign key constraints et al? Does this lead to conflicts for example, when deleting records that have a :dependent => :destroy set in the model? Are there any Ruby ORM's that are meant to run on top of these databases?
July 6th, 2010 at 9:59 am
@Srdjan: actually 'references' doesn't create a foreign key, it just creates an integer column. See the source.
July 6th, 2010 at 4:40 pm
I think these ideas lead directly to a question I've had recently: "If Rails has always treated the database as a hash in the sky, then why are relational databases still the default in Rails?"
NoSQL databases seem perfect for the way Rails wants to work with data, but instead we're in a no man's land between solid database design and pure persistable objects.
The answer probably has to do with the immaturity of most NoSQL databases, making it hard to pick one or properly build an API that utilizes the strengths of each.
Another point that was neglected in this otherwise insightful article is the reason that Rails was advertised as being the opposite of enterprise software. I think it was not a technical argument but an emotional one for the purposes of marketing.
If Rails was truly anti-enterprise on a technical level, would it have taken its most significant idea from a book called "Principles of Enterprise Architecture"? http://martinfowler.com/eaaCatalog/activeRecord.html
So it was not the case that they "studied enterprise software, and built the opposite", but rather that they implemented a few salient enterprise concepts and then (successfully) advertised Rails as being the opposite of enterprise based on the few concepts that were not implemented.
As you noted, XML was a prime scapegoat of early Rails evangelism, yet XML was (and still is) the default file format for APIs produced by Rails and is the default for the now neglected ActiveResource.
In summary, trying to understand Rails on a purely technical level will inevitably be confusing. A full explanation must also consider the ideological and promotional goals of the framework (for further examples, see the Merb merger).
July 6th, 2010 at 4:51 pm
To all who've pointed out the :references usage, I do apologise, you're right. I did use the RedHillOnRails plugin to enforce the FK constraint in a Postgres database and have had no problems writing a pretty big app with it in place for every migration.
@Xavier, as to my second issue, the point is that you implied that Rails should have stronger ties to a relational database. This is what I took issue with. There are plugins available to provide those ties, should they be needed/wanted, but that functionality shouldn't be in the core itself. That's the relevance of it.
As for the third point, "software professionals" as you define them should also push the boundaries of their frameworks/languages/domains. This is why distinguishing between the two is, in my mind, slightly offensive. No programmer or software engineer should aim for the middle and be content with what they have and what they use to solve problems. That's one of the major problems in our industry, this unwillingness to learn new technologies and to push those new technologies into corporations that one might work in.
July 6th, 2010 at 9:37 pm
Tim, "getting a rails app to do the exact contraints that he describes, in the manner that he describes, through the db, is... trivial." but you need to know all about them before Rails, because Rails doesn't tell you about them. This is my point that if you follow the Rails Way, you won't have data integrity, which is probably (but not necessarily) not what you want. There are very few use cases where, all things considered and considering the "cost" of adding a constraint, you should have a null column in your database. Null is the default not because it's a good idea, but because it is consistent with the Rails Way.
I also believe many of the apps running without foreign keys are driving without seat belts, and are just lucky not to have crashed yet (the crash hopefully is only wasted developer time). When data goes missing in production, will you feel comfortable justifying your decision not to implement basic data integrity controls?
Martijn, :dependent => :destroy works best with ON DELETE restrict. It's weird conceptually, but forces all cascading destroys to go through AR.
Geoff, great point, thanks.
Srdjan, ah I see what happened there. My intention was to highlight the distinction between what should be the motivating factors between opinionated framework developers, and people solving problems with that framework. Both motivations are required, and indeed the same person can switch between the two.
July 6th, 2010 at 11:25 pm
@Xavier, now we're getting somewhere.
"This is my point that if you follow the Rails Way, you won't have data integrity, which is probably (but not necessarily) not what you want."
I'm starting to understand why you wrote this article. I've long held an opinion that to use Rails effectively, one should know all facets of web development, including understanding how the database works and how to use said database effectively. This includes understanding how constraints and foreign keys work.
Using Rails without all this knowledge approaches cargo culting. It has spawned an entire industry where knowledgeable, competent web devs fix hopelessly broken Rails projects.
July 6th, 2010 at 11:36 pm
I take an issue with ActiveRecord being called Rails as well. It's in Rails, but it's not Rails. Nowhere in the piece any other aspect of Rails is even touched. Perhaps non-configuration, yes, but given no actual issues with anything else other than the default ORM are pointed out, I'm assuming that's what this is about.
That may inspire confusion if you don't know better. Just saying.
July 8th, 2010 at 6:13 am
Srdjan, exactly, that's what I was trying to get across. Glad we ended up on the same page :)
Julio, ActiveRecord may as well be Rails. 90% of projects use it, new developers will certainly use it. I recognise that Rails 3 in particular has made it much easier to sub in other components, but the defaults still represent the "Rails Way". There's other non-AR issues too (sending emails inside transactions for instance - can be convenient, but not necessarily a great idea).
July 12th, 2010 at 2:48 pm
@Xavier, thanks for posting this. You bring up some interesting points and at the very least, it makes for great discussion, as we've seen.
July 13th, 2010 at 4:46 pm
When I first discovered Rails about the same time last year, I thought it was pretty cool. By now I think it is way to conservative in sticking to the same default framework components as the first Rails version, namely Erubis for views, test-unit for testing and Active Record with an SQL database. By now, this is very "old school" and looks similar to a typical php stack.
Personally I think it´s time to evolve Rails in a new direction, similar to what Merb did, but this time focusing on creating a new default stack which might be somethin like:
ExtJS or Sproutcore for the View (with full Ajax/Web socket and HTML 5 support integrated). Datamapper with perhaps Riak/Mongo Document DB and Redis as key-value store, then let each "data segment" be persisted in whatever store is most suitable instead of picking only one to suit all. RSpec 2 and Cucumber for BDD and testing.
The problem with this approach would be, that the learning curve might be pretty steep? On the other hand I think these frameworks come with some nice DSLs that ae more intuitvie and easier to understand than using the Rails default stack. With the right abstraction layers on top and better use of generators, I think it would be feasible without making it too difficult. Rails should be seen as an infrastructure with parts you can pull out and replace to create your own super framework IMO.
Thoughts?
Kristian