Is soft-delete fail?

Posted in code, ruby on rails by mtjhax on March 1, 2010

Brakeman by artist Will Enns,
It appears that the Rails community has hit the brakes hard on soft-delete (marking database records as ‘deleted’ instead of actually deleting them). Is the entire concept of soft-delete fail? Long before the Rails community embraced soft-delete, there were discussions of its various pros and cons.

I’m a big believer in the concept of never permanently deleting data. It’s a pretty easy bandwagon to jump on–why risk losing data when you can safely tuck it away somewhere or simply mark it deleted, then update all of your database queries to ignore the “deleted” rows? As a believer, I’ve worked quite a bit with soft deletion in both web and stand-alone apps, and it has always been an ongoing maintenance headache. Everyone in the organization who touches the data ends up needing to be aware of the soft-deletion scheme to avoid causing problems–programmers, DBAs, even support staff. Even if you move deleted records into a separate table, you are still likely to end up with a system where one forgetful programmer can cause headaches for everyone.

For a while it looked like Rails to the rescue, first with technoweenie’s (Rick Olson) acts_as_paranoid which started as an experiment back in 2005, and more recently semanticart’s (Jeff Chupp) is_paranoid, which cleverly exploited the new ActiveRecord default_scope feature to improve on the concept. Over the years, there has been some criticism of acts_as_paranoid’s problems and limitations but devotees have worked around them. More recently, some plugins and gems have popped up with alternative schemes such as moving deleted records to an archive table instead of marking them as deleted. In the last few months, however, Rails community support for soft deletion seems to be having a meltdown. Jeff of semanticart publicly stated that he is killing is_paranoid and rumored that technoweenie is no longer using acts_as_paranoid. A quick search of the net finds a number of very recent articles slamming soft-delete in general (for example, see Luke Francl’s list of softdelete Bookmarks) and there are an increasing number of Rails add-ons dedicated to the concept of moving deleted records to an archive table.

Ultimately, the argument for killing is_paranoid boiled down to how making soft-delete transparent added one too many layers of Rails magic, shielding the programmer from important data considerations that should have been kept explicit. Soft-delete doesn’t get off the hook that easily though–it still presented problems even when wrapped in Rails magic. For example, you can get into trouble if you use unique indexes in your database. Imagine a row is soft-deleted, then you try to create a new row with the same attributes that were supposed to be unique. Also, as Rails has rapidly evolved new and interesting methods of associating and finding data, the soft-delete add-ons required constant updating.

Personally, I’m starting to think the best answer is to remove all consideration of soft-delete / archive-delete from your apps and handle this externally. There are quite a few tools out there for backing up data in real-time as it is created, using SQL triggers so deleted rows are automatically journaled or moved into an archive, etc. There are also some pretty compelling articles asking an important meta-question, why are we deleting data at all?

For further reading with some interesting suggestions for alternatives, I recommend Luke Francl’s post on Rail Spikes.