Security and Objects

Mobile code is one of the great challenges for software security. Lets say  you are writing an email application. The idea that people could send little apps to each other in email messages might seem like a potentially interesting feature: users could build polls, schedule meetings, play games, share interactive documents. Kind of cool.

And if the platform you are building upon supports reflectively evaluating code, it could be as easy as something like this (in OO pseudocode):

    define load_message(message)
        ...
        eval(message.code)

Of course it can't be that easy. What if the code in the message does something like:

    new stdlib.io.File("/").delete()

The standard way to avoid the vulnerability is to put the code in a so-called sandbox. It sounds very secure, but in practice this usually amounts to gathering up a list of "dangerous" call sites and inserting in each some code to check if the caller has permission to proceed. So the implementation of delete would include code along the lines of:

    define delete()
        if VM.callStackContainsEvilCode?()
            raise YouShallNotPassException()

        ...

This is fraught with problems. It requires runtime support for inspecting the call stack and a system for declaring that certain code has some level of authorization while some other code has a lower level. Not to mention the busywork of going trough the code and peppering that little snippet over every suspect call site. If you miss one — say, for instance, a method that gets the addresses of all contacts on your email application — and you have a security bug on your hands.

A better way ?

Perhaps there is a better way. Take another look at the offending line: new stdlib.io.File("/").delete(). It is only able to call the dangerous delete() method because it has a reference to a file object pointing to the root of the filesystem. And it only has that reference because it could reach for the File class on a global namespace. What if there was no global namespace?

It might seem weird, but it's not that hard to imagine a programming system lacking a global namespace. Many object-oriented languages, following Smalltalk's lead, have a notion of a metaclass, an object that represents a class. Many of them (also following Smalltak) also get by without a "new" operator. Objects are created by calling a method — usually named new() — on the metaclass object.

We are very close now. The last step, unfortunately not taken by most common languages, is to avoid anchoring the metaclass object onto a global namespace. The result is that code can only create objects of the classes it holds a reference to. And it only has a reference if it is given one via a method or constructor parameter.

Proceeding recursively, we end up with a stratified program. There is an entry point that receives a reference to the entire standard library, and each call site decides how much authority to grant each callee. On our example, when we evaluate external code we can grant very little authority, meaning we can pass the evaluated code just a handful of references. Care must be taken so that none of them will direct or indirectly provide a way to create a File. In a way, object design becomes security policy.

And we get very fine-grained control over such policy. We could, for instance, grant loaded code authority to write on a designated directory just by passing it a reference to the Directory object for that directory. Our choices get even more interesting when we realize we can pass references to proxies instead of real objects in order to attenuate authority. Continuing with our example, hoping it doesn't get too contrived, we could build a proxy for the Directory that checks if callers exceed a given quota of disk space.

Research

I have mentioned above that most common languages don't fit this post's description. But there are languages that do, a prime example is E. In fact, there is a whole area of research for dealing with security in this manner, it's called "object capability security".

I'm not really a security guy, I got interested in the area due to the implications for language and system design. If you got interested, for any reason, please check out Mark Miller's work. He is the creator of the E language and the javascript-based Caja project. His thesis is very readable.

Experience report: Ruby

For a long time I've been curious about how the supposed benefits and liabilities of programming in a dynamically typed language¹ actually play out in practice. I'm now getting a chance to find it out, since I'm involved in a largish project using Ruby (lots of Rails plus some Sinatra and a couple of random daemons). My professional background has largely been in Java, but I spend a good chunk of my free time learning about different programming languages (and some PL theory), mostly on the static side of the fence. Anyway, here are some notes:

The big question is whether dynamic typing allows for more bugs to pass trough. My answer to this has to be put in context: we are a medium-sized team (between half and a full dozen of devs), all experienced in, and practicing, TDD. As with the rest of this blog post, I don't have hard data to show, but my impression is that the unit tests do indeed catch all the bugs that the Java type system would help to catch; with the caveat that it's often harder to pinpoint the source of a regression. I don't have much real experience in languages whose type systems help to enforce strong guarantees², but I would imagine they would catch a larger fraction of the bugs that are caught by the unit tests, while not really avoiding further bugs. The reason is twofold: firstly, in my experience, many of the bugs are found in the interaction of separate pieces of software (such as Javascript in the browser talking to the web server), secondly, even when the bug is located within process boundaries, it is most often related to a forgotten invariant than to breaking a established one. But that's all conjecture.

On the matter of productivity, the abstraction mechanisms offered by the Ruby language help to structure code and avoid repetition and that's certainly noticeable in comparison to Java (though, in my opinion, not in comparison to Scala, and probably also not in comparison to Smalltalk, ML, Haskell or Oz). That gain is offset to a point by the lack of refactoring tools. There are some who argue that those tools are made necessary by the verbosity of Java, and aren't needed in better languages. That's nonsense. If a language has an abstraction mechanism, that mechanism is used to define an abstraction and to use the defined abstraction elsewhere. If we then want to change the abstraction we must change code at the use sites as well, and that's where such tools can help. This is all so obvious that I find it almost silly to have to write it down.

Many of the abstraction gains in Ruby come from metaprogramming techniques. I'm not completely sold that they are necessary to achieve the level of abstraction attained, and I'm sure that they hinder readability. It's much easier to gain a footing in a library written with straighforward composition mechanisms (be it functions or classes and objects) than in a mess of Strings and calls to define_method and cousins.

While on the subject of abstraction, I have to comment on Rails. It's a mature web framework that does a masterful job of making the common cases easy. This is a much harder feat than it sounds, as we can glean from the failure of JSF, WebForms and similar unsuccessful attempts to abstract the web. The hidden cost of the bargain is when we get to the uncommon cases. It isn't so much the case that there are specific application features that are hard to code in Rails, though it happens, but that the structure of the code that Rails mandates sometimes isn't a good fit to the problem being solved.

I'm sounding a little negative, so let me balance it out by saying that I believe Ruby and Rails were as adequate choices for our project as any. The main reason is the availability of decent quality libraries and tools (refactoring and code navigation notwithstanding), specially in the often overlooked front of deployment and configuration management.

Apologies for an opinionated blog post.

1. Unittyped, for the pedantic.

2. Such as Agda or Coq, or some styles of programming in ML, Haskell or Scala.

Book review: Growing Object Oriented Software Guided by Tests

I’m a curious kind of guy. It is with some surprise, then, that I catch myself re-reading a technical book: Nat Pryce and Steve Freeman’s Growing Object Oriented Software Guided By Tests. It’s a very practical book, stuffed with code and useful advice, but it’s also more than that.

The first section of the book gives an overview of the basic principles of object-oriented design and test-driven development; not much is new but everything is clearly explained. The third and final section is a potpourri of test-centric techniques to identify problems and improve code quality. But the meat of the book is in the second section, a long walkthrough the development of a sample program. It’s a much larger example than usually found on programming books, tackling thorny issues such as asynchornous inter-process communication and end-to-end testing of GUIs. And it reads like a real software project, there are missteps, refactorings are rolled-back, some of the work is almost clerical, but then there are the great design breakthoughs, the elegant ideas that simultaneously solve several difficulties, the joy in seeing the product grow. It felt like reading a good novel. And that’s kind of what it is, a programming book that is a narrative, not an exposition. Besides making for a good read, the narrative aspect of the book is important for showing how modern object-orientation practice takes place.

OO criticisms nowadays are a dime a dozen, and though they sometimes present good points, often many of the arguments are directed at practices that aren’t so common, or at least that shouldn’t be so common. For instance, inheritance hierarchies, rampant mutable state, and patternitis (the FactoryFactory syndrome), are not indictments of object orientation, just symptoms of bad programming practice. One of the great things about GOOS is that it provides a great example of actual non-trivial object-orientation. And the same goes for test-driven-development, any abstract discussion of the benefits of TDD is bound to seem hand-wavey; this book helps to ground the understanding in real coding, done step by step in front of the reader’s eyes.

Anyway, I've read some pretty good technical books this year, but this one was the best.

Comments on Comments on the Previous post

• Henry Ware suggested a modification to the builder with abstract members removing a lot of the boilerplate. Incidentally, this is a nice illustration of how nested types can be put to a good use in Scala.
• Justin ported the code to Haskell, which was very cool.
• A couple of commenters suggested that languages with support for default parameter values (like Python and Groovy) don't need elaborate constructs such as the builder pattern. There are two ways to respond. One is to remind that the intent of the pattern, specially as originally described in the GoF book, has little to do with optional data. The other is to acknowledge that I probably put too much emphasis on this issue and forgot to mention a very common idiom for building objects in Scala: just declare mandatory "parameters" as abstract vals and optional ones as concrete vals with default values, like so:
  

  abstract class OrderOfScotch {
    val brand:String
    val mode:Preparation
    val isDouble:Boolean 
    val glass:Option[Glass] = None
  }

  
  And to instantiate:
  

  val myDose = new OrderOfScotch {val brand = "Bobby Runner"; val mode = OnTheRocks; val isDouble = false}

• I guess that's it. Thanks y'all.
  

Type-safe Builder Pattern in Scala

The Builder Pattern is an increasingly popular idiom for object creation. Traditionally, one of it's shortcomings in relation to simple constructors is that clients can try to build incomplete objects, by omitting mandatory parameters, and that error will only show up in runtime. I'll show how to make this verification statically in Scala.


So, let's say you want to order a shot of scotch. You'll need to ask for a few things: the brand of the whiskey, how it should be prepared (neat, on the rocks or with water) and if you want it doubled. Unless, of course, you are a pretentious snob, in that case you'll probably also ask for a specific kind of glass, brand and temperature of the water and who knows what else. Limiting the snobbery to the kind of glass, here is one way to represent the order in scala.

sealed abstract class Preparation  /* This is one way of coding enum-like things in scala */
case object Neat extends Preparation
case object OnTheRocks extends Preparation
case object WithWater extends Preparation

sealed abstract class Glass
case object Short extends Glass
case object Tall extends Glass
case object Tulip extends Glass

case class OrderOfScotch(val brand:String, val mode:Preparation, val isDouble:Boolean, val glass:Option[Glass])

A client can instantiate their orders like this:

val normal = new OrderOfScotch("Bobby Runner", OnTheRocks, false, None)
val snooty = new OrderOfScotch("Glenfoobar", WithWater, false, Option(Tulip));

Note that if the client doesn't want to specify the glass he can pass None as an argument, since the parameter was declared as Option[Glass]. This isn't so bad, but it can get annoying to remember the position of each argument, specially if many are optional. There are two traditional ways to circumvent this problem — define telescoping constructors or set the values post-instantiation with accessors — but both idioms have their shortcomings. Recently, in Java circles, it has become popular to use a variant of the GoF Builder pattern. So popular that it is Item 2 in the second edition of Joshua Bloch's Effective Java. A Java-ish implementation in Scala would be something like this:

class ScotchBuilder {
  private var theBrand:Option[String] = None
  private var theMode:Option[Preparation] = None
  private var theDoubleStatus:Option[Boolean] = None
  private var theGlass:Option[Glass] = None

  def withBrand(b:Brand) = {theBrand = Some(b); this} /* returning this to enable method chaining. */
  def withMode(p:Preparation) = {theMode = Some(p); this}
  def isDouble(b:Boolean) = {theDoubleStatus = some(b); this}
  def withGlass(g:Glass) = {theGlass = Some(g); this}

  def build() = new OrderOfScotch(theBrand.get, theMode.get, theDoubleStatus.get, theGlass);
}

This is almost self-explanatory, the only caveat is that verifying the presence of non-optional parameters (everything but the glass) is done by the Option.get method. If a field is still None, an exception will be thrown. Keep this in mind, we'll come back to it later.

The var keyword prefixing the fields means that they are mutable references. Indeed, we mutate them in each of the building methods. We can make it more functional in the traditional way:

object BuilderPattern {
  class ScotchBuilder(theBrand:Option[String], theMode:Option[Preparation], theDoubleStatus:Option[Boolean], theGlass:Option[Glass]) {
    def withBrand(b:String) = new ScotchBuilder(Some(b), theMode, theDoubleStatus, theGlass)
    def withMode(p:Preparation) = new ScotchBuilder(theBrand, Some(p), theDoubleStatus, theGlass)
    def isDouble(b:Boolean) = new ScotchBuilder(theBrand, theMode, Some(b), theGlass)
    def withGlass(g:Glass) = new ScotchBuilder(theBrand, theMode, theDoubleStatus, Some(g))

    def build() = new OrderOfScotch(theBrand.get, theMode.get, theDoubleStatus.get, theGlass);
  }

  def builder = new ScotchBuilder(None, None, None, None)
}

The scotch builder is now enclosed in an object, this is standard practice in Scala to isolate modules. In this enclosing object we also find a factory method for the builder, which should be called like so:

import BuilderPattern._

val order =  builder withBrand("Takes") isDouble(true) withGlass(Tall)  withMode(OnTheRocks) build()

Looking back at the ScotchBuilder class and it's implementation, it might seem that we just moved the huge constructor mess from one place (clients) to another (the builder). And yes, that is exactly what we did. I guess that is the very definition of encapsulation, sweeping the dirt under the rug and keeping the rug well hidden. On the other hand, we haven't gained all the much from this "functionalization" of our builder; the main failure mode is still present. That is, having clients forget to set mandatory information, which is a particular concern since we obviously can't fully trust the sobriety of said clients*. Ideally the type system would prevent this problem, refusing to typecheck a call to build() when any of the non-optional fields aren't set. That's what we are going to do now.

One technique, which is very common in Java fluent interfaces, would be to write an interface for each intermediate state containing only applicable methods. So we would begin with an interface VoidBuilder having all our withFoo() methods but no build() method, and a call to, say, withMode() would return another interface (maybe BuilderWithMode), and so on, until we call the last withBar() for a mandatory Bar, which would return an interface that finally has the build() method. This technique works, but it requires a metric buttload of code — for n mandatory fields 2ⁿ interfaces should be created. This could be automated via code generation, but there is no need for such heroic efforts, we can make the typesystem work in our favor by applying some generics magic. First, we define two abstract classes:

abstract class TRUE
abstract class FALSE

Then, for each mandatory field, we add to our builder a generic parameter:

class ScotchBuilder[HB, HM, HD](val theBrand:Option[String], val theMode:Option[Preparation], val theDoubleStatus:Option[Boolean], val theGlass:Option[Glass]) {

  /* ... body of the scotch builder .... */

}

Next, have each withFoo method pass ScotchBuilder's type parameters as type arguments to the builders they return. But, and here is where the magic happens, there is a twist on the methods for mandatory parameters: they should, for their respective generic parameters, pass instead TRUE:

class ScotchBuilder[HB, HM, HD](val theBrand:Option[String], val theMode:Option[Preparation], val theDoubleStatus:Option[Boolean], val theGlass:Option[Glass]) {
  def withBrand(b:String) = 
      new ScotchBuilder[TRUE, HM, HD](Some(b), theMode, theDoubleStatus, theGlass)

  def withMode(p:Preparation) = 
    new ScotchBuilder[HB, TRUE, HD](theBrand, Some(p), theDoubleStatus, theGlass)

  def isDouble(b:Boolean) = 
    new ScotchBuilder[HB, HM, TRUE](theBrand, theMode, Some(b), theGlass)

  def withGlass(g:Glass) = 
    new ScotchBuilder[HB, HM, HD](theBrand, theMode, theDoubleStatus, Some(g))
}

The second part of the magic act is to apply the world famous pimp-my-library idiom and move the build() method to an implicitly created class, which will be anonymous for the sake of simplicity:

implicit def enableBuild(builder:ScotchBuilder[TRUE, TRUE, TRUE]) = new {
  def build() = 
    new OrderOfScotch(builder.theBrand.get, builder.theMode.get, builder.theDoubleStatus.get, builder.theGlass);
}

Note the type of the parameter for this implicit method: ScotchBuilder[TRUE, TRUE, TRUE]. This is the point where we "declare" that we can only build an object if all the mandatory parameters are specified. And it really works:

scala> builder withBrand("hi") isDouble(false) withGlass(Tall) withMode(Neat) build()
res5: BuilderPattern.OrderOfScotch = OrderOfScotch(hi,Neat,false,Some(Tall))

scala> builder withBrand("hi") isDouble(false) withGlass(Tall)  build()                
<console>:9: error: value build is not a member of BuilderPattern.ScotchBuilder[BuilderPattern.TRUE,BuilderPattern.FALSE,BuilderPattern.TRUE]
       builder withBrand("hi") isDouble(false) withGlass(Tall)  build()

So, we achieved our goal (see the full listing below). If you are worried about the enormous parameter lists inside the builder, I've posted here an alternative implementation with abstract members instead. It is more verbose, but also cleaner.

Now, remember those abstract classes TRUE and FALSE? We never did subclass or instantiate them at any point. If I'm not mistaken, this is an idiom named Phantom Types, commonly used in the ML family of programming languages. Even though this application of phantom types is fairly trivial, we can glimpse at the power of the mechanism. We have, in fact, codified all 2ⁿ states (one for each combination of mandatory fields) as types. ScotchBuilder's subtyping relation forms a lattice structure and the enableBuild() implicit method requires the supremum of the poset (namely, ScotchBuilder[TRUE, TRUE, TRUE]). If the domain requires, we could specify any other point in the lattice — say we can doll-out a dose of any cheap whiskey if the brand is not given, this point is represented by ScotchBuilder[_, TRUE, TRUE]. And we can even escape the lattice structure by using Scala inheritance. Of course, I didn't invent any of this; the idea came to me in this article by Matthew Fluet and Riccardo Pucella, where they use phantom types to encode subtyping in a language that lacks it.

---

object BuilderPattern {
  sealed abstract class Preparation
  case object Neat extends Preparation
  case object OnTheRocks extends Preparation
  case object WithWater extends Preparation

  sealed abstract class Glass
  case object Short extends Glass
  case object Tall extends Glass
  case object Tulip extends Glass

  case class OrderOfScotch(val brand:String, val mode:Preparation, val isDouble:Boolean, val glass:Option[Glass])

  abstract class TRUE
  abstract class FALSE

  class ScotchBuilder
  [HB, HM, HD]
  (val theBrand:Option[String], val theMode:Option[Preparation], val theDoubleStatus:Option[Boolean], val theGlass:Option[Glass]) {
    def withBrand(b:String) = 
      new ScotchBuilder[TRUE, HM, HD](Some(b), theMode, theDoubleStatus, theGlass)

    def withMode(p:Preparation) = 
      new ScotchBuilder[HB, TRUE, HD](theBrand, Some(p), theDoubleStatus, theGlass)

    def isDouble(b:Boolean) = 
      new ScotchBuilder[HB, HM, TRUE](theBrand, theMode, Some(b), theGlass)

    def withGlass(g:Glass) = new ScotchBuilder[HB, HM, HD](theBrand, theMode, theDoubleStatus, Some(g))
  }

  implicit def enableBuild(builder:ScotchBuilder[TRUE, TRUE, TRUE]) = new {
    def build() = 
      new OrderOfScotch(builder.theBrand.get, builder.theMode.get, builder.theDoubleStatus.get, builder.theGlass);
  }

  def builder = new ScotchBuilder[FALSE, FALSE, FALSE](None, None, None, None)
}

* Did you hear that noise? It's the sound of my metaphor shattering into a million pieces

EDIT 2008-07-09 at 19h00min: Added introductory paragraph.

Pull my Strings

My favorite Joel Spolsky article is The Law of Leaky Abstractions. It may not be the wittiest or the most controversial, but makes up for it in shear importance. Joel's site is down right now (something that doesn't happen frequently), but wikipedia says that the Law was worded as:

"All non-trivial abstractions, to some degree, are leaky."

Well, some pretty trivial abstractions are leaky too. What is the type you use most in your programs? Probably int, but after that? For me it is String (currently java.lang.String or scala.String to be annoyingly more precise).



That is a pretty shitty* abstraction, don't you think? String is supposed to be short for StringOfCharacters. Leaving aside the fact that shorthanding is a bad practice, the name may not be wrong, per se, but it does nothing to characterize the meaning of the type in our programs. Why not Text? It is semantically more accurate and actually shorter than "String".



* To retain the plumbing analogy.

Design Patterns are not Recipes

The controversy

Over the past few months, I've been seeing some Design Patterns backlash on the blogosphere, I guess it is accompanying the agile anti-hype phase that's also strong these days. The most rabid attack came from Mark Dominus, making the case that many innovations in programming languages in the past decades were in response to common practices that could be described as patterns, had the pattern format been known back then. Take, for instance, what we now know as a procedure call: code to stack up register values and return address followed by a jump instruction. Before assemblers became available, programmers used to recode this every time they wanted a subroutine. Dominus sees this observation as a sign that something is wrong with the patterns movement:

"Identification of patterns is an important driver of progress in programming languages. As in all programming, the idea is to notice when the same solution is appearing repeatedly in different contexts and to understand the commonalities. This is admirable and valuable. The problem with the "Design Patterns" movement is the use to which the patterns are put afterward: programmers are trained to identify and apply the patterns when possible. Instead, the patterns should be used as signposts to the failures of the programming language. As in all programming, the identification of commonalities should be followed by an abstraction step in which the common parts are merged into a single solution."

Professor Ralph Johnson, one of the original "Gang of Four" members, responded:

"No matter how complicated your language will be, there will always be things that are not in the language. These things will have to be patterns. So, we can eliminate one set of patterns by moving them into the language, but then we'll just have to focus on other patterns. We don't know what patterns will be important 50 years from now, but it is a safe bet that programmers will still be using patterns of some sort."

Dominus retorts with subtler arguments, and I encourage you to read his article to avoid any misinterpretations of my part. But I'll still quote what I believe is the core of his thinking:

"What I imagine is that when pattern P applies to language L, then, to the extent that some programmer on some project finds themselves needing to use P in their project, the use of P indicates a deficiency in language L for that project.

The absence of a convenient and simple way to do P in language L is not always a problem. You might do a project in language L that does not require the use of pattern P. Then the problem does not manifest, and, whatever L's deficiencies might be for other projects, it is not deficient in that way for your project.

(...)

But to the extent that some deficiency does come up in your project, it is a problem, because you are implementing the same design over and over, the same arrangement of objects and classes, to accomplish the same purpose. If the language provided more support for solving this recurring design problem, you wouldn't need to use a "pattern". Consider again the example of the "subroutine" pattern in assembly language: don't you have anything better to do than redesign and re-implement the process of saving the register values in a stack frame, over and over?"

My take

Mark seems to be worried that programming language innovation will be stifled by the patterns movement, on account of programmers being taught to mimic patterns on their code instead of applying that energy on incorporating them into programming languages. It is important to first acknowledge, as himself put it, that it is valuable to identify commonalities in software design, one could even say this is a precondition for evolving a programming language. So, he sees the problem in not working to embody these commonalities in the language. From my exposure to the patterns literature, I believe the movement is not in any way against incorporating patterns as programming language features. It is implicit in this passage from GoF, page 4:

"Our patterns assume Smalltalk/C++-level language features, and that choice determines what can and cannot be implemented easily. If we assumed procedural languages, we might have included design patterns called "Inheritance," "Encapsulation," and "Polymorphism." similarly, some of our patterns are supported directly by the less common object-oriented languages. CLOS has multi-methods, for example, which lessen the need for a pattern such as Visitor"

And explicit in Ralph Johnson's aforementioned blog post: "Many of the patterns will get subsumed by future languages, but probably not all of them". Now that we know that design patterns are contextual and can be incorporated as language features, we must ask if the emphasis of the patterns movement isn't displaced. Why go to the trouble of publishing large books describing at length things like intent, motivation, participants, consequences, related patterns, etc, when we could just inventory all patterns and shove our favorites in our programming languages? One very simple reason is that not all patterns would yield benefit from being reified. Dominus says that people often mentioned to him MVC as an example of a pattern so complex that it could not be absorbed in a programming languge. But, of course, they were all wrong and he was right, since novel systems like Ruby on Rails or subtext do exactly that, definitively confirming his thesis that the only impediment was an atrocious lack of imagination on the part of his opponents! This is all very strange to me, since "MVC" came to life as pretty concrete software - it was the GUI development framework bundled with Smalltalk-80. Many years later, after several separate versions were developed, it was described and published as an (architectural) pattern. Leaving object orientation early history aside, lets get back to the matter of patterns that aren't targets for reification. One example is the Façade(185) design pattern. As one of the most frequently applied GoF patterns, described having as it's Intent to "provide a unified interface to a set of interfaces in a subsystem", there can be no doubt that it is indeed recurring. Looking at the Structure and Participants sections, we can see that it is very simple, just a Façade class that depends on many other (unidentified) subsystem classes inside a boundary. A consequence of this simplicity is that there would be little gain in incorporating this pattern into a programming language, as it doesn't really represent specific interactions that have to be recoded each time the pattern is applied.

One could respond that if that is the case, then the pattern is useless. It describes a solution so simple that a programmer who has never heard of the "Façade design pattern" would probably be able to construct it by himself. The error in this judgment is that a pattern's merit isn't measured by how interestingly or usefully it gives a solution to a problem. Knowing that its not hard to invent Façade does not detract from the fact that being able to talk about Façade is a big design win in itself. Patterns are as much an aid to communication as they are vehicles to disseminate knowledge.:

"Naming a pattern immediately increases our design vocabulary. It lets us design at a higher level of abstraction. Having a vocabulary for patterns lets us talk about them with our colleagues, in our documentation, and even to ourselves. It makes it easier to think about designs and to communicate them and their trade-offs to others."[GoF, page 3]

On the other hand, we already mentioned that some of the patterns are indeed amenable to reification. In this case there are two paths to choose from: coding it as a library atop usual language abstraction mechanisms or incorporating it as a basic language feature. Dominus second post argues that this choice is a mere implementation detail. Now, to paraphrase him, his point seems completely daft, which may I interpret to mean that there's something that went completely over my head. The difference is obvious: if some feature can be implemented as a library, any programmer that wants it can write it exactly one time and reuse as appropriate; no language modification required. As a contributor to CPAN , he should know well that nowadays much software reuse is done through open source libraries, lowering the recode count to zero for it's users. This is a crucial distinction, as adding something to the language has a huge impact on the whole technical ecosystem, from increased barrier to entry to pernicious interaction between features (non-orthogonality).

Lets take another look at Dominus' central proposition:

"(...) when pattern P applies to language L, then, to the extent that some programmer on some project finds themselves needing to use P in their project, the use of P indicates a deficiency in language L for that project. "

I believe the most important question here is: what does it mean to "use [pattern] P"? If you take it to mean follow the steps from P's description, possibly copying some code from the "Implementation" section, then this reasoning makes sense. But patterns are not merely recipes, and a pattern language is not a cookbook! One way in that patterns differ from recipes is that, as we saw in Façade, the Solution section can be of little importance compared to the communication gain from just naming the pattern. Another difference is that there is wide latitude in implementation strategies for each pattern, not to mention cases where multiple patterns present themselves as design alternatives for a set of forces.

Speculating a bit, I believe the origin of this antagonism toward design patterns is that the idea of a recurrent structure is antithetical to a certain ethos present in the software development community. Mark Dominus summarizes this feeling well in this passage "As in all programming, the identification of commonalities should be followed by an abstraction step in which the common parts are merged into a single solution." It can also be formulated as a call to arms: Don't repeat yourself! Or better yet, DRY! (the love for TLAs is another part of the programming ethos :). I don't dispute that eliminating repetition is an important design heuristic, but I sometimes wonder if it can be taken too far, becoming a sort of factorization fetishism that is detrimental to productivity.

Domain Specific Languages e Estratificação por Estabilidade

2007 mal começou e o meme das DSLs já está provocando polêmica. Eric Evans, em entrevista na InfoQ, disse:

"More out on the cutting-edge are the efforts in the area of domain-specific languages (DSLs), which I have long believed could be the next big step for DDD. To date, we still don't have a tool that really gives us what we need. But people are experimenting more than ever in this area, and that makes me hopeful."

Philip Calçado citou o Evans e foi além, afirmando que "DSLs são iminentes mas as ferramentas simplesmente ainda não chegaram lá". Escrevendo em tom mais cético, Rodrigo Kumpera comentou que DSLs ainda parecem uma tecnologia de um exemplo só; da mesma maneira que toda palestra de AOP fala de logging, toda introdução à DSLs fala em configuração. Citando:

"Todos estão falando o tempo todo sobre como configuração é um problema a ser resolvido por linguagens de domínio. Até o Martin Fowler comete essa gafe em sua palestra na JAOO 2006! O problema é o mesmo que logging, eu diria, representa nenhum risco a complexidade de um projeto. Quantos realmente já tiveram problemas relacionados a dificuldade de colocar logging em uma aplicação? Configuração idem."

Eu acho que a analogia com AOP não procede. Tracing/logging é de fato um caso de uso (quase trivial, BTW) para AOP. Precisamos analisar configuração com mais cuidado. Tome por exemplo uma aplicação de eCommerce B2C tradicional (você pode imaginar que é, quem sabe, uma loja de, sei lá.., animais de estimação). Na primeira iteração do software, cada item à venda tem um preço de catálogo cadastrado. Essa decisão logo se mostra inflexível, pois uma variação sazonal nos preços é comum no varejo. O requisito é atendido com uma nova feature permitindo que se estabeleça um perentual de desconto/acréscimo por mês para cada produto. A interface com o usuário para esta feature se localiza numa tela entitulada "Configurações". Mais adiante percebe-se que a flexibilidade ainda está aquém do ideal, os clientes querem definir regras de preço mais complicadas, como:

se vendeu mais de 500 na semana passada, acréscimo de 10%.

Pode-se inserir estas regras (hard-coded) no meio do sistema ou decidir por manter a parametrização externa e usar uma DSL. Em qualquer caso, ainda podemos chamar isto de configuração?
Sabendo que código e dados são basicamente a mesma coisa, eu vejo a aplicação de DSLs como decisões a respeito de que tipo de informação faz parte do sistema, e que tipo de informação consiste em parametrização "externa". No fim das contas, é técnica para atender ao "Stable Dependencies Principle (S0P)", deve-se isolar as porções de um software com maior volatilidade e uma linguagem específica - talvez até editável por usuários finais - é uma boa alternativa para implementar os módulos menos estáveis.

A taste of Scala

I've recently been trying to learn Scala, a programming language developed at a Swiss university. It has many (many!) cool features, such as seamless interoperability with Java - a result of being compiled to JVM bytecodes -, strong support for functional programming, sophisticated object oriented characteristics and a strong static type system. Rather than continue listing the language's capabilities I will, instead, share a personal use case.

OK, so one of the first items things I looked at was the unit testing framework, SUnit. It comes bundled in the standard library in the scala.testing.SUnit package and is dead simple. I hope the following snippet is self-explanatory enough:

object StackTest {
  import scala.testing.SUnit._

  def main(args:Array[String]): Unit = {
    val tr = new TestResult
    new TestSuite(
    new Test01,
      new Test02,
  ).run(tr)

    for(val f <- tr.failures())
    Console println f
  }

  class Test01 extends TestCase("pushing an element onto an empty stack") {
  override def runTest() = {
      val stack = new Stack()
      val element = "asdf"
      stack push element
      assertEquals(stack.peek(), element)
    }
  }

  class Test02 extends TestCase("popping an element from a stack") {
  override def runTest() = {
      val stack = new Stack()
      val element = "asdf"
      stack push element
      stack.pop()
      assertEquals(stack.isEmpty, true)
  }
  }
}

It really is pretty simple; the whole testing framework sits on a single 200-line file. But is also is a bit verbose, isn't it? All those inner classes, cluttering the code... I tried to simplify things a bit. Here is what I came up with:

import sorg.testing._;

object StackTests extends Tests with ConsoleDriver {
  test ("pushing an element onto an empty stack") {
    val stack = new Stack()
    val element = "asdf"
   stack push element
    assertEquals(stack.peek(), element)
  }

  test ("popping an element from a stack") {
  val stack = new Stack()
    val element = "asdf"
    stack push element
   stack.pop()
    assertEquals(stack.isEmpty, true)
  }
}

I think it looks better. Sort of like those DSLs that are so fashionable these days... But the really cool thing is that it only took a couple dozen lines of code and a couple of hours to extend SUnit. Mind you that someone really proficient in Scala could probably do it much more quickly. See the whole unit testing domain specific language: (the name is almost larger than the code itself :)

package sorg.testing;
import scala.testing.SUnit._;

abstract class Tests extends Test with Assert {
  type TestExp = () => Unit;
  var tests = List[Pair[String,TestExp]]();

  def test(desc: String)(t: => Unit) : Unit = {tests = Pair(desc,()=>t) :: tests};

  override def run(tr: TestResult) = {
    for (val Pair(desc, expression) <- tests) new TestCase(desc) {
       override def runTest() = {Console println "running (" + desc + ")"; expression()}
    }.run(tr)
  }
}

trait ConsoleDriver extends Test {
 def main(args:Array[String]): Unit = {
   val results = new TestResult
   Console println "running tests..."

   this.run( results )

    if (!results.failures.hasNext)
     Console println "Success!";
    else {
     Console println "The following tests failed:";
     for(val each:TestFailure <- results.failures)
       Console println (each.toString + ":\n" + each.trace);
   }
 }
}

Expressiveness and power, what more can one ask of a programming language?

Componentes e objetos

Muito já se escreveu sobre a distinção entre programação orientada a objetos (OO), desenvolvimento baseado em componentes (CBD) e service oriented architectures (SOA). Isso significa que este post vai indubitavelmente chover no molhado. Entretanto, não se pode dizer que o assunto está resolvido e acabado; eu só espero confundir menos do que esclareço.

Como bem lembrou Philip Calçado, não podemos nos esquecer de que estes conceitos todos foram concebidos em contextos diferentes por grupos distintos e sem maiores preocupações com uma coesão formal. OO por exemplo, tem origem na Noruega, mas foi conceptualizada de maneira mais completa no laboratório da Xerox em Palo Alto, sendo reificada numa sequência de protótipos da linguagem Smalltalk. O líder desse projeto foi o lendário Alan Kay, que também tem a distinção de ter cunhado o termo "programação orientada a objetos". Kay, que na época já tinha um background em biologia e matemática, imaginou um paradigma computacional centrado em entidades similares a células biológicas, que se comunicavam trocando mensagens químicas. Apesar da paternidade reconhecida, não há consenso sobre o que seria a definição de OO, então trabalharei com uma definição improvisada (não confie muito). Um objeto é caracterizado por:

1. Comportamento. Ele é capaz de receber invocações* de outros objetos para efetuar uma computação e (a) responder com um outro objeto ou (b) alterar o seu estado. Para efetuar a computação, o objeto pode invocar outros objetos. Ele obtém acesso a estes objetos de três maneiras: criando - "instanciando" - novos objetos na hora, usando uma referência que é parte de seu estado ou recebendo-os no momento em que é invocado.
2. Estado. Um objeto pode armazenar um conjunto de referências para outros objetos. Através dessas referências ele pode invocar comportamento dos outros objetos conforme descrito no ítem anterior.
3. Identidade. Mesmo que o estado todo de um objeto se altere, ele ainda existe como a "mesma" entidade.
4. Ciclo de vida. Um objeto é instanciado em algum momento, quando ele passa a existir (manter uma identidade). Ele também pode ser destruído em algum momento. As linguagens de programação que seguem o paradigma OO apresentam uma grande variedade de mecanismos para instanciar e destruir objetos (classes, protótipos, prefixos, gerenciamento manual de memória, coletor de lixo, ...).

Enquanto que objetos são entidades fundamentais de um modelo computacional, componentes são apenas uma alternativa para atender a certos requisitos encontrados em algumas aplicações. Essa afirmação pode parecer polêmica, por isso apelo para uma autoridade maior, Ralph "GoF" Johnson, que expôs bem essa consideração num post recente em seu blog:

It is important to realize the "components" are not technology, they are a customer requirement. Customers want to add on to their software system by adding a new component, and adding this component must not require that they change their existing system. It is like adding new speakers to a stereo system.

Ou seja, componentes são nada mais que uma unidade de extensão e/ou substituição dinâmica de um sistema de software. O adjetivo "dinâmico" é uma daquelas palavras que são jogadas por aí em contextos tão diversos a ponto de perder o seu significado, mas a noção aqui é bem simples: estamos considerando alterações realizadas em ambiente de produção, feitas por operadores do sistema. A partir dessa caracterização, podemos dezfazer uma série de confusões:

• Uma aplicação pode usar componentes sem ser baseada em componentes. Isto é, é possível definir alguns poucos pontos de substituição e/ou extensão mantendo o resto do software completamente estático. Dentre os muitos exemplos estão a maioria dos programas de desktop; um exemplar típico é o Photoshop e seus plugins. É claro que também encontramos diversos casos no extremo oposto - aplicações totalmente baseadas em componentes - veja por exemplo os últimos servidores de aplicações e IDEs.
• Um sistema pode ser modular mas não baseado em componentes, na acepção que estamos considerando. Basta que seja composto de partes pouco acopladas.
• Não faz muito sentido projetar um sistema baseado em componentes fortemente acoplados. O motivo é simples: componentes fortemente acoplados exigem que um desenvolvedor independendente se preocupe com grandes porções do sistema, o que prejudica o atendimento ao requisito de reconfigurabilidade. Mesmo no caso de aplicações que limitam seu uso de componentes a extensões localizadas, é uma boa prática estabelecer uma interface reduzida entre o núcleo do sistema e os componentes.
  
• A definição sugerida exige apenas que um sistema possa ser reconfigurado em ambiente de produção, não é necessário que ele tenha a capacidade de sofrer alterações enquanto está executando. Não obstante, existem algumas aplicações onde isto é um requisito real e soluções tecnológicas sofisticadas são necessárias.
• O professor Johnson deixa bem claro que tecnologias como COM ou EJB não são necessárias para um sistema que use componentes. A rigor, para antender ao requisito delineado é suficiente um suporte tecnológico mínimo. Podemos imaginar até um sistema onde a escolha de componentes é feita através da substituição de arquivos objeto (em java seria um .class) no diretório de instalação.
• Apesar do ponto anterior, é frequente que um sistema com componentes se beneficie de uma infra-estrutura básica para auxiliar na solução de problemas comuns como: controle do ciclo de vida, gerenciamento de dependências, suporte à modos de comunicação entre componentes, parametrização local (propriedades), etc.

Como já demorei muito mais do que imaginava para escrever este post - incluíndo um tempo desperdiçado reescrevendo uma boa porção do texto quando eu fiz besteira e perdi o draft - vou deixar para uma entrada futura a discussão sobre sistemas distribuídos e serviços.

Uma última observação: o leitor atento notará que a palavra reuso não aparece nesse post; isso não foi um acidente.

* Estou evitando a terminologia baseada em "mensagens" que é padrão em Smalltalk para evitar mais confusão.

Model View Confusion

The development community is a strange beast. And I'm not talking about hitchhiker's jokes and D&D tournaments. The strangeness I'm referring to is that many of the most talked about concepts are used in ways that differ a lot from their original, intended, meanings. Many patterns fall in this category, but one is specially abused: MVC. This is one of the first TLA's a java developer learns on his first days in the craft. By "learn", I really mean "kinda sorta knows that it has something to do with that Struts crap".

I was in that exact position a few years ago (crazy coincidence, innit?). After a few hours of wikistumbling through c2.com, I got considerably more confused. The next step I took was to chase down a copy of the first published description. Google wasn't my friend in this process* and I had to resort to the old-fashioned route of requesting a loan from some - the only one, in fact - college library that had the aug/1988 copy of the Journal of Object Oriented Programming. I only half-understood it because at the time I wasn't conversant in Smalltalk. But I undestood enough to see that MVC was somewhat different from what the Java guys talked about.

One big difference is that Smalltalk-80 was self-hosted - it was it's own operating system**. This meant that input device processing was a responsability of objects. And that, my little friends, is how the Controller came to be. Quoting the paper:

In particular, controllers coordinate the models and views with the input devices and handles scheduling tasks. (...) Class Controller does include default scheduling behaviour...

What motivated me to write this entry was Fowler's superb article on GUI architectures. In there you'll find a thorough discussion on MVC and its subsequent variants. It will be a chapter on the upcoming second volume of his Patterns of Enterprise Application Architecture.




* Of course, now the bastard is online here.

** This control obsession can be seen still today, just look at Squeak's UI and fondly remember the glorious Win3.1 days...