Where scala left me wanting

n the past few years, I've grown to enjoy more and more programming in a functional style. Even the java code I write nowadays is mostly free of mutable state. But before that, I spent a good chunk of time learning about object orientation and found some great ideas there. Ideas that can carry over to the functional world. One of them is Domain Driven Design, where we strive to have the symbols in the code respresent conncepts in the domain. Of course, I'm oversimplifying DDD, but the ideal of a bijection between the reality our software models and constructs in a programming language is one I believe we should maintain in many, perhaps most, applications.

A natural consequence is that our systems architecture tends to resemble an onion, with the domain model in the center and code to make it interact with the rest of the world surrounding it:

I've labeled the code that interacts with the rest of the world "Adapters", as it should do little more than adapt external representations into concepts the domain model can understand. This architecture is not in any way novel. In fact, it is a version of Alistair Cockburn Hexagonal Architecture. In the common case where the only interactions the domain model has with the rest of the world are through a Database on one end and an User Interface on the other, we've just described the 3-layer architecture that was so popular in the 90s. Some examples of adapters are — to increase our shot at the buzzword bingo — Web MVC Controllers, Repositories or DAOs, message endpoints, GUI listeners, etc., you get the picture.

Since Scala is a truly hybrid language, marrying quite elegantly OO and FP, it would appear to be the perfect vehicle to write this kind of software. I've found this premise to be correct, for the most part. As you might have guessed, there is an exception; (If there weren't, I wouldn't be writing this post, would I?). The problem is in the code for the adapters. As mentioned, they tend to be simple shims, translating some external representation into domain objects. Coding such translations every single time for every adapter instance in every application is very dull. We desperately need our old friend, lady abstraction, to help us out here. In the Java world, she lends us a hand though dynamic reflection, allowing the construction of objects and invocation of methods to be done generically. Unfortunately, Scala has no reflection API, so we have no alternative but to resort to Java reflection, in a sense reverse-engineering the Scala compilation process.

This isn't a major gripe, as Scala constructs tend to map to Java constructs in a straightforward fashion. Also, rumor has it that a Scala reflection API is being developed for 2.8.1. But that's only half the story. Powerful as it is, dynamic reflection alone is not enough to solve the Adapter problem once and for all. We often need to parameterize the translation in some way. For instance, when translating objects to database rows, we must discover the Column names corresponding to object properties. In many cases we can trust convention, for instance we could expect the columns to match exactly the names of the properties. But in other cases there is no option but to somehow configure the translation with an explicit mapping.

 In the Java world this kind of thing used to be done with verbose and annoying XML configuration files. As the language evolved, annotations were introduced, and they are now the main way to configure such translations.

Annotations are a great improvement over external XML files, but they can't be the end of the story. There is the relatively minor issue that annotations pollute the domain model we strive so much to keep clean and organized, specially when the same domain objects will be active in many adapters. Beyond that, there is the larger problem that annotations are just metadata. Sometimes we want to parameterize our adapters in richer ways. Take, for instance, the proposal for a typesafe API for database queries to be added to the new version of the Java Persistence API. It requires a special pre-processor to generate a metamodel that can be used to parameterize a database adapter. 

A better and more generic approach would be to have the language itself provide this metamodel: a kind of static reflection. I would love to see something like this in Scala. Some time ago I toyed with the idea of writing a compiler plugin to provide a static meta-model of Scala classes, but apparently compiler plugins have issues with non-transparent code generation. 

Postscript. As with all matters regarding programming languages on the web, we must tread lightly. I am not ranting against Scala, in fact I rather enjoy it, as can be gleaned from some of the previous posts. I am only relating a very specific domain where I believe the language can be much improved. 

On Exceptions, Mythological Monsters and Household Appliances

Philosophy aims at the logical clarification of thoughts. Philosophy is not a body of doctrine but an activity. A philosophical work consists essentially of elucidations. Philosophy does not result in 'philosophical propositions', but rather in the clarification of propositions. Without philosophy thoughts are, as it were, cloudy and indistinct: its task is to make them clear and to give them sharp boundaries.
Wittgenstein

If language is not correct, then what is said is not what is meant; if what is said is not what is meant, then what must be done remains undone; if this remains undone, morals and art will deteriorate; if justice goes astray, the people will stand about in helpless confusion. Hence there must be no arbitrariness in what is said. This matters above everything.
Confucius

Shit happens
Forrest Gump?

Exceptions are a controversial matter among programmers. We disagree about how to deal with them. We disagree if the should be checked by the compiler. We even disagree if they are useful at all. Sometimes controversy arises from real disagreement: I may argue that test-driven-development is the best way to write software while you might say that formal proofs are the way to go. Both sides can productively debate the issue and present thoughtful arguments either way. I posit that the polemics surrounding exceptions are not of this sort. They are, rather, brought about by the inherent confusion in the mechanism. Exceptions are the Hydra of current programming languages, under a simple banner — to handle exceptional conditions — they mix several mechanisms.

The heads

Let's begin with the humble throw term, which is a sort of procedure exit statement, causing the current routine to throw his hands up in the air, call it quits, and immediately return to the caller. The only other statement with a similar effect is return. Of course when calling return, we must return something! In statically typed languages, this something must have a type: the return type of the function or method. Exceptions muddle the picture, if we think of a throw as a kind of return then what is the type being returned? There can be many answers to this question, but let me put forth a proposal: the "return type" of a "throw" is, for all intents and purposes, a variant type, and that is the second head of the Hydra.

This warrants a small digression. Most functional languages have a way of saying "this type FOO is really either a BAR or a BAZ". So every place in the code that is handed a FOO must check whether it is a BAR or a BAZ and deal appropriately. We call FOO a variant type, and BAR and BAZ are its variants. The syntax for the check is called called a type-case¹.  And where have we seen syntax for checking the type of something and then acting on it? Catch clauses!

But of course many languages lack the concept of variant types. I'll argue that in practice, exception handling code in effect greenspuns variant types. First, we can observe that in languages with checked exceptions the declaration of the exceptions a method can throw — the throws clause in Java — is basically a variant type declaration in disguise. Even when working with unchecked exceptions variants lurk behind, encoded through subtyping. Just notice how little polymorphism is exploited in exception class hierarchies. Different exceptions are given different types for the sole reason of being distinguished in catch clauses. Just like variant types.

Ok, time for the next head of the monster. We'll continue to look at the catch clause, but this time we'll look at it as a control structure. As such, it is a strange beast, neither a repetition nor purely a decision structure. Once an exception is thrown, the calling method will stop everything it was doing and jump straight to a catch block. In a way, every exception-throwing method call can be seen as a regular call paired with a conditional jump. Dijkstra may not be rolling in his grave, but he is probably feeling a little tingly.

Since we've talked about control flow, let's direct our attention to the mischievous head of the Hydra: stack unwinding. After a throw happens, if the calling method doesn't care to handle an exception, his caller will have to bare the burden or leave it up to his caller, recursively unwinding the stack until some frame decides it can be bothered to catch the exception and deal with it. This is a pretty powerful feature, sometimes it's even exploited to simulate continuations in languages that lack them.²

One more head left: call stack introspection. Besides unwinding the call stack, exceptions may reify it. It is a limited form of reflection, typically used only to print stack traces to a log someplace where they can be attentively ignored. Common lore says capturing the stack frames is an expensive operation, and so should be executed only on, well, exceptional circumstances.

Exceptional Situations

The confusion bus doesn't stop here, I'm afraid. All the complexity we've examined so far is justified under a deceptively simple banner: to handle exceptional conditions. Hidden in these four words is the vast range of situations that may be interpreted as "exceptional". Like any piece that deals with the subject of exceptions, we can't escape the temptation to suggest a taxonomy. At least this one is short, these are the three main categories of "exceptional situations":

• Programming error: A traditional example would be code trying to index an array beyond it's bounds. Ideally we would design our abstractions so as to make it impossible to commit such coding errors. From formal grammars to dependent type systems this ideal has been driven much of the research on programming languages over the years. Unfortunately, at least for now, we cannot always evade the need for run-time checks.
• External failure: Something bad happened to some component not within control of the application. Examples: network cable cut, disk crapped out, database died, sysadmin was drunk and lost a round of UNIX Russian roulette.
• Invalid user entry: "If an error is possible, someone will make it. The designer must assume that all possible errors will occur and design so as to minimize
  the chance of the error in the first place, or its effects once it gets made. Errors should be easy to detect, they should have minimal consequences, and, if possible, their effects should be reversible." This is from Don Norman's superb The Design of Everyday Things. The book is so good I'm going to slip another quote in here: "The designer shouldn't think of a simple dichotomy between errors and correct behaviour; rather, the entire interaction should be treated as a cooperative endeavor between person and machine, one in which misconceptions can arise on either side"

Let's not forget we are classifying "exceptional situations", not exception types. Here is an example that I hope will make the distinction clear. Take the always popular FileNotFoundException, that is thrown when application code attempts to read a file and the file isn't there: where do we place it on our little taxonomy? Imagine the application is a boring old word processor and the user types in the wrong file name. That obviously is a case of invalid user entry. Now, consider the case of a mail server attempting to open the file where it stores all queued outgoing mail. This file was probably created by the server application itself, maybe during the installation procedure, and its absence would be an indication that something is very borked in the server environment. The same File Not Found event is now clearly an external failure. This matters, because the strategy used for dealing with the three categories is different, as we'll soon see.

The Inevitable Cross Product

Time to look at how the heads of the hydra fare up to our new categories:

Programming errors are, by definition, unexpected. If we aren't expecting them, there is no point in writing code to handle different exception types. What could you do when you know some code tripped a NullPointerException that's different from what you would do if the offence was an IndexOutOfBounds? In other words, the variant types head is useless here. Exceptions triggered by programming errors are usually caught in a generic stack frame, in the bowels of infrastructure code — through the magic of stack unwinding — and dealt with by logging the error type, message, and stack trace. After logging there is little left to do but to roll over and die. In multi-user systems, killing the whole process is not very polite, so we might nuke just the current work item. This is the gist of the fault barrier pattern. Call stack introspection is a very valuable aid to pinpoint the bug. So valuable that I go as far as point its absence as the single most important reason to avoid C for teaching programming to beginners.

Practical strategies for dealing with external failures aren't too different. We tend to exploit stack unwinding to catch the exceptions in a frame deep in infrastructure land and log them in pretty much the same way. But, every so often, we might want to code a different action, such as to retry the operation or to trigger a compensation transaction. In these relatively rare cases it might be useful to deal distinctly with different variants. Another aspect where external failures are distinct from programming errors is that call stack introspection isn't so helpful: if the failure is external, the exact place in the code where it was detected is of little significance. Of course, it is important that the exception carries enough information to enable a system administrator to find the origin of the fault by just looking at the log.

Invalid user input is a different beast altogether. Variants are definitely useful; after all it's important to let the user know if their request was over quota rather than under quota. On run-of-the mill interactive OLTP systems, we handle invalid user entry by providing immediate feedback on what was wrong. But not every case is so simple: in some circumstances it may be important to identify suspicious input and raise security flags; in others, we could offer suggestions for the user to correct the data; in asynchronous interactions, we might want to proceed with the action substituting a default or normalized value for the invalid input; and so on. Altogether, there is a wide range of handling strategies, mostly domain-specific. This is a key observation, as not only handling strategies, but also the detection and the very definition of what values are invalid, are full of domain specific decisions. I would go as far as proposing that the controversial domain exceptions all boil down to cases of invalid user input.

What about the other heads of the monster, are they helpful for invalid user inputs? Call stack introspection is clearly not needed. Moreover, the performance costs of reifying the stack can be detrimental for the not-so-rare instances of invalid input. This isn't just premature optimization, as the conscience of these costs can prevent users from even considering exceptions for the domain logic. What about stack unwinding? I'm afraid I don't really have an answer for this one. On the one hand, domain specific handling logic seems to fit naturally in the calling method. On the other paw, if all we do is inform the user that something happened, a generic code further down the stack is awfully convenient.³ 

So what?

We've seen that the feature by the name exceptions is really a hodgepodge of language constructs. Beyond the technical aspects, the very raison d'être for the feature is also muddled. It's as if some crazy nineteenth century inventor decided that since people need to be protected from the heat and, hey, food needs cooling to be preserved as well, why not make buildings like large refrigerators? Just keep the food near the bottom where it's colder, the humans near the top where it's not freezing, and presto! His solution is absurd, but that is because the problem itself is ill-formulated: "to keep stuff cool" is not an useful problem statement.  "To deal with exceptional circumstances" isn't either.

I don't know if the above ramblings are of any practical significance. I certainly don't have any concrete proposal to make things better. My only hope is that language designers will keep in mind context and necessity when devising new constructs.

¹ In languages where pattern matching is available, it subsumes type-cases. In fact, one way of thinking about pattern matching is that it is a sophisticated form of type-case.

² Some languages without full support for exceptions do offer a feature called long jumps, that is very close to what I'm calling stack unwinding.

³ To further complicate the issue, languages offering variant types usually allow different program structuring techniques. For instance, instead of a direct chain of method calls we may thread a monad through a series of functions.

Software

This is my Quality is Dead hypothesis: a pleasing level of quality for end users has become too hard to achieve while demand for it has simultaneously evaporated and penalties for not achieving it are weak. The entropy caused by mindboggling change and innovation in computing has reached a point where it is extremely expensive to use traditional development and testing methods to create reasonably good products and get a reasonable return on investment. Meanwhile, user expectations of quality have been beaten out of them. When I say quality is dead, I don’t mean that it’s dying, or that it’s under threat. What I mean is that we have collectively– and rationally– ceased to expect that software normally works well, even under normal conditions. Furthermore, there is very little any one user can do about it.

James Bach

If you look at engineering or maths, we've been doing that for thousands of years, so we now know how to build a building and make it solid. With code, we've been doing computer science for 70-75 years, so we are still scratching the surface - we don't have a real theory or like physics, where they have a good foundation. We have the Turing machine that doesn't really reflect distributed computation. The lambda calculus captures certain parts, then there is a lot of process algebra, but it's not yet clear that we really have understood everything, which I think is fantastic, because that means there is opportunity to discover new things.

Erik Meijer

Much of what is wrong about our field is that many of the ideas that happened before 1975 are still the current paradigm. He [Alan Kay] has a strong feeling that our field has been mired for some time, but because of Moore’s law, there are plenty of things to work on. The commercialization of personal computing was a tremendous distraction to our field and we haven’t, and may not, recover from it.

One of Alan’s undergraduate degrees is in molecular biology. He can’t understand it anymore despite having tried to review new developments every few years. That’s not true in computer science. The basics are still mostly the same. If you go to most campuses, there is a single computer science department and the first course in computer science is almost indistinguishable from the first course in 1960. They’re about data structures and algorithms despite the fact that almost nothing exciting about computing today has to do with data structures and algorithms.

Alan Kay (paraphrased by Phil Windley)

[Actually, check the comments for Alan's clarification].

REST Beyond the Obvious

Now that the heat of the REST vs. SOA battle seems to be dissipating, we can try to shed some light into how the choice of a resource centric design affects overall enterprise architecture. We can start by following a thought experiment. Imagine a company growing so fast the HR staff is overwhelmed by the task of forwarding all the new job openings to recruiting agencies. Our job is to automate this problem away. Using REST.

The basics

As RESTful Web Services go, this one seems pretty simple. One resource per job opening, represented with a simple custom XML format; maybe a collection resource, listing all current openings; and possibly also a search resource, to look for openings with specific attributes. Something like this:

URI Template	Method	Representation
/job_opening/{id}	GET	Job Opening XML Schema
/job_openings	GET	Links to all openings as a custom schema or with XOXO.
/job_openings/find?{query_params}	GET	Custom schema or XOXO or atom feed, all w/ Opensearch elements.

So, how do we create the job opening resources? The obvious choice would be to apply the traditional RESTful collections pattern: a POST to the /job_openings collection with an entity body containing the xml representation of the opening triggers the creation of a new resource, whose URL would then be returned in the location header of the response.

Shaking things up

But there is an alternate model: let client departments simply publish job_opening resources by themselves, on their departmental web servers. There is a vast array of options to do such a thing, none of them requiring users to write XML, of course. IT could whip up a word processor macro to convert documents to our xml format, and save them to a web server. In the MsWorld, one could use Word's XML support for inserting elements from the job_opening namespace into a document and publish it to an WebDav enabled server, maybe using Sharepoint. In the OpenOffice universe, we could just as easily save a document as ODF, pass it trough an xml transformation to the job_opening format, and then publish it to an Web Server. The publishing itself can be made with a number of techniques, from FTPing to a shared directory to using an Atompub client with a compatible server to the aforementioned WebDav protocol. A completely different, and much more complex, option would be to have the department run an instance of a custom job openings application, maybe even something similar to the “obvious choice” outlined above, but I really don't see how it could be useful.

So far, all very cute and distributed, postmodern even, but does it work? I mean, the end goal is to send all the openings info to external recruiting companies, and just throwing the resources at the internal web doesn't quite accomplish that. The missing piece of the puzzle is a simple crawler; a small piece of software that scans the web by following links and looks for resources that accept being viewed as job_openings. A side effect of most publishing options mentioned in the past paragraph is that a link is created to the newly available document in some sort of listing. Our crawler needs only to know how to get at those listings and how to parse them looking for links. If you think parsing is complicated, I urge you to think about the following code snippet: /(src|href)="(.*?)"/

What If...

Since this is a thought experiment, we can get more, well, experimental. Let's see what could be gained from shunning XML altogether and using an HTML microformat for the job_opening resource representations. This would expand even more the publishing options. For instance, a plain Wordpress blog, maybe already used as a bulletin board of sorts by some department, could be repurposed as a recruiting server. Another benefit would be to have every document in the system in human-readable form, not XML “human-readable”, but really human-readable.

Now, suppose the company decided it just wasn't growing fast enough, went ahead and bought a smaller competitor. And of course, this competitor already had an in-house recruiting application. Being of a more conservative nature, which just might explain why they weren't market leaders, their IT department built it as a traditional Web front-ed / RDBMS backed application. How do we integrate that with our REST-to-the-bone job openings system? First we note that there is no need to have the data available in real-time, after all, no company needs to hire new people by the minute. Given that, the simplest solution would probably be a periodic process (someone said cron?) that extracts the data directly from the database, transforms it to our job_opening format and shoves it in some web server.

So what?

I can't say if this scenario were for real would I choose such a distributed approach. Maybe a quick Rails app running on a single server would better fit the bill. But that's not the point of our little exercise in architectural astronautics, we are here to think about the effects of REST's constraints to overall architecture. So, let's recap some of them:

• The line between publishing content and using an application blurs. “Static” documents and active programs have equal footing as members of the system.
  

• Thanks to the uniform interface constraint, the mere definition of a data format allows for unplanned integration.

• If we add a standard media type to the mix, we can take advantage of established infrastructure, as in the case of a blogging platform reused as a “service endpoint” simply by adoption of HTML microformats.

• REST in HTTP encourages pull-based architectures (think of our HR crawler), which aren't all that common outside of, well, HTTP web applications

• The very idea of a service may fade away in a network of distributed connected resources. The closest thing to a service in our system is the crawler, but note that it does its work autonomously, no one ever actually calls the service.

• Links (aka hypermidia) are a crucial enabler for loosely-coupled distributed services. In some sense, everything is a service registry.

• One of the forgotten buzzwords of the 90's, intranet, may come to have a new meaning.

Pondering

"Formal methods will never have any impact until they can be used by people that don’t understand them."

Tom Melham

"The original study that showed huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson and Grant (1968). They studied professional programmers with an average of 7 years' experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1, the ration of debugging times over 25 to 1, of program size 5 to 1, and of program execution speed about 10 to 1. They found no relationship between a programmer's amount of experience and code quality or productivity.

Although specific rations such as 25 to 1 aren't particularly meaningful , more general statements such as "There are order-of-magnitude differences among programmers'" are meaningful and have been confirmed by many other studies of professional programmers (Curtis 1981, Mills 1983, DeMarco and Lister 1985, Curtis et al. 1986, Card 1987, Boehm and Papaccio 1988, Valett and McGarry 1989, Boehm et al. 2000)."

Steve McConnell
Code Complete, Second Edition

"If you look at the way software gets written in most organizations, it's almost as if they were deliberately trying to do things wrong. In a sense, they are. One of the defining qualities of organizations since there have been such a thing is to treat individuals as interchangeable parts. This works well for more parallelizable tasks, like fighting wars. For most of history a well-drilled army of professional soldiers could be counted on to beat an army of individual warriors, no matter how valorous. But having ideas is not very parallelizable. And that's what programs are: ideas."

Paul Graham
Holding a Program in One's Head

"Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine--open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound. [...]

The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence. For three centuries, mathematics and the physical sciences made great strides by constructing simplified models of complex phenomena, deriving properties from the models, and verifying those properties by experiment. This paradigm worked because the complexities ignored in the models were not the essential properties of the phenomena. It does not work when the complexities are the essence."

Fred Brooks
No Silver Bullet

"Architecture is intended to be facilitative, of course, in that a good architecture should enable developers to build applications quickly and easily, without having to spend significant amounts of time re-inventing similar infrastructure across multiple projects. [...]

But an architecture is also intended to be restrictive, in that it should channel software developers in a direction that leads to all of these successes, and away from potential decisions that would lead to problems later. In other words, as Microsoft's CLR architect Rico Mariani put it, a good architecture should enable developers to "fall into the pit of success", where if you just (to quote the proverbial surfer) "go with the flow", you make decisions that lead to all of those good qualities we just discussed. "

Ted Neward
Hard Questions About Architects

"The more interesting your types get, the less fun it is to write them down!"

Benjamin C. Pierce
Types and Programming Languages - The Next Generation

"If you don’t know the difference between a group, a ring, and a field, you have no business overloading operators.

Now I’m not saying that one has to take a course in abstract algebra before you can be a competent programmer. You don’t as long as the language you program in doesn’t support operator overloading (or as long as you’re wise enough not to use it if it does). However since most programmers are didn’t get beyond ODEs in college (if indeed they got that far–some of my comp sci friends struggled mightily with calculus and had to retake it repeatedly), one can’t responsibly design a language that requires mathematical sophistication in the 99th percentile for proper use."

Elliotte Rusty Harold
Operator Overloading: Trickier Than it Looks

"You used to start out in college with a course in data structures, with linked lists and hash tables and whatnot, with extensive use of pointers. Those courses were often used as weedout courses: they were so hard that anyone that couldn't handle the mental challenge of a CS degree would give up, which was a good thing, because if you thought pointers are hard, wait until you try to prove things about fixed point theory.

All the kids who did great in high school writing pong games in BASIC for their Apple II would get to college, take CompSci 101, a data structures course, and when they hit the pointers business their brains would just totally explode, and the next thing you knew, they were majoring in Political Science because law school seemed like a better idea. I've seen all kinds of figures for drop-out rates in CS and they're usually between 40% and 70%. The universities tend to see this as a waste; I think it's just a necessary culling of the people who aren't going to be happy or successful in programming careers."

Joel Spolsky
The Perils of JavaSchools

Concurrency and the demand for computing

On my previous post I expressed some doubt with the “market” for rich internet applications. The skepticism was rooted on an observation that the demand side of the equation isn't looking so promising: what applications will emerge benefiting from those wonderful technologies?

I find myself thinking along similar lines with regard to concurrency. For the sake of analysis, lets split the space of applications in server-based and client-based. Members of the first group basically deal with responding to requests coming over the network. This means that there is a naturally high degree of parallelism and, of course, this has been exploited for a long time. The typical scenario is some serial business application code atop a middleware platform that handles threading and I/O. This means that on the server front, the “multicore revolution” will impact little on most software development efforts. Now, desktop software developers don't have such luck – the era of surfing on Moore's Law is really over. And so what? The way I see it,* raw computing has ceased to be an important bottleneck, long gone are the days of watching a crude hourglass animation while the CPU labored away. Not that we do any less waiting by now, these days we spend our time waiting for the network.

Anyway, maybe the concurrency boogieman is less scary than we think.


*(this is a blog, after all)

IDLs para REST?

[Edit: O Aristotle Pagaltzis tratou deste mesmo tema neste post.]

Antes de mais nada, um aviso para quem usa a porcaria do blogger: o “auto-save” não funciona! Como o leitor já deve ter deduzido, eu descobri esse problema de maneira pouco agradável, perdendo o resultado de algumas horas de digitação. Vamos tentar de novo...

Se eu ainda me lembro, comecei explicando que fui motivado por dois posts com que esbarrei por aí na internet. Primeiro foi o Roy Fielding na lista REST-discuss comentando que um dos princípios do estilo não é bem compreendido. O segundo foi o Daniel Quirino bloggando sobre uma deficiência do REST, a ausência de linguagens de descrição de interface. Citando o Daniel citando ele próprio:

“WSDL cumpre um papel muito importante neste contexto de servir outros sistemas, uma vez que existe uma maneira padrão para se descrever para outras máquinas todos os serviços que estão sendo oferecidos. “

Parece razoável, não? O provedor de um serviço escreve um documento WSDL descrevendo as operações oferecidas e os tipos de dados de entrada e saída de cada. A partir daí, para se desenvolver um cliente do serviço basta obter uma cópia do WSDL, usar alguma ferramenta para gerar artefatos correspondentes e invocar as operações como se fossem chamadas de procedimento locais(*).

Essa descrição não se aplica a serviços RESTful. Para começar, o conceito de operação é muito diferente – só existe um conjunto fixo delas para todos os recursos no sistema inteiro. No caso do HTTP, as operações são os famosos verbos GET, PUT, POST e DELETE(**). Ok, mas para onde vai a semântica que estava nas operações? A princípio parece que vai parar na identificação de recursos: uma chamada getStockQuote(“SUNW”) se transforma em um GET na URL http://example.com/stock/SUNW.

Mas REST é um pouco mais complicado do que isto. Aliás, os argumentos a favor do estilo por ser mais simples do que WS-* estão ligeiramente equivocados, IMHO. O foco principal do REST é a escalabilidade, em vários sentidos, desde uma preocupação em controlar a carga nos servidores em relação à demanda(***) até um entendimento de como habilitar network effects para aumentar o valor da rede inteira à cada contribuição individual. Na tese do Roy Fielding este foco leva à quatro restrições que guiam a arquitetura: “identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state”. Uma destas é o tal princípio que o Roy observou ser mal compreendido, consegue adivinhar qual?

O ganhador é “hypermedia as the engine of application state”, que de fato parece bem mais obscuro que os demais. A idéia aqui é, no meu entendimento, que o estado da aplicação é conhecido somente pelo cliente mas é dirigido pelo servidor. Conhecido somente pelo cliente porque o estilo exige que o servidor não mantenha informação alguma sobre “sessões” com um cliente. Isso mesmo, cookies com sessionids e assemelhados não são RESTful. A parte de “dirigido pelo servidor” pede um exemplo; vamos considerar o funcionamento dos forms html. O browser recebe um documento contendo uma tag <form action="alguma_uri"> definindo o destino e subtags <input ...> especificando os dados a serem preenchidos. O usuário entra com seus dados, aperta submit e o browser faz um POST no recurso identificado por alguma_uri passando uma representação application/x-www-form-urlencoded das informações. Percebam que o cliente não precisava saber a priori para qual URI POSTar seus dados, ela foi obtida do servidor. Em outras palavras, a transição de estado se deu no cliente seguindo um link (hipermídia). O Sam Ruby explica isso bem melhor que eu aqui.

Essa restrição ajuda a permitir que porções diferentes do sistema evoluam independentemente, por exemplo se um servidor decidir mudar sua estrutura de navegação basta começar a servir URIs diferentes em seus links e os clientes continuam operando sem sofrer alterações. Outras características do REST, como content-negotiation baseada em tipos padronizados, também contribuem para facilitar este requisito. E é claro que evolução independente é crucial para um sistema que pretende atingir escala global. Aí está o link com a escalabilidade, no pun intended.

Por isso que muitos RESTfarians criticam as linguagens de descrição de interface, inclusive WADL. Como já vimos no caso do WSDL, elas acoplam um cliente com uma definição de serviços em “tempo de compilação”, por assim dizer. E mais, há quem defenda o uso de documentos análogos à estes padrões de descrição de interfaces, mas pensados para serem interpretados por clientes em tempo de execução. Uma iniciativa deste tipo é o RDF-forms, a respeito do qual eu não sei absolutamente nada.

Isso ainda é controverso, e os oponentes apresentam um argumento forte – IMHO de novo – que comunicação programática entre aplicações, estilo EAI, é muito diferente do sistema interativo de hipermídia global para o qual REST foi concebido. Na minha opinião pessoal, independente de ortodoxias, considere adotar REST para suas APIs públicas, mesmo se não seguir rigorosamente todos os preceitos do doutor Fielding.

* Uma equipe esperta vai saber que a confiabilidade e a latência destas chamadas são muito diferentes de invocações locais, mas isso não muda muito o processo em linhas gerais.

** E mais HEAD, OPTIONS e outros primos pobres.

*** Caching, caching, caching

Design Patterns are not Recipes

The controversy

Over the past few months, I've been seeing some Design Patterns backlash on the blogosphere, I guess it is accompanying the agile anti-hype phase that's also strong these days. The most rabid attack came from Mark Dominus, making the case that many innovations in programming languages in the past decades were in response to common practices that could be described as patterns, had the pattern format been known back then. Take, for instance, what we now know as a procedure call: code to stack up register values and return address followed by a jump instruction. Before assemblers became available, programmers used to recode this every time they wanted a subroutine. Dominus sees this observation as a sign that something is wrong with the patterns movement:

"Identification of patterns is an important driver of progress in programming languages. As in all programming, the idea is to notice when the same solution is appearing repeatedly in different contexts and to understand the commonalities. This is admirable and valuable. The problem with the "Design Patterns" movement is the use to which the patterns are put afterward: programmers are trained to identify and apply the patterns when possible. Instead, the patterns should be used as signposts to the failures of the programming language. As in all programming, the identification of commonalities should be followed by an abstraction step in which the common parts are merged into a single solution."

Professor Ralph Johnson, one of the original "Gang of Four" members, responded:

"No matter how complicated your language will be, there will always be things that are not in the language. These things will have to be patterns. So, we can eliminate one set of patterns by moving them into the language, but then we'll just have to focus on other patterns. We don't know what patterns will be important 50 years from now, but it is a safe bet that programmers will still be using patterns of some sort."

Dominus retorts with subtler arguments, and I encourage you to read his article to avoid any misinterpretations of my part. But I'll still quote what I believe is the core of his thinking:

"What I imagine is that when pattern P applies to language L, then, to the extent that some programmer on some project finds themselves needing to use P in their project, the use of P indicates a deficiency in language L for that project.

The absence of a convenient and simple way to do P in language L is not always a problem. You might do a project in language L that does not require the use of pattern P. Then the problem does not manifest, and, whatever L's deficiencies might be for other projects, it is not deficient in that way for your project.

(...)

But to the extent that some deficiency does come up in your project, it is a problem, because you are implementing the same design over and over, the same arrangement of objects and classes, to accomplish the same purpose. If the language provided more support for solving this recurring design problem, you wouldn't need to use a "pattern". Consider again the example of the "subroutine" pattern in assembly language: don't you have anything better to do than redesign and re-implement the process of saving the register values in a stack frame, over and over?"

My take

Mark seems to be worried that programming language innovation will be stifled by the patterns movement, on account of programmers being taught to mimic patterns on their code instead of applying that energy on incorporating them into programming languages. It is important to first acknowledge, as himself put it, that it is valuable to identify commonalities in software design, one could even say this is a precondition for evolving a programming language. So, he sees the problem in not working to embody these commonalities in the language. From my exposure to the patterns literature, I believe the movement is not in any way against incorporating patterns as programming language features. It is implicit in this passage from GoF, page 4:

"Our patterns assume Smalltalk/C++-level language features, and that choice determines what can and cannot be implemented easily. If we assumed procedural languages, we might have included design patterns called "Inheritance," "Encapsulation," and "Polymorphism." similarly, some of our patterns are supported directly by the less common object-oriented languages. CLOS has multi-methods, for example, which lessen the need for a pattern such as Visitor"

And explicit in Ralph Johnson's aforementioned blog post: "Many of the patterns will get subsumed by future languages, but probably not all of them". Now that we know that design patterns are contextual and can be incorporated as language features, we must ask if the emphasis of the patterns movement isn't displaced. Why go to the trouble of publishing large books describing at length things like intent, motivation, participants, consequences, related patterns, etc, when we could just inventory all patterns and shove our favorites in our programming languages? One very simple reason is that not all patterns would yield benefit from being reified. Dominus says that people often mentioned to him MVC as an example of a pattern so complex that it could not be absorbed in a programming languge. But, of course, they were all wrong and he was right, since novel systems like Ruby on Rails or subtext do exactly that, definitively confirming his thesis that the only impediment was an atrocious lack of imagination on the part of his opponents! This is all very strange to me, since "MVC" came to life as pretty concrete software - it was the GUI development framework bundled with Smalltalk-80. Many years later, after several separate versions were developed, it was described and published as an (architectural) pattern. Leaving object orientation early history aside, lets get back to the matter of patterns that aren't targets for reification. One example is the Façade(185) design pattern. As one of the most frequently applied GoF patterns, described having as it's Intent to "provide a unified interface to a set of interfaces in a subsystem", there can be no doubt that it is indeed recurring. Looking at the Structure and Participants sections, we can see that it is very simple, just a Façade class that depends on many other (unidentified) subsystem classes inside a boundary. A consequence of this simplicity is that there would be little gain in incorporating this pattern into a programming language, as it doesn't really represent specific interactions that have to be recoded each time the pattern is applied.

One could respond that if that is the case, then the pattern is useless. It describes a solution so simple that a programmer who has never heard of the "Façade design pattern" would probably be able to construct it by himself. The error in this judgment is that a pattern's merit isn't measured by how interestingly or usefully it gives a solution to a problem. Knowing that its not hard to invent Façade does not detract from the fact that being able to talk about Façade is a big design win in itself. Patterns are as much an aid to communication as they are vehicles to disseminate knowledge.:

"Naming a pattern immediately increases our design vocabulary. It lets us design at a higher level of abstraction. Having a vocabulary for patterns lets us talk about them with our colleagues, in our documentation, and even to ourselves. It makes it easier to think about designs and to communicate them and their trade-offs to others."[GoF, page 3]

On the other hand, we already mentioned that some of the patterns are indeed amenable to reification. In this case there are two paths to choose from: coding it as a library atop usual language abstraction mechanisms or incorporating it as a basic language feature. Dominus second post argues that this choice is a mere implementation detail. Now, to paraphrase him, his point seems completely daft, which may I interpret to mean that there's something that went completely over my head. The difference is obvious: if some feature can be implemented as a library, any programmer that wants it can write it exactly one time and reuse as appropriate; no language modification required. As a contributor to CPAN , he should know well that nowadays much software reuse is done through open source libraries, lowering the recode count to zero for it's users. This is a crucial distinction, as adding something to the language has a huge impact on the whole technical ecosystem, from increased barrier to entry to pernicious interaction between features (non-orthogonality).

Lets take another look at Dominus' central proposition:

"(...) when pattern P applies to language L, then, to the extent that some programmer on some project finds themselves needing to use P in their project, the use of P indicates a deficiency in language L for that project. "

I believe the most important question here is: what does it mean to "use [pattern] P"? If you take it to mean follow the steps from P's description, possibly copying some code from the "Implementation" section, then this reasoning makes sense. But patterns are not merely recipes, and a pattern language is not a cookbook! One way in that patterns differ from recipes is that, as we saw in Façade, the Solution section can be of little importance compared to the communication gain from just naming the pattern. Another difference is that there is wide latitude in implementation strategies for each pattern, not to mention cases where multiple patterns present themselves as design alternatives for a set of forces.

Speculating a bit, I believe the origin of this antagonism toward design patterns is that the idea of a recurrent structure is antithetical to a certain ethos present in the software development community. Mark Dominus summarizes this feeling well in this passage "As in all programming, the identification of commonalities should be followed by an abstraction step in which the common parts are merged into a single solution." It can also be formulated as a call to arms: Don't repeat yourself! Or better yet, DRY! (the love for TLAs is another part of the programming ethos :). I don't dispute that eliminating repetition is an important design heuristic, but I sometimes wonder if it can be taken too far, becoming a sort of factorization fetishism that is detrimental to productivity.

Domain Specific Languages e Estratificação por Estabilidade

2007 mal começou e o meme das DSLs já está provocando polêmica. Eric Evans, em entrevista na InfoQ, disse:

"More out on the cutting-edge are the efforts in the area of domain-specific languages (DSLs), which I have long believed could be the next big step for DDD. To date, we still don't have a tool that really gives us what we need. But people are experimenting more than ever in this area, and that makes me hopeful."

Philip Calçado citou o Evans e foi além, afirmando que "DSLs são iminentes mas as ferramentas simplesmente ainda não chegaram lá". Escrevendo em tom mais cético, Rodrigo Kumpera comentou que DSLs ainda parecem uma tecnologia de um exemplo só; da mesma maneira que toda palestra de AOP fala de logging, toda introdução à DSLs fala em configuração. Citando:

"Todos estão falando o tempo todo sobre como configuração é um problema a ser resolvido por linguagens de domínio. Até o Martin Fowler comete essa gafe em sua palestra na JAOO 2006! O problema é o mesmo que logging, eu diria, representa nenhum risco a complexidade de um projeto. Quantos realmente já tiveram problemas relacionados a dificuldade de colocar logging em uma aplicação? Configuração idem."

Eu acho que a analogia com AOP não procede. Tracing/logging é de fato um caso de uso (quase trivial, BTW) para AOP. Precisamos analisar configuração com mais cuidado. Tome por exemplo uma aplicação de eCommerce B2C tradicional (você pode imaginar que é, quem sabe, uma loja de, sei lá.., animais de estimação). Na primeira iteração do software, cada item à venda tem um preço de catálogo cadastrado. Essa decisão logo se mostra inflexível, pois uma variação sazonal nos preços é comum no varejo. O requisito é atendido com uma nova feature permitindo que se estabeleça um perentual de desconto/acréscimo por mês para cada produto. A interface com o usuário para esta feature se localiza numa tela entitulada "Configurações". Mais adiante percebe-se que a flexibilidade ainda está aquém do ideal, os clientes querem definir regras de preço mais complicadas, como:

se vendeu mais de 500 na semana passada, acréscimo de 10%.

Pode-se inserir estas regras (hard-coded) no meio do sistema ou decidir por manter a parametrização externa e usar uma DSL. Em qualquer caso, ainda podemos chamar isto de configuração?
Sabendo que código e dados são basicamente a mesma coisa, eu vejo a aplicação de DSLs como decisões a respeito de que tipo de informação faz parte do sistema, e que tipo de informação consiste em parametrização "externa". No fim das contas, é técnica para atender ao "Stable Dependencies Principle (S0P)", deve-se isolar as porções de um software com maior volatilidade e uma linguagem específica - talvez até editável por usuários finais - é uma boa alternativa para implementar os módulos menos estáveis.

Componentes e objetos

Muito já se escreveu sobre a distinção entre programação orientada a objetos (OO), desenvolvimento baseado em componentes (CBD) e service oriented architectures (SOA). Isso significa que este post vai indubitavelmente chover no molhado. Entretanto, não se pode dizer que o assunto está resolvido e acabado; eu só espero confundir menos do que esclareço.

Como bem lembrou Philip Calçado, não podemos nos esquecer de que estes conceitos todos foram concebidos em contextos diferentes por grupos distintos e sem maiores preocupações com uma coesão formal. OO por exemplo, tem origem na Noruega, mas foi conceptualizada de maneira mais completa no laboratório da Xerox em Palo Alto, sendo reificada numa sequência de protótipos da linguagem Smalltalk. O líder desse projeto foi o lendário Alan Kay, que também tem a distinção de ter cunhado o termo "programação orientada a objetos". Kay, que na época já tinha um background em biologia e matemática, imaginou um paradigma computacional centrado em entidades similares a células biológicas, que se comunicavam trocando mensagens químicas. Apesar da paternidade reconhecida, não há consenso sobre o que seria a definição de OO, então trabalharei com uma definição improvisada (não confie muito). Um objeto é caracterizado por:

1. Comportamento. Ele é capaz de receber invocações* de outros objetos para efetuar uma computação e (a) responder com um outro objeto ou (b) alterar o seu estado. Para efetuar a computação, o objeto pode invocar outros objetos. Ele obtém acesso a estes objetos de três maneiras: criando - "instanciando" - novos objetos na hora, usando uma referência que é parte de seu estado ou recebendo-os no momento em que é invocado.
2. Estado. Um objeto pode armazenar um conjunto de referências para outros objetos. Através dessas referências ele pode invocar comportamento dos outros objetos conforme descrito no ítem anterior.
3. Identidade. Mesmo que o estado todo de um objeto se altere, ele ainda existe como a "mesma" entidade.
4. Ciclo de vida. Um objeto é instanciado em algum momento, quando ele passa a existir (manter uma identidade). Ele também pode ser destruído em algum momento. As linguagens de programação que seguem o paradigma OO apresentam uma grande variedade de mecanismos para instanciar e destruir objetos (classes, protótipos, prefixos, gerenciamento manual de memória, coletor de lixo, ...).

Enquanto que objetos são entidades fundamentais de um modelo computacional, componentes são apenas uma alternativa para atender a certos requisitos encontrados em algumas aplicações. Essa afirmação pode parecer polêmica, por isso apelo para uma autoridade maior, Ralph "GoF" Johnson, que expôs bem essa consideração num post recente em seu blog:

It is important to realize the "components" are not technology, they are a customer requirement. Customers want to add on to their software system by adding a new component, and adding this component must not require that they change their existing system. It is like adding new speakers to a stereo system.

Ou seja, componentes são nada mais que uma unidade de extensão e/ou substituição dinâmica de um sistema de software. O adjetivo "dinâmico" é uma daquelas palavras que são jogadas por aí em contextos tão diversos a ponto de perder o seu significado, mas a noção aqui é bem simples: estamos considerando alterações realizadas em ambiente de produção, feitas por operadores do sistema. A partir dessa caracterização, podemos dezfazer uma série de confusões:

• Uma aplicação pode usar componentes sem ser baseada em componentes. Isto é, é possível definir alguns poucos pontos de substituição e/ou extensão mantendo o resto do software completamente estático. Dentre os muitos exemplos estão a maioria dos programas de desktop; um exemplar típico é o Photoshop e seus plugins. É claro que também encontramos diversos casos no extremo oposto - aplicações totalmente baseadas em componentes - veja por exemplo os últimos servidores de aplicações e IDEs.
• Um sistema pode ser modular mas não baseado em componentes, na acepção que estamos considerando. Basta que seja composto de partes pouco acopladas.
• Não faz muito sentido projetar um sistema baseado em componentes fortemente acoplados. O motivo é simples: componentes fortemente acoplados exigem que um desenvolvedor independendente se preocupe com grandes porções do sistema, o que prejudica o atendimento ao requisito de reconfigurabilidade. Mesmo no caso de aplicações que limitam seu uso de componentes a extensões localizadas, é uma boa prática estabelecer uma interface reduzida entre o núcleo do sistema e os componentes.
  
• A definição sugerida exige apenas que um sistema possa ser reconfigurado em ambiente de produção, não é necessário que ele tenha a capacidade de sofrer alterações enquanto está executando. Não obstante, existem algumas aplicações onde isto é um requisito real e soluções tecnológicas sofisticadas são necessárias.
• O professor Johnson deixa bem claro que tecnologias como COM ou EJB não são necessárias para um sistema que use componentes. A rigor, para antender ao requisito delineado é suficiente um suporte tecnológico mínimo. Podemos imaginar até um sistema onde a escolha de componentes é feita através da substituição de arquivos objeto (em java seria um .class) no diretório de instalação.
• Apesar do ponto anterior, é frequente que um sistema com componentes se beneficie de uma infra-estrutura básica para auxiliar na solução de problemas comuns como: controle do ciclo de vida, gerenciamento de dependências, suporte à modos de comunicação entre componentes, parametrização local (propriedades), etc.

Como já demorei muito mais do que imaginava para escrever este post - incluíndo um tempo desperdiçado reescrevendo uma boa porção do texto quando eu fiz besteira e perdi o draft - vou deixar para uma entrada futura a discussão sobre sistemas distribuídos e serviços.

Uma última observação: o leitor atento notará que a palavra reuso não aparece nesse post; isso não foi um acidente.

* Estou evitando a terminologia baseada em "mensagens" que é padrão em Smalltalk para evitar mais confusão.

Model View Confusion

The development community is a strange beast. And I'm not talking about hitchhiker's jokes and D&D tournaments. The strangeness I'm referring to is that many of the most talked about concepts are used in ways that differ a lot from their original, intended, meanings. Many patterns fall in this category, but one is specially abused: MVC. This is one of the first TLA's a java developer learns on his first days in the craft. By "learn", I really mean "kinda sorta knows that it has something to do with that Struts crap".

I was in that exact position a few years ago (crazy coincidence, innit?). After a few hours of wikistumbling through c2.com, I got considerably more confused. The next step I took was to chase down a copy of the first published description. Google wasn't my friend in this process* and I had to resort to the old-fashioned route of requesting a loan from some - the only one, in fact - college library that had the aug/1988 copy of the Journal of Object Oriented Programming. I only half-understood it because at the time I wasn't conversant in Smalltalk. But I undestood enough to see that MVC was somewhat different from what the Java guys talked about.

One big difference is that Smalltalk-80 was self-hosted - it was it's own operating system**. This meant that input device processing was a responsability of objects. And that, my little friends, is how the Controller came to be. Quoting the paper:

In particular, controllers coordinate the models and views with the input devices and handles scheduling tasks. (...) Class Controller does include default scheduling behaviour...

What motivated me to write this entry was Fowler's superb article on GUI architectures. In there you'll find a thorough discussion on MVC and its subsequent variants. It will be a chapter on the upcoming second volume of his Patterns of Enterprise Application Architecture.




* Of course, now the bastard is online here.

** This control obsession can be seen still today, just look at Squeak's UI and fondly remember the glorious Win3.1 days...

Don't forget to use your brain

Keith Braithwaite, one of the postmodern programming guys, wrote this interesting blog post. I tried to comment on the post page but, sadly, my wise and deep words didn't find it's way to the world wide web. So, in the immortal words of the prophet, I'm just gonna have to wing it here.

A reasonable amount of bytes and brain cells have been lost, over the years, on the object-relational mapping battleground. Keith's argument is that many of those casualties could be prevented if developers let go of the RDMS dogma and adopt simpler solutions when possible. If the requirements don't indicate a heavy load of concurrent updates, then there is no need for powerful (and expensive) transactional capabilities. He gives as an example an online shopping web site. Usually, additions and changes to the product catalog don't need to be immediately reflected to the users. Also, the possibility of simultaneous modifications can be negligible here. He proposes* that, in cases like this, the data can be made available in a simple format locally, on the front-end servers. Updates can be periodically pushed from a central database to the front-ends.

The only nit I have to pick with his post is when he talks about queries:

"For instance, if you want to get hold of an object that (you have to know) happens to be backed by the underlying database then what you do is obtain an entry point to the data, and search over it for the object you want. Unlike the navigation that you might expect to do in an object graph, instead you build...queries. (...)"

Yes, building a whole query just to get hold of one object reference is too much trouble and a violation of DRY (parenthesis: this sort of thing eases somewhat this pain, minus all the factories and spurious abstractions). But I think he overlooks a bit the fact that often we really need to do a query. It's frequently part of the problem domain, not the solution domain, to put it in other (more pompous) words.

I view queries as inherently declarative operations (given x information, get me Y more information). As such, they are better expressed through declarative means. So, aCollection select: aBlock in Smalltalk is better than the equivalent for loop in Java. Still in OOland, Evans and Fowler's Specification pattern is even better for more complex cases. Advancing to the logical conclusion, a language specifically designed for searching would be even better. Unfortunatly, SQL falls short of reaching this goal in practice, because of the mess that is integrating it with the rest of the application. Microsoft's linq project is an intriguing technology in this space.

Anyway, what I wanted to point out is that there is no one size fits all software architecture and every project needs to be thought out** by a team who knows what it's doing and isn't afraid to think outside the vendor-supplied box.


* barring possible misunderstandings of my part.
** This is not a defense of BDUF

Clássicos

Minha mais nova tentativa de tentar transformar esta porta de banheiro binária em algo minimamente útil será uma série de comentários breves sobre artigos clássicos na área da computação. O gerador de clichês dentro do meu cérebro, condicionado por milhares de horas de hipnóticas aulas de redação, não me deixa continuar este post sem explicar o que entendo por "artigos clássicos", então vamos tentar uma definição por oposição: não são selecionados de acordo com qualquer métrica acadêmica; não são os 'melhores' textos, apenas os mais importantes dentre os pouquíssimos que lí; não busco cobrir nenhum escopo prévio, especialmente não tenho ilusões sobre cobrir o todo da ciência da computação; e nem todo clássico é velho (pelo menos aqui...).

Para facilitar, vou listar aqui os links para os posts nesta série:

• End-to-end Arguments in System Design, por Reed, Saltzer e Clark,
• Growing a Language, por Guy Steele,
  
• No Silver Bullet, por Fred Brooks.
  

Começo pelo "End-to-end arguments in system design", artigo de Reed, Saltzer e Clark, todos professores do MIT, publicado inicialmente em 1981. A tese principal do artigo é que em sistemas de software em rede arquitetados em camadas, adicionar funções nas camadas inferiores pode trazer pouco ganho ou até se revelar prejudicial. Isso porque a camada de mais alto nível tem mais informações sobre o problema em questão, às vezes tanto que ela precisa implementar completamente a funcão independente dos esforços dos níveis inferiores. Essa discussão está muito teórica e convido o leitor a read the f*cking paper para aproveitar os exemplos catalogados pela turma do Reed.

Numa primeira leitura, o artigo parece tratar de uma consideração arquitetural relevante, mas não mais importante que as tantas outras restrições que tiram o sono de desenvolvedores há decadas. O que torna o argumento dos professores especialmente importante é que entre suas implicações econômicas encontramos (suspense...) a Internet. Antes que alguém dispare o seu cliente de email na direção de um inocente, aviso que, sim, eu sei que a internet e o TCP/IP já existiam há um bom tempinho quando os nobres professores escreveram o paper, mas seu objetivo é descrever um fenômeno observado, não apresentar nenhuma nova e mirabolante idéia. Aliás, a maioria dos artigos nesta série não versam sobre novidades e invenções. Do que eu estava falando mesmo?

Pop(), pop(). Ok, a Internet. Pergunte a qualquer engenheiro de comunicações o que ele acha sobre a estrutura da internet e ele contentemente respoderá que é um negócio que até funciona mas poderia ser muito melhor. Ela não contempla, por exemplo, segurança (ip-sec?) e garantia de qualidade de transmissão (QoS? RSVP? ATM?). É, em outras palavras, uma rede bastante burra. Entretanto, é inegável que modificou violentamente a economia e a sociedade na última década (para o bem e para o mal). Imagine um cenário alternativo onde QoS fosse implementado de ponta a ponta na internet, permitindo que se reservem faixas de banda para determinadas aplicações. Finalmente video-on-demand seria uma realidade! O preço a pagar é que a rede deixaria de ser um campo neutro onde empresas florescem a todo tempo (que meigo) para se tornar um mercado imobiliário onde quem pode pagar mais por naco de banda excluí a priori qualquer competidor potencial. Mais sobre esse assunto dos próprios autores do clássico ou, mais eloquentemente, do senhor Lessig.

Esse último texto foi apresentado ao congresso americano pedindo legislação que impeça o comprometimento da isonomia entre aplicações, contrariando os planos das grandes empresas telefônicas. Donas dos backbones da Internet, elas ameaçam cobrar para dar preferência a determinados serviços em detrimento de outros nas "suas" redes. Confirmando sua vocação para clássico, vemos que as observações do "End-to-end arguments in system design" continuam, mais que atuais, prementes.