Recipes to Reuse Thomas K*uhne *kuehne*isa*informatik*th*darmstadt*de* Department of Computer Science*TU Darmstadt Magdalenenstr***c* D****** Darmstadt Abstract We propose to use objects as closures for behavior parameterization*In contrast to reuse by inherit ance*they realize reuse by composition*Closures allow black*box behavior parameterization with en capsulated components*promote function reuse*al low calculations on demand*represent *rst*class behavior*i*e** feature protocol** undo** and per sistence mechanisms*and can represent *business transactions**In this paper*we present closures as the object*oriented design pattern Recipe*In con trast to the Command pattern*Recipe establishes a useful collaboration with iterators*We show in particular how to use generic recipes with iterators in order to allow multi*dispatching operations on heterogeneous data structures* *Introduction Smalltalk programmers use closures every day* They are used to parameterize behavior*For in stance*the action to be performed during an it eration on a collection*selements is passed to the iteration method as a closure* This paper explains the concept of closures and makes it accessible to any object*oriented language by describing it as the design pattern *** *Recipe* In Smalltalk*closures are called blocks *** ** A block allows to inject behavior into an object* When a block is received and subsequently executed by an object*we can think of it as executing a new method in the object*There are vital di*erences* though*First*in contrast to a constant method*we can even inject freshly created blocks*This allows us to exchange and extend object behavior at run time*Second*encapsulation is fully preserved*So* blocks can use the public object interface only*but both object and block are protected against imple mentation changes* Recent Smalltalk implementations implement blocks as lexical closures*What does this mean* A closure may refer to variables which are neither declared as parameters nor as local variables*These are called free variables*A lexical closure binds the values of these variables in its creation environment* What does this static binding buy us*It enables us to not only inject behavior into objects*but also data*Hence*the closure code can operate on two data spaces while these remain completely isolated from each other* Up to now*we described closures with the same functionality as available in functional lan guages *** ** There is more to object*oriented clos ures*Imperative closures can produce side*e*ect s and carry state*As a result*they can be used for parameterization of side*e*ect s and to accumulate results internally* The pattern description in section *will give applications for the remarkable properties listed above*While we use behavior re*nement with in heritance as a negative pattern to motivate Recipes in section **** parameterization is clearly not the only problem solved by Recipes*We illustrate fur ther applicabilities and consequences with examples of their own*Section *summarizes Recipes as a powerful abstraction for software reuse* *Design Pattern*Recipe *** Intent Encapsulate a procedure or function with an object* This is useful for parameterization of algorithms* partial parameterization of functions*lazy evalu ation*lifting methods to *rst*class citizens* and for separating functions from data* * *** Also Known As Lexical Closure ****Functor ** ** Agent *** ** Agent*Ob ject *** ** Functionoid ** ** Functoid *** ** Function*Ob ject *** * ** ** *** Motivation Behavior parameterization occurs in almost every program*Iterators are a good example*Consider a collection of books*We might be interested in *whether a given title is in the collection* *a list of all books written by one author* *the date of the oldest book*etc* All these operations need to traverse the collection structure*In fact*all operations can use the same traversal algorithm*The operations di*er* however* in the test they apply to books and the sort of res ult they produce*Thus we decompose the opera tions into a common traversal algorithm and a set of functions that perform tests and produce results* Concerning the traversal algorithm*we may provide an abstraction called external iterator*It will yield the elements of the collection one by one so we have to write a loop in order to access all elements*Yet*we expect to use the traversal al gorithm many times*which will result in many ex plicitly written loops*Moreover*writing the loops is error*prone*since it is easy to use an incorrect exit condition or to forget to step to the next ele ment *see also *Write a Loop Once**** *and the discussion in the Iterator pattern *** *** Consequently* we provide one more level of ab straction with an internal iterator*Given a func tion*it will apply the function to all elements in the collection* Now*how do we compose an internal iterator and a function to be applied*We also want the books to be sortable according to various criteria*like author* title*and date and we want to choose the criterion at run time*In order to avoid switch statements we may use dynamic binding of iterators*The actual iteration method of an ITERATOR class is a Template Method *** ** which depends on an abstract func tion method*The implementation for the abstract function method*and thus the speci*cfunction to be applied to the elements*is given in descendants of ITERATOR *** ****** ** Composing traversal al gorithm and functions then works through dynamic binding of the abstract function method*Selecting one of several functions corresponds to selecting one of the existing ITERATOR subclasses* In this case*however*the application of an object*oriented design*using inheritance and dy namic binding*has some severe drawbacks* *Static combination*All possible combinations of iteration schemes and functions are *xed at compile time*Neither is it possible to create a new function at run time* *Combinatorial explosion*Sometimes it is use ful to select not just one*but a combination of functions or tests and functions*With sub classing*it is not feasible to provide any inde pendent combination*since it leads to an ex ponentially growing number of subclasses* *Subclass proliferation*Each new function de mands a new subclass of ITERATOR*The name space for classes is cluttered by many concrete ITERATOR subclasses*We may combine all functions in one subclass using repeated inher itance *** ** but this only makes things worse* First* it is non*local design to lump all func tions in one class*Second*we have to apply heavy renaming for iteration schemes and func tions in the subclass*any combination of itera tion scheme and function must be given a dis tinct name*Third*we lose the ability to use dynamic binding for the selection of a function* Since all functions belong to one class*we no longer can uniformly send a message to a vari able of type abstract-ITERATOR and use con crete ITERATOR instances to select the actual combination of iteration and function* *Awkward Reuse*Reusing the functions for other iteration schemes or di*erent purposes is practically impossible if they are de*ned in ITERATOR subclasses*The solution is to ex tract the functions in classes of their own*But now multiple inheritance is necessary in order to inherit from ITERATOR and to inherit from a particular function*At least multiple tests or functions can be *mixed*in** but scope resolu tion is needed* and each function combination results in a combinator subclass* * *Poor encapsulation*Composing an iteration scheme and a function with inheritance joins the name spaces of both*In fact*the multiple inheritance solution causes iterator*function* and combinator class to share the same name space*Implementation changes to either of the classes can easily invalidate the other*An in terface between super- and subclasses*as the private parts in C ** *** **alleviates the prob lem considerably* *Unrestricted *exibility* Creating a designated class for the combination of an iteration scheme and a function opens up the possibility of over riding the iteration scheme for particular ac tions*An iteration used to count the elements in a collection could be replaced by just return ing the value of an attribute count* Unfortunately*this advantage for the designer of a library is a disadvantage for the user of a library*The user may rely on properties of the original iteration scheme*If the iteration func tion not only counts the elements*but in addi tion produces some side*e*ect* the side*e*ect s will not be executed in the optimized version described above*Pre- and postconditions *** * can help to enforce behavioral identity between iteration schemes*but problems like the above are hard to cover and checking contracts at run time boils down to testing*as opposed to rig orous proo*ng* *Identity changes*In order to change the itera tion function a di*erent iterator instance must be used*While one would seldom need to rely on an unchanging iterator instance*this prop erty is inhibiting in other settings of parameter ization*For instance*it might be mandatory to keep the same instance of a cook*while being able to process di*erent recipes* *Obligatory Source*code*Some languages en force iteration classes to be available as source code*Otherwise it is not possible to derive sub classes*Hence*vendors do not have the option to sell pre*compiled library code only* We can get rid of all these disadvantages if we sacri*cethe ability to adapt iteration schemes for particular functions*We accomplish this by using higher*order functions*well*known from functional programming ***and as blocks from Smalltalk*In stead of subclassing the iterator*we pass a function to be applied to the elements*We pass a Recipe that describes what to do with the elements*Instead of an *is*a** we establish a *uses*a* relationship*Ac tually*we call this special usage of a parameter for behavior parameterization *takes*a* *** ** Since we usually can not pass methods as functions and many object*oriented languages *e*g** C ** * Ei*el* do not feature blocks*we pass an object representing the function ** *** ** ** The object*sinterface makes it possible to receive arguments and to return a res ult*As the object represents a way of doing things we call it Recipe*Technically Recipes are closures* i*e** they can store any parameters or variables from their creation environment in order to use them even beyond the existence of environments* Why is it useful to pass closures instead of plain functions*i*e** what is the use of being able to re member the value of free variables*Remember that we also want to sort books according to titles*We certainly need a function that compares the titles of two books*Now we can partially apply this function to one book we look for*We thus create a Recipe that only takes one more book and produces a cer tain result*depending on whether or not the titles match*Given this function and an iterator*we can realize the *rstlibrary operation mentioned at the beginning of this section* This is quite pleasing*since we reused a sort ing predicate for a totally di*erent purpose*The iteration does not have to worry about the number of parameters of recipes*For instance*in order to collect a list of books whose dates are in a given interval*we can use a function with three paramet ers*The *rsttwo specify the interval*the third is a book*and the function checks whether the book*s date is in the interval*When the interval data is supplied in advance*we can use the resulting recipe for a standard library iteration* *** Applicability Recipes are free functions*They are not members of a particular data abstraction*They resemble so*called design*objects *** ** like iterator objects* event handler*and Commands*which similarly con stitute entities of their own right*The separation of functions from data can be bene*cialin several ways* * *Parameterization*Recipes are a good candid ate whenever general behavior can be adapted to special behavior*Use Recipe if one of the following aspects is desirable* Dynamics*In addition to run time selection of existing Recipes*new Recipes can also be created at run time*A user may dynamic ally compose a multi*media Recipe from text** graphic** and sound*producing Recipes* Orthogonality*Having more than one behavior parameter creates the problem of handling all possible combinations of the individual cases* Recipes can freely be mixed without interfering and without the need for combinator classes* Reuse*Recipes can be used by any adaptable algorithm that knows their interface*Even if the algorithm was not designed to supply the Recipe with necessary parameters*it is often possible to supply them to the Recipe in ad vance*For instance*consider an error reporter* parameterized by output format Recipes*only intended for generating text messages*We can upgrade the reporter to create a graphical alert*box by passing a Recipe that already re ceived information about box*size*colors*etc* Identity*When the behavior of an object should change while keeping its identity*Re cipes can be used as behavior parameters to the object*In contrast*encoding behavior in subclasses calls for something like Smalltalk*s *become**method in order to achieve the same e*ect* *Uniform invocation*Imposing a Recipe*sin terface on related operations makes it possible to uniformly invoke them*Instead of switching to di*erent method names *e*g** editor com mands** we invoke evaluate on an abstract Recipe and rely on dynamic binding *** *** ** Consequently* when we add new operations* we do not need to change the event handler*If spe ci*cevaluation names *e*g** execute*evaluate* solve*are considered important*then they can be provided as aliases* *First*Classmethods*Recipes make methods amenable to persistent command logging*com mand histories for undoing*network distri bution of commands*etc*Like Commands* Recipes can provide an undo method*which will use information in the Recipe*sstate to undo operations *** ***** A perfect candid ate for *rst*class methods are so*called *busi ness transactions* ** ** Often the functions are the stable concepts of a system and repres ent good maintainance spots*in order to cope with changing functionality*Instead of being a well*de*ned operation on one single object* transactions are *anorchestration of objects working together toward a common goal***** When transactions do not naturally *tinto ex isting data abstractions*Recipes can lift them to *rst*class status while providing a uniform interface* *Monolithic Algorithms*When a data struc ture is stable*but the operations on it often change*it is not a good idea to use the standard object*oriented method to distribute the oper ations over the object types involved*For in stance*each change or addition of an algorithm on abstract syntax trees *such as typeCheck* compile*demands a change in all node*object types*In addition*it is not possible to ex change the operations at run time* If we turn the algorithm into a Recipe*we must use a generic Recipe *see subsection Multi dispatch*to dispatch on the node types*but in analogy to the Strategy pattern *** ** * the algorithm logic is localized* *Recipe*sstate can accumulate results*and *we can dynamically choose an algorithm* *Small interfaces*When an object potentially supports many extrinsic operations *e*g** CAD objects may support di*erent viewing meth ods*cost calculations*etc*** but its interface preferably should contain the basic* intrinsic functions only *e*g** geometric data** then the functionality can be implemented in Recipes that take the object as an argument* *Method simpli*cation*If a large method*con taining many lines of code*can not be split into smaller*more simple methods*because the code heavily uses temporary variables for communication*then the method can be trans formed into a Recipe*The main transforma tion is to replace the temporary variables with * Recipe attributes*As a result*the method can be split up into more manageable sub methods*without passing parameters between inter*method invocations*since communication still can take place via Recipe attributes*The main computation method *e*g** evaluate* simply puts the pieces together*itself being as clear as documentation **** *Call*by*ne ed Semantics* One aspect of call*by need is to calculate a result only once*no mat ter how many times the calculation is reques ted*Recipes can do this*but class methods can also perform this using a technique called memoization*Class methods*however*can not realize the second aspect of call*by*need which is to postpone a calculation until the result is actually needed*If the result is never needed* this pays o*in run time e*ciency*A Recipe represents a calculation that is performed only if someone is in need of the result* Note that lazy evaluation enables in*nite data structures and supports modularization by de coupling data generation from data consump tion *** ** *Multi*dispatch*Sometimes an operation de pends on more than one argument type*For instance*adding two numbers works di*er ently for various pairs of integers*reals*and complex numbers*Simulating multi*dispatch with standard single dispatch *** ** where only the type of the receiver objects is taken into account*results in many additional methods *like add Integer*add real** The dispatch ing code is thus distributed over all involved classes*If*as in the above example*the opera tion must cope with a symmetric type relation *e*g** real*int *int*real** each class has to know all other argument types*As a result* the classes involved are unnecessarily coupled* A generic * Recipe removes the dispatching code from the argument types and concentrates it in one place*It uses run time type identi *cation to select the correct code for a given combination of argument types*As such*it is not simply an overloaded Recipe*which would statically resolve the types* * Named after CLOS***** generic functions* Unfortunately*the necessary switch statements on argument types are sensitive to the in troduction of new types * *Yet*in the case of single*dispatch simulation*new dispatching methods *e*g** add complex*are necessary as well* Generic Recipes may use coercions to reduce the possible number of combinations ** ** and employ partial parameterization to avoid nes ted type switches*Upon receipt of an argu ment*the generic Recipe uses one type switch statement to create a corresponding new gen eric Recipe that will handle the rest of the ar guments* *** Structure Client Invoker state Application(argument) Application(argument) ConcreteRecipe Recipe *** Participants *Recipe *declares an interface for application* *ConcreteRecipe *e*g** *printBook** * implements a procedure or function* *carries state for free variables and results* *Client *e*g** Application* *creates a ConcreteRecipe* *possibly applies it to arguments* *calls an invoker method*passing Con creteRecipe* *Invoker *e*g** Iterator* *applies a ConcreteRecipe to more argu ments or simply evaluates it* * A more *exibleapproach is to use dynamically extend ible dictionaries that associate types with code* * *** Collaborations *A client creates a Recipe*Free variables are bound in the client*senvironment* *An invoker takes the Recipe as a parameter* *The invoker applies the Recipe to arguments* *The invoker gets a result from the Recipe and both invoker and client may optionally request the Recipe for further *accumulated*results* aClient aRecipe anInvoker Application(arg) Application Parameterization Creation Imperative Result new Recipe(free variables) InvokerMethod(aRecipe) getOptionalResult() *** Consequences *Abstraction*Recipes abstract from func tion pointers and in particular from point ers to class methods*Instead of the C ** code*aFilter*** aFilter*current**t** we can write aFilter*t* ** ** * Simplicity* Recipes do not introduce inherit ance relationships and do not create spurious combinator classes* *Explicitness*The code cook*prepare*fish* is easy to understand*When Recipes are wired into COOK subclasses*cook*prepare depends on the actual cook type*Clever variable names *e*g** fish cook*prepare*often are not an op tion*e*g** cook*prepare*fish** followed by cook*prepare*desert** * Compositionality*As Macro*Commands *** ** Recipes can be dynamically composed to form a sequence of actions*Unlike Macro Commands*Recipes may form a calculation pipeline by forwarding intermediate results to the next processing Recipe*A composite Re cipe can also apply several component Recipes in parallel*producing multiple results at once* *Encapsulation*As Recipes establish client re lationships only*they are protected from im plementation changes to algorithms that use them*Likewise*the implementation of Recipes can change without invalidating the algorithms* Hence*Recipes allow so*called black*box re use *** *and help to advance reuse by inher itance to reuse by composition *** ** * Security* A client of an adaptable algorithm can be sure not to change the algorithm se mantics*It is impossible to be given an optim ized version which does not fully comply to the original semantics *see section ***** *Flexibility* *A statement like iterator*map*c*is polymorphic in three ways* ** iterator may internally reference any data structure that conforms to a particular interface* ** The actual instance of iterator de termines the iteration strategy *e*g** pre- or post*order traversal on trees** ** The actual instance of Recipe c de termines the iteration function* *It is not possible to automatically optim ize algorithms for speci*cfunctions*Nev ertheless*one more level of indirection can explicitly construct combinations of func tions and optimized algorithms* *When a function has been extracted from a data structure*it is no longer possible to simply rede*ne it in future derivations* One way to account for this is to make the extracted function a generic Recipe that will discriminate between data structure variations* *Separation*Partial parameterization allows to separate concerns between arguments* *The ability to carry data *e*g** alert box size*enables Recipes to operate on data from outside an *error*rep orter*al gorithm as well as *errortext*data from inside the algorithm*The data from out side the algorithm can be local to the al gorithm caller*There is no need to com municate via global data*Since Recipes can combine local data spaces*they allow decoupling data spaces from each other while still supporting communication* * *Generic Recipes separate type dispatch ing code from data*Even di*erent aspects of dispatching can be separated*For in stance*a state transition function*applied to a state*will yield a function that maps inputs to new states *the principle of pat tern State *** *** Type and*orvalue of a state can be used to decide between res ulting mappings* *Reuse*Recipe*simpact on reuse is twofold* First*adaptable algorithms become more re usable because they do not need to know about additional parameters for Recipes* Second*Recipes are multi*purpose* *Recipes are not bound to a particular ad aptable algorithm*e*g** comparing book titles is useful for sorting and for member ship testing in collections* *Since Recipes are so easy to compose*it is feasible to form useful composite Re cipes*For instance*we may combine a test- with an action*Recipe*in order to perform conditional actions on elements of a collection*Then*we do not necessar ily need to provide special iterations as *do if***** ** * One Recipe with n parameters actually represents n Recipes and one value*The *rstRecipe has n parameters*The second*created by applying the *rstto an argument*has n **parameters*and so on*until the last Recipe is applied to an argument and produces the result* An example from physics shows the useful functions which can be created from the gravitational force function *** ** Grav law m * r m * * G m * m * r * force earth *Grav law mass earth force surface *force earth radius earth force my *force surface mass my *Iteration*Recipes suggest the use of internal* rather than external*iterators*Internal iterat ors avoid explicit state and avoid reoccurring explicit control loops*Often external iterators are promoted to be more *exible* It is said to be practically impossible to compare two data structures with an internal iterator *** ** However* we simply propose to extend an iter ator to accept not just one*but n data struc tures*A transfold * *method may access the *rst* second* etc** elements of all data struc tures simultaneously*When used to compare data structures*it can stop the iteration as soon as a mismatch has been found* Since Recipes do not rely on inheritance*it is possible to make iteration a method of data structures*This facilitates the use of iterat ors and allows to rede*ne iteration algorithms for special data structures*Moreover*the data structure *e*g** DICTIONARY*then does not need to export methods *e*g** first*next*in order to allow iterators to access its elements* *E*ciency* * A Recipe may calculate partial results from arguments and pass these to a res ult Recipe*Hence*the partial result is computed only once*no matter how many times the resulting Recipe will be ap plied to di*erent arguments in the future* e*g** force surface * force earth costly calc* and then force my * force surface mass my * force your *force surface mass your * *Passing client parameters to Recipes can be more ine*cient than to*e*g** directly access internal attributes of an iterator superclass*In principle this could be tackled by compiler optimizations *** ** * An overhead exists in creating and call ing a Recipe*instead of just invoking a method*This suggests only to use Re cipes when truly needed* *Care should be taken not to unnecessar ily keep references to unevaluated calcu lations*i*e** Recipes*Otherwise*the oc cupied space can not be reclaimed* *Finally*Recipes access the public inter face of their servers only*This represents positive decoupling*but can be more inef *cient than unrestricted access*However* selective export *Ei*el* or friends *C** ** allow to trade in e*ciencyfor safety* * Its functional de*nition shall be*transfold f a g * *foldr f a***map g**trans * *** Implementation *C ** allows to overload the **** operator* which gives a nice syntax for Recipe applic ation ****Ei*elo*ersthe in*xoperator *** * *In order to provide true static binding*Recipes must copy their arguments*Otherwise*their behavior will depend on side*e*ects on their arguments*In some cases*however*this may be desirable*The Recipe then plays the role of a future variable*which is passed to a client before all data needed to compute the result is available*Long after the Recipe has been passed to the client*it can be supplied with the necessary arguments by producing side*e*ects on bound values* *How shall the initially free variables of a Recipe be bound*When closures are emulated with objects*the free variables can not be bound im plicitly at the place of creation as usual*One way out is to forbid free variables and demand that they be parameters*This results in a uni form treatment of parameters and free vari ables*Alternatively*one can tie the binding of the free variables to the creation of the Re cipe *e*g** C ** constructor or Ei*elcreation method**This saves implementing the inter mediate classes*which may result from partial application of free variables*Of course*both variants can coexist* Note that explicit binding is equivalent to im plicit binding*In fact*explicit binding does not force the Recipe to use the same names for free variables*as the environment prescribes* Ergo*a Recipe can be used in various environ ments without the need to make the free vari ables names match the environment* *In addition to standard application*Recipes may provide keyword parameters*Accord ingly*parameters can be passed in any order* This makes sense*in order to create useful ab stractions*If the de*nition of the gravitational force in the example of section *** had been Grav law m * m * r * G m * m * r * **note the dif ferent order of parameters*we can not de*ne* force surface *Grav law mass earth radius earth * With keyword parameters we can write* f**Grav law*m**e- mass**r* e-radius** * Commonly a function is evaluated after it has received its last argument*Yet*Recipe applic ation and evaluation can be separated by cor responding methods apply and evaluate*As a result*the client*invoking evaluate*does not have to know about the last argument*In addition*the supplier of the last argument does not need to enforce the calculation of the result* which is crucial for lazy evaluation* In order to enable automatic evaluation on full argument supply while still supporting the above separation*it is possible to evaluate on the last parameter and allow a kind of dummy parameter *called unit in ML *** *** * Recipes do not break the encapsulation of the objects they are passed to*If a particular Re cipe is closely coupled to a speci*cobject type* then declaring the Recipe as the object*sfriend* i*e** using selective export* allows e*cient ac cess to internal data nevertheless*As a result the object can keep its public interface to other clients to a minimum* *Two extremes to implement partial paramet erization exist*The less verbose is to always keep the same instance of Recipe and assign incoming arguments to corresponding internal attributes*This will cause trouble when the same Recipe is used by several clients*As the application of a Recipe does not create a new instance*the clients will get confused at the shared state*Furthermore*static typing be comes impossible* If each application of a Recipe produces a new instance of a di*erent type*then static typing is enabled again and no unwanted sharing of Re cipe state can occur*Unfortunately*this forces us to write at least n **classes for a Recipe with n parameters* *Generic Recipes must use run time type iden ti*cation to deduce the type of actual ar guments*There are many ways to account for this*type*casestatement *Sather *** *** typeid *C ** * ***speci*ca tion**reverseas signment attempt *Ei*el *** *** *How shall we accumulate the results of an it eration*A functional programmer would im plement all of the three library operations *see * section **** as a fold over a list of books*The result type of fold is determined by the function supplied*It will return whatever the function returns *e*g*