EclipseZone - When is a closure not a closure? ...

The Rich Engineering Heritage Behind Dependency Injection

Andrew McVeigh takes us on a tour of the rich heritage behind dependency injection, what it represents, and tells us why its here to stay.

Java, the OLPC, and community responsibility

The "One Laptop Per Child" project has a great device ready to ship, but there's no Java on there. Let's think about working together to put Java on OLPC!

Replies: 52 - Pages: 4 [ 1 2 3 4 | Next ]
Threads: [ Previous | Next ]

Reply 1. At 6:48 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Guido van Rossum, creator of Python , wrote:

Have you tried looking it up on wikipedia?

Python is considered by most to support closures ever since we started
supporting access to an outer function's locals from within a nested
function. Anonymous blocks are a separate concept (though they would
be pretty useless without closures).

Alex: Note that there's a distinction between 'supporting closures' and 'a lambda expression is a closure' to be made here, but it's not clear from the answer, unfortunately. And anonymous blocks without closures can be useful; for example, to pass an in-line 'comparator' like function to sort members of an array. Having said that, in Python, it's just as easy to define an ordinary function to do that anyway, which I believe was the point.
Reply 2. At 6:52 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Yukihiro Matsumoto, creator of the Ruby language , wrote:

Hmm, I use the term 'closure' for anonymous code block with the
ABILITY to capture the outer local variables. In mathematical sense,
the code block without capturing any variables are _not_ closure, but
that distinction is not practical in programming languages. Calling
Python lambda prior to 2.0 as closure is plain wrong, since they
cannot capture variables. But I'm not sure that calling code block
without reference to outer variables closure is wrong or not, when the
language provides the ability to create closures.

Alex: Note that Matz appreciates the difference between a code block, and a closure obtained by binding/capturing local variables inside a code block at runtime. Also note the common use of the phrase 'lambda' as used by Python to refer to such anonymous code blocks. In fact, he answers this question in a separate Artima interview .
Reply 3. At 6:56 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Ehud Lamm, contact for Lambda the Ultimate (a website devoted to research and implementation of programming languages), wrote:

I can understand your frustration. The misunderstandings concerning
closures are widespread and very hard to combat. In my experience, the
only way to convince someone about this is practically to enlighten
them, by having them experience programming patterns that require
closures - and having them try to translate them into languages that
do not support first class functions (note that closure is a semantic
concept, not really the language feature).

Here, however, is the definition from Essentials of Programming
Languages (Friedman, Wand and Haynes, 2nd ed.), one of the standard
textbooks in the field:

"In order for a procedure to retain the bindings that its free
variables had at the time it was created, it must be a _closed_
package, independent of the environment in which it is used. Such a
package is called a _closure_. In order to be self contained, a
closure must contain the procedure body, the list of formal
parameters, and the bindings of its free variables... We sometimes
say the procedure is _closed over_ or _closed in_ its creation
environment" (sec. 3.5, p. 85).

I hope this helps. I am forwarding this message to Anton, who might be
able to help more in your quest. If all fails, you might want to raise
the question on LtU. Keep in mind, though, that this issue was
discussed at length many times. A couple of recent discussions were
about the idea of extending Java with closures - a proposal that
shared the lack of clarity regarding the difference between closures
as a semantic construct and anonymous functions (and first-class
functions) as a language feature.

Alex: In particular, the idea of Java using the term 'closure' is pretty horrible, especially if it's being used in the wrong way. Fortunately, other terms (like functor , stand just as good a chance as being used). Note also the last paragraph, explicitly highlighting the problems between the semantic construct of closures and the language features of anonymous functions/first-class function objects that Groovy completely conflates.
Reply 4. At 7:00 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Matthias Lang, maintainer of the FAQ on Erlang , had this to say about closures:

The Erlang spec doesn't use the word closure at all.

The term 'closure' covers multiple concepts and trying to pin down a
single definition seems futile. In particular, saying "mathematical
term closure" seems unwise. Abelson and Sussman (SICP) warn:

> The use of the word ``closure'' here comes from abstract algebra,
> where a set of elements is said to be closed under an operation if
> applying the operation to elements in the set produces an element that
> is again an element of the set. The Lisp community also
> (unfortunately) uses the word ``closure'' to describe a totally
> unrelated concept: A closure is an implementation technique for representing
> procedures with free variables.

Alex: it's certainly true that closures can cause headaches; there is the concept of a closed set of operators across a mathematical set, too. For example, given the number 1 and the operator +, then a closure of this set includes all the positive integer values (i.e. it contains 1, 1+1, 1+(1+1), 1+(1+(1+1)) ...). Either way around, using 'closure' can lead to problems; Erlang stays away from such problems by not using it.
Reply 5. At 7:05 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Anton van Straaten, who is associated with Lambda the ultimate , had this to say:

Applying common terminology across languages can be a tricky business, and this case demonstrates that. While I agree that the Groovy docs leave much to be desired, I think that the fundamentals of the given definition of "closure" can reasonably be defended, in the Groovy context.

First, in my experience, the use of the term "closure" to refer to any first-class function, regardless of whether it actually closes over any variables in its definition environment, is pretty widespread. I think the reason for this is that if a programming construct is capable of defining first-class functions which close over their environment, then the case in which it closes over zero variables can be considered a special, degenerate case, which is often not worth distinguishing via a separate term (such as "combinator"). This may be an informal or casual use of "closure", but it's not uncommon.

But in addition to using "closure" as synonymous with first-class function, Groovy apparently goes a step further and makes it synonymous with "anonymous function". IMO, this is more problematic from the perspective of communication between programmers in different languages, since most languages with closures also allow named function definitions to generate closures. A Groovy programmer trained to believe that "closure" == "anonymous function" is likely to experience and/or create confusion when moving beyond Groovy.

However, within the context of Groovy, there's some justification for treating the two terms as equivalent. Here I run into my limited knowledge of Groovy, but afaict (please correct me if I'm wrong), there doesn't seem to be any other way to create a closure in Groovy (whether in the strict or loose sense). If that's the case, then it's not really wrong to say e.g. "In Groovy [...] A closure definition is one or more program statements enclosed in curly brackets" (from http://groovy.codehaus.org/Closures).

If that is indeed the only way to create closures in Groovy, then to the average Groovy programmer, making a distinction between closures and anonymous code blocks is likely to seem irrelevant.

One way to deal with this might be to add a note to the Groovy docs which says something like "In Groovy, this is the only way to create closures, but in some other languages, such as Javascript, Perl, Scheme (etc...) named function definitions can also create closures."

More generally, as Ehud pointed out, education on this issue should center around closure as a semantic concept. To illustrate the usefulness of the semantic concept, it may help to point out other instances of closure: for example, in the Java/Groovy context, methods close over instance variables. This is what makes it so easy to implement objects using closures, the language feature. (However, such are the vagaries of terminology that going around referring to methods as closures will simply confuse most people.)

Alex: I agree with most of this, since (as he points out): "But in addition to using "closure" as synonymous with first-class function, Groovy apparently goes a step further and makes it synonymous with "anonymous function". IMO, this is more problematic from the perspective of communication between programmers in different languages, since most languages with closures also allow named function definitions to generate closures." In fact, Java too supports closures; any time you have an instance method with access to instance fields, that's a closure. So attempting to combine the concept of 'anonymous function' with the semantic concept of closure can cause problems with other languages, and mixes the two concepts needlessly. It's also why the grammar should be changed, since grammar is about syntactical construct and not semantic construct.
Reply 6. At 7:09 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Philip Wadler, from the University of Edinburgh , (and contacted as someone writing about Haskell wrote:

I found this on the web page you cite http://groovy.codehaus.org/Closures:

"A closure may reference any variables defined within its enclosing lexical scope. Any such variable is said to be bound to the closure. Any variables bound to a closure are available to the closure even when the closure is returned outside of the enclosing scope."

That sounds like a proper closure to me. -- P

Alex: You get both sides of the opinions here However, it's worth remembering tha the closure is the instantiated form (when the program is run and bound) as distinct from the code block in the program itself; see also points elsewhere re: difference between constructor vs. instance.

Alex: Disclaimer: I was at the school of informatics at the University of Edinburgh, although I didn't know Philip at the time.
Reply 7. At 7:12 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Martin Odersky, of Scala language fame, wrote:

I prefer the definition in Wikipedia:

Closure (computer science) , an abstraction binding a function to its scope.

And I would call the construction used by the Groovy people ``an anonymous function''. Anonymous code block sound more like the Smalltalk terminology; it works just as fine, I think.

Alex: Again, the important thing is the binding, as opposed to the code used to create that binding.
Reply 8. At 7:14 PM on Dec 19, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
Robin Milner, creator of ML , wrote:

I think the term `closure' was invented by Peter Landin, or maybe by Christopher Strachey, probably in the sixties. It arose from the fact that, when you declare a function or procedure, e.g. in ALGOL60, then its body uses identifiers x,y,... that are not necessarily in scope at the places where F is invoked. So, what's the `value' or `meaning' of F that is used in all the places from which F is invoked? The answer is that it's a `closure', which means the function body of F paired with the environment (i.e. identifier/value pairs) that pertained at the place where F was declared.

Of course, other people may have used the term closure for other things. Also, the term `anonymous code block' could mean the same thing; after all, a closure in the above sense doesn't contain the name F, it just contains the `value' denoted by F. Some care is needed if F is recursive, i.e. calls itself.

If you look in the ` Semantics of Standard ML ' (by others and me) the concept is very carefully defined. I haven't a copy to hand, but I think we use the term `closure' for it. If we didn't we ought to have!

Alex: Disclaimer; I was at Cambridge University, and was lectured by Robin Milner.
Reply 9. At 7:29 AM on Dec 20, 2006, Paul King wrote:: Re: When is a closure not a closure?
Alex, do you have any issues with the terminology used in the Java proposal. Groovy is pretty much the same (or could be made the same with very minor word-smithing):

http://www.javac.info/closures-v04.html
Reply 10. At 7:43 AM on Dec 20, 2006, John Wilson wrote:: Re: When is a closure not a closure?
It might be useful if I clarified a couple of points about Groovy.

This information may already have been provided in the previous "thread from hell" I'm afraid I haven't read all of the previous thread. If I'm repeating stuff you already know I apologise.

Firstly "closures" are not the only form anonymous function provided by Groovy. It also supports method pointers:

def o = new Object()
def h = o.&hashCode
println h() // prints the hash code of o

Secondly a Groovy "closure" always captures a reference to the enclosing object.

Now there is an argument that says that even if the "closure" captures state it's not really a closure unless the state is used.

If you take that position then Groovy can provide you with something of a conundrum...

class C {
private int i = 10;

void f() {
def c = {println i}
......
}
}

Now the question is "is the thing assigned to c a closure?"

It captures the reference to the instance of C in which it is created (as do all Groovy "closures"). However, it does not necessarily use the reference.

The name resolution of 'i' is done at run time via the "closure's" MetaClass. The standard MetaClass will use the reference to the enclosing instance to resolve 'i'. However this MetaClass can be replaces with one which resolves names differently.

So it is perfectly possible to make the object assigned to c in the example above resolve 'i' against the enclosing object the first time it's called but against some other object the next time it's called.

In this case is it a closure the first time it's called any an anonymous lambda the second?
Reply 11. At 9:02 AM on Dec 20, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
The Java proposal is almost as mixed up as the Groovy one. You'll also note the disparaging comments to that effect here too. Two wrongs don't make a right ...

Alex.
Reply 12. At 9:05 AM on Dec 20, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
No, in the strict sense, they're both lambda expressions. It's only a closure at run-time when the value of the 'i' in the lambda expression is bound to the value. This is the problem with the Groovy's use of 'closure' as syntactic construct; the "{print i}" is a lambda expression with a free variable "i", and when bound (at run-time) that's when you have a closure.

Alex.
Reply 13. At 9:09 AM on Dec 20, 2006, Alex Blewitt wrote:: Re: When is a closure not a closure?
A door may be open or closed. An open door may be closed by pushing; this achieves closure. A closed door cannot be closed any further; pushing on a closed door has no effect (and thus is not a closure). In both cases, the door remains a door at all times; it does not become a closure. A closure is the combination of a door and pushing.

:1,$s/door/lambda expression/g
:1,$s/pushing/binding to an environment/g

A lambda expression may be open or closed. An lambda expression may be closed by binding to an environment; this achieves closure. A closed lambda expression cannot be closed any further; binding to an environment on a closed lambda expression has no effect (and thus is not a closure). In both cases, the lambda expression remains a lambda expression at all times; it does not become a closure. A closure is the combination of a lambda expression and binding to an environment.

Alex.
Reply 14. At 2:05 PM on Dec 20, 2006, Dave Webb wrote:: Re: When is a closure not a closure?
Alex,

I'm trying to understand your reason for highlighting Groovy so much on this subject, as opposed to other languages like Ruby.

Matz commented: "I use the term 'closure' for anonymous code block with the ABILITY to capture the outer local variables"

So, strictly on the subject of the use of the term 'closure' being loosely used as a synonym for 'closure-able', is Groovy any worse than Ruby?