News for hackers. Stuff that matters.
Re:Talk with Rich Hickey: Time is the New Memory by vdm on Tuesday September 22, 2009 @07:42AM (#836731)
I think the Date class is an unfortunate example to use, its too prone to being confused with the idea of a timeline of mutable state. Maybe a better example to use would be the concept of a document, and that parts of it don't change while you're reading it. On the other hand, his audience will always be familiar with the Date class...

I've been following Rich's work for a while and I felt myself almost slip into confusion.

Re:Talk with Rich Hickey: Time is the New Memory by thunk on Tuesday September 22, 2009 @08:37AM (#836795)
I think he chose it specifically because of the pleasing self-similarity -- it's sort of a joke. After all, what kind of problem is a mutable date? A time problem, of course.
Re:Talk with Rich Hickey: Time is the New Memory by gruseom on Tuesday September 22, 2009 @11:35AM (#837172)
What does this add to the discourse about mutable state? Is there any genuine insight here, other "hey it's kind of like about time"? If so, it ought to give rise to some new construct or technique that helps solve the problem, or even just a simplification of some well-known example.

I guess he's trying to popularize the first principle of functional programming, but he's doing it in a confusing way that will inevitably breed further confusion. Look at the absolute silliness of the title for evidence of that.

Re:Talk with Rich Hickey: Time is the New Memory by fauigerzigerk on Tuesday September 22, 2009 @01:49PM (#837516)
What's your issue with the title? We did manual memory management in the past and we do manual time management now.

I agree with him completely that managing the timing of parallel processes is the most pressing complexity issue in computing today.

Re:Talk with Rich Hickey: Time is the New Memory by gruseom on Tuesday September 22, 2009 @04:27PM (#837856)
Absolutely no one uses the phrase "time management" in this way. "I found a bug in my time management." <- Huh?

That mutable state has something to do with time is a tautology. Nothing in the recent discussion shows it to add anything significant to what we do or think about the problem. The interview title takes this doodling and amplifies it into some sort of grand theory. I'd be embarrassed.

I'm not criticizing Clojure at all, I'm criticizing this way of drawing attention to it (if that's what it is).

Re:Talk with Rich Hickey: Time is the New Memory by richhickey on Tuesday September 22, 2009 @06:29PM (#838147)
Yes, there is a tautology, and I'm glad you understand it, but such understanding is not widespread I assure you.

You are right, no one says "time management". I don't think they said "memory management" as often either, until there was automatic memory management (GC). But one thing is certain, most OO languages don't yet offer automatic _____ management, whatever it is to be called.

People do say they have a "race condition" or a "deadlock", or simply "data corruption" often without connecting it at all to the unmanaged mutation of their objects. And before there was GC there was just "heap corruption", "double deletes" and "memory leaks". Management implies a manager, and there isn't one yet, so why would anyone say it? What I am advocating is that languages provide automatic management of "time". It's not a novel idea, yet it is still missing, and I think it is missing because there is no place to put it in OO strategies which conflate identity with state, with weak or missing notions of object values.

But the phrase "mutable state" is broken, in a way that keeps people from understanding the problem and especially the potential solution. As soon as you consider state the value of something at a point in time, then mutation isn't the correct term, succession is, and the connection to time becomes clearer.

FYI, I neither wrote nor uttered the title. The interview was a recording of a casual, no prep conversation following my keynote. It was not meant to be profound, nor an advertisement.

Re:Talk with Rich Hickey: Time is the New Memory by gruseom on Wednesday September 23, 2009 @01:43AM (#838698)
I agree with much of this, but "time management" still seems like a half-baked metaphor for FP immutability and not a clarifying model, at least not yet. It's easy to explain the X to which "memory management" refers: allocating and freeing memory. Surely that was easy to understand even before the advent of GC. It's also easy to explain what memory is and point to where and how it is manipulated in code. I don't see how time is a programming construct in the way that memory or state is.
Re:Talk with Rich Hickey: Time is the New Memory by cschep on Wednesday September 23, 2009 @03:24AM (#838824)
Isn't time a product of memory and state?
Re:Talk with Rich Hickey: Time is the New Memory by crc on Tuesday September 22, 2009 @12:38PM (#837347)
"Its kind of like about time". That is the point. This is just concurrency control, and looking at as time is just a different perspective. (Actually, it is more than concurrency control, since he also talks about aliasing, but anyway..)I find that perspective intriguing.

I am not worried that it will confuse people. Ideas and perspectives that don't have merit will lose out to more powerful and clearer ideas over time. Also, I have always had more clarity only after passing through a state of confusion.

Re:Talk with Rich Hickey: Time is the New Memory by gruseom on Tuesday September 22, 2009 @01:26PM (#837469)
Ideas and perspectives that don't have merit will lose out to more powerful and clearer ideas over time.

If there's one statement that the history of programming languages (to date) contradicts... :)

Actually, I think I'm reacting more to the slides of Hickey's recent talk about this, which he called "deconstruction" and kept comparing to Whitehead. I think Clojure deserves better than that. The signs of guru worship are already appearing. I would have said that's a bad thing. On the other hand, maybe it's the way to grow.

Re:Talk with Rich Hickey: Time is the New Memory by reginaldo on Tuesday September 22, 2009 @10:52AM (#837076)
Funny that Rich used the number 42 as a recurring example in the interview. After all, 42 is "the answer to life, the universe and everything" (really, just search on google), which basically is what he is trying to find.
Re:Talk with Rich Hickey: Time is the New Memory by 10ren on Tuesday September 22, 2009 @08:43AM (#836801)
I wonder if there is an entirely different way to reason about computation, other than mutable state, which coincides with our intuitions based on everyday experience of the physical world; and immutable state, which coincides with the powerful and abstract approach of our mathematics.

Alas, mutable and immutable would appear to cover all the cases.

Re:Talk with Rich Hickey: Time is the New Memory by lucifer on Tuesday September 22, 2009 @09:22AM (#836859)
The issue isn't computation, it is modeling. The next order of evolution for computation is quantum computation. What R Hickey is addressing is modeling.

I think the modeling problem could be alternatively addressed by unlearning the notion (conceit?) that every program is a representation/simulation (which is where (domain) state creeps in) and instead embrace a descriptive/measurement approach. Or more precisely, to accept that a program is always orthogonal to the domain model and give up chasing after the holy grail/folly of seeking a paradigm to express "reality" in computer instructions.

One could argue that this is what FP is all about, but suggestion here is that the issue is fundamentally the conceptual view of the programmer and it can apply equally to OO or any other paradigm of programming.

For example, the "Date" case he outlines conceptually works if one is to suspend disbelief and accept the notion that a "Date" type object is really a point in time. "How can September 22, 2009 change?" But at some level this is just semantics. Would calling the "Date" object "DateDatum" cause a similar conceptual problem?

DateDatum lastUpdate = now (); //... doSomething(); lastUpdate = now();

Re:Talk with Rich Hickey: Time is the New Memory by richhickey on Tuesday September 22, 2009 @11:44AM (#837200)
> DateDatum lastUpdate = now (); //... doSomething(); lastUpdate = now();

Yes, this highlights the crux of the argument:

MutableDate lastUpdate = now(); doSomething(); lastUpdate.changeInPlace();

versus:

ImmutableDate lastUpdate = now(); doSomething(); lastUpdate = changedVersionOf(lastUpdate);

In the first case, when someone reads lastUpdate they get a chunk of memory that could change right out from under them, with no clear recipe as to how to obtain a consistent value. In the second case, they get a consistent value they can use without worrying or synchronization. The first conflates identity and state, the second separates them.

Now, all is not right with the second case, as there still aren't coordination semantics for putting new values in lastUpdate:

TransactionalRef<ImmutableDate> lastUpdate = now();

or

AtomicRef<ImmutableDate> lastUpdate = now();

or something similar would fix that. It's not a matter of removing mutable memory, but one of using it in a way with clear coordination semantics, other than "good luck with that".

Re:Talk with Rich Hickey: Time is the New Memory by radu_floricica on Tuesday September 22, 2009 @09:09AM (#836840)
I just started reading Norvig's PAIP, and one of the things that surprised me was seeing lisp code written in a non-functional manner. Of course the code in PAIP evolves and the early iterations are "bad" on purpose, but it's amazing how obviously wrong some early patterns seemed just because I was used to Clojure's more functional style.
Re:Talk with Rich Hickey: Time is the New Memory by lispm on Tuesday September 22, 2009 @05:11PM (#837956)
no, they are not 'bad on purpose'. Peter Norvig shows how to get from quick bottom up prototypes to optimized code. The purpose is to quickly and cleanly as possible get a first version going and then improve that.

Common Lisp programs make heavy use of side effects from imperative code, over destructive operations to the CLOS object system.

Re:Talk with Rich Hickey: Time is the New Memory by radu_floricica on Tuesday September 22, 2009 @06:21PM (#838135)
I haven't programmed at all in lisp and less then a year in Clojure so my opinion doesn't carry much weight, but I did notice that a) imperative code is harder to read and b) later versions look more functional. So for what's worth I'm happy Clojure considers functional style and immutable data as defaults.
Re:Talk with Rich Hickey: Time is the New Memory by tybris on Tuesday September 22, 2009 @08:49AM (#836810)
I'm sure he's a really smart guy, but he sounds like a Computer Science freshman who is just beginning to grasp the problems of concurrency and consistency (could be his way of explaining). We already know these things are hard. No reason to give it a new, vague name like "Time management".
Re:Talk with Rich Hickey: Time is the New Memory by gizmo on Tuesday September 22, 2009 @09:00AM (#836823)
A computer science freshman? I'd rather argue the opposite -- that he's one of the few guys in Computer Science who comes up with something original AND builds a language that supports his ideas to see if they work in practice.

Even if you don't think Hickley is on the right track, how can you not like his combination of hands on implementation and search for theoretical soundness?

Re:Talk with Rich Hickey: Time is the New Memory by jrockway on Tuesday September 22, 2009 @10:26AM (#837004)
that he's one of the few guys in Computer Science who comes up with something original AND builds a language that supports his ideas to see if they work in practice.

Not to discount Rich's work, but all of the ideas in Clojure have been well-known forever, and Haskell and ML implemented these ideas many, many years ago. Clojure lets you use these ideas with some Java libraries; that's its innovation.

Also, I don't really get the whole time analogy, but I can't help but think he wants Comonads, which Clojure does not have. (And due to its type system, will never be able to support well.)

Re:Talk with Rich Hickey: Time is the New Memory by richhickey on Tuesday September 22, 2009 @12:30PM (#837323)
Yes, Clojure is full of old ideas. I've never claimed otherwise. Unfortunately, they have yet to gain significant traction as currently embodied. Clojure is just an attempt to make them palatable and practical (and fun). I would never argue against someone who preferred Haskell.

As far as comonads, no - monads and comonads are essentially threaded through a single computation. The reference types I'm talking about are like, and for similar situations as, Haskell's MVars and STM TVars, which wouldn't exist if, e.g., state monads were sufficient.

Re:Talk with Rich Hickey: Time is the New Memory by jrockway on Tuesday September 22, 2009 @01:18PM (#837448)
OK, now that you mention MVar/TVar, I think I understand I understand your argument better. I think MVars and TVars are mostly a performance hack (like most other uses of state in general and ST or IO in Haskell specifically). Theoretically, many programs written in terms of MVars could be written in terms of spawning a new thread for each read/write pair. This would be easier to reason about, but probably slower.

A common use of MVars:

   f x = pure computation
   g h m = forever $ hGetChar h >>= putMvar m

   do mvar <- newEmptyMVar
      forkIO $ liftM f (takeMVar m)
      forkIO $ g a mvar
      forkIO $ g b mvar
      <wait for exit condition>
All we do is read from two handles, concurrently, and eval f for every byte on either handle. The MVar is a natural way of expressing this, but it does lead to potentially undesirable side effects.

Writing it in a functional way would probably be better:

  f x = pure computation
  g h f = forever $ liftM f (hGetChar h)
  
  do forkIO $ g a f
     forkIO $ g b f
     ...
There is now no shared state, since the problem doesn't actually require any. (We could use continuation passing style if we needed to keep state.)

Anyway, I am still not sure how Clojure encourages the reference-less style.

Re:Talk with Rich Hickey: Time is the New Memory by lucifer on Tuesday September 22, 2009 @09:33AM (#836874)
You recognize, of course, that the ironic fact of Clojure's implementation on top of JVM (using JVM concurrency mechanisms) argues against the very thesis he is putting forward.
Re:Talk with Rich Hickey: Time is the New Memory by loup-vaillant on Tuesday September 22, 2009 @09:40AM (#836891)
How so ? Do you say the manual implementation of automatic time management is contradictory ? Automating things requires manual work. Interpreters, compilers, garbage collectors, STM etc are the same in this respect.
Re:Talk with Rich Hickey: Time is the New Memory by lucifer on Tuesday September 22, 2009 @09:49AM (#836910)
Sorry, replied to the wrong comment. (Was addressing the "concurrency is hard" bit.)
Re:Talk with Rich Hickey: Time is the New Memory by jimbokun on Tuesday September 22, 2009 @09:21AM (#836856)
If "we" already know these things are hard, why are "we" all programming in languages with mutable state? That seems to indicate that "we" are all idiots, or masochists.

I found the analogy to garbage collection just right. It's obvious now that for large parts of the problem space of writing computer programs, having garbage collection built into the language makes programs significantly easier to write and understand. Joel Spolsky made the argument that object oriented programming didn't do nearly as much to make programming more productive as automated memory management did, and I agree.

So why not take this other problem that is very hard to reason about, and build constructs into the language that take care of it for you automatically? I think the fact this seems so obvious to you that it's hardly worth saying points to how powerful the argument is. If it's so obvious, why is almost everyone still programming with mutable state and with so little support for "time management" constructs? Isn't this the same thing as stubbornly clinging to free and malloc in programs where garbage collection would work just fine?

Re:Talk with Rich Hickey: Time is the New Memory by tybris on Tuesday September 22, 2009 @10:02AM (#836941)
> why are "we" all programming in languages with mutable state?

My argument is not against closure or using immutable objects. My argument is against trying to rebrand a problem that has been well-known and well-studied for over 40 years.

Clojure does a bit of copying (memory management, not "time management") to force you to avoid concurrency problems that arise in some scenarios. Arguably that is a good thing, because people tend to think of concurrency when it's too late. However, it too ultimately has a mutable authoritative source where the same issues arise. Moreover, the hardest concurrency and consistency problems arise in distributed systems. Using Clojure certainly isn't the answer to that.

> Isn't this the same thing as stubbornly clinging to free and malloc in programs where garbage collection would work just fine?

People "stubbornly clinge" to programming with free and malloc (in C), because it allows them to have very specific control over how their program will behave. This allows you to perform very effective optimizations. Moreover, because it is cumbersome, C programmers will avoid dynamic memory, which will again make your program faster and more durable. The same thing will apply to mutable objects. Clojure simply restricts you to a certain model, which is not necessarily most efficient.

Does that mean Clojure is bad? No it means it's not the right tool for every job.

Re:Talk with Rich Hickey: Time is the New Memory by loup-vaillant on Tuesday September 22, 2009 @12:20PM (#837298)
> Clojure simply restricts you to a certain model, which is not necessarily most efficient.

So does C. You don't use mutable array because they are the best solution. You use them because they're the best supported datastructure in C. Sometimes even when a shared immutable tree would be most efficient.

For a well-know, well-studied, and still unsolved problem, rebranding isn't wrong. It's the only way. I think pervasive mutation falls in this category. So does the widespread use of the Qwerty layout.

Re:Talk with Rich Hickey: Time is the New Memory by drewr on Tuesday September 22, 2009 @12:28PM (#837316)
No, it's not the right tool for every job. Rich isn't arguing that kernel drivers be written in Clojure. His work addresses the pain he feels -- writing concurrent, high-level applications that model the real world.

Also, you'd be amazed at how much control you have over the underlying system in Clojure; you just rarely have to wield it.

Re:Talk with Rich Hickey: Time is the New Memory by jrockway on Tuesday September 22, 2009 @10:20AM (#836988)
If "we" already know these things are hard, why are "we" all programming in languages with mutable state? That seems to indicate that "we" are all idiots, or masochists.

The same reason people still use C -- perceived efficiency, and 30+ years of history. Computers are just a big block of mutable state that runs instructions, after all, so the first thing that comes to mind is to program them without abstracting that away.

The reason we stick with this is that nobody wants to give up their favorite language (mostly for perceived employment reasons, if you don't know Java, you are unemployable or something), and "their favorite language" doesn't support immutability well. Try writing composible effectful computations in Java. Doesn't work. So you have to do something else. (If you have effectful computations that know about each other, you can put them in a class. If they don't know about each other, you basically have to manage the state manually.)

I experimented with this recently in Perl, via MooseX::Data (http://github.com/jrockway/MooseX-Data/). It ports various category-theoretic type classes to Perl. (You could do this in any language, although Java is not one of them unless you want to copypaste the standard liftM-in-terms-of->>= code into every class that implements "Monad", etc. Scala fixes this particular problem.)

Anyway, without special syntax, it ends up being a real mess. For example, consider incrementing a global state variable by one:

   use MooseX::Data::Function::State;
   sub S() { MooseX::Data::Function::State:: } # save typing

   my $inc = S->get->bind(
      S->new(
          function => sub {
              my ($state) = @_;
              S->mreturn($state+1);
          },
      )->bind( S->put ));
And printing it:

   my $print = S->get->bind(
      S->new(
         function => sub {
             my ($state) = @_;
             say $state;
             S->mreturn($state);
         },
      ),
   );
So now to inject 42 into the environment, increment it twice, and print it:

   my $result = $inc->bind($inc)->bind($print)->runState(42);
This prints 44 and returns [44, 44], as expected. (44 is the state, 44 is the unlifted return value of $print->apply(44).)

Anyway, this is an awful lot of code compared to the imperative version:

  sub inc { $_++ }
  sub prnt { print }

  $_ = 42;
  inc;
  print;
  return $_
(The functional version composes better, though; you can write a function that acts on "the" shared state, and you can compose these arbitrarily. In Perl, you only get one $_ per program.

The tree-numbering example here is an excellent demonstration of composablity:

http://www.haskell.org/ghc/docs/latest/html/libraries/mtl/Co...

Note how there is no mutable state here, just a bunch of functions that pass (s,a) pairs to each other. And yet, you don't see that, you "see" composable mutable state. In this case, we are composing numberTree with itself recursively.)

To get back to my original point, with the right language, immutability is as expressive as mutable state. The increment example in Haskell:

  import Control.Monad.State

  type Step = StateT Int IO Int -- save typing

  inc, prnt :: Step
  inc = do
    x <- get
    let x' = x + 1
    put x'
    return x'

  prnt = do
    x <- get
    liftIO.putStrLn $ show x
    return x

  main = runStateT (inc >> inc >> prnt) 42
  -- or prnt << inc << inc
This prints:

  44
  (44,44)
It is worth pointing out that I combined the function calls with >> instead of >>= so you can see that the values are being passed implicitly (via get and put), rather than as arguments to the individual functions, which would look like

  (putStrLn.show) . (1+) . (1+) $ 42
Anyway, it's a function of languages. If everyone was using Haskell, very few programs would be using mutable state. But since they are using Java, and the only state encapsulation tool Java supports well are classes, that's what everyone uses. Very few people care about "doing it right", they care about having a working program. And while class-based state encapsulation is inferior to immutability in many ways, most developers don't know this, and don't care.
Re:Talk with Rich Hickey: Time is the New Memory by jimbokun on Tuesday September 22, 2009 @11:44AM (#837198)
I think this still fits into the comparison with garbage collection. To really get the benefits of garbage collection, programmers had to switch from C++ to Java (or something else). Hickey, I think, is suggesting the same is true of time management, so he created Clojure.
Re:Talk with Rich Hickey: Time is the New Memory by loup-vaillant on Tuesday September 22, 2009 @09:52AM (#836916)
Basically, the guy is saying that pervasive mutation hurts. If it takes buzzwords like "management" to make programmers understand that, I'm glad he uses them.

We may already know concurrency and consistency are hard, but we do little to identify and eliminate the root causes. Those who do can show the bright side, like Rich Hickey does, or they can finger-point the dark side, like I do: http://theroleplayingprogrammer.blogspot.com/2009/06/assignm...

Re:Talk with Rich Hickey: Time is the New Memory by mbrubeck on Tuesday September 22, 2009 @01:10PM (#837426)
Maybe part of your problem is that his explanation in this particular interview is on a very basic level, using concepts that any freshman can understand. For a deeper version of Hickey's ideas, see his talk from the JVM language summit, or some of the docs on clojure.org. (Or better yet, read the Clojure source code...)

http://news.ycombinator.com/item?id=829268

http://clojure.org/state

Re:Talk with Rich Hickey: Time is the New Memory by lucifer on Tuesday September 22, 2009 @09:06AM (#836836)