News for hackers. Stuff that matters.
Re:Automatic Differentiation with Julia by ncfausti on Tuesday October 23, 2018 @06:23AM (#18282176)
For those curious about Julia, I just found this:

https://www.infoworld.com/article/3284380/data-science/what-...

Close to C speed in a dynamic language? Seems pretty great on paper. Is this generally the case?

Re:Automatic Differentiation with Julia by jules on Tuesday October 23, 2018 @07:35AM (#18282474)
As long as you write code that's type stable, yea. Type stable means that the compiler can deduce the types of the variables used in a function given the types of the arguments passed to the function. An example of code that's not type stable is code something like this:

  function test1(n)
    x = 0
    for i = 1:n
      if i == 10
        x = x + 0.1
      else
        x = x + 1
      end
    end
    return x
  end
It isn't type stable because x starts out as an int but then changes to a float in the middle of the loop. This code compiles to 78 instructions.

The following code is type stable:

  function test2(n)
    x = 0.0
    for i = 1:n
      if i == 10
        x = x + 0.1
      else
        x = x + 1.0
      end
    end
    return x
  end
This code compiles to 14 instructions.
Re:Automatic Differentiation with Julia by kristofferc on Tuesday October 23, 2018 @10:56AM (#18283889)
Julia is pretty good at dealing with this now though

    julia> @btime test1(10^5)
      147.427 μs (0 allocations: 0 bytes)
    99999.1

    julia> @btime test2(10^5)
      88.472 μs (0 allocations: 0 bytes)
    99999.1
In earlier versions of Julia, the penalty here would be order of magnitudes worse.
Re:Automatic Differentiation with Julia by FridgeSeal on Tuesday October 23, 2018 @07:21AM (#18282398)
I've been using it at work and it's pretty great.

The performance is awesome, the REPL is super great as well. There's libraries that I'm itching to try out as soon as I've got a relevant project (Flux ML being the top of that list).

There's been a lot of situations in using it where I've just gone "this is everything I needed/wanted": the performance, the language features and API design, etc.

Plus there's a lot of very interesting things going on with the language and ecosystem, I definitely recommend trying it out.

Re:Automatic Differentiation with Julia by sischoel on Tuesday October 23, 2018 @06:45AM (#18282263)
While Julia is indeed a dynamically typed language, it also supports type annotations and can in most cases infer the static type of a variable. If you write a function

  function foo(x)
    #do something
  end
and you call foo(10) and foo("some string"), then the compiler will create specialized methods foo(x::Int) and foo(x::String). Then there is no need for tracking the dynamic type of x inside these functions.
Re:Automatic Differentiation with Julia by one-more-minute on Tuesday October 23, 2018 @06:35AM (#18282218)
Generally, yes. It sounds like magic at first, but it's really just like a very lazy C++ compiler. You can happily look through the generated LLVM and machine code if you want to make sure it's reasonable.
Re:Automatic Differentiation with Julia by currymj on Tuesday October 23, 2018 @10:26AM (#18283612)
surprisingly yes, it really is. Julia is smart about type inference, so it compiles a specialized version of a function as needed for whatever argument types actually get used.

the catch is, the first time you run a function there is often a noticeable compile time. but it’s cached after that.

other problems with Julia include a somewhat immature/unfinished set of libraries, in part because the language was constantly changing underneath people.

but now that 1.0 has been released, the language will be stable for a long time and you can expect that to improve quickly.

good language! it gets a lot of hype on HN but that’s because it is actually very nice.

Re:Automatic Differentiation with Julia by ska on Tuesday October 23, 2018 @01:30PM (#18285451)
Julia is hardly the first (cf some common lisp environments, 20+ years ago). To really get near (say within 2x) of c you need some sort of type annotation or inference of course, and (more importantly) you have to structure your code such that this works, but it's quite do-able. Your code does tend to end up a bit c-like in those performance critical sections, but that's hardly surprising.
Re:Automatic Differentiation with Julia by byt143 on Tuesday October 23, 2018 @02:24PM (#18286014)
>Your code does tend to end up a bit c-like in those performance critical sections, but that's hardly surprising.

That isn't true. Closures, higher order functions and fused broadcast array expressions are all very fast except in some corner cases.

Re:Automatic Differentiation with Julia by ska on Wednesday October 24, 2018 @02:38PM (#18295141)
I'm not sure we disagree. In my experience you can get "pretty fast" but not "c-like fast" while keeping many language features. The more you chase c-like performance (or even better fortran), the more you start restructuring things in a c-like way. This isn't universal of course, and I don't think it has anything specific to do with c language, just that it is a semi reasonable proxy for hardware architecture (ignoring SIMD).

Anyway, this isn't really Julia specific, and I haven't tried with Julia recently so I may be wrong :)

Re:Automatic Differentiation with Julia by kazagistar on Tuesday October 23, 2018 @11:23AM (#18284139)
It's "cost" is in startup times (compiling your code with LLVM at runtime slows things down, no surprise there), and you still have to write somewhat type-stable code (outputs uniquely inferrable from inputs) if you want it to actually run full speed.