News for hackers. Stuff that matters.
Re:Microsoft Supercharges Bing Search With Programmable Chips by zackmorris on Monday June 16, 2014 @02:25PM (#7900615)
I've been ranting about the inadequacies of mainstream processors for almost twenty years. I remember even back in the late 90s, seeing processors that were 3/4 cache memory, with barely any transistors used for logic. It's surely worse than that now, with the vast majority of logic gates on chips just sitting around idle. To put it in perspective, a typical chip today has close to a billion transistors (the Intel Core i7 has 731 million):

https://en.wikipedia.org/wiki/Transistor_count

A bare minimum CPU that can do at least one operation per clock cycle probably has between 100,000 (SPARC) and 1 million (the PowerPC 602) transistors and runs at 1 watt. So chips today have 1,000 or 10,000 that number of transistors, but do they run that much faster? No of course not.

And we can even take that a step further, because those chips suffered from the same inefficiencies that hinder processors today. A full adder takes 28 (yes, twenty eight) transistors. Could we build an ALU that did one simple operation per clock cycle with 1000 transistors? 10,000? How many of those could we fit on a billion transistor chip?

Modern CPUs are so many orders of magnitude slower than they could be with a parallel architecture that I’m amazed data centers even use them. GPUs are sort of going the FPGA route with 512 cores or more, but they are still a couple of orders of magnitude less powerful than they could be. And their proprietary/closed nature will someday relegate them to history, even with OpenCL/CUDA because it frankly sucks to do any real programming when all you have at your disposal is DSP concepts.

I really want an open source billion transistor FPGA running at 1 GHz that doesn’t hold my hand with a bunch of proprietary middleware, so that I can program it in a parallel language like Go or MATLAB (Octave). There would be some difficulties with things like interconnect but that’s what things like map reduce are for, to do computation in place rather than transferring data needlessly. Also with diffs or other hash-based algorithms, only portions of data would need to be sent. And it’s time to let go of VHDL/Verilog because it’s one level too low. We really need a language above them that lets us wire up basic logic without fear of the chip burning up.

And don’t forget the most important part of all: since the chip is reprogrammable, cores can be multi-purpose, so they store their configuration as code instead of hardwired gates. A few hundred gates can reconfigure themselves on the fly to be ALUs, FPUs, anything really. So instead of wasting vast swaths of the chips for something stupid like cache, it can go to storage for logic layouts.

What would I use a chip like this for? Oh I don’t know, AI, physics simulations, formula discovery, protein folding, basically all of the problems that current single threaded architectures can’t touch in a cost-effective manor. The right architecture would bring computing power we don’t expect to see for 50 years to right now. I have a dream of someday being able to run genetic algorithms that take hours to complete in a millisecond, and being able to guide the computer rather than program it directly. That was sort of the promise with quantum computing but I think FPGAs are more feasible.

Re:Microsoft Supercharges Bing Search With Programmable Chips by jacquesm on Monday June 16, 2014 @03:11PM (#7900879)
> I really want an open source billion transistor FPGA running at 1 GHz that doesn’t hold my hand with a bunch of proprietary middleware, so that I can program it in a parallel language like Go or MATLAB (Octave).

I fail to see how not having proprietary middleware will enable you to program an FPGA in Go or MATLAB.

FPGA's are not well suited to being programmed in conventional languages (and go is not a parallel language, it employs a clever model that may give you that impression but under the hood it is fairly regular, nothing you could not achieve using co-routines and threads in a different language, maybe syntactically cleaner and easier to understand). MATLAB might be more feasible but still not an easy match. You could conceivably make an FPGA co-processor though that you access from those languages through some library.

If you want to program an FPGA in something that looks like a high level language syntactically there are a number of solutions:

http://stackoverflow.com/questions/5603285/c-to-hardware-com...

And some newer developments. But none of those change the essence of the chip.

I'm not sure how to convey the difference between an FPGA and something like a high level language any better than 'imagine all your code executing at once'.

FPGAs are something you tell what to be, not what to do (and high level languages tell CPUs what to do, not what to be).

Re:Microsoft Supercharges Bing Search With Programmable Chips by gabbo on Monday June 16, 2014 @05:49PM (#7901677)

  FPGAs are something you tell what to be, not what
  to do (and high level languages tell CPUs what to
  do, not what to be).
That's a really great way of putting it. I never thought of it that way before, but you're spot on.
Re:Microsoft Supercharges Bing Search With Programmable Chips by joshu on Monday June 16, 2014 @03:33PM (#7900995)
Yeah, this rant makes me wonder if poster actually understands how these things work in deep detail.
Re:Microsoft Supercharges Bing Search With Programmable Chips by zackmorris on Monday June 16, 2014 @07:44PM (#7902112)
I used VHDL for a semester back in 98 or 99 for my ECE degree. I remember it being extraordinarily brittle compared to something like C, because there were so many ways to trigger unreliability with the wrong clock edges etc. So for example we did almost everything as state machines instead of math. The code was basically unmaintainable by today's standards but was an important learning tool. As I recall, we wrote a VGA signal generator, and if you had less than half a dozen incorrect pixels onscreen, you were doing pretty well. So in fairness, I USED to know this stuff in nauseating detail.
Re:Microsoft Supercharges Bing Search With Programmable Chips by joshu on Monday June 16, 2014 @08:58PM (#7902331)
Yes, this is because VHDL is not code. It's a different paradigm entirely. The FPGA doesn't "run" the VHDL, etc.

The units in an FPGA are just a little bit of logic with a bunch of connectivity to local busses.

I haven't done much VHDL lately (ECE '96 here) but I do remember how the stuff works. But then again I focused mainly on CPU architecture.

Re:Microsoft Supercharges Bing Search With Programmable Chips by colanderman on Monday June 16, 2014 @03:23PM (#7900956)
> one operation per clock cycle […] Modern CPUs are so many orders of magnitude slower than they could be with a parallel architecture

Modern CPUs are superscalar, meaning they can execute multiple operations per clock cycle.

> processors that were 3/4 cache memory, with barely any transistors used for logic. It's surely worse than that now, with the vast majority of logic gates on chips just sitting around idle

That's because memory is the bottleneck for 99% of compute workloads. The only things that memory is not the bottleneck for is silly arithmetic (e.g. graphics) and network processing. And we have processors for those workloads. (Source: I've programmed Tilera network processors.)

> even with OpenCL/CUDA because it frankly sucks to do any real programming when all you have at your disposal is DSP concepts.

OpenCL/CUDA are so unlike DSP programming it's not even funny. SIMT and SIMD are very different programming models. (Source: I've programmed both CUDA and DSPs (Blackfin specifically).)

> it’s time to let go of VHDL/Verilog because it’s one level too low. We really need a language above them that lets us wire up basic logic without fear of the chip burning up.

Huh? VHDL and Verilog are very high-level languages that expose concepts like open-drain/collector logic and different voltage domains only unwillingly. Unless you have a really crappy elaborator, "chips burning up" just ain't gonna happen. (Source: I've done mixed-signal ASIC design.)

> So instead of wasting vast swaths of the chips for something stupid like cache, it can go to storage for logic layouts.

I don't think you understand the importance of cache. Memory is S-L-O-W. (Source: I've hand-optimized high-performance network processing code.)

> AI, physics simulations, formula discovery, protein folding, basically all of the problems that current single threaded architectures can’t touch in a cost-effective manor

All those things you list are bottlenecked by memory bandwidth, not CPU time.

Re:Microsoft Supercharges Bing Search With Programmable Chips by zackmorris on Monday June 16, 2014 @07:57PM (#7902156)
I just meant that OpenCL/CUDA are moving towards a narrower scope than what I would like. There is too much emphasis on manually arranging your data with the correct interleaving, or doing things in batches, pipelined in just the right order. It all reminds me of DSP, which I was never a huge fan of. That's because all of the supposed gains can easily be replicated with a general purpose language and an order of magnitude more computing power (which is easily attainable with billions of transistors at your disposal, getting cheaper every year).

You are right about memory bandwidth though. I'm excited that memory has come so far, with wide channels running at a GHz or more. But, that has come at the expense of latency. I think this problem is intractable, because everything is moving toward data centers with even slower interconnect.

I’m reaching here, but I think the endgame will be memory that works more like bittorrent, where the latency won’t be how far the nearest neighbor is, but how much the data differs from what you already have. Then we can eliminate most of the problems with slow interconnect and move to a big dumb hash system that uses diffs but results in orders of magnitude higher throughput. If processing is really not a bottleneck, then this will be trivial.

Re:Microsoft Supercharges Bing Search With Programmable Chips by colanderman on Monday June 16, 2014 @11:17PM (#7902708)
I think what's more likely is that processing will move into the memory itself. We already see this somewhat with manycore shared-cache architectures like Tilera, which effectively provide one core per ¼ MiB of L3 cache. (Though unfortunately L3 cache latency is only ~twice as fast as RAM, but at least the bandwidth is much better.)
Re:Microsoft Supercharges Bing Search With Programmable Chips by lallysingh on Monday June 16, 2014 @03:21PM (#7900943)
More ALUs don't help: we have a few that can run in a cycle on modern CPUs, but we still get IPCs < 1.0. The bottleneck is getting the arguments and operation into the ALU for it to run. That's what the cache is for. As you pointed out, it's horribly inefficient (in % silicon space, but that's an odd metric, isn't it?) in some ways, but that's what you get for generality.

If you want something faster, you'll need to alter the upper level systems to make feeding the ALU(s) easier. (And you'll still have Amdahl's law to contend with -- which is a major factor even at the single-core scale.) AFAICT, you're asking for a system as high level as Go or MATLAB, but have it compile itself into some sequence of FPGA programs and I/Os on it. Altering those upper level systems (to the point of rewrite with a substantial amount of research work in the middle) is a lot of work, and efficiency at the silicon level isn't enough justification for it. Crude work partitioning by the programmer and more cores/machines often works well for those who actually need the additional performance.

But, there's something reasonable on regular silicon that you may like. Haskell's got auto-parallelization features, as well as the ability to compile code to run on CUDA. http://chimera.labs.oreilly.com/books/1230000000929 Is a book (available to read freely online) covering these features. You can get the background here http://book.realworldhaskell.org/

Re:Microsoft Supercharges Bing Search With Programmable Chips by al2o3cr on Monday June 16, 2014 @03:20PM (#7900934)
"I really want an open source billion transistor FPGA running at 1 GHz that doesn’t hold my hand with a bunch of proprietary middleware"

Yeah, and I want a pony. The pony just happens to be about 10000x more likely in terms of capital expense.

I look forward to the Kickstarter to raise the $10 million+ needed just to spin a mask set for this chip, not to mention the licensing costs for EDA tools that can do routing + signal integrity on this scale.

Re:Microsoft Supercharges Bing Search With Programmable Chips by jacquesm on Monday June 16, 2014 @03:31PM (#7900990)
And then it still won't deliver on the intended range of applications. An open source FPGA won't magically do what a closed source one can't, there is a fundamental impedance mismatch here between thinking about what an FPGA could do versus what it actually does.
Re:Microsoft Supercharges Bing Search With Programmable Chips by rdrdss23 on Monday June 16, 2014 @02:59PM (#7900803)
I think the general priority has been to improve single thread performance and for good reasons.

While there are plenty of problems that are parallelizable, a great majority of problems are sequential. Even if you think of say on the OS level - of running every process on a separate CPU/core/whatever - the single thread performance still comes out being the most important factor