News for hackers. Stuff that matters.
Re:Choosing Functional Programming for Our Game by reitzensteinm on Monday December 19, 2016 @07:42PM (#13216365)
I'm a massive fan of Clojure (I spoke at EuroClojure this year), and have used Unity3D for 5 years... I was also the developer of the first big iOS Haxe game, another unproven platform at the time, years ago.

I would be very concerned about straying too far from the garden path while also trying to ship a commercial game.

Unity's GC isn't even generational, a very bad mix with garbage heavy Clojure code. Expect confusing, bizarre and transient performance and memory use issues. Debugging will be a nightmare.

And on top of that, Stack Overflow doesn't have an unlimited supply of Clojure JVM answers; Clojure .Net through IL2CPP? Forget it; you'll have to figure everything out yourself.

It's generally a mistake to make the initial development more pleasant at the expense of long term maintainability; and I think that's the trade off on the table here.

Re:Choosing Functional Programming for Our Game by buzzybee on Monday December 19, 2016 @08:17PM (#13216528)
I would concur and add that the main risk one runs in deploying any new, widespread abstraction to game code is an architectural meltdown, where perf bottlenecks are appearing everywhere, too many code paths have been locked in by the abstraction so you can't iterate on them fast anymore, you have race conditions you can't analyze properly, etc.

Iteration times across the whole development lifecycle are hard to optimize because in the near term the iteration problem is specific to writing basic functionality, then later on it's adding and adjusting a wide variety of assets, and then even later it's debugging and optimizing. Care too much about one and you mess up the others.

If you soak up pain up front to keep the core of the code in an unadorned imperative style, crunching on plain data, avoiding deeply nested callstacks or polymorphism, you get a huge assurance that you can handle anything that happens later on - it follows the grain of the tooling and the happy path of the optimizer.

The way in which I last approached the live-reload problem was to write a data model, and then bolt a scripting language on to generate data. Then reload the scripts - you get the fast iteration, without being particularly invasive on the main loop.

Edit: And similarly I've focused most of my abstraction braincycles on making it easier to access and swap out assets. One of the biggest recurring problems in engine code is deriving runtime assets from combinations of static assets and parameters. This has led me down the road of developing an addressing scheme.

Re:Choosing Functional Programming for Our Game by s_m_t on Monday December 19, 2016 @08:53PM (#13216712)
I've just started to program a game engine and to my surprise the most difficult to structure aspect of the programming has been essentially, I'm not really sure how to word this, but mapping things to other things.

Making sure that a keypress eventually leads to the right function being called is a good example. It seemed really simple at first but then I figured that the keypress shouldn't directly call the function because then you'd need to recode or reparameterize some giant state machine because in certain contexts a keypress can do many things or control many different objects in the game (even controlling multiple objects at the same time, and what if you want to be able to control the game remotely or from two controllers at once?).

Before I started I thought the 3d movement and physics stuff would be the hardest... but I've found it was actually pretty simple, I just iterate through a list of objects and just call their respective member functions to make the movement happen and it seems to work fine for now.

Re:Choosing Functional Programming for Our Game by jstimpfle on Monday December 19, 2016 @09:15PM (#13216840)
See my simple Tetris Game, https://github.com/jstimpfle/tetris-on-a-plane/blob/master/t..., at the very bottom.

This implements a state machine in Javascript. It's a very simple case with only 2 states, "playing" and "paused". Each state registers its own event handlers on entrance, and unregisters them on leave.

This approach avoids the need to answer the question "where am I? what should I do?" every time an event handler is called.

While I think the approach is nice, the code is not so nice from a superficial point of view. The more general problem is that event-style sucks. Look up CSP by Tony Hoare, and Occam the language. What we need is better formalisms for sequential programs that include non-determinism like user events (i.e. view the user as part of the program).

(And, of course, also parallel execution of these sequential threads, and also sub-states that can access parent states in a way. Look up Harel's Statecharts. A good example would be a form with two text input boxes. We could view them as parallel sub-threads of the form thread).

Re:Choosing Functional Programming for Our Game by s_m_t on Monday December 19, 2016 @09:40PM (#13216975)
Nice, I implemented something very similar, except in my game not every controllable entity a) implements every control, and b) implements controls the same way, so I store what a control would do as a member function of each object and store an array with the names of these functions in each object. The indices of the array with the names of the functions match the indices of an array that stores the key codes I listen for, from there I match the indices, grab the name of the function and call it from the object being controlled.
Re:Choosing Functional Programming for Our Game by jstimpfle on Monday December 19, 2016 @09:57PM (#13217058)
In Javascript, you have to install event handlers. There is no control when they are executed, which is a problem.

When writing a game engine, you typically create events yourself at the top of the game loop (reading the input sources in a non-blocking fashion). You have better control what handler is called and when, but what handler should be called probably still shouldn't be decided in a central location. So I figure your object-based approach is a good solution. (The individual handlers can rely on the fact that they are only called at the start of the loop).

I realize a Javascript app could easily emulate the game loop style by letting handlers write to a queue.

Re:Choosing Functional Programming for Our Game by buzzybee on Monday December 19, 2016 @09:45PM (#13217002)
Yeah. There is a lot of routing/binding action going on in an engine when you get past the "hardcode everything for a single context" stage. I'm currently mixing about four big ideas together in my engine - they haven't all been deployed in the actual game project I have right now, but they're getting developed alongside what's there:

* Unix-esque hierarchical path routing - the addressing system I alluded to in parent comment. It's used as a mapping substrate - you can swap out parts of the route to quickly change what's being addressed, and it stores just a little bit of data so that leaf nodes mostly act as keys into other containers. I changed it from int-per-node to int-array-per-node today, which may or may not have been a great idea, but came up because it was irritating to have to make a special store for string keys.

* Micro-database system for allocating various types of engine data. This is the biggest source of friction in the engine right now - each data type has a custom container, and in theory, when allocated it'll automatically map to a path address. Engine scenes are currently loaded up through a script saying "allocate these things, relate these things together", but the addressing isn't present, you just access the containers more-or-less directly(ad-hoc per container).

* J. Paul Morrison-style "classic FBP" to compile data graphs. This is more of a "I want to program with Lego Bricks" kind of goal - it's the most directly interesting one for the keybinding problem, and it's used in existing engines for aspects of rendering graphics and sound. But it ultimately depends on having the addressing system functioning on real assets, so it's on the backburner.

* A "stack of small compilers" as in the VPRI STEPS project. This is used to turn game design assets(event scripts etc.) into runtime data by way of a behavior tree compiler that emits opcodes for a small VM interpreter. This is the stuff I'm really using the most, right now, and it actually inspired the addressing system, as compiling all that stuff created a need for it.

Re:Choosing Functional Programming for Our Game by adrianoconnor on Tuesday December 20, 2016 @08:49AM (#13219693)
Have you seen the Game Programming Patterns book (http://gameprogrammingpatterns.com/). It was originally a blog that later become a book, and I think you can browse most of the content online. It's full of interesting lessons and patterns and deals with coupling and state management. It won't have the answers to copy and paste, but it gives you a lot of food for thought and it's really well written.

I'm not a game programmer by trade, but I'm interested in that stuff and I really enjoyed it all the same. It made me rethink some stuff about enterprise patterns too.

Re:Choosing Functional Programming for Our Game by optionalparens on Monday December 19, 2016 @09:42PM (#13216986)
A keypress should never do much other than to update some game state to say that whatever key was pressed and it is mapped to whatever game concept in your input mapping. Nearly any code I've ever seen with an if/keypress, do this is an anti-pattern. Once you have your state, you just need to handle it somewhere later during a game tick, and keep in mind that might be even multiple places.

I would recommend thinking in loops, so it sounds you're already on the right track, you just need to expand that more to everything else. You generally want to do the same "thing" to many "things." Since you are newer to programming engines, I won't get too technical and I'll say that the gist is that the CPU likes this. The bi-product is if you take this approach, you'll tend to have better architecture as well as performance and solve some of the problems you listed. This is really the basis of an entity system approach if you want an example architecture.

Whether using entity systems or something else, my recommendation would be to separate out functionality and make them rather agnostic to each other. Again, this fits in with the loops paradigm. You still often need things to communicate as your problem suggests, but the way to do that is to often take some state, update it with new state, and pass it down the chain to see if anyone cares. That chain itself can be a loop, i.e. a bunch of systems in your game loop calling a method like update(tick, state, ... ). That said, you also need to be careful what and how much state gets passed around for performance reasons (ex: avoid excessive copying) and to prevent weird things from happening if you start introducing concurrent and/or parallel programming. To that end, I tend to keep things minimal as I can, or at least have some sort of way to hand things just what they need, not everything.

Queues also relate to all of this and can be your friend. A common communication approach is to use mailboxes for example and tends to be more straight forward to debug and optimize than using callbacks and event-based approaches (they tend to murder the cache and your stack traces). Queues also play nicer for being building blocks for concurrent programming.

Also keep in mind you may need to flip your thinking. Sometimes you don't actually need some action that happens during a tick to be processed the same tick. You can often or will often want to defer things to the next tick, or even several ticks after that. There's a Naughty Dog presentation on Last of Us port to PS4 that relates to this and how they flipped even the way frames are processed on an existing codebase to get better performance. The point is not for you to do what they did, rather to start thinking in terms of what the user will see as the end result. The user often doesn't care that you didn't finish processing some action last tick if it had no bearing on the gameplay.

Re:Choosing Functional Programming for Our Game by s_m_t on Monday December 19, 2016 @10:20PM (#13217184)
Thanks, right now every game tick I check a sort of register to see if any keys have been pressed, held, or released and then immediately call the associated functions to implement the controls. If I'm interpreting what you said correctly and I understood the Naughty Dog presentation correctly (I did see that one!) I guess I should instead add those function calls to a queue and then call them at my convenience?

That makes sense to me, I could split up the queue and send it to different threads to be worked on or perhaps delay doing things in the queue if a frame was taking too long.

Re:Choosing Functional Programming for Our Game by optionalparens on Tuesday December 20, 2016 @01:03PM (#13222264)
I wrote a long comment, but was rate limited at the time.

I'll summarize instead with a simple list:

- It's better to map your input to some game actions than pass around the raw value

- You don't need to do anything creative like queue function pointers themselves, just the raw values.

- You can have an input "queue" but that is typically different from a queue that is going to act on those inputs.

- You need to deal with multiple inputs potentially, like someone pounding a button. Hopefully your input lib deals with this some already, but always be aware of validating the input and deciding which one matters most. To that end, priority queues are good sometimes. You also sometimes want to cancel inputs. If you want to read more about it all, I'd read some about "intention systems" as related to input.

- Once your input is mapped, you can operate it in multiple pieces of codes, systems, etc. that might be downstream in your game loop

- You can transform an action into yet other new state as you go along

- Queues are indeed great for doing multi-threading, just be sure you select the appropriate kinds of queues. Mostly this ends up being thread-safe queues that are lockless. But sometimes you want to just use queues for FIFO, and these queues are generally faster and/or more flexible if they use a non-thread safe approach.

- You can have many queues in your code as mailboxes to distinct systems. Again, think something that is agnostic to the world around it and just receives some state it needs, and has its own "inbox" of things it might need to process this frame. It certainly could be that you can't process them all that frame.

- Keep in mind with queues generally once you pop something off, that's it. So you need other ways of hanging on to state and things that aren't ready. To that end, simple arrays are your friend, especially if you can pull things out quickly by index in relation to some id (ex: entity id) and keep them tightly packed. Dynamic vectors are garbage and mostly not used in serious game engines unless the developer had a wtf moment or the use case specifically calls for it. That's a longer topic though.

- Attaching functions directly to input again is a pretty bad approach. If you need more than one function, you'll have to implement something more complex. If order of function calls matter, you again need yet more. The order should be well-defined in the game loop itself.

- Regarding order, it also relates to communication as well. Note that you don't need to always process every new input or piece of state every tick. Sometimes you actually don't want to do this and defer it to a later time. There's no 100% rule, but an easy thing that helps is to think if the player or game sim will suffer because you didn't process that state during that tick. For some physics related things, that can be bad, but for other things, it's more of a "as long as this happens very soon, it's alright."

- There's no 100% rules for any of this.

Re:Choosing Functional Programming for Our Game by aschearer on Monday December 19, 2016 @08:50PM (#13216695)
Authors please heed the above warning. You are entering an extremely competitive, challenging marketplace and opting to play on "hard mode" for limited benefit. Worse you're doing it with your first project. You should focus on becoming an expert with vanilla Unity and shipping as many titles as quickly as possible.
Re:Choosing Functional Programming for Our Game by jplur on Monday December 19, 2016 @10:14PM (#13217152)
So I've shipped a few small games with clojure and unity. GC does come up, but it's very easy to optimize your tight update functions with Unity's profiler.

As for finding answers, the clojure CLR interop is pretty similar to JVM. I spend most my time looking at the Unity scripting docs.

Re:Choosing Functional Programming for Our Game by optionalparens on Monday December 19, 2016 @09:28PM (#13216908)
I couldn't agree more. If anything, the pleasant experience argument tends to come up more in the context of discussing a secondary scripting language to be added on top of a fast, already working foundation. Even then, you mostly just end up using something reasonably fast like Lua or one of its cousins. I can't see Clojure really being a good fit for any existing engine I am familiar with as a main or scripting language (maybe ClojureScript if we stretch it for something using JS, which I disagree with), and I love Clojure.

FWIW, scanning the Arcadia source there's also some garbage being created that can be easily avoided. I didn't really trace to see what each thing was "doing" or if it even runs during a game, but it certainly felt eye opening to me to see lots of temporary garbage created in something that is supposed to be lower-level. I am not sure at this point what would get optimized away anyway (ex: foreach loops, extra tolist/arrays used to create garbage in the past), so here's to hoping the devs just didn't get around to optimizing that stuff away yet. Seems to me you would want to minimize garbage from Aracadia itself given Clojure is going to create yet more, not to mention cut stylistic corners for pure speed in this context.

Re:Choosing Functional Programming for Our Game by ramseynasser on Tuesday December 20, 2016 @10:58PM (#13226682)
To your first point, Mooi is planning on using Clojure as a scripting language. Clojure compiles to CLR bytecode, and is in theory as fast as C# is as a language. Using the idiomatic persistent data structures has an overhead, of course, but Clojure allows you to be flexible with how and when you do that. I wouldn't say Lua or JavaScript are "reasonably faster" than Clojure is on the CLR.

To your second point, we're aware that there's room to optimize and we're actively working on it. We profile and optimize our code paths regularly, and the results are often surprising. If there's "obvious" stuff left in there it's because we're tackling bigger fish than e.g. a foreach loop. None of these are show stoppers, in practice, but this is part of why we're in pre-alpha.

Re:Choosing Functional Programming for Our Game by optionalparens on Wednesday December 21, 2016 @02:26PM (#13231873)
I am aware of the compilation into CLR bytecode and I've been a .NET user since the beginning, though I don't work with it much these days. The fact that it someone is using Clojure though doesn't mean it will translate always into the most efficient byte code. On the JVM, this is a problem sometimes forcing you to either write your Clojure differently to produce something more efficient, or requiring you to write a bit in Java and call it from Clojure (contrary to amateur practices w/Unity, people do need to share code that is not just abstracted by the engine such as some behavior). The same paradigm would be true of having to drop to C# and then call it from Clojure CLR. The problem here just becomes you get further from what is good about Clojure and instead write a version of Clojure that is closer to C# or something else.

Regarding speed, I think you misunderstood part of my comment. I simply mean you pick something decently fast when picking a scripting language with regard to the host language. Obviously Clojure is decently fast on the CLR in theory, but in practice may be another matter given the properties of a game which are far different than general use computing.

As for the speed with regard to Clojure, I've covered much of the general stuff in other comments, but I was in part writing to the point of using it independent of Arcadia in a scripting context. For this, I am contending there aren't many advantages to using Clojure over other approaches as a general scripting language for most engines, ex: Unreal, CryEngine, in-house, etc. Unity obviously benefits because of the CLR, but there are still issues with both Clojure CLR and using something that is yet another abstraction on top of Unity as others have pointed out. This makes life hard for newer or less-skilled game devs especially which it tends to sound like the original post authors are in general. More than anything, unless you have unlimited time, do you want to get a game done or do you want to help fix a new runtime?

Additionally, I am not comparing Lua to Clojure for the CLR specifically, rather stating that most of the time you add a scripting language to an engine in general it depends to be languages with certain properties. The reasons have little to do with Clojure as a language and more to do with scripting specific needs, which Unity itself doesn't really handle very well depending on how you define "scripting." That is any decent scripting engine must address some of the following below in a game, or it is not a scripting engine but rather just part of the main compilation target.

- Support hot code reloading, ex: from file system, user scripts, in game commands, etc.

- Compile/JIT reasonably fast if required

- Provide bi-directional calling semantics

- Easily integrate with the primary host language of the engine

- Decent experience for script authors

- Easy tool integrations, including with 3rd party middleware

- Minimize garbage if generating any, and make collection predictable if possible

- Not require the game code core to be recompiled in tandem (a primary motivator for using one from C/C++)

Clojure on the CLR certainly does some, but I'd argue not all well to be a huge benefit over C# in terms of productivity, performance, user experience, or user-facing scripting features. Of course there are some cool things due to Clojure's reader and macros, REPL, and so on, but the point is more the aggregate benefits. The reason people use Lua, Squirrel, AngelScript, and many others is not that they are the best, or fastest languages, but are generally good in all those points with regard to the primary language of the engine they are attached. Personally, I'd rather use other languages like Clojure, but there are tons of drawbacks that don't bite you until you are working on a real project, many that I listed elsewhere in this thread.

As I noted (and you repeated), I assume the code is alpha and there are still items to address. Nonetheless, most of these things are obvious and easy up-front, and many well-known in the Unity community, so I'm not 100% buying everything is a "later," just some of it. While in a normal game I wouldn't expect various optimizations or micro-optimizations, in an engine you definitely do. Even when working in C or C++, all the engines I've worked on we used our own allocators, minimized general allocations, added indexing tricks, branch prediction and inlining tricks, array packing, alignment tricks, and so on from the start, because that's the job of the engine and the end-user (game dev) cannot optimize away all of those things if the engine is the issue. Of course Unity doesn't do all these things well and things like Clojure layers are a lot further from the actual engine, but I'd argue that it means that everything needs to strive for equal parity or better performance when and if possible. Those are things you design around mostly from the start, the other optimizations that I think you are referring to happen later. I'm simply contending I see a mix of both.

Finally, following up again on the use of Clojure, as I've said in this thread, I'm a big proponent of Clojure. That aside, the CLR vs JVM implementations do not have parity because of lack of resources more than anything else. This automatically puts a lot more work in the pipeline and ultimately on the game developer, at least until things mature more. This isn't about using something like Arcadia at all, but rather using it right now given the original constraints and resources the article author mentions or implies.

Re:Choosing Functional Programming for Our Game by jackmott on Monday December 19, 2016 @08:05PM (#13216476)
F# with monogame will get you better performance and less weirdness, it is a pleasant lighter weight environment. unity gc and runtime are rapidly improving now that ms owns xamarin though.
Re:Choosing Functional Programming for Our Game by psyc on Monday December 19, 2016 @08:11PM (#13216502)
Where did you hear of the GC improving at all? I've heard vague intentions for years, but never anything concrete.
Re:Choosing Functional Programming for Our Game by jackmott on Monday December 19, 2016 @08:12PM (#13216512)
they have a roadmap with the plans laid out on their website. it was impossible in the past because xamarin did not want to sell them licenses for newer mono. now Microsoft owns xamarin, and they are happy to. some improvements have already landed at least in beta.
Re:Choosing Functional Programming for Our Game by psyc on Monday December 19, 2016 @08:36PM (#13216608)
Unfortunately, although it's on a page named "roadmap", it doesn't look like a GC upgrade is part of any concrete roadmap. It's in a big bucket labeled "research".

On topic, I just want to affirm that it's hard to overstate the inadequacy of Unity's GC, or the amount of time one can sink into optimizing for it.

Re:Choosing Functional Programming for Our Game by mquander on Monday December 19, 2016 @10:16PM (#13217168)
The GC upgrade is going to happen when they upgrade Mono, and the Mono upgrade is definitely going to happen, although I don't know if it's going to be before Unity 6. (Source: Unity people have suggested this to me.)
Re:Choosing Functional Programming for Our Game by elcapitan on Tuesday December 20, 2016 @05:37AM (#13218773)
Does anybody here have experience with F# and Unity? Unity uses older versions of .net, and F# is not officially supported, so it's hard to find real experiences with it, although it sounds very tempting. I would imagine that it would be most effective to have the performance-critical stuff in C# and then F# for developing higher-level game mechanics.
Re:Choosing Functional Programming for Our Game by hota_mazi on Monday December 19, 2016 @11:41PM (#13217505)
> I would be very concerned about straying too far from the garden path while also trying to ship a commercial game.

Agreed. OP is choosing the dangerous path of picking heart over reason.

It's not just that using a dynamically typed language for a game is suicidal, it's that you don't even have a choice in picking a statically typed language: C++ is pretty much the only credible game in town if you want 60 fps and a chance at not being ridiculed technically before game testers even look at your game from a purely gaming standpoint.

Re:Choosing Functional Programming for Our Game by AimHere on Tuesday December 20, 2016 @04:43AM (#13218580)
It's nowhere near that suicidal.

In this particular case, the game is being run using modifications to Unity, a very well-established game-engine, so C++ is already doing the heavy lifting (alongside Unity's way of writing GPU shaders).

Clojure is being used in place of the usual C# for game logic scripting and the like. It's much less performance-critical and so using something like Clojure is a lot less nonsensical.

Whether they're optimizing for the right things in picking Clojure over some other language is maybe up for debate (I'd guess their way would be easier to iterate changes, but harder to debug, and the clojure-on-unity stack they're using is pretty much alpha quality), but Naughty Dog already showed that commercial games can use a Lisp in a similar part of the stack with some measure of success.

Re:Choosing Functional Programming for Our Game by deepaksurti on Tuesday December 20, 2016 @07:45AM (#13219319)
>> It's generally a mistake to make the initial development more pleasant at the expense of long term maintainability;

Me and my friend are working on this game [1]. I have prototyped all the game logic in Common Lisp (using both LW Mobile trial edition and MOCL). We could really quickly iterate the game play and reach a design where the experience will be fun.

But to get it working with 60 FPS, we are writing the game now in Swift. A lot of our game asset pipeline is also in CL and Python. So as per my limited experience, I would prototype the game idea in CL; also use more powerful tools for offline utilities etc but ship the game in whatever is the best toolkit for that platform. In our case, Swift, SceneKit and SpriteKit.

[1] http://www.isongames.com

Re:Choosing Functional Programming for Our Game by tlarkworthy on Monday December 19, 2016 @08:05PM (#13216479)
agree, GC is going to slay this approach.
Re:Choosing Functional Programming for Our Game by j_m_b on Tuesday December 20, 2016 @12:58AM (#13217804)
Isn't there a good profiler for .NET? VisualVM works well for profiling Clojure JVM code.