Johnathan Blow on a Programming Language for Games

Just thought you might find this interesting, Jonathan Blow talks about game programming, languages and writing a game specific programming language.

Does he have a point the Garbage Collection should not be a feature of a game programming language, due to it's impact on performance with larger asset loads?

Still watching it, but he does mention Unity and that GC is a performance "bottleneck" for higher level games.

 

Not watching it yet, but I wouldn't argue that garbage collection is necessary. There is a spectrum of convenience to performance with GC's that I could understand not wanting a GC just to get every once of performance you can.

 

Garbage Collection simply needs more manual control. If I want to keep it from running or only run it at specific times, I should be able to do so.

 

Or you could just go for a language that does not have a GC, but makes using memory extremely simple.

 

Yes, but why remove the GC when you can simply achieve the same thing with an off switch?

 

Or you could just go with D.

Sounds pretty sweet, if you ask me.

 

This started a major talk on the RUST Programming Language subreddit.

http://www.reddit.com/r/rust/comments/2gwi11/jonathan_blow_ideas_about_a_new_programming/

 

It's much ado about nothing. There's nothing here that's important, that hasn't been solved repeatedly. You've got blitzmax, swift, C# (Unity) and many other solutions. Even UE4 allows for GC on/off within C++ code. I've rolled my own contained memory management in C++ before. It's not something new being discussed here and I'm not sure why this is suddenly something new. It's been done to death by people who really know what they're talking about.

 

Something he did not mention is the potential for a game specific language to provide concurrent programming on the CPU and GPU. e.g. imagine just writing a shader in C# or Unityscript.

Let's face it a modern game language should have a very simple syntax to allow developers to write multithreaded code at any level and it will run on the CPU and/or GPU.

 

I disagree allot of things can be improved in terms of both quality of life and simplicity of the language. C++ is complex - I am currently taking a 101 course in it. While I may not be eye to eye with his views on things in general. I can see why we need to get away from C++.

 

How much of the complexity do you use on a regular basis though? There are some keywords in C++ that aren't even remotely useful in most situations.

I spent about a month learning in back in 2001/2002 and despite rarely touching it over the years it is one of the few languages I am able to jump into at any moment and immediately use.

 

1. node based shader editors are the future and the present, not writing shaders in c#. In any case there's already people writing shaders in c#. Compute shaders are also the future and use a c style syntax. Shaders are already solved. He's added nothing.

2. C++? Isn't Unity in C# ? Don't most major engines use LUA and custom high level script languages? who the hell uses pure C++ for games? Even UE4 has blueprint and managed C++, enabling you to dip in and out of complexity as you wish. Then there's literally dozens of managed game-specific programming languages already that do what he suggests, and have been available for decades.

I don't understand why its being discussed like it's something new, or why people are thinking it's new. Go do some research.

 

Unity uses C# as a scripting language but the engine itself is written in C++.

Primarily those who don't use game engines. Making a 2D game with C++ and SDL is pretty straight forward.

 

I do like the idea of possession in the language, although how would you manage multiple pointers claiming ownership? The general idea that the language and simple declarations during setup would account for a good chunk of what anyone needs garbage collection for sounds like a smart idea to me.

It's mostly just food for though, but let's face it, most people don't put a ton of thought into what language they use. Whatever language is in front of them is usually the one they keep using, unless it is such an abomination that they just have to switch.

 

Thing is, these are questions AAA ask themselves on a daily basis, and things that giants like Unity and Epic make their business out of. While I understand Jonathan Blow is still relatively new to game dev itself, and is an influential voice, his discussion is a little bit naive, as if it's not a subject that's already quite mature. It's kind of cute really.

You don't want to make another language at this point. That's pretty much suicide unless you own an entire platform (Apple/swift) or you're able to use a C++ core with a high level language of your design (what most of AAA does).

There's no point in making engine work easier. That should stay C++ until a more optimised language appears. Currently, there's nothing more optimised than C++ out there. The compiler work that's gone into C++ is insane, decades of research and work by the brightest minds in the world.

So solutions center around splitting the load between low level (C++ engine etc) and high level (scripting) and very high level (blueprint).

This tiered approach is far wiser, because anyone making an engine from scratch in this day an age has probably got a) million/billion dollar resources b) is japanese - it's kind of fashionable to do it over there or c) doesn't really know what they're doing and will find out the hard way later.

Tiered is better because:

1. The Engine gets to be as fast as possible
2. gameplay can use a high level scripting language
3. artists can still contribute to behaviour of game and level design

So you're catering across a wide range of people, and this is something common to all big game engines. Rather than try to reinvent the wheel, and make a S*** job (it would be a S*** job, don't fool yourself) by making a new language, simply use the right tool for the job.

I think it's admirable an indie would consider a change of this magnitude even if it's not realistic and is terrible business sense. I'd rather pour time and effort into evolving what we have, that is proven to work with even better tools, optimisations and workflows. I don't actually have a problem with Unity, and that's mind blowingly easy. The GC is a monster that can be tamed quite well, and is deserving of R&D.

That's my piece said.

 

I don't think there's any need to be so hard on it. Sure, it's all been said before, has been repeatedly visited by far more experienced minds, etc. etc., but what was ever wrong with discussing something that's been discussed before?

The content of a discussion is only half of what's important. The other half is the people who are having the discussion. If this is bringing the discussion to a bunch of new people then why is that a bad thing? We who've already had it don't have to jump in again, and those who haven't may very well learn some cool stuff.

 

I'm all for thinking about how to reinvent a tool used every day, however when I heard that he uses C++ and knows some guys who still insist that C is the way I had to turn it off. I don't have 2 hours at the moment to spend listening to someone who hasn't heard of Actionscript, Javascript, UnityScript, C#/VB, Unreal Script/Blueprints, Objective C, swift, Python (esp. stackless), SQL, PHP, Java, Erlang, Lua etc.

If you want to invent a programming language for games, first learn about what programming languages games use.

To reiterate: That's not to say he might not make good points, there is value in thinking about the tools you use (I do it all the time), just poor set-up given the topic.

 

There are so many languages in use though. Practically every language has been tried for game development at some point regardless of whether or not it was practical.

 

Hi my name is hippocoder. I'm the best at everything and everything and everyone else is rubbish.

Aren't moderators supposed to be nice? How are you helping Unity with that attitude?

Or are you playing the devils advocate? aka just trolling to make the discussion more interesting?

 

I am personally tired of the constant flaming / Trolling going on these forums.

That said - It needs to stop.

 

Really not getting the hate. Most of what he's talking about is taking C#, removing the value and reference type associations from structs and classes, letting pointers be a perfectly normal thing with any pointer manipulation being declared unsafe, and adding ownership to objects in the heap so that it isn't overly laborious to have control over memory management.

 

I agree with angrypenguin. And I don't see this discussion as so much about what's been done before, or already done. I see this discussion as about how to move forward in the future. How to construct a tool for programmers that gives us the maximum amount of control over every aspect of game development (including memory management) with the easiest possible way of doing it.

I think that's a much desired and admirable goal.

 

Interesting, I skimmed through it a bit so might have missed a few things but he is probably right in that something like Rust in which security is paramount is probably not the most efficient for games. Just look at the huge rewards browser makers offer for double free's, buffer overflows, use after free, etc. It is a huge problem that language is trying to help solve but not quite as important to games.

 

I only have experience with Unity when it comes to what I consider real game development. And I mostly do C#. I've done other stuff in the past that really doesn't apply here.

I don't really need a new language, or even a different one to be honest. In this area, what I really need is for Unity to take advantage of threading and encourage it, not just barely tolerant.

I would like a little more control of GC in general. But in my ever so humble opinion, the problem is not GC. The problem is the CURRENT GC. Which I realize they are working on.

But C# is awesome.

 

You say this as if you don't think that games still heavily use C++. Or was he talking about it in the context of the high-level mechanics implementation stuff?

I agree with hippo that the high-level and low-level stuff are different problem domains and thus have different ideal solutions.

 

What he's talking about is essentially engine development. If you're using Unity, then his discussion really has nothing to do with you. Not that it isn't an interesting topic, but just not very relevant to what we're doing here.

 

Jonathan Blow has been in game dev for almost 20 years, going back to a port of Doom II he did for set top boxes in 1995. He was a programmer on Oddworld, Deus Ex Invisible War, and co founded the company that did one of the early online multiplayer games for TEN called ShockForce.

I don't know if you watched the video, but he makes a very compelling case for game programmers to take control of their own future and happiness with a better language than C++. If C++ wasn't crap to deal with, we'd be switching to Unreal, right?

 

But people are already doing that, on a daily basis. I don't understand how repeating it makes it somehow better or more.

DOING something about it, now that I can understand.

 

First, I agree with everything you've said, without exception. And, I speak as a hobbyist indie. I'm not saying C# (or C++, or Lua, or whatever else out there) is perfect, but the problem has already been solved, and people use the solution every day. Even I can see that.

As far as workflows, I agree that refining those is the true problem, specifically art workflows. Writing engine code is tough, and should be, it's the part of a game that needs to run the fastest.

However, with engines like Unity, UE4, and the competition out there, this has already been solved, for a while now. A complete neophyte to game dev (me, in 2009 when I joined, funny enough) can now get a modern game engine, graphics software, music software, sound editing software, productivity software, and probably a partridge in a pear tree, for free or very little money.

What does cost (beyond practicing until you're good enough), is artwork. Modern game art is, in my opinion as a programmer, many times more complex than anything we programmers do, and for good reason: our tools have been optimized over decades by the brightest minds in the world, which has culminated in 4th Generation Programming Languages (like C#). I don't feel that the artist's tools have matured nearly as well or as quickly, and it may not be possible.

The amount of redundant legwork that you have to do in any modern art software is ridiculous. In programming, we typically take groups of related instructions and spin them into a function, which when we call it, it does all those things. Artists have no such recourse, and artistic actions are typically intransitive. In fact, it may not even be possible to refine art software to be as efficient as programming software.

If the artistic pipeline can be refined at a broad level to be less convoluted, require less steps, and require less technicality, I feel that we can be able to make games, virtual artwork, and simulations much more cheaply and effectively, without sacrificing visual fidelity. Unfortunately, I'm not nearly arrogant enough to claim to know where to start solving that problem; I merely sidestep it by sticking with my 2D-character-on-3D-background style, which I find to be cheap, easy to reproduce, but still palatable to a reasonably wide audience.

 

are you serious?? "be nice" lol chill out! he didn't say anything that bad hahaha actually I'm happy that hippo isn't talking like a tosser, and by him helping out moderating the forum doesn't mean he should become "Mr nice". I appreciate his honest opinion and prefer it.

"Hi my name is hippocoder. I'm the best at everything and everything and everyone else is rubbish" don't get me started on this sort of thing.. manipulation is not needed in this forum.

 

I've a thick skin I can take it. Moderators aren't necessarily supposed to be nice, no. We're not hired by Unity Keep on topic please, and if you're going to post (yoonitee) then I'd expect some form of contribution to the discussion like everyone else has done

If I'm not right, or you disagree then explain it and I may change my mind, it's called a discussion forum.

 

Well the idea of the topic was to get people thinking and talking about the possibility of a game programming language.

What features and style of syntax would your ideal game programming language have?

Or are there features and syntax elements of C# and UnityScript that you don't like or could be improved?

Would you want to keep the GC or have an optional GC with manual/automatic controls and/or have enough control to do your own memory management?

 

Well you've used Blitzmax and there's swift...

 

Can you state the problem that you think has been solved, and the solution you think has solved it?

This does still get discussed from time to time by industry veteran game programmers, which suggests that it's not done and dusted in their eyes. Nobody's saying that they can't get their job done because of available languages, but it's perfectly healthy to discuss imperfections in current approaches and potential ways to improve them in the future.

 

I think this whole argument of using C++ or not is based on the two following points:

- Wanting to be a complex engine developer.
- Being in the AAA industry.

If you're an indie and you have tools like Unity available, don't worry about C++'s relevance since it does not relate to you.

 

The problem I think has been solved, is that games somehow warrant a specific language.

The languages/tools that I think have 'solved' it*, are C++, which pretty much is the lingua franca of engine work due to its sheer speed and power. Lua, on the other hand, is extremely well-regarded as a higher-level scripting language that gives designers a simplified instruction set with which to make the game come alive.

On that note, a thought I had earlier today, is that a singular language for all aspects of game development might appear to be (perhaps) convenient on the surface, but it isn't all that it's cracked up to be. As a programmer who is involved in web development on a daily basis, I look at each language I use as a different tool, as each is good at different things. JavaScript (not UnityScript) is a functional language, well-suited to deliver behaviors to a webpage due to the more esoteric aspects of that language. It doesn't feature multithreading, or a bunch of other cool concepts, but if you're just building a business webpage, you don't really need that.

Conversely, the object-oriented C# code I write for the server has useful abilities, like being able to assert locks on blocks of code, declare 'unsafe' regions where I can do more advanced memory management. I need the C# to take care of itself, but I'm also not worried about it being super-simple.

You might say, 'Well, why not code your web apps in all JavaScript? It's perfectly doable using Node.js! It all uses the same coding paradigm. It would make your life easier.' All of the above are true. What makes server-side JavaScript inferior to compiled C#, are those little things, the capability to do lower-level things. Sometimes, it's necessary to implement a bit mask across an enumeration. Even though there is a well-known phenomena called the Object-Relational Impedance Mismatch, it's often much easier to prepare and work with data on our servers using an object-based model, instead of worrying with the various types of objects you can use in JavaScript (Object object, function object/closure, prototypes, JavaScript's weird and poorly notated version of 'inheritance'...the list goes on.) Just because I can use JavaScript for everything does not make it a good idea, and does not automatically imply readable code.

TL;DR of all of that - there are different domains that a general-purpose language would fail to help the human programmers explain to the computer what they want to do.

Back on the topic of the video, though, I can't help but wonder if the games community might not be able to drive standards for some languages, to either A) improve said languages to the point that they're more efficient for games use, without precluding other non-gaming uses, and/or B) adopt some of the mechanisms that were talked about by Joe Blow?

For instance, the speed improvements that JavaScript is undergoing through Mozilla's efforts to bring direct-in-browser gaming may not only make direct-in-browser gaming more of a thing than it already is, it means that the business apps my team is serving to our clients will perform better, which (if we have the time) means we can make more impressive business apps.

*: Caveat being, nothing is ever 'truly' solved, it's just in a state where we can't yet envision a more optimal approach that is economically feasible. Theoretically Joe Blow could introduce his game programming language tomorrow, but would it make sense for you and your staff to drop your C++ expertise in the hopes that this new language solves all of your problems with a new compiler and a 1.0 specification?

 

Not sure what to make of this thread reading some of the responses. If you're not interested in new language development then don't pay attention to threads discussing that topic.

It is not fair to say everything has been solved so there is no need for a new language, with thinking like that nothing new would ever be created. I thought he did a reasonable job explaining the features he looks for in a language that would optimize his performance whether I agree with them or not. I remember when C++ came out and it was all "blah who needs this, we have C!", same thing.

 

But what's the problem? "A specific language for games" is a proposed solution. What is the pain that it's trying to solve? What is it trying to make easier, less risky, or faster? What are the downsides to how it's currently done?

People do write games in a variety of languages already, and have been doing so for decades. So the problem isn't that we need a language to be able to write games - as you say, that's "solved" at least in so far as people are repeatedly and successfully making games with the tools we've already got. That's not what this is about.

Here's a discussion by someone who clearly understands some problems related to C++ in the use case of game programming discussing the current state of potential alternatives, to give some idea of the kind of problems people think about in this area.

Also, this...

... is covered extensively in the linked discussion. The thing is that like tiggus says, refusing to even consider this stuff means that we'll never advance in those areas.

 

There is such a language. It's called KL.
It basically has:
- just in time compilation
- Can run on CPU or GPU
- Can do multithreading (automatic or manual)
- Is multiplatform (PC, Mac, Linux.. based on LLvM)
- it's a high level language (more like java script than C++)
- highly optimized for performance (LLvM)

The guys who developed it, at Fabric Engine made quite an impact in the VFX industry. Where a lot of the big studio make custom tools / programs. (A very performance critical environment)

I think they showed that code, optimized by the computer is more flexible and often faster than optimized by humans (low level C++)

http://fabricengine.com/

http://en.wikipedia.org/wiki/LLVM

 

@danybittel KL looks like it's doing the right thing, but would it be a good game development language?

Should a next generation game development language also aim to be the lingua franca of Virtual Reality, where you're code can be running on a device or on the cloud to provide the user/player with an immersive presence within your game/virtual world?

If so would it need to have networking and serialisation built in?

It sounds to me like the 'ideal' game programming language would need to provide
run on CPU/GPU/CLOUD
Multithreading,
networking,
serialisation,
high performance memory management either GC or Manual,
FSM's as a primitive

 

Having read the article, what I'm taking away about the current problems with extant languages (C++ and Lua)...

1. C++ is painful to use (You'll get no argument from me on that one.)
2. C++ does not have either a robust-enough type system, or a very poor system of working with types. (He commented on a number of 'type'-related subjects, like how OOP is passe, and how D has a pretty awesome type system)
3. More effective means of dictating memory management would help achieve better performance.
4. A performance-efficient asynchronous computing setup would allow for all sorts of useful features.
5. C++ has very long iteration times (build...compile...test...document results...fix bug...check in...unit tests...build...etc. I remember this well, and not in a game programming context.)

What didn't occur to me was that Lua itself was a partially-successful attempt to resolve the iteration times problem. I also wasn't aware that data-driven logic was even a thing in the first place.

I found it interesting that he says that C# in and of itself is great as a language, it just lacks features that would make it really useful for hardcore game development. And, as someone whose work is primarily C#, I think he's right.

So, I didn't understand the problems, and I still may not - low-level development isn't something I do - but, I feel I've got better perspective on the matter. So, thanks bro.

 

I think his talk will be useful to language designers to see what kind of thing game developers want from a language.

Although I would probably disagree with him where he says writing a compiler is easier than writing a AAA game.

 

I think he's referring to the fact that we now have .net style intermediate language layers that are then compiled to native code or can be compiled at runtime. So in theory a new language is just a new top layer on the compiler stack.

Or you could consider that once a compiler has de-tokenised a language they will all start to look the same as now it is constructing assembly code for the cpu/gpu.

And as long as you don't want some amazing new features for your language then the work will already have been been done for other compiled languages.

 

One thing that's bad about C++ is that it is not easy to return a tuple or struct on the fly. i.e. a function can have several input variables but just one output variable. (Unless you specify a class somewhere else). It would not be so hard to modify C++ so that you could do this:

Code (CSharp):

1. (r:radius, c:color) getCircleInfo( id:int, group:int){
2.     r = circle[id].radius;
3.     c = pattern[group].color;
4. }

for example. Other languages such as Go can do this. There are lots of other examples where C++ could be improved. They have put in lambdas but I don't much like the syntax. It is not very mathematical.

As for memory, allowing you to set pointers to null or zero seems like asking for trouble.

 

Not sure why that would be needed or even a good idea for a strongly typed language. Aside from setup in C++ being a pain in the ass, so you're too lazy to write the proper objects/objects properly, I don't see why it would be necessary.

And why would null pointers be bad? They are only bad if you assume it will only be assigned, but you know what happens when you assume... RIGHT?

 

The thing is, null pointers are a rather bug-introducing part of a language (and, I'm not saying that because of the guy whose document Angrypenguin linked.) This is why some newer languages - like Rust - only support 'null' 0x0 pointers in unsafe code blocks.

(Another round of thanks to Angrypenguin. That article made me investigate Rust more closely, and it seems like a pretty awesome, if immature, language. I have high hopes, but low expectations for it.)

 

Apparently I just like nulls then, because I would probably have more bugs that aren't easily diagnosed if I couldn't flush a reference. I would have to spend much more time figuring out whether a pointer is actually there to be worked on or if it's a leftover, or I end up making special instances of an object to shortcut processes that I would just have to take as a given.

All of that to replace going "hey, you null? you are? sweet, then I have nothing to do."

 

I don't like nulls, and one of the bits of boilerplate that I always worry about in my C# work - both games, and my Unity projects - is finding cool ways to avoid null stuff.

The way I see it, if I declare a variable of some type, there are very, very few reasons for it to be null (unless it's a nullable type, but even then, I rarely see reasons outside of web development for that.) A default (yes, that's a C# key word) value? That's OK; at least it's the default of that type.

The point the guy who wrote the article above made is that nulls fundamentally undermine the point of a type system in the first place, since null is by definition, not whatever that type is. I would extend that as a logical criticism of C#'s nullable types, since you can read 'int? foo' as 'Foo is a variable that can be a 32-bit integer or null'.

That's why LINQ extends IEnumerable with FirstOrDefault or SingleOrDefault - not FirstOrNull or SingleOrNull. It makes no sense that, if you're querying an enumerable collection of [insert type here] for a member of [insert type here], that if you find nothing to get null; it's way more reasonable to have a default object of [insert type here], and have a mechanism built into the language to check for exactly that. Even if your type is nullable, null is nothing, and therefore not that type. It breaks the logic of the code (I'm not talking about logic in a business way, I'm talking in a literacy way. Go back to my nullable int example above, it adds an unnecessary caveat to what should be a number.)

Of course, these are just my opinions. And, at present I am a junior developer. And, I don't do low-level engine work. But I have had questions that apparently others share about why nulls are necessary.

 

I would read that as Foo is a container of type 32 bit integer. I don't care that it's a bottle of gin when there is no gin in it. At that point, the type of the container has no bearing because there is nothing I can do with it.

The problem with default is it appears just as valid an object as any other. It requires far more work to actually realize something isn't right. It's pretty much finding out the bottle of gin is actually water while you're drinking it. If you aren't drunk, you're going to be pissed, but if you are, you're getting ripped off while being unaware.

At the end of the day, I would much rather see a null ref exception than presume everything is peachy.

 

The problem with having default values instead of nulls is that you can't distinguish between a result that gives you the default value and a result that gives you a legitimate value which happens to be equal to the default value.