What's your take on Software Complexity?

See, this is the weird sort of disconnect going on in this thread.

I said "most games are somewhat more than a skin on a database." I don't see how that's even debatable, and when people disagree with it, they distance themselves from the only claim that makes any sense as a disagreement: that most games are nothing more than a skin on a database.

If you're not going to make that claim, then it looks to me like you don't actually disagree.

 

Of course it makes it worse, but the topic is complexity. If you want to change the oil in your car, you don't build a lube shop. I assumed it would be obvious that don't you don't create deep systems for simple tasks/games.

But by the same token, there are fairly complex clickers (which makes me shudder a bit), that do utilize that type of system.

 

Okay, so it's "not technically a game," but "a fairly advanced MMO by any perspective."

I find your core position incoherent. It's an MMO, but not a game. We're already stretching the definition of MMO because if the drivers are the players and the users are the quests, none of the quests are multiplayer. And then we have to wrestle with whether Uber is "massive" and whether the use of an app makes it "online." I can see an argument for Uber being an ARG, except for the part where there's a G in it and that stands for something you just said Uber is not.

I don't see what you're trying to say. It feels like you're dancing and jumping around saying things that aren't really connected to a rational point.

 

As I said earlier, I misunderstood what you posted.

 

Was it this thread or another one where I mentioned the inherently indistinct nature of the second-person pronoun in the English language?

 

Actually there are. But regardless, my point was only that there are non-gaming apps that rival the complexity of games using the same techniques/methods, and scale. Yes, it was hyperbolic calling it an MMO, it would more accurate to say it has the hallmarks of an mmo, gamification, systems and scale. It is massive, multiplayer (player interaction is the only mechanic) and online (real-time synchronous).

 

Well, as I said, complexity is relative to both the developer and game. Lerp is just a percentage of progress between to values. Your phone bill lerps, interest on loans/bills lerps. The phase of the moon lerps. Yes, there is complexity there, but again to my original point, planning and building the model reduces complexity by reducing the specifics. If your game is complex enough that you can get hit with a rock, while sliding and it affects controls, just like in math, you reduce. Your system doesn't need to know that it is a rock or sliding. It just becomes how a type of damage affects a type of motion. Could be a rock, or a ping-pong ball, or a boulder. Could be sliding, skipping or swimming.

 

See, that's just disingenuous. You say something is an MMO, then that it's not a game, and then that you didn't really mean it was an MMO so much as that it has some things in common with an MMO. There's no substance there. Does Uber have some things in common with an MMO? Sure! Who cares? What use is that to anyone? What does it say about complexity?

 

Yea,.. ok.

 

Could Unity do with an interaction or event grid system, it has a physics layered collision grid but would a object/tag/layers interaction grid be a good way to manage complexity in games?

It could be a quick reference to code or FX that is triggered, a meta to do list that automatically expands with every new physical object in your game allowing you to manage your complexity.

With a good API it could even be used for bespoke game interactions, e.g. Spells, Cards, Events

So would an interaction grid help Unity developers manage and tame their games complexity as it grows?

It seems strange that we have this powerful 3D game engine and we could potentially be using some of it's power to provide better visualisations of our game's complexity...

 

I find it rather depends on what it is you're trying to achieve/implement. There is a gap inbetween the artistic vision and the user interface of the development software. Sometimes when the vision is very simple and refined or abstract or high-level, there can be an increased gap to overcome. You have to translate the vision into computer-speak and if there's a lot of difference between the vision and the computer then you have a S***load of extra programming to do implementing abstractions and tools and stuff to help make it efficient, or possible. Also similarly if the project is not necessarily simple and high-level refined but is perhaps complex - has a lot of parts etc, then it also can similarly require a lot of lower level stuff to be implemented closer to the user interface... not as much of a gap to cross perhaps, but more instances of gaps to cross. So it adds up.

 

@Arowx maybe if we all discussed specifically the areas of complexity we deal with in our projects (particularly as scope increases) this will help to focus the thread more on solutions.

I think we all have a good general idea of what you are referring to with complexity increasing with the development life / overall (revised) scope of the project. That's a given and the reason there has been so much focus on the area including the design patterns being presented as a way to manage the risk involved in continually bolting on new features.

Honestly, I think in games (and other software for that matter) one way to get around this that would be accepted by the end users is to go with a sequel (version 2, 3, etc). Basically gather up all of the enhancements and design the new version from scratch dealing with the new features and changes to the features in the current version. For part-time Indie Devs I think this makes a lot of sense.

As far as could having more of a framework help in minimizing the cost I think that is likely. I've been thinking along these same lines for how to approach Unity this time around. Basically identify all of the areas of weakness in the current model (things I want to be able to do that are not directly supported and must be built) first. However, in my case, I know a part of the issue is not having a deep understanding of Unity. Truly I think many newbies start using Unity and knock out things in a few days or weeks through simple copy & paste of various snippets here and elsewhere that I probably would not be able to. And if did manage to complete the same thing in the same time my solution would likely see me doing much more work than they are.

So part of it is, of course, learning what all is available. But it is more than that. The reason I take the approach I do is because typically canned solutions only carry a person so far. They may work fine for ultra simple use cases and bare bones scope that relies fully only on what is directly available out of the box.

It is when you want to go beyond that... even a tiny bit... that you find yourself (or at least I do) either building wrappers around the existing functionality to extend it or completely writing that piece from scratch. And that is the beginning of the hidden cost I think.

Which approach is more efficient? Most people would probably say to build a wrapper and use as much of the existing functionality provided by Unity as possible is the better approach. But we don't know that. If we ultimately end up running into a brick wall or have say 80% or more custom code then it would almost certainly have been better to have created our own solution from scratch to begin with even if we base it around the core concepts of the Unity feature.

At this point, I am not back into Unity enough to really even know what the issues are. I need to start playing around maybe with a 3D Cube Sky Fox game or something like that.

 

Lerp isn't progress, it's just the percent, lerp( a, b, .3 ) is "give me 30% of a and 70% of b". I'm sure you know this as a long timer, but it's worth mentioning since I think a lot of people actually don't really understand what lerp is - they just copy/paste it with time.delta after seeing it used that way in examples.

"planning and building the model reduces complexity by reducing the specifics"

I've found this to be an inefficient method for approaching code unless you are a legit expert on the specific problem that you are planning to solve.

It's like this, if you aren't super familiar with the specifics already, you're very unlikely to understand a complex problem. The more complicated the problem is, the less likely you are to understand it, and the less likely you are to be able to reduce it properly.

The hardest code to work with (in my experience) is code that reduces the problem incorrectly. It's just a nightmare to fix something when the core assumptions are flawed since so much stuff tends to depend on those core assumptions.

When it's really hard to fix a problem, the more likely it is to never be fixed, instead the problem is swept under the carpet in one form or another. This is the main reason why code bases atrophy. The sloppier things get, the more OK it becomes to not bother fixing other problems and stop caring in general.

The second hardest code to work with is code that tries to be too flexible. The more flexibility you write into a system the less decisive it is. More flexibility is almost always about deferring decision making. The problem is that later on you need to a decision, and the actual decision making is further and further removed from stuff that gets affected. "damn it, where is the code that actually does something" is a problem I have spent enough time dealing with.

I don't want a hundred knobs and configuration settings. I don't want a billion empty methods passing things around. I want to make a decision as quickly as possible and never even look at it again unless absolutely necessary.

I try to approach code (at least when solving an unfamiliar problem) with the understanding that it's a coin flip on if it's right or wrong. Maybe I guessed right, but maybe not. So, the goal is singular, make it as easy to understand and clear as possible. This lets me identify, fix and corrected problems as fast and completely as possible. Is the public interface wrong or misleading? Fix it. Delete it and make a different one if you have to. Don't try to salvage a public interface that's wrong. Correct the mistake.

Should a boulder be treated the same as a rock? Should swimming be treated the same as sliding?

Do you really know if that's true?

Maybe you do, maybe you're working on "Swimming and Sliding with Rocks and Boulders 4: Revenge of the Ping Pong Ball" and you really know your S*** when it comes to rocks and boulders. Most of the time though, we just talk out of our ass, and make a guess. But the code pretends to be more than a guess, the code pretends like you're really certain about the answer.

Who knows though, maybe I'm just talking out of my ass.

 

@frosted great stuff in your reply above. And this is one of the reasons I question the modern best practices as far as using design patterns to refactor things. It is a choice. Not the only way. Definitely there is value in it. But I think a best practice should be to simply fix things at the root as directly & simply as possible without layers of abstraction accepting this may well have a large impact that needs to be dealt with. Bolting on additional layers of complexity to abstract things as the patterns show works and can minimize the impact but it has also added complexity an additional layer that must now be learned and maintained.

I often think the biggest problem in software development has little to do with the actual development but rather it is the expectations of the non-developer people. The people requesting enhancements and other changes and then choosing a deadline for the implementation. That is what causes the bulk of issues in software at every company I have worked. X amount of work to do in Y amount of time where Y represents somewhere between X*0.2 and X*0.8 = rush / cut corners / focus on an arbitary date instead of quality.

Sorry for the ramble.

 

Fixed. The challenge is always to understand the goal. Sometimes the goal is in the specs, sometimes the specs are wrong. The guy writing the specs isn't perfect either, he might mess up. Happens.

Playtesting shoudnt just be for games!

 

Ha ha. That is a better way to put it. Very true. The main objective is always to make the moolah and to do that we need to meet the deadline (someone made up) and then also add in the extra features that nobody knew about but the sales folks promised they would be in there (just forgot to mention that until 2 weeks from the deadline). lol

 

Playtesting brand new code on a multimillion dollar machine that handles hazardous chemicals is always fun.

A decent programmer tends to be able to educate the customer on what the specs should be, where to look for edge cases, and where things are just plain dumb. Some of the worst projects I've worked on are when the programmer just goes ahead and does what was speced.

In PLC land you always have at least three obvious edge cases: start up, shut down and estop. And that's before you get into anything domain specific. Any programmer who can't educate their customer on these isn't worth their wages.

Of course that's not even where the real complexity starts for me. The real complexity is in the various different systems. The program has to interact with electronics, pneumatics, mechanical and optical devices. Plus it has to be safe under all circumstances. On a properly designed process automation system, I should be able to cut any random wire, network cable or air tube without anything bad happening.

But anyway, I'm getting firmly into non games. Don't want to upset the monkey too much. .

 

@BoredMormon a lot of truth there. Perhaps a difference in industries. The majority of the projects at my current job are done in what I think is the right way. It took a bit of time to get the process in place.

Each new project comes in at a research stage to estimate scope of work. So development actually gives the timeframe... the estimate of the scope of work in the form of the number of hours.

Sometimes things still stack up and there are multiple projects all with a high priority and sometimes there are other requirements that come in late. Not often but it happens. Not perfect just always striving to improve.

Anway this is the biggest step in the right direction for any software development business. Far too many companies have someone outside of development driving the deadlines. And it just makes no sense.

To just talk to end users come up with a deadline and then give it to the people actually doing the work. It is completely illogical yet all too common. I'm glad we are not doing that but I've worked like that enough in the past to know how insane and stressful it is.

 

In one way, games are trivial - it's sugarcoating on a dynamical database, as correctly noted. There are gazillions of ways that will work out to be "entertaining", as long as it's a dynamic process. Because it's a very non-exact process.

On the other hand, games have this absolute requirement of real-timedness. Which means, that EVERY damn calculation has to be extended and apportioned over time, and sometimes a half-assed calc result would have to be woven back in earlier than is optimal. Seems like multithreading and games are natural partners all the way, from this aspect. And multithreading is a theoretically complicated matter. Because, one has to face the fact, that these calculation chains get longer and longer, partly offloaded into GPU:s. Individual NPCs "thinking" on their own, distant unit/army logistics and pathfinding in complex environments, and so on. But as environment dynamics is bound to expand in radius from player (an increasingly dynamic world), the same communication and results recombination bottleneck that parallel programming researchers are fighting with will be creeping in. Well, what can I say? Certainly there will be more and more Academia within the development sphere. Iffy stuff like optimal node partitionings, workload balancing. Can't get away from the whitepaper stuff there, unlike the sugarcoats.

From that aspect, games seem a bit like operating systems, in fact - management of resources in real time.

 

"A decent programmer tends to be able to educate the customer on what the specs should be" will often not at all be the case in heavily specialized fields where the customer is an expert on the subject/process and the programmer lacks deep domain knowledge. I've found that many programmers (or spec writers) misjudge their own domain expertise and design systems that simply don't do what their users want.

Another instance is when the system is sufficiently complex, in these cases it's silly to think that any given person has the ability to predict the result accurately enough to judge a spec at all in the abstract.

The best thing we can do in these cases is be realistic about the idea that the spec is likely flawed and attempt to shorten the feedback loop between the user and the dev team. The value here is that problems are caught earlier and addressed.

It's kind of like saying "A decent programmer will not write bugs". The fastest way to improve the productivity of a developer and the quality of his code is not to pretend he'll never write a bug, but to find a workflow that allows bugs to be identified as quickly as possible, so they can be fixed.

Bad Spec == Another Kind of Bug

Tight feedback loops almost always produces a better result in the end. Although, admittedly, your domain might be really different from anything I've ever worked in.

 

It's possible this is the case. Programmers in process automation do tend to have a fair bit of domain specific knowledge. While I don't expect them to know what happens to the process if temperature TT05 gets higher then 180C, I do expect them to consider how I restart the process once the temperature comes down.

Process automation tends to be very collaborative.

 

As far as I see it, software engineering is, in most part, an art of keeping complexity of a software to manageable level. Language paradigms, design patterns, algorithms, best practices, and etc are there not because they are the only way to write a software, but because they prevent your software from becoming a complete mess which even you can hardly understand.

But even they cannot really stop a software becoming more complex as the codebase grows. The best they can do is to keep it growing linearly to the size, and that's in an ideal situation. Usually, it becomes complex much quicker, as the developers hardly can eliminate 'technical debts' as they appear but have to build on top of them, thus making it more difficult to deal with the problem later. They always say "I'll refactor this code as soon as I have some free time", but that time usually never comes.

In addition to this common problem, it seems to me that the gaming industry has some peculiar characteristics that might contribute to worsen the situation. In particular, it seems that unit testing has not become a common practice among game developers, at least not as widely accepted as in enterprise business field.

Lack of unit tests, and the reliance on a game engine makes it more difficult to keep refactoring vigorously as the codoebase grows, since you never know what might break if you change some fundamental part of your code without manually inspecting every game objects which might use that script, or without looking into some serialized values.

And I suspect there could be a certain cultural trait among game developers that they tend to see software components as a collection of 'scripts' - an isolated or disposable piece of code you can attach to some game object, rather than a persisting part of a coherent whole.

Even MonoBehavior, probably the backbone of Unity API, is such an ugly, monolithic mess of random features, so it almost seems like Unity itself is encouraging its developers to adopt that "Just attach another MonoBehavior script to your game object be done with it" kind of approach, which might not be the ideal practice to write a complex but manageable software.

Maybe I'm just babbling nonsense here, since I've yet to make a single sample project in Unity. But coming from the business software industry, I had some difficulties in adjusting myself to those peculiar aspects of game development, so it's just my observations so far.

 

Where it is honestly not as widely accepted as it should be, and then we would have to get into whether it's being done properly anyway.

There is definitely a higher degree of respect in the game dev community for "starting over." Many great game series are explained as being great precisely because every time they began work on a sequel, they threw out the engine and started from scratch.

I've been making both games and business software for 25 years, and I think your observations here are spot-on. But that's probably because, like you, all of my success has happened on the business end. I have two hammers, but only one of them has ever really worked for me when it comes to making a living, so naturally I'm inclined to believe it's a better hammer. This may not be an objective fact.

 

@mysticfall & @cdarklock It is good to hear these words. I never considered it before but perhaps the reason I had my views of Unity that were so different from the majority is because many of the other folks don't have the same background. Having made games (for fun) in many other languages and apis even several different computers plus being a developer professionally in the business, government & emergency operations sectors for a long time... to me this thing was about the most illogical thing I had ever encountered.

I'm not saying every 3rd party product I've interfaced with in my IT career is perfect but they are generally pretty clean and reasonably well documented. More importantly they work in a logical manner.

I am slowly getting into it. The light bulb moment I had several days ago that I should just plaster the project/scene with GameObjects for anything & everything was a major step in understanding how it is intended to be used. Still I find it to be so different from everything else I have ever worked with.

Anyway, didn't mean to derail or anything just glad to hear I am not the only one who had these kind of thoughts.

 

I really wish I could throw away everything I wrote for my current project and just start over

I'm still not feeling comfortable with the idea myself, so I just decided I should use a DI container instead. Currently, I'm writing my first Unity sample project with Zenject and I'm quite content with it as it enabled me to write everything as a POCO, which also made unit testing them a bit easier (wish I could run those tests inside my IDE, though).

I'm not entirely happy with how wiring and event handling part of my code, however. I wish there were more people adopting this approach, so I could just learn best practices from their sample projects or tutorials, and so on.

 

Chances are you can. The question is primarily whether you're willing to accept the consequences of that choice.

I put my current WIP out for commentary, got some commentary, then promptly initialised a brand new source depot and checked in an empty project. It's taken me very little time to reimplement what I had, and it's a lot cleaner. Also the new version has localised text and a better architecture for the save-load implementation.

I always feel like there's a best practice everybody knows except me, and when I say "here's what I do" everyone will point and laugh.

 

@cdarklock The previous project of mine contains some 280k+ lines of Java code and I'm in the middle of banging my head against a 65k+ lines Scala project. So I certainly don't want to accept the consequences myself.

They might not look so big to some people, but as there's only another developer in my team, so rewriting it from scratch would take me considerable efforts.

And more importantly, I just want to minimize the time I spend on my real work and do a full time game development, instead (hope my colleagues won't read this )

 

That sounds similar to the way I used to develop in Unity before deciding to try to wrap my head around the way it is supposed to be used. I had just plain classes that didn't inherit from monobehavior but instead they had fields that were references to all of the relevant Unity specific stuff. GameObject, etc. I just handled everything myself. It actually worked quite well. But was also completely not the way one is supposed to be developing in Unity. lol