I’ve enjoyed today.

We (the TaleSpire team) had a quick meeting to look at what was feasible in the days remaining. We are still going to try the closed alpha out on the 31st so it’s all hands (all 6 hands :D) in order to make that. It’s gonna be tight but it could work. Minimal viable product is the name of the game.

Campaign history is in a very basic working state and so I jumped back to looking at unique creatures. The UI had a few requirements I hadn’t met yet so I did a bunch of work getting that in shape. I finished that about half an hour ago. In short the list of data describing the unique creatures in the campaign can be used to spawn the previews we use when placing creatures. When the preview is placed we spawn the actual creature object at that spot passing in the ‘unique creature info’ this let’s us populate all the correct values (name, hp, etc) and tell the backend that the creature has moved boards.

UI is getting tighter as Jonny is hammering at it night an day. We are not going to have the final look by the 31st but we will be iterating on it pretty hard (along with everything else) in the new year.

Tomorrow my primary two tickets are getting the Steam signup/login finalized for the alpha and fix the ‘sync on quit’ feature. If neither of those cause any drama then I’ll probably look into a rather cute fog of war issue. Currently there is a case where if the obstruction object is on the boundary of a zone it may not show, which is terribly confusing when it’s a locked door :p

Goodnight folks!

Today has gone well. Not a whole bunch of stuff to tell but the invite panel is fine now and ui work is continuing steadily.

I found some bugs that require a tiny bit of context to explain.

Every tile in the board has an ID the game can use to address it. These IDs are how the network code specifies a tile when it wants to say something like ‘delete that tile’. Naturally IDs need to keep the same on every client and no two tiles should have the same id.

The bugs resulted in duplicate IDs. What was happening was that we synchronized the IDs when sending the board and we weren’t including the ones from things that had been deleted and were in the undo/redo history. This could result in the following sitation:

• Player 1 places 10 tiles (ids 0 - 10)
• Player 1 leaves
• Player 2 deletes some of the tiles (ids 5 - 10)
• Player 1 returns and Player 2 sends Player 1 the board (sending ids 0 - 4)
• Player 1 places 3 tiles (starting at 5 as it thinks that is free)
• Player 2 hits undo (restoring tiles 5-10)
• tiles 5-8 now have duplicates.

We could either issue new IDs on undo/redo or just make sure we keep better track of ids in the undo/redo history. The latter felt like less of a band-aid so I refactored the code to handle that. It took a while to test enough to feel comfortable that it was behaving properly again but it seems ok (comforting no?).

This delayed some things but is always worth it as it’s such an ugly bug when it appears as it normal manifests differently on the different player’s clients.

Tomorrow I’m back on session history which in theory should be simple. I think after that I’ll need to tackle one of the remaining tickets which could require significant re-engineer in the worst case (most of the tickets have fairly understood scope).

Until next time!

Today we landed the new UI, we are now hooking it up and squashing bugs.

The tickets are rather specific but here are a few as the daily so far is a tad short :)

• Start hooking up the campaign invite panel
• Allow a lone Gm who is in player mode to perform actions that would normally require Gm approval without the request.
• Add hooks for providing strings to the session history panel (only the lua state-machine scripts are using it right now)
• Handle more cases where board loading can be interrupted
• Tweaks to the radial menu to make it feel nicer (such an insightful daily)
• Change under what circumstance we focus the camera on a creature (For some reason I had it set up to focus on the first time you selected any creature)
• More things that are so specific to the internals that it would be meaningless

Ok. That’s enough that I’m not entirely wasting people’s time by posting this.

Tomorrow we’ll be working on more of the same. I also need to look at the code to react to a player’s rights (e.g. if they can gm) being changed during the session.

Peace

I hadn’t expected to be doing a fairly hefty refactor of the board loading today but that’s where I’ve found myself. I’m trying to nail down the behavior that occurs when the user you are syncing the board from leaves during that sync, or if the sync fails, or.. well you get the idea, we are looking at failure cases and how they behave.

The downside is having to retest a bunch of basic functionality but naturally the upside is that we are fixing issues the old one had.

The new UI hasn’t landed yet so nothing new to report there yet.

The only other important thing I’ve been working on today is adding a retry callback to our http request so that, in event of failure, you can choose if the request is retried or not. This was a pretty quick change as a bunch of the csharp code that wraps the calls to the backend is generated from a description on the erlang side. This has made it easy to make frequent changes to the backend api and know that the frontend code matches it exactly. More on that another time though.

Goodnight folks. Tomorrow is just gonna be more of this robustness work.

p.s. Reentrancy in state machines is a pain

Hey again, back after a few days without dailies.

Development is still going ok. Jonny has been working hard on getting the new UI to the point where we can merge it into master, looks like we are close there now. Once that lands we can hook things up and write the features that were waiting on that.

I mainly been working on query retries and managing the connection to the backend. I’ve merged in the branch with support for Steam login so I can get back to that soon. Board checkpointing took a little longer than planned but is working nicely now. At the start of each play session a new file for the board is created on the backend so we keep a history of those boards. For now we will just let these pile up but we’ll have to look into how many we want to keep around in future.

Another boring but necessary feature from today was adding limits to the number of campaign invite codes a person can make per minute and per day.

Tomorrow I want to look at having queries check with the connection manager to see if they should bother retrying on failure. I also need to look at the error codes from the backend queries so we can give meaning messages or take specific actions. After that it might be making the version of the steam login that we will use in the alpha.

Along with this we are squashing bugs but also finding more so the deadline is still looking very tight. I mean the worst thing that happens is the alpha is in January, but man.. I’d really love to get it out this year.

Ah well, gotta keep hammering.

Seeya tomorrow

I haven’t been on r/lisp for a week or so due to being rather low on free time as we try and get an alpha of our game out of the door. When I did pop back I saw ‘5 Reasons why Lisp Games Suffer and proposed solution’ which I thought would be interesting, but what I found was bizarre and misinforming.

This article is, in my opinion, wrong from end to end so I’d like to go through it a bit. I realize this may come across as shitting on the authors intent which isn’t my goal however being named explicitly in the article I feel like I can respond in some fashion. Also it’ll sound like I’m shitting on lisp, which I don’t feel I am. I adore working in this language but it is also just a piece of technology, just a tool, and we should be realistic about our tools when it comes to a given context.

The article

Let’s start with the first line

Lisp games, like the Lisp AI and Lisp Machine hype in the 1980’s, has proven yet again to fail to deliver.

This implies some massive lisp game hype akin to that of the lisp machines in the ai boom..so where is this hype? I’ve been around the lisp game community for a while now and none of the people who have been most active in making things are spouting anything like that line suggests. If there are people exposing lisp as some way to fix the game industry then they probably havent been making games in lisp. A lot goes into a game and decent programming languages are one part.

Let’s say I’m a lisp noob, or even an average gamer. I think, hey, what’s the state of Lisp in games?

Ultimately gamers don’t give a shit about your tech choices. If it runs well and is fun it could be TCL and MUMPS for all most people will care. Yes there are higher profile articles diving into the tech of games and these get a lot of followers but this doesn’t mean the majority are peering into the internals and judging games based on this.

sidenote: for those interested in this stuff I highly recommend Adrian Courrèges articles

The Poverty of #Lispgames TLDR: Lisp games has many channels, but not enough content

I’m not sure what ‘not enough’ means in this case, not enough for what? for who? If it’s not enough to satisfy the fallacious first line of the article then there is no doubt that this will never be satisfied.

Setting up channels for sharing content is a cheap endeavor compared to making the content. People are having fun, they enjoy lisp and they want to share stuff and big platforms make that easy, I’m not sure that holding off making those channels until the community reaches some assumed level of success makes any sense. Unless your goals and assumptions are rather out of sync with what those making these channels seems to be.

But where are the games?

SURPRISE! games are hard and just having a nice language doesn’t make that go away.

Also a lot of us enjoy working on problems, and games afford us a lot of interesting limitations which allow us to stretch our minds by finding solutions that would fit in this language we love.

CEPL is a prime example of this. CEPL doesn’t need to exist at all, it’s purely my attempt to make a high level api with which I could play with GL and yet which doesn’t hide any of GL’s features. This is tricky, fun problem as I feel that many apis I have worked with provide simplicity by hiding things and I don’t want that.

If my drive was to make a specific game I would shut up and make that game… BUT, and this is an important bit, if your goal is to make a specific game then you should be using the best tools available to make that game, and that likely is not going to include lisp.

That might sound odd given my background but please bear with me, I’ll rant a bit more about this below. Let’s wade through the rest of this thing first.

Outnumbered by Blub

This section is just a bunch of BS based on false assumptions.

Lisp is a niche language, and making games is a niche within it. Golly gosh I wonder why there isnt much being made.

Even within that sub-niche there is a significant percentage of people who just want to make tools, not games, so there goes a few more potential games.

Even the use of the term ‘blub’ pisses me off here. When we talk about making a game we have left the realm of being academic about a language and have entered the world of making a product. That product has to run on real devices used by real people. Those devices have actual hardware that have actual characteristic and so a language’s ability to leverage those devices is what dictates how good it can be for a given task. When you speak of non lisps are ‘blub’ you disregard the actual characteristic of those languages and how they fit to the task.

There are many folks in the game industry who don’t like c++, but it’s still the number one language used for 2 core reasons (there are more but for my narrative 2 will suffice):

1. Because it remains one of the few major languages to give the kind of control over data and instruction layout that is mandatory if you want to ship a high end game on all platforms.

2. Momentum. There are a lot of existing libraries written in C++ and C++ is difficult to interface with efficiently from other languages. And yes I know about extern "C" but try taking the bullet physics library or even the fbx sdk sdk and expose those with a sane C api. It’s a monster of a task and just not worth it for studios with the kinds of time and money budgets they have.

The Sorry Tale of Those Who Tread the Great Lisp Path

The entitlement in this section is strong.

I don’t know how someone could call out a game creator by saying ‘So much for sticking to common lisp’ with a straight face. They don’t owe you, or lisp, anything. Lisp is a tool.

Same for warweasel’s work on Clinch. Clinch had crazily large ambitions and it didn’t work out. That happens, and it’s fine. Now he’s got a specific game he wants to work on and so has picked a tool to make that feasible, that is a good move.

Then there is just random bitching that they don’t like Shinmera’s video style. Not sure what that has to do with anything, however as a regular viewer I liked them. The world has room for long and shortform videos.

As someone who does a little of that myself I can tell you that my 5 minutes videos can often take longer to make than my 2 hour videos due to the number of takes and tweaks required to make sure the content hits all of the beats it needs to in such a short time slot. Even putting that aside, somehow equating this to a reason that lisp games are failing is a seriously warped outlook.

Next let’s take a quick detour before I get to the bit that really raised my eyebrows

Difficult to install, difficult to run, difficult to mod, difficult to use

Nothing here is in any way limited to games in lisp. It only applies to the developers so doesn’t affect potential popularity of the resulting game, and the whole ‘modding’ aspect of it doesn’t make any sense. I’ll spare you a diatribe on this unless it becomes necessary.

And Finally!

Is Baggers leaving?

Fuck off.

Why not just contact me if that was something so noteworthy you think it belongs in here. Although I’m pretty slow to reply to emails I do get to them. I’m also always logged into the lisp discord channel and am DM’able on twitter. In short there are a pile of ways to reach me and lack of even an attempt to do that is lazy at best.

I’d mind less if this was an off the cuff twitter comment, but this is an article purporting to shed light on some issue and provide some answers. Even the minimum amount of work would have answered your question. I even explained I wasn’t leaving in many of my streams (not that I expect people to be watching them, but if you make claims a minimum of due diligence wouldn’t hurt).

This whole article reeks of this kind of laziness, it points the finger at all the wrong things whilst missing everything that does stop me using lisp for serious games today.

For what it’s worth, I’m currently working my ass off to try and get a game out the door. I’ve saved up enough money that I could quit my job and try this but that stash wont last for long. So fuck yeah I’m going to pour myself into that work, fuck yeah I’m gonna work my hardest to see this thing succeed and fuck you for calling me out just because I haven’t given you significant free work in 4 months.

Actual thoughts on lisp’s applicability for games

Phew, with that out of the way I’ll try and speak a little to the original question. Why not lisp?

I am not going to make any bigger claims on the community or industry as a whole so this is just where I’m at right now. I’ll also touch a little on why I’m not using lisp for my current games.

1. Performance matters

You need to hit your target fps, whether that is 30 or 60 but it’s usually the latter. A game that runs too slow or that inconsistent performance doesn’t just stop being fun, it stops being a game. Of course many games don’t push the machine that hard, but we are talking about games as a whole and a vast number do.

We also need to run on mobile devices which don’t have the grunt your desktop gaming rig does.

Performance (beyond the efficiency gained from appropriate algorithms) is going to be dictated from your ability to control what is being processed and when. Maybe I can’t afford a GC in the middle of my frame, maybe to get the entity count I need I really have to maintain data locality.

ANSI CL doesn’t have many provisions to allow for this kind of control. Certain implementations do have some and you can do a lot with CFFI, but at what point do the restrictions you impose on yourself making another language a better candidate?

There is a lot of room for lisp to do good things in this space and I’m looking forward to working on some things in this area but for those who just want write a game today.. maybe it’s not for them.

2. Games of any real size are rarely single person endeavors

Making a game from scratch is a lot of work. Even with libraries and engines (see point 2) you are likely to be collaborating with people. Given a limited budget this means you all need to be working at your peak, so trade-offs will be made to optimize for the team as a whole.

Yeah I’d love to code in lisp every day. But (even putting aside lisp not running everywhere I need it) i’d be throwing away 13 years of experience of another member of my team. There are 3 of us and so I just kneecapped a third of my team for what?

3. Libraries and Engines

This is really a continuation of the last point but when you use an engine or library you lean on the work of hundreds or thousands of other developers. Have a peek at this changelog for ONE sub-version of Unreal, then think about the tens or hundreds of man-years on work that has gone into that one version.

We have less of these libraries in lisp and so a lisp game developer today is going to need to do far more work than one using Unity (for example) due to being forced to create more from scratch.

4. High level api design is a fickle beast

In lisp it’s very easy to make high level apis and DSL over systems. However how relevant a particular solution is to a given person varies wildly. It’s not just aesthetic either, the features of the language we choose have performance implications and not every game can afford to make the choices you can afford for yours. Generic methods are an example here, a great feature but everything has a cost.

In my opinion the approach of providing low level bindings over C libraries separately from the high level api is a way better way of fostering code reuse in lisp (borodust is a shining example of this).

5. Portability matters

I need to get code on mobile platforms & consoles. Lisp doesn’t help me here. I could shell out a shit tonne of money for lispworks or dabble with the relatively unknown mocl. But that only gets me mobile and it’s still only the language, I need to port all the tools and many of my C dependencies probably don’t work on the PS4.

The alternative is I can use Unity or Unreal for free and use code that was tuned for games on those platforms today.

6. Content is king

The content pipeline is staggeringly absent from the provoking article. The ability for you to consume models, video, images etc from the swathe of authoring tools out there is a serious challenge and having assimp and sdl_image is not nearly enough.

Many of the libraries you will need are C++ and not super amenable to being wrapped. Even for the ones that are that work still needs to be done and be maintained.

Remember that you are a member of a team and telling your artist to just use blender and obj is not an option.

With unity I can drag in FBX files (in the latest format) and I get what I expect. And the representation of the data in memory is being handled based on how I’m handling and rendering those objects.

Something a little more upbeat

CL is a great language that still has a lot to give. I’m gonna be using it for a long time including for games. It lives in the place of being fantastic and performant for scripting and also having the control to be truly viable for higher performance code. The implantation specific extensions to the language open up a ton of possibilities for squeezing more out of the machine and the abilities that macros afford us means there is a lot of meaningful stuff we can do trivially that require complex and unwieldy systems in other languages.

For the right project and with the right support, CL can be a great tool in your journey to make a game. But it makes perfect sense that it isn’t that for everyone right now. And it’s ok if it doesn’t ever become that.

I’ll be back to lisping early next year and I’d love to natter more about these things but this post was too long a thousand words ago.

To the original author

I’d heartily recommend making a non-trivial game in lisp. Like something that was mainstream 10 years ago and try and remake it. The problems you run into will tell you so much and your fixes (if you choose to share them) will help the community in measurable ways.

Personally I’d recommend halflife 2. It’s 14 years old now but you should be able to find rips of the assets which removes that whole part of the equation.

Enough, lets wrap this up

To close I’d like to say the following:

If you’d like to make things, I’d advise trying to gain an understanding of yourself and what you need. What is driving you, what do you feel incomplete without. If you need a specific game to exist then ignore the pseudo-religious posturings of various technology advocates and just focus on making it a reality. Use the best tools for the job and get it out there. I’ll be rooting for you.

A quick one today. I got TaleSpire up and running again with the latest backend changes. Next we want to store multiple versions of each of your boards (we may expose this in future so you can restore an earlier version). All the code required is pretty much there but I just need to work out the strategy we want to use for when we make a new checkpoint and then plumb what we have together to do that.

Then it’s back to refactoring the queries for handling retries etc.

Time for sleep. Ciao

Some serious yak shaving has been taking place today.

The goal was to add the ‘manager’ that would keep track of the connection to the backend and handle cases where we lose connection. One task was to be able to inform a request if it should retry, but hmm.. that means I should really refactor the queries a little.

With my new goal I dug into the queries a bit and saw that I should really take care of the changes to board I needed to make first. We want to be able to keep multiple versions of the board on the server so we have a backup history of sorts. This requires some work on the database.

With my new goal I started writing the board changes and saw that a bunch of this code will need revisiting anyway as we expose some guids we use as keys and that it’s a good plan. So I made the basic changes and then switched task.

With my new goal..well it’s a monster. The guids were used in a tonne of places and so about 70% of the queries in the backend needed to be refactored. This has taken me the rest of the day but, as it’s work we needed to do for the release anyway, I’m not worried.

I’m not at the point where I have pushed a new staging build and am testing TaleSpire (spoiler there are still bugs). There are a good few hours of work before this is working properly again and I’m not sure how much I will get done tomorrow as I need to visit the doctor. We shall see anyway.

Until next time,

Peace

Evening all! Today has seen a bunch of bug fixes which are rather specific but I’ll mention a few.

• Changing floor rapidly on board load could cause assets not to load
• Deleting the board you are in now takes you to the campaigns default board (currently a new empty board)
• Fix for case some requested data arrives for a board asset which has been deleted
• Grabbing a board asset whilst board loading caused error as creature landed in invalid area
• This is half the fix, the other half is a UI fix to make sure you can’t pick things up before the zone is fully loaded
• Add inspector button for checking for duplicate network ids in board assets
• Fix case which resulted in duplicate network ids after reloading board
• Add compression to network messages for building/deleting multiple tiles
• batch multi-tile messages to avoid making single message too large
• not as robust as I’d like but good enough for alpha
• Fix case where other player deleting assets we have selected caused our selected region to grow
• A bunch of code cleanups

I also hit an annoying aspect of mono. In c# you are able to set the validator used when a HttpWebRequest needs to check if a http request is valid. One of the things that is meant to be passed to that callback is the certificate chain. I use this to verify the intermediates on top of our certificate being checked. On MS c# this works perfectly, but in the mono version unity is using it isn’t. Super aggravating but not much we can do right now, hopefully it will improve as the .net version support in Unity gets better.

The standard certificate validation is fine of course so no worries on the security side. I just wanted the option to do a bit extra.

Anyhoo that’s all for today. Tomorrow I am looking at managing the connection to the backend and the handling/retrying/etc of failed requests.

Peace.

Ah today has been fun again. A few things that were done were:

• A few more little fixes to serialization of unique characters,
• If you delete a character that is unique, remove it’s ‘uniqueness’ server side
• Added text to help debug fog of war (this was great, more below)
• fix two bugs in fog of war search when the search crossed zones
• make sure that when the octree node is larger than a tile that the fog of war search takes that into account in calculating distance walked (just fixed up some lazyness when coding the feature)
• Fix case where creatures you don’t control seem to load in an incorrect position when you join and existing play session on a board
• Fix dumb bug where non GMs weren’t loading unique creatures properly if they were the first player to reach a board
• Fix bug in quadtree deserialization
• Serialize fog of war for unique creatures. Sync it to players on join and store it in board so you can leave and rejoin board much later and it is remembered.
• Add code to recognise and upgrade boards stored in an old format
• other misc small fixes

Of these the most fun was the adding some additional info to the debug view of the fog of war so I’ll jsut rabbit on a bit about that. This really is one of the places where Unity shines, its so easy to add additional UI to the editor, not just in the inspectors but in the scene view too. I was struggling working out why the fog of war was behaving strangely so I decided to show a number at every ‘step’ of the search to show how far the character could still walk (how many steps remaining). It took a few tweaks as the fog of war search is done on a different thread and UI elements have to be drawn from the UI thread, also I wanted the text to remain visible for a number of seconds. Anyhoo the result was that every time you placed a creature the scene view looked like this:

the red circle is where the creature is

When I did this in the morning I could immediately see that the numbers were wrong when the search crossed a boundary. Turns out that was a simple case of not initializing an int in one of the constructors for the search. Also there was a case where the number went from 0 to over 4 billion, that’s an easy one, unsigned integer wrap around when subtracting 1 from 0. Each is a simple issue but it just became obvious as soon as I could see a tiny bit more info. Yay tools!

Out of curiosity I added a little slider to the creature’s inspector to control how far the search would walk, this was great to fine tune and find a default value that felt nice.

The other stuff doesnt make for nice pictures but is all obviously needed for the alpha.

Here’s to tomorrow, hopefully it’s as productive.

Peace