Heya folks, yesterday and today I have spent getting familiar with Unity’s Job Systems and new ECS.
I’ve written toy ECS’ before and am currently working on a little optimizing compiler for querying table data so a lot of things were very familiar. The bulk of the video and prose content for Unity’s new systems is focused on the fact that the new thing is ‘different, but don’t worry it’s fast and not that hard’ etc, etc. As I’m already sold on the premise this is nice but of limited here. Beyond that, you better be comfortable reading other people’s code :p
The best resource so far has been the ECS Samples. It’s a small of samples evolving a piece of code from a
foreach on the main thread, through different flavors of jobified tasks. One of the biggest advantages of the code is just seeing what is still in use. If you glanced at the cheatsheet you’d be missing certain things (like sync points).
The reference docs are passable but there are plenty of things where the description of a method is the name. When I hit walls in C# I often read Mono’s source to get an idea of what to expect, but naturally many of the things in the new ECS are either proprietary or are in c++ code I can’t reach easily so those missing doc-strings become a pain point.
Given that, in TaleSpire, we load most assets dynamically I’d like to avoid having to use the Entity conversion routines on load of each asset, I’m fine with running them in TaleSpire and serializing something useful, but I’m not sure how we prefabs play with ECS entities at this stage. I’d be very surprised if Unity’s mega-city demo was using conversion routines so I’m sure there is still more to understand here. Also the lack of documentation around the
Hybrid.Renderer is a pain in the proverbials, I’d really like to look into zone culling in TaleSpire to potentially improve rendering performance but information seems scarce here. I’m sure it’s all in the mega-city demo but that doesn’t feel like the best introductory material to be using.
Regardless, between the samples and these docs, I’m making decent enough progress. We’ll definitely be using this in TaleSpire and I’ll have this under my belt well before the campaign ends.
Let’s get back into the swing of these!
During the Kickstarter I’m using any free time I have to study and make little systems to get familiar with systems we may end up using
I got interested in how easy it would be to de/serialize such data, given these types can have precise memory layouts specified for them. The that end I was making little
BinaryWriters and readers that had Write methods that could take any blittable type or array of blittables and handle them correctly.
I’m going to make a dump of some useful links as most of the stuff I was doing was simple enough, but was a good exercise for familiarity. It was the first time I’ve significant use of pointers in c# and it was a very simple and unsurprising experience.
The first handy thing was that the unmanaged constraint is available. Whilst unmanaged types are not the same as blittable types there is a useful overlap. Using this allows us to have signatures like
unsafe void Foo<T>(T value) where T : unmanaged. Also note that
sizeof is defined to work on unmanaged types. It was very easy to write generic methods doing useful stuff with c#, I’m please with it so far.
- Scoped GC pin
- Heap allocate, check out all of Marshal, it’s pretty handy
- more info on blittables
- create a struct from data referenced by pointer
- You can explicitly cast typed pointers to
- Unsafe seems like a handy class until you see that it’s not implemented in mono
- But then you find UnsafeUtility in Unity itself and it’s way better.
mallocetc are all there
The big disappointment of the day is how quickly one ends up having to reinvent the wheel as most Streams, Buffers, Writers, etc, never expose their buffer or provide overloads taking
IntPtr. The more you poke to work around this the more it feels like you should just make your own. It’s not ideal. Saying that, I have spent most of today in dotnet code rather than Unity’s so I do still need to see what is there.
Until tomorrow, Ciao.
So here we are! The clock ticks and, baring any disaster, in the next 18 hours the Kickstarter will go live.
We are currently beavering away shining edges and triple checking wording.
It’s an odd feeling to be this close to finding out the world makes of this but it’s a privilege to be here.
Thank you to everyone that showed us that they were interested in this. Thank you to the alpha testers who delved deep and proved the limitations of the systems And thank you to you for reading these little posts.
The new day is upon us, The gates are soon to open, Let us see what is on the other side
Dwarf, Baggers & Ree (The rather sleepy Bouncyrock crew)
Not really a dev log as right now there’s not a whole lot of dev going on. I’ve been just sitting writing an rewriting bits of the kickstarter page. It’s an odd feeling as I flit between thinking ‘ah your thinking about it too hard’ and thinking ‘man this kickstarter decides whether I have to refresh my CV soon’. Heavy.
It’s odd writing this when there isn’t much to report. Assets for the trailer have been made, the trailer itself is going well. Really I’m writing as it feels weird going a few days without reporting the latest to you.
We’ll be letting you know the kickstarter date one week before it happens.. which is soon.
Heya all, time for a quick behind the scenes again.
Those who have been keeping count since the dev stream will know that we are only a couple of weeks to go before we should be doing our kickstarter. We really need to focus on this so we will be pretty much stopping updates to the game for the next couple of weeks. I’m going to get the ‘duplicate board’ feature shipped this weekend and then I hope to fit in some bugfixes, but I think this is what we need to do to make sure we get everything done.
Other than that I finished v1 of the ‘duplicate board’ feature mentioned above. It’s a little rough around the edges but nothing not acceptable for an alpha. We just need to make an icon for that now and it’s ready to ship.
Not much more to say other than thank-you once again for caring so much about our little game.
We’ll do our best.
I was too tired to writeup last night’s dev log so here it is a little late.
I really want to get a user visible feature out this week so I implemented ‘duplicate board’. This will let you click on an icon in the board panel and duplicate the chosen board. This will be handy for a lot of cases but I love the idea of making a town, copying it and then destroying the town for the players to visit again after the calamity.
The hard parts had been researched or implemented when I was looking into the S3 part of cleaning up old board files. All that was left to do was write the database queries and expose it to the game itself.
With the core feature now working I just need to do cleanup and handle the fact that it currently doesnt copy the board description (an easy fix).
Back later today with more news
Today I spent most of my time working on the code that deletes old boards from the backend storage.
Every time you load a board TaleSpire takes a new snapshot of the board that you then modify during that session. We keep hold of these snapshots so that we can at some point give you the ability to roll back to one of these backups in case something drastic happens.
Of course this means we are slowly accruing more and more board snapshots so they have to be cleared up. The goal was to keep only the 10 most recent snapshots. Any older than that will be deleted.
Now even though a snapshot is old it would still be good to be able to recover it for a short window of time so we have set up ‘buckets’ on S3 which auto-delete files that have been in there more than 14days. This means the task is to identify the old board snapshots and move them to these buckets.
As this operation needs to touch every row in the ‘board files’ table in the database it could be rather costly so we are using postgresql’s materialized views to help with this. This lets us chose when to update the view at the cost of it being out of date after that. This is fine for us as the worst that happens is that we don’t delete a few snapshots as soon as we could, it will never result in us deleting something we shouldn’t.
After that it was a case of setting up the erlang process that would take care of this at regular intervals. I got the hard part done, namely identifying and moving the files, but for whatever reason the event isn’t firing automatically. This will be some simple mistake on my part so I’m not worried about fixing that relatively soon.
This also means I’ve tested the bulk of the code I would need to implement ‘duplicate board’ so if all goes well I’ll get that out this weekend.
Until next time,
Heya folks, @Baggers here again,
I didn’t write a log last night as what I thought was a quick nap on the sofa turned into the kind of sedation usually reserved for surgery.
On Friday I wrapped up the experiments I was doing with resource loading in Unity. I was able to load the icons and index of assets at startup which significantly dropped the pause at first campaign load. The branch i was doing the experiments on is rather a mess so I’ll need to clean that up but it looks promising.
I also took the morning to take on some contracting work. I’ve done these very rarely since starting on TaleSpire but it keeps bread on the table. Here’s to the Kickstarter making that unnecessary :)
Behind the scenes @Ree has been doing great work on the campaign trailer and the assets and tech surrounding that.
Right, back to the weekend!
Alright today I was mainly experimenting with asset loading. We currently have a big pause when we load assets and so naturally we need to fix that. Some of it is just moving things earlier, spreading things over frames etc, but we need to try this stuff out to see what works.
There’s not really a lot more to tell for now. Research is cool but make for pretty generic dev logs ‘TODAY I DUN STUFF’.. good job me.
Tomorrow is a national holiday but we’ll be back with more on Friday.
Heya folks tiny log today as I don’t have much of note to report, lots of poking around with Unity and reading up on a bunch of things we may need networking wise in future.
The good thing is that the dungeon tile set is progressing well. It’s been delayed by wanting to add more texture and variation to it but hopefully we get something out tomorrow.
I best run for now.