Heya folks, I hope the first week of the year is treating you well.

Things are going well over here. I feel well back into the swing of things now, so it is a good time for an update on what I’m doing.

Before Christmas, I had been pushing hard to get creature modding shipped, but clearly, I didn’t succeed. I thought I could hack around some of the tricky cases and get something made to see us through the holidays. If that had worked, my January would have been rewriting all the hacks into reasonable code.

As I failed to hit that target, I felt I had a couple of options.

1. Continue the crunch and try and get the hacks to work. And then immediately rewrite them all.
2. Just bite the bullet and do it more correctly, with the downside being that it would take more of January to do it.

I chose option two.

I chose that as my personal goal this year is to get better balance and organization in my life, something I have always struggled with. If I’m in doubt[0], I always find it easier just to work as I know what I’m doing with code as opposed to… well, everything else. But you can only do that for so long before cracks appear elsewhere.

This choice is feeling good for me so far.

That’s quite enough of the human meat issues; let’s talk technical stuff instead!

Content Addresses

Until now, most of the content you interact with in TaleSpire has been identified by a UUID. This gives us a (hopefully) globally unique identifier (name) for each creature, tile, prop, etc.

When you copy a slab, those IDs are included in the weird string that TaleSpire produces, and when you paste it back in, we use the IDs to find the data for the tiles/props/etc that you are pasting.

Now imagine how it will be with mods. A person may copy a slab with tiles or props made by the community or (soon) may be hosted on a repository that Bouncyrock does not control. How can TaleSpire find the data it needs only using the UUID from before?

There are a bunch of possibilities. One is that we could host a central address book that lets TaleSpire look up an ID and find the content pack and repository that contains it. This, however, means registering all content and IDs with use, centralizing information, and giving us too much power over creators. Instead, we are borrowing from the web and moving to URIs.

With the new scheme, every piece of content has a content-address:

Here is an example one: :46be6663d03843b6bb320a97716e5ace#72fbaa19-806d-4816-8d7f-91f17a459b93

Without the scheme, this identifies a piece of content named 72fbaa19-806d-4816-8d7f-91f17a459b93 in the internal content-pack 46be6663d03843b6bb320a97716e5ace (An internal content-pack is one that is shipped as part of the game)

The # is the start of a fragment, often used in URLs to point to some content within whatever thing the main part of the URL was pointing to. That’s precisely what we are doing, too.

Here is a different one: repo://talesbazaar.com/api/7d745de08e494e7ea7dce871519c09f0#d0bd95b7-0ef3-4144-a1ee-63aeedc93707

This doesn’t work yet, as we haven’t yet published the community repository REST API specification. But, once we do, a site like talestavern.com or talesbazaar could implement it, and then TaleSpire would be able to fetch content from places that are entirely out of our hands (We think this is critical for the longevity of TaleSpire).

• The repo: scheme tells us it is a public repository.
• The talesbazaar.com/api/ portion tells us the location of the endpoint (https is always used, so that doesn’t need to be specified in our URI).
• The 7d745de08e494e7ea7dce871519c09f0 is a hex-encoded content-pack ID. It’s not globally unique, but it will uniquely identify a pack in the repository
• Once again, the fragment d0bd95b7-0ef3-4144-a1ee-63aeedc93707 is the UUID of the content inside the pack we are fetching.

We have other schemes for mod.io, HeroForge, and we are even tinkering with fetching content from arbitrary web addresses. This is proving useful for icons that are hosted separately from the content itself.

This is just example syntax, but I’m experimenting with URIs like this: web://some-site.com/some-place/some-image.png#64,64,32,32

In this case, the “content-pack” is an image at https://some-site.com/some-place/some-image.png, and the fragment identifies a region of the image that is then used as an icon.

Async everywhere

TaleSpire grew out of a simple, single-floor, dungeon-building, TTRPG-esque game. Initially, all of its content was built in, and there is still code that assumes some resources are immediately available. Over the years, we’ve been moving away from that as we tackled HeroForge and the community mod browser, but there is plenty of old code around.

As you can imagine, modding breaks all of that!

I won’t be able to fix all these old assumptions this time. I will leave some for when we add modding for Tiles and Props, but still, it’s a lot of work.

Each time I finish a feature like the mod browser, I think, “Ah, yeah, now we’ve got all the bits we need to make the real solution,” and each time, something else comes up. I’m sure there will be something else I need to handle. I’ll be sure to let you know when it happens!

Progress

All in all, though, it’s going well. In a game that is so much about displaying and navigating content, it’s no surprise that changing the fundamentals of how content is addressed, obtained, and loaded touches a significant portion of the project.

I’m working through hundreds of files and tens of thousands of lines of code and doing so carefully, as I never want to break anything you’ve been building.

I think I’ll be through the first wave of TaleSpire refactorings in a few days, and then I’ll move on to TaleWeaver, where there is a TON of work to do as the internal content-pack formats have completely changed.

I’m very optimistic, though, and I’ll keep you posted as it all quickly gets closer to release.

Man, this got so long, and I haven’t even talked about the ways we intern the URIs to make them fast to work with at runtime or any of the other stuff we are doing, but that will have to keep for another day.

Have a great night folks,

Peace.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

[0] and I’m always in doubt :P

Heya folks,

Today was a tedious one.

I finished up last week by moving a whole bunch of code out of our Unity project and into a separate C# project that referenced the Unity DLLs. We are doing this as we want to ship TaleWeaver as compiled code[0].

I finished that task, and the TaleSpire project compiled, but when I ran it, it crashed immediately. The reason is that by moving all the code to a separate project, all of the references from prefabs and scenes to that code broke.

To understand why, we have to know that Unity wants developers to be able to rename and move things without breaking all those references. So, rather than referring to the classes by name[1], they create a GUID for everything in the project. As they can’t push those GUIDs into the files they are identifying, they instead make a separate file called a meta file, which stores the ID and any other metadata for that thing[2].

As meta files are a Unity concept, I can’t include them in our new C# library. So, by moving the code, I broke all the references!

Of course, Unity does need a way to identify types inside a C# library, and I found this unofficial info talking about how it works: https://www.robinryf.com/blog/2017/10/30/unity-behaviour-in-dlls.html

I now have the tools I need to fix this issue.

• First, I will check out a build from before I moved the code out to the separate library.
• Then, I will write a script to collect the GUID for every script in the project[3]. I will make a lookup table between the GUIDs and the fully-qualified type names.
• I can then check out the most recent code (the one with all the broken references), scan the project for any use broken references, and use the GUID to find the fully qualified type name.
• With the type and the sample code from the link above, I can calculate the new reference information and replace the broken reference.

If I’m correct, I can automate the entire repair, which is a relief as I have done fixes for issues like this before, and they SUCKED.

In the future, I might even be able to use this knowledge to make tools to help fix issues that occur during merges sometimes.

Well, wish me luck for tomorrow. We’ll see how this all turns out.

Peace.

p.s. The MD4 hashing code from the linked blog post was helpful but performed a bunch of unnecessary heap allocations. I ended up procrastinating earlier today by tweaking it to be more efficient. You can find that here: https://gist.github.com/baggers-br/48bba51a9e0b1c5a0632c57537f84534 It is almost entirely untested, so please beware.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

[0] I know, I know. Byte-code. But still, you know what I mean. [1] Fully-qualified or otherwise [2] If you are very curious, you can find some info about this here: https://unityatscale.com/unity-meta-file-guide [3] In fact I’m only including subclasses of MonoBehaviour

Heya folks!

After a few weeks of public activity, it feels weird to be quiet again, but things are progressing well.

My big task is restructuring our projects so we can ship the creature modding tools as DLLs.

This task involves some decoupling from Unity, so I’ve been removing the dependency on Unity.Entities. We didn’t use the ECS they provided at all, but they had a zero-copy data format, which was pretty handy.

I wrote our own format the other day, so for the last couple of days, I have been updating all the code to use that. I’ll admit that a lot of that time was me trying to find what I thought were bugs but turned out to be me screwing up.

My task for today is to work out how I will handle conditional compilation (e.g. ConditionalAttribute and #if) with this library. We’ll see how that goes.

Catch ya soon.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

‘Allo folks.

Today went well. I’ve got a texture format I think we can work with, and I’ve made a de/serializer for it. Here is a quick pic.

There isn’t really a way to tell it’s from our format, but I guess that’s the point :P

I need to put this all together into a file format, but that is easy with the work from the other day.

The next unknown I need to tackle is building dlls from Unity projects. This is required for the project we will ship to let you make creature mods.

We’ll keep you posted on the progress.

Peace

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

Hi again folks,

Today, I continued working on our custom mesh format. The image below shows some of the progress.

This is the first mesh I’ve taken from Unity to our format and back again.

With that working, my next task for creature modding is to work on how we will store textures on disk. I should be able to work on that tomorrow, although there may be other Bouncyrock work that will take precedence. We shall see :)

That’s all for today,

Ciao.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

Heya folks! I’ve not written in the last couple of weeks as we’ve been going hell for leather, trying to ship two patches a week.

That’s been very successful, and it’s a lot of fun to do from time to time, but it is pretty unsustainable when you hit bigger features. So this week, we are only aiming to ship the board folders. The good news, however, is that the feature I’m chipping away at now is creature mods.

So, where are we at?

Unity can pack stuff into things called AssetBundles. They are very flexible, but one downside is that you need to build one for each platform you are targeting. That isn’t nice for modding, and it also makes upgrading the Unity version stressful, as the AssetBundle format can change over time.

As we don’t need[0] all the features AssetBundles provide, we are opting to make our own cross-platform format that only supports the very few things we need.

The first task was defining a low-level format that only specifies blocks of bytes and an index. That was nice and simple to design, and I’ve left the save/loading code for another day, as it should be pretty straightforward.

From there, I’m making a format for storing meshes that is compatible with Unity’s low-level mesh APIs. We want to use that low-level format as it allows us to create meshes across multiple threads and also do so with a minimal number of copies. I got a nice draft of that format finished in the morning and then started fleshing out the implementation of a zero-copy de/serializer[1].

Until now, we have used Unity’s blob asset reference system. However, it means we need a dependency on an additional Unity library, and I’m no longer comfortable with any unnecessary dependency on Unity systems (due to how Unity has acted this year[2]).

I think I’m about 70% of the way through writing the zero-copy stuff. Tomorrow, I’ll start pushing meshes through this thing and see how it handles it. I’m sure I’ll hit plenty of things I forgot up to that point.

As soon as meshes are working, I’m looking into how we will handle textures. I’ve been giving it some thought already, so it should be quick to make some progress.

I’ll be sure to come back and update you once I’ve got some news to share on the texture progress. Especially as, from there, I should be able to dive into the UI that you folks will be interacting with!

I hope you’re having a good day,

Peace.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

[0] And actually we actively do not want all the features

[1] Feels weird to call it that when the point is that you don’t need an explicit deserialize step as you just load it into memory and use it.. but meh. I can’t recall a better term right now.

[2] This might seem moot after they rolled back their proposed fee changes for many folks, but I disagree, and I furthermore think the same applies to WOTC. The company is still run by mostly the same people, they still want money, and it stands to reason that sooner or later, they are going to try and extract it from us again. “Fool me once” can be overused, but it feels appropriate here.

Well yesterday was nice! Borodust migrated the database without any major hitches and Golbin shipped another asset pack. I’ve been helping out but my main focus has been on board folders and the slab-browser.

My first pass at my server-side code was a little more broken than I thought, but all the fixes were easy enough. The UI side is also coming along.

That’s all from me for today, see you folks around.

Disclaimer: This DevLog is from the perspective of one developer. It doesn’t reflect everything going on with the team

Heya folks,

It felt good to get the slab-browser beta out, and while I do have things to fix before it is ready for prime time, I couldn’t help but take a swing at a different feature. One we have all wanted for a while. Board folders.

Today was mostly spent writing server code for this feature. I tested some of it but wanted to get some UI in before trying the rest. To that end, I’ve been hacking the basics together.

For the first version, changing a board’s folder will be done via a dropdown. Really, I’d prefer to make it possible to just drag boards into folders, but I’m not familiar with how to do that yet, and I don’t want to spend too much time on this as I NEED to get moving with the creature modding support. So, the first version will be functional, if a bit basic. I’m also not adding arbitrary nesting of folders for now.

Jumping back to the creature modding, I read more about what kinds of access Unity gives us to mesh data. I’m satisfied that I can make something to cover all the bases very quickly. It’s also tempting to look into Zeux’s excellent mesh optimizer.

That’s enough for today.

See you in the next one.

Disclaimer: This DevLog is from the perspective of one developer. It doesn’t reflect everything going on with the team

Heya folks,

For the last two days, I’ve been working on the slab upload. It’s going very well.

I’m currently cleaning up a lot of little issues with it, and then it’ll be ready to show it off. The main focus the last two days was being able to edit the details of mods you have already published. That was a slog until about six hours ago when it all started to click.

I’m also pleased to say we support multiple kinds of authentication with mod.io. This means you can sign in via Steam, email, browser, and even QR code.

This is all very meh without some video to show it working, however, so I’ll come back when I have fixed the remaining visual issues.

The big issue right now is that the list that shows published mods doesn’t like certain kinds of filtering and thus can leave gaps between items. I know what to do, but it’s gonna take some hours to code.

Right, seeya folks soon.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team

With yesterday’s realization that some of Unity’s handling of overlapping keybindings is broken, I needed to add our own workaround.

I knew I didn’t want to try and build a complete solution, so the first version was just to handle the issue that caused undo/redo on Mac to be broken.

To recap on Mac, Undo is Command+z, and Redo is Command+Shift+z. This is an overlap; in order to hit the keys for Redo on Mac, you also need to hit the keys for Undo.

I’ve added a simple handler that detects multiple actions happening on the same frame and picks the one with the longest chain of modifiers. This works for the Mac undo/redo issue as both actions will fire on the same frame as z is the last key pressed and is the one that causes both to fire.

We will need to handle more cases in the future, but that’s a problem for another day.

I’ve got some new builds progressing through the build pipeline, so tomorrow, I should be able to ship a fix for that. Then, I can get back to business.

Hope you folks are well,

Seeya in the next log.

Disclaimer: This DevLog is from the perspective of one developer. So it doesn’t reflect everything going on with the team