Modding:Serialization (Saving/Loading)

This page is about modding. See the modding overview for an abstract on modding.

This page is about modding. See the modding overview for an abstract on modding.

For best results, it's recommended to have read the following topics before this one:

• Modding:C Sharp Scripting

What is Serialization?

Serialization is how games and other programs convert data (in Qud's case, C# class fields) into a format that can be written to a file, so that it can be stored between sessions. Effectively, it's how a game saves its data. The opposite process, the process of loading data from a file, is called deserialization.

There are many different ways that data can be (de)serialized. One particular format that Qud uses for its persistent data files is XML. One advantage of XML is that the data can be read by the game independent of the game version, however, it also takes up more storage space. Another consideration is that it makes the data human-readable and -editable, which is useful for modding.

Qud uses binary blobs for save data. The format is not easily readable or editable by humans, but it is highly flexible in programming terms and can be used for a wide variety of data. However, to some extent it also relies on certain things not changing across game and mod versions, so it can be easily invalidated by certain types of changes.

Due to the modular way in which Qud does serialization, as a mod author you don't have to concern yourself with all the specifics of the save format. The remainder of this article describes how to use the tools that Qud and its engine provide for telling it how you would like to serialize your mod data.

How Serialization Works for Qud

This section assumes you have the following using directives at the top of your code, which tell C# where to find the attributes you will be using:

using System;
using SerializeField = UnityEngine.SerializeField;

In many cases, you can simply include the the [Serializable] attribute at the top of your class definition, and Qud's game engine will take care of serializing all of your public class fields for you. Specifically, Qud is capable of correctly serializing all intrinsic types (such as string, int, etc.) as well as containers of those types (such as List<string>). If your class only uses intrinsic types, the [Serializable] attribute should be all that you need. Qud will properly save and load your data without further effort. Note that private, protected, and static class fields are not serialized by default. If you want the values associated with private or protected class fields to be retained through a save and load, you need to mark each individual private or protected field with a [SerializeField] attribute. Static fields cannot be serialized in this manner. See the Unity docs for more information about serializable fields (note that the GameObject discussed on that linked page is not related to the GameObject type commonly used in Qud).

Here is an example of the basic concepts discussed above:

using System;
using SerializeField = UnityEngine.SerializeField;

namespace XRL.World.Parts
{
    [Serializable] //this attribute causes all public fields to be serialized automatically
    public class MyCoolModPart : IPart
    {
        public float CoolnessRatio;  //this field gets serialized!
        public int AbilityLevel;     //so does this field!
        private string CurrentValue; //because it's marked private, this field does NOT get serialized - the value is cleared out after a saved game is reloaded! Be careful with how you use this value.
        [SerializeField]
        private string SecretID;     //this field DOES get serialized even though it's private, because it's marked with the SerializeField attribute!

        public void MyMethod()
        {
            //...
        }
    }
}

Object fields are not serialized automatically. If your class introduces a new field for an object type or a container of objects, such as a GameObject field that stores a reference to some particular object, a List<LiquidVolume> that stores a list of liquid volumes, or a completely new type of object introduced by your mod, you must implement custom serialization for those object fields, or your object references will no longer be valid after the game is reloaded.

However, if your class extends a built-in class, such as IPart, object fields from the built-in class will already have the required custom serialization logic (typically implemented in their Save/Load and Read/Write functions). For example, IPart contains the field public GameObject _ParentObject, which stores a reference to the GameObject that the part is a component of. IPart.Write() contains the special logic necessary to serialize that GameObject when the game is saved, and IPart.Read() contains the logic required to deserialize that object so that it remains valid after a player reloads that save. This means that if your class extends IPart, you can safely rely on the fact that references to the parent object, such as this.ParentObject, will always be valid in your code.

If you can find a way to reference objects in your code without explicitly saving a reference to those objects in a new field, it's generally a good idea to avoid creating that field so you don't have to serialize it. For example, there's pretty much always a convenient way to retrieve a reference to the GameObject that your part is attached to, or a reference to the Player object, without saving those objects in your own discrete fields.

Custom Serialization

As described above, Qud's default serialization is not sufficient for object fields introduced by your custom classes. In those cases, you need to tell the game to let you serialize a field yourself. By putting the attribute [NonSerialized] above a field, it tells the game not to save that field. You can then add your own custom save behavior.

If your class extends IComponent (which includes descendants like IPart) and Effect), you should override the Read and Write methods to handle your custom serialization. These are the only virtual methods the game currently makes available for serialization. If you need to serialize data outside of a class that extends IComponent, you will need to look at the SerializationReader and SerializationWriter classes and implement your own serialization logic.

Qud does provide some serialization helper functions. In particular, serializing GameObject fields or lists is particularly easy if you use the WriteGameObject or WriteGameObjectList functions. Here's an example straight from the game that shows how Qud serializes a character's inventory, which is stored as a list of GameObjects in the Inventory class:

namespace XRL.World.Parts
{
	[Serializable]
	public class Inventory : IPart
	{
        public override void Write(GameObject Basis, SerializationWriter Writer)
        {
            Writer.WriteGameObjectList(Objects);
            base.Write(Basis, Writer);
        }

		//...

        public override void Read(GameObject Basis, SerializationReader Reader)
        {
            Reader.ReadGameObjectList(Objects);
            for (int num = Objects.Count - 1; num >= 0; num--)
            {
                if (Objects[num] == null)
                {
                    Objects.RemoveAt(num);
                }
            }
            base.Read(Basis, Reader);
        }

		//...

		[NonSerialized]
		public List<GameObject> Objects = new List<GameObject>();

		//...
	}
}

Below is an additional example segment of code with comments that shows how you might serialize a custom list.

This article has information that is missing or not up to par.
Reason: The code below assumes that StandAbility has implemented a meaningful Write method when it calls ability.Write(Basis, Writer). It's not really a complete example.

...
        // The non-serialized attribute signals not to save this list when Write is called.
        [NonSerialized]
        public List<StandAbility> abilities = new List<StandAbility>();

        // Write is called when the game is ready to save this object, so we override it here.
        public override void Write(GameObject Basis, SerializationWriter Writer)
        {
            // We have to call base.SaveData to save all normally serialized fields on our class
            base.Write(Basis, Writer);
            // Writing out the number of items in this list lets us know how many items we need to read back in on Load
            Writer.Write(abilities.Count);
            foreach (StandAbility ability in abilities)
            {
                // Here, we call the save function on each ability because they are game objects
                ability.Write(Basis, Writer);
                // If our list was full of basic types, such as integers, instead, we would call Writer.Write, for example:
                // Writer.Write(someNumber)
            }
        }

        // Read is called when loading the save game, we also need to override this
        public override void Read(GameObject Basis, SerializationReader Reader)
        {
            // Load our normal data
            base.Read(Basis, Reader);
            // Read the number we wrote earlier telling us how many items there were
            int arraySize = Reader.ReadInt32();
            for (int i = 0; i < arraySize; i++)
            {
                // Load returns a generic object, so we have to cast it to our object type before we add it to our list.
                abilities.Add((StandAbility) Reader.ReadObject());
                // If we had a basic type in our list, we would instead use the Reader.Read function specific to our object type.
            }
        }
...

There may be other instances where you need custom serialization. The methods shown above can be applied to other situations as well.

Migrating between mod versions

A major issue when modding with C# is migration between mod versions. We use "migration" here in a similar sense to database migration: you want players' saves to seamlessly update the fields of a particular class between an old version of your mod and a new one. For example, you could have a part MyPart designed as such:

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        public string S1 = "foo";  // added in 0.1.0
    }
}

And then decide that you want to add a new field to your part for v0.2.0 of your mod:

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        public string S1 = "foo";  // added in 0.1.0
        public string S2 = "bar";  // added in 0.2.0
    }
}

Historically this has been a major issue, as replacing fields in a part could permanently corrupt its parent object. In the example above, if the game tried to read a save that had been made under v0.1.0 of the mod using the v0.2.0 code, it would be unable to read instances of MyPart as they would not contain the S2 field. Post-Spring Molting, this issue is much less severe as the game is able to remove parts that it doesn't know, so disabling a mod doesn't permanently corrupt all impacted objects. The game will also back itself up to help prevent save corruption.

However, a separate issue arises as the game approaches 1.0. In the past, modders could rely on aligning migrations with major game updates, since major updates did not preserve save compatibility. Post-1.0, modders need to be able to perform migrations without relying on a major game update.

This section is intended to give you some strategies for performing migrations on your own. Note that it is possible to migrate simply by renaming classes, e.g. renaming MyPart to MyPart2, and then relying on the game's own mechanism for removing the unknown MyPart part. However this will surface errors to the user. Moreover, in some situations removing a part altogether may be undesirable if its functionality is critical to the operation of an object.

Simple migration with the IScribed* interfaces

In version 207.69, three new interfaces were added to the game to make migrating between mod versions much simpler: IScribedEffect, IScribedPart, and IScribedSystem. These classes work like their non-"scribed" counterparts, except that you can add and remove fields from them at will.

It is strongly recommended that you use these new interfaces in place of Effect or IPart for new code. This will make future updates to your mods much simpler. You should take note of the following for IScribed classes:

• You can add and remove fields at will from an IScribed class, but you cannot change the type of an existing field.
• It is possible, but nontrivial, to migrate a non-IScribed class to an IScribed class. Therefore it is strongly recommended that you start by writing your classes using the IScribed interfaces.
• You can use multiple inheritance to inherit from an IScribed class and another class, e.g. public class MyPart : IActivePart, IScribedPart.

When should I not use the IScribed* interfaces?

Under the hood, the only difference between the IScribed and non-IScribed classes is that the former writes fields alongside their names during serialization, while the latter does not. As a result, IScribed classes increase save size, and reduce serialization/deserialization speed.

Save files are gzip-compressed, which mitigates much of these costs, and in 99% of cases the advantages to future save migration will outweigh the disadvantages. You should in general only consider avoiding the IScribed interfaces for IComponents that have many fields (the size of the fields themselves is unimportant) and are added to many objects. If you are planning a very large mod (i.e. one that has dozens or hundreds of parts) you may also wish to consider avoiding IScribed interfaces.

Migrating components: a simple example

If you are not using an IScribed interface, then you will need to rely on the Read method for custom part migration.

Note that you cannot remove a part when calling Read. Moreover, certain fields may not be initialized and other objects/effects may not have been deserialized at the point where Read is called. Thus, you will typically want to use Read only for the purpose of deserializing fields, and defer the actual migration to a later point.

Let's consider a simple example: suppose that you write a mod that includes the following part:

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        public string S1 = "foo";  // added in v0.1.0
    }
}

Now in version 0.2.0 and 0.3.0 of the mods, suppose that you added new fields S2 and S3, so that your class now looks like this:

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        public string S1 = "foo";  // added in 0.1.0
        public string S2 = "bar";  // added in 0.2.0
        public string S3 = "baz";  // added in 0.3.0
    }
}

You will want to ensure that you are able to safely migrate players' saves from v0.1.0 and v0.2.0 of your mod to v0.3.0. For the sake of argument, we'll suppose that

• For players on version >= 0.2.0, < 0.3.0, you will convert their old MyPart to the new version and populate the new S3 field with null.
• For players on version >= 0.1.0, < 0.2.0, you will simply remove their old MyPart part altogether.

Finally, we'll assume that you're using semantic versioning for your mod so that you can easily compare mod version differences. You do not strictly need to use SemVer, but it is easier to compare version numbers if your version is parseable by System.Version. See the API pages for System.Version for more details.

Our methodology for performing migration will be this:

• When the part is being deserialized, we'll check the mod version using Reader.ModVersions. If the mod version is >= 0.3.0, we simply deserialize the part as usual.
• Otherwise, we will read off fields from the part as appropriate for the version that we're migrating from. For example, for v0.1.0 we will only read off S1; for v0.2.0 we will also read off S2.
• Then, once the game is loaded, we will run an AfterGameLoadedEvent handler to complete the migration.

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        public string S1 = "foo";  // added in v0.1.0
        public string S2 = "bar";  // added in v0.2.0
        public string S3 = "baz";  // added in v0.3.0

        /* Everything below this is new! */
        private Version? MigrateFrom = null;

        public override bool WantEvent(int ID, int cascade) {
            return base.WantEvent(ID, cascade)
                || ID == AfterGameLoadedEvent.ID;
        }

        public override bool HandleEvent(AfterGameLoadedEvent E) {
            if (MigrateFrom == null || MigrateFrom >= (new Version("0.3.0"))
                // Player was already on the latest mod version.
                return base.HandleEvent(E);

            if (MigrateFrom >= (new Version("0.2.0"))) {
                // Player was on v0.2.0 or later, so we can migrate the part
                // cleanly.
                S3 = null;
                return base.HandleEvent(E);
            }

            // Player was on a version before v0.2.0, so we remove the part
            ParentObject.RemovePart(this);
            return true;
        }

        public override void Read(GameObject Basis, SerializationReader Reader) {
            var modVersion = Reader.ModVersions["ExampleMod"];

            if (modVersion >= (new Version("0.3.0"))) {
                base.Read(Basis, Reader);
                return;
            }

            // Set MigrateFrom so that we know we'll need to migrate from an
            // older version of the part.
            MigrateFrom = modVersion;

            // S1 has existed in all versions of the part
            S1 = Reader.ReadString();

            // S2 was only added in v0.2.0
            if (modVersion >= (new Version("0.2.0")))
                S2 = Reader.ReadString();
        }
    }
}

Migrating components with parent data: example with IActivePart

In some cases, migration may be complicated by the fact that your component inherits from a class that provides its own data. For example, parts that inherit from IActivePart will inherit the (dozens) of fields specified by the IActivePart class.

In these situations, one strategy is to mark all of your class's fields as NonSerialized. Then you can manually deserialize the fields for your child class, and deserialize the fields of the parent classes via reflection with base.Read. Here's an example using a modified MyPart that is actually an IActivePart rather than an IPart:

using System;

namespace XRL.World.Parts {
    [Serializable]
    public class MyPart : IPart {
        // Now we mark the following fields NonSerialized, so that when we call
        // base.Read we don't attempt to read them via reflection.
        [NonSerialized]
        public string S1 = "foo";  // added in v0.1.0
        [NonSerialized]
        public string S2 = "bar";  // added in v0.2.0
        [NonSerialized]
        public string S3 = "baz";  // added in v0.3.0

        private Version? MigrateFrom = null;

        public override bool WantEvent(int ID, int cascade) {
            return base.WantEvent(ID, cascade)
                || ID == AfterGameLoadedEvent.ID;
        }

        public override bool HandleEvent(AfterGameLoadedEvent E) {
            if (MigrateFrom == null || MigrateFrom >= (new Version("0.3.0"))
                return base.HandleEvent(E);

            if (MigrateFrom >= (new Version("0.2.0"))) {
                S3 = null;
                return base.HandleEvent(E);
            }

            ParentObject.RemovePart(this);
            return true;
        }

        public override void Read(GameObject Basis, SerializationReader Reader) {
            // The logic here has been reorganized a bit. Now in all paths we
            // do base.Read at the very end so that we can deserialize the
            // fields from IActivePart.
            //
            // We also manually deserialize S1, S2, and S3 when we're on the
            // latest version of the game, rather than rely on reflection.
            var modVersion = Reader.ModVersions["ExampleMod"];

            if (modVersion < (new Version("0.3.0")))
                MigrateFrom = modVersion;

            S1 = Reader.ReadString();
            if (modVersion >= (new Version("0.2.0")))
                S2 = Reader.ReadString();
            if (modVersion >= (new Version("0.3.0")))
                S3 = Reader.ReadString();

            // Now we deserialize all of the fields from IActivePart
            base.Read(Basis, Reader);
        }

        /* This is new! */
        public override void Write(GameObject Basis, SerializationWriter Writer) {
            // Since our fields are NonSerialized, we have to manually write them
            // as well.
            Writer.Write(S1);
            Writer.Write(S2);
            Writer.Write(S3);
            base.Write(Basis, Writer);
        }
    }
}