.blmod file format spec

blmod.md

.blmod file format (v1)

The .blmod file format is designed as a replacement to the .blcm format, and to be generic enough that it could be applied to the existing .bl3hotfix/.wlhotfix file format, or possible future ones.

The main thing it's designed around is mods which consist of text files holding one "command" per line. Commands can be arbitrary strings (as long as they don't contain newlines). These commands can then be grouped into categories, which users can enable/disable to configure their mods - i.e. what BLCMM does.

This document only goes over the base file format. Tools interacting with it may define their own extensions.

For background info/early ideas which were incorporated into this spec, see this old proposal document.

Rough Overview

A .blmod file is a yaml file, consisting of two separate documents.

'blmod':
'version': 1
'encoding': ascii
---
'category': root
'contains':
  - 'enabled': |-
      say This is a command

The first document is the file header, which contains overall information about the file. The second document contains all the mod content.

Using a yaml-based format allows us to use multi-line strings to store commands in line. This does rely on a few assumptions on how commands are parsed:

• Leading whitespace is ignored
• No command starts with a ' or -

This has not been found to be a problem in practice.

.blmod files do not use any "advanced" yaml features such as tags. Each document has a one-to-one mapping to json, and can be round-trip converted (though of course it's not valid in json format).

Schema

The documents within the file can be validated against the following schema. You can convert both the document and the schema to json to use a standard json schema validator

Header

title: blmod header
description: .blmod file header
type: object
properties:
  blmod:
    description: >-
      A magic string which MUST appear at the top of the file. The value is ignored.
  encoding:
    description: >-
      What encoding the rest of the file uses.

      While we have some predefined values (which SHOULD be case insensitive), any value is
      acceptable. Custom values MUST also be written in ascii, to allow interpreting without fully
      knowing the correct encoding. If a tool does not understand the encoding string, it MUST throw
      an error.
    anyOf:
      - enum:
          - ascii
          - utf8
          - utf16
          - utf16le
          - utf16be
          - utf32
          - utf32le
          - utf32be
      - type: string
        pattern: ^[\x00-\x7F]+$
  version:
    description: >-
      The file's format version. Parsers MUST always check this, and reject any value other than 1.
    type: integer
    minimum: 1
    maximum: 1
  games:
    description: >-
      An array of all games this mod file supports.
      
      While we have some predefined values (which SHOULD be case insensitive), any value is
      acceptable. Values which a tool does not recognise MUST be ignored.
    type: array
    minItems: 1
    items:
      anyOf:
        - enum:
            - bl1
            - bl1e
            - bl2
            - tps
            - aodk
            - bl3
            - wl
            - bl4
        - type: string
  metadata:
    description: >-
      Arbitrary user-defined file-specific metadata.
    type: object
    default: {}
required:
  - blmod
  - encoding
  - version
  - games

Contents

title: blmod contents
description: .blmod file contents
$ref: "#/definitions/category"
definitions:
  comment:
    description: >-
      A comment.

      Comments are only information for users, they MUST NOT be taken to have any meaning.
    type: object
    properties:
      comment:
        description: The comment string.
        type: string
    required:
      - comment
  enabled:
    description: An enabled command.
    type: object
    properties:
      enabled:
        description: The command string. MUST be formatted as a multiline string.
        type: string
    required:
      - enabled
  disabled:
    description: A disabled command.
    type: object
    properties:
      disabled:
        description: The command string. MUST NOT be formatted as a multiline string.
        type: string
    required:
      - disabled
  category:
    description: >-
      A nested category.

      Categories have no inherent enabled/disabled status, instead they inherit it recursively from
      their contents:
      - If there are no 'enabled' commands, the category is considered to be disabled. This includes
        the case where there are no recursive child commands at all.
      - If there are some recursive child commands, and there are no 'disabled' commands, the
        category is considered to be enabled.
      - Otherwise, if there are recursive child commands, but there are both 'enabled' and
        'disabled' commands, the category is considered to be partially enabled.
    type: object
    properties:
      category:
        description: The category's name.
        type: string
      contains:
        description: This category's contents.
        type: array
        items:
          oneOf:
            - "$ref": "#/definitions/comment"
            - "$ref": "#/definitions/enabled"
            - "$ref": "#/definitions/disabled"
            - "$ref": "#/definitions/category"
      locked:
        description: >-
          When true, the category is locked. An editor MUST NOT allow changing the enabled status of
          any child commands recursively.
        type: boolean
        default: false
      mut:
        description: >-
          When true, the category contains mutually exclusive options. An editor MUST only allow
          exactly one of this category's direct children to be enabled or partially enabled. All
          other direct children MUST be disabled.
        type: boolean
        default: false
      metadata:
        description: >-
          When a category is obtained by importing another mod file into this one, the metadata in
          the importee's header SHOULD be copied to this.

          The presence of this property MAY be used to infer that a category is an entire mod, which
          might for example adjust styling.

          Generally, metadata in this property SHOULD be ignored, in favour of that in this file's
          header. In some cases there may be reason to check mod-specific metadata - e.g. an editor
          might look for a mod's description and demo screenshots, and display those.
        type: object
    required:
      - category
      - contains

Both of these schema deliberately allow additional properties. Future updates to this spec may add new ones, and older tools MUST accept and ignore them. If a new property requires parsing changes, the version property in the header will be bumped, hence tools MUST reject any version they don't understand.

Additionally, tools working with .blmod files can add their own custom properties. Any property starting with an underscore _ is a custom property, no "official" properties will ever start with one. Tools adding custom properties SHOULD generally prefix them using the tool's name - e.g. BLCMM might define a _blcmm_custom_prop. As with new "official" properties, all tools MUST accept and ignore any custom properties they don't understand.

Determining Encoding

Different games have different encoding requirements, so we can't simply require all .blmod files use the same one. To handle this we have an encoding property in the header - but we have a bit of a chicken-and-egg problem. In order to be able to parse the header to find out what encoding we want, we need to know what encoding to read it as.

The way we resolve this is by parsing the header twice, the first time with a temporary encoding to find out the actual encoding. All encodings MUST be written in ascii, so that they won't get corrupted if the temporary encoding is wrong. In order to do this we must still match the actual encoding's width and endianness however.

Exact steps for determining encoding are as follows:

1. If the file begins with a unicode BOM, we can immediately identify it as that encoding.

   Bytes	Encoding
   EF BB BF	utf8
   FF FE	utf16le
   FE FF	utf16be
   FF FE 00 00	utf32le
   00 00 FE FF	utf32be

   All parsers MUST recognise all forms of the BOM, to make sure they don't try read the rest of the file using the wrong width/endianness. Parsers MAY however only implement some of these, and simply throw an error on encountering the other formats.

   1. If there isn't a BOM, continue on to step 2.

   2. If there is a BOM, select the relevant encoding as the temporary encoding.

      The first 8 characters of any .blmod file MUST be the magic string 'blmod':. Use the temporary encoding to read them out and validate this. If this is not the case, the file is considered not to be a .blmod, and a parser MUST NOT continue parsing it as one. Tools may choose to continue attempting to parse it as other file formats, this need not be a user facing error.

      If the prefix is valid, jump to step 4.

2. We did not have a unicode BOM. As mentioned above, the first 8 characters of any .blmod file MUST be the magic string 'blmod':. Since these are all ascii, we can use them to determine width/endianness.

   Bytes	Encoding
   27 62 6c 6d ...	Single byte
   27 00 62 00 ...	2-byte little endian
   00 27 00 62 ...	2-byte big endian
   27 00 00 00 ...	4-byte little endian
   00 00 00 27 ...	4-byte big endian

   As before, all parsers MUST recognise all these forms, however they MAY only implement some of them, and throw errors on the other formats. And if none of these match, the file is considered not to be a .blmod, and a parser MUST NOT continue parsing it as one.

3. Select an arbitrary encoding matching the detected width/endianness characteristics. This is the temporary encoding.

4. Seek back to the start of the file, then attempt to parse the first document.

   If the parser encounters any undecodeable bytes (if the temporary encoding is wrong), it MUST NOT error. Instead, it MAY corrupt the given data in an arbitrary manner.

   If yaml decoding fails, the file is ill formed and the parser MUST throw an error.

5. Read out the encoding property, to get the actual encoding. This is an arbitrary (ascii) string, however we predefine the following formats (case-insensitively):

   • ascii
   • utf8
   • utf16, utf16le
   • utf16be
   • utf32, utf32le
   • utf32be

   A parser MAY only implement some of these, it MAY accept other aliases to the same encodings, and it MAY accept other strings mapping to completely new encodings.

   If a parser doesn't recognise the encoding (including if it's one of the predefined ones it chose not to implement), or if the encoding property is not present or of the wrong type, the file is ill formed and the parser MUST throw an error.

6. If the actual encoding is different to the temporary encoding, throw away all the data, seek back to the start of the file, and parse the header again using the actual encoding.

Parsing

After determining the encoding, the file can be parsed using a standard yaml decoder.

If the file is not valid yaml, a parser MUST throw an error.

If the file parses, but does not fit the schema, a parser SHOULD throw an error. It is permitted not to if the mistake occurs in an inconsequential location - for example, if a file contains a tag, a parser may simply ignore it. It is also permitted to "best-effort" parse the rest of the file - for example a single malformed command might not invalidate the best of the file. Recovery in this scenario is implementation defined.

The version property in the header MUST be equal to the integer 1. If it is not, a parser MUST throw an error, which SHOULD state that the file was made for a newer version.

Serializing

Serializing a .blmod file is a little more involved.

Before writing anything, the serializer MUST update the encoding property to the encoding it's outputting as.

Then there are a few yaml settings that must be configured:

• All properties MUST be quoted, using single quotes.
• All arrays/mappings MUST be in their multiline form (no json-style).
• All strings under an enabled property MUST be multiline strings. A serializer MAY choose any multiline format, as is appropriate for the contained data.
• All other strings MUST appear on a single line, even if they contain newlines.
• The serializer MUST NOT output any yaml tags.

Finally, the very first line of output MUST be the blmod property - all files MUST start with the string 'blmod':.

Round Tripping

A mod editor, or any tool that supports both parsing and serializing, has a few more round-tripping rules to obey (assuming the user hasn't made any changes).

A tool MAY change the following data - i.e. they're not required to round-trip:

• The blmod property's value.
• The predefined encoding and game enum's values - though their meanings SHOULD NOT be changed without user action. For example, normalizing bl1 to BL1 or utf-8 to utf8 is allowed.
• The order of properties in any mapping. The blmod property MUST still be the first property at the top of the file however.
• Any custom property the tool understands, which is specified as ok to change.

An tool MUST NOT change anything else, including (but not limited to):

• Any values in metadata.
• Any unrecognised standard or custom properties.
• The order of commands/comments within a category.

After an tool outputs a file once, running it back through the same tool SHOULD round-trip, changes are only generally expected when first loading from one into another.