Archipelago/worlds/_bizhawk

BizHawk Client

BizHawkClient is an abstract base class for a client that can access the memory of a ROM running in BizHawk. It does the legwork of connecting Python to a Lua connector script, letting you focus on the loop of checking locations and making on-the-fly modifications based on updates from the server. It also provides the same experience to users across multiple games that use it, and was built in response to a growing number of similar but separate bespoke game clients which are/were largely exclusive to BizHawk anyway.

It's similar to SNIClient, but where SNIClient is designed to work for specifically SNES games across different emulators/hardware, BizHawkClient is designed to work for specifically BizHawk across the different systems BizHawk supports.

The idea is that BizHawkClient connects to and communicates with a Lua script running in BizHawk. It provides an API that will call BizHawk functions for you to do things like read and write memory. And on an interval, control will be handed to a function you write for your game (game_watcher) which should interact with the game's memory to check what locations have been checked, give the player items, detect and send deathlinks, etc...

Table of Contents:

• Connector Requests
  • Requests that depend on other requests
• Implementing a Client
  • Example
• Tips

Connector Requests

Communication with BizHawk is done through connector_bizhawk_generic.lua. The client sends requests to the Lua script via sockets; the Lua script processes the request and sends the corresponding responses.

The Lua script includes its own documentation, but you probably don't need to worry about the specifics. Instead, you'll be using the functions in worlds/_bizhawk/__init__.py. If you do need more control over the specific requests being sent or their order, you can still use send_requests to directly communicate with the connector script.

It's not necessary to use the UI or client context if you only want to interact with the connector script. You can import and use just worlds/_bizhawk/__init__.py, which only depends on default modules.

Here's a list of the included classes and functions. I would highly recommend looking at the actual function signatures and docstrings to learn more about each function.

class ConnectionStatus
class BizHawkContext

class NotConnectedError
class RequestFailedError
class ConnectorError
class SyncError

async def read(ctx, read_list) -> list[bytes]
async def write(ctx, write_list) -> None:
async def guarded_read(ctx, read_list, guard_list) -> (list[bytes] | None)
async def guarded_write(ctx, write_list, guard_list) -> bool

async def lock(ctx) -> None
async def unlock(ctx) -> None

async def get_hash(ctx) -> str
async def get_system(ctx) -> str
async def get_cores(ctx) -> dict[str, str]
async def ping(ctx) -> None

async def display_message(ctx, message: str) -> None
async def set_message_interval(ctx, value: float) -> None

async def connect(ctx) -> bool
def disconnect(ctx) -> None

async def get_script_version(ctx) -> int
async def send_requests(ctx, req_list) -> list[dict[str, Any]]

send_requests is what actually communicates with the connector, and any functions like guarded_read will build the requests and then call send_requests for you. You can call send_requests yourself for more direct control, but make sure to read the docs in connector_bizhawk_generic.lua.

A bundle of requests sent by send_requests will all be executed on the same frame, and by extension, so will any helper that calls send_requests. For example, if you were to call read with 3 items on your read_list, all 3 addresses will be read on the same frame and then sent back.

It also means that, by default, the only way to run multiple requests on the same frame is for them to be included in the same send_requests call. As soon as the connector finishes responding to a list of requests, it will advance the frame before checking for the next batch.

Requests that depend on other requests

The fact that you have to wait at least a frame to act on any response may raise concerns. For example, Pokemon Emerald's save data is at a dynamic location in memory; it moves around when you load a new map. There is a static variable that holds the address of the save data, so we want to read the static variable to get the save address, and then use that address in a write to send the player an item. But between the read that tells us the address of the save data and the write to save data itself, an arbitrary number of frames have been executed, and the player may have loaded a new map, meaning we've written data to who knows where.

There are two solutions to this problem.

1. Use guarded_write instead of write. We can include a guard against the address changing, and the script will only perform the write if the data in memory matches what's in the guard. In the below example, write_result will be True if the guard validated and the data was written, and False if the guard failed to validate.

# Get the address of the save data
read_result: bytes = (await _bizhawk.read(ctx, [(0x3001111, 4, "System Bus")]))[0]
save_data_address = int.from_bytes(read_result, "little")

# Write to `save_data_address` if it hasn't changed
write_result: bool = await _bizhawk.guarded_write(
    ctx,
    [(save_data_address, [0xAA, 0xBB], "System Bus")],
    [(0x3001111, read_result, "System Bus")]
)

if write_result:
    # The data at 0x3001111 was still the same value as
    # what was returned from the first `_bizhawk.read`,
    # so the data was written.
    ...
else:
    # The data at 0x3001111 has changed since the
    # first `_bizhawk.read`, so the data was not written.
    ...

2. Use lock and unlock (discouraged if not necessary). When you call lock, you tell the emulator to stop advancing frames and just process requests until it receives an unlock request. This means you can lock, read the address, write the data, and then unlock on a single frame. However, this is slow. If you can't get in and get out quickly enough, players will notice a stutter in the emulation.

# Pause emulation
await _bizhawk.lock(ctx)

# Get the address of the save data
read_result: bytes = (await _bizhawk.read(ctx, [(0x3001111, 4, "System Bus")]))[0]
save_data_address = int.from_bytes(read_result, "little")

# Write to `save_data_address`
await _bizhawk.write(ctx, [(save_data_address, [0xAA, 0xBB], "System Bus")])

# Resume emulation
await _bizhawk.unlock(ctx)

You should always use guarded_read and guarded_write instead of locking the emulator if possible. It may be unreliable, but that's by design. Most of the time you should have no problem giving up and retrying. Data that is volatile but only changes occasionally is the perfect use case.

If data is almost guaranteed to change between frames, locking may be the better solution. You can lower the time spent locked by using send_requests directly to include as many requests alongside the LOCK and UNLOCK requests as possible. But in general it's probably worth doing some extra asm hacking and designing to make guards work instead.

Implementing a Client

BizHawkClient itself is built on CommonClient and inspired heavily by SNIClient. Your world's client should inherit from BizHawkClient in worlds/_bizhawk/client.py. It must implement validate_rom and game_watcher, and must define values for system and game.

As with the functions and classes in the previous section, I would highly recommend looking at the types and docstrings of the code itself.

game should be the same value you use for your world definition.

system can either be a string or a tuple of strings. This is the system (or systems) that your client is intended to handle games on (SNES, GBA, etc.). It's used to prevent validators from running on unknown systems and crashing. The actual abbreviation corresponds to whatever BizHawk returns from emu.getsystemid().

patch_suffix is an optional ClassVar meant to specify the file extensions you want to register. It can be a string or tuple of strings. When a player clicks "Open Patch" in a launcher, the suffix(es) will be whitelisted in the file select dialog and they will be associated with BizHawkClient. This does not affect whether the user's computer will associate the file extension with Archipelago.

validate_rom is called to figure out whether a given ROM belongs to your client. It will only be called when a ROM is running on a system you specified in your system class variable. In most cases, that will be a single system and you can be sure that you're not about to try to read from nonexistent domains or out of bounds. If you decide to claim this ROM as yours, this is where you should do setup for things like items_handling.

game_watcher is the "main loop" of your client where you should be checking memory and sending new items to the ROM. BizHawkClient will make sure that your game_watcher only runs when your client has validated the ROM, and will do its best to make sure you're connected to the connector script before calling your watcher. It runs this loop either immediately once it receives a message from the server, or a specified amount of time after the last iteration of the loop finished.

validate_rom, game_watcher, and other methods will be passed an instance of BizHawkClientContext, which is a subclass of CommonContext. It additionally includes slot_data (if you are connected and asked for slot data), bizhawk_ctx (the instance of BizHawkContext that you should be giving to functions like guarded_read), and watcher_timeout (the amount of time in seconds between iterations of the game watcher loop).

Example

A very simple client might look like this. All addresses here are made up; you should instead be using addresses that make sense for your specific ROM. The validate_rom here tries to read the name of the ROM. If it gets the value it wanted, it sets a couple values on ctx and returns True. The game_watcher reads some data from memory and acts on it by sending messages to AP. You should be smarter than this example, which will send LocationChecks messages even if there's nothing new since the last loop.

from typing import TYPE_CHECKING

from NetUtils import ClientStatus

import worlds._bizhawk as bizhawk
from worlds._bizhawk.client import BizHawkClient

if TYPE_CHECKING:
    from worlds._bizhawk.context import BizHawkClientContext


class MyGameClient(BizHawkClient):
    game = "My Game"
    system = "GBA"
    patch_suffix = ".apextension"

    async def validate_rom(self, ctx: "BizHawkClientContext") -> bool:
        try:
            # Check ROM name/patch version
            rom_name = ((await bizhawk.read(ctx.bizhawk_ctx, [(0x100, 6, "ROM")]))[0]).decode("ascii")
            if rom_name != "MYGAME":
                return False  # Not a MYGAME ROM
        except bizhawk.RequestFailedError:
            return False  # Not able to get a response, say no for now

        # This is a MYGAME ROM
        ctx.game = self.game
        ctx.items_handling = 0b001
        ctx.want_slot_data = True

        return True

    async def game_watcher(self, ctx: "BizHawkClientContext") -> None:
        try:
            # Read save data
            save_data = await bizhawk.read(
                ctx.bizhawk_ctx,
                [(0x3000100, 20, "System Bus")]
            )[0]

            # Check locations
            if save_data[2] & 0x04:
                await ctx.send_msgs([{
                    "cmd": "LocationChecks",
                    "locations": [23]
                }])

            # Send game clear
            if not ctx.finished_game and (save_data[5] & 0x01):
                await ctx.send_msgs([{
                    "cmd": "StatusUpdate",
                    "status": ClientStatus.CLIENT_GOAL
                }])

        except bizhawk.RequestFailedError:
            # The connector didn't respond. Exit handler and return to main loop to reconnect
            pass

Tips

• Make sure your client gets imported when your world is imported. You probably don't need to actually use anything in your client.py elsewhere, but you still have to import the file for your client to register itself.
• When it comes to performance, there are two directions to optimize:
  1. If you need to execute multiple commands on the same frame, do as little work as possible. Only read and write necessary data, and if you have to use locks, unlock as soon as it's okay to advance frames. This is probably the obvious one.
  2. Multiple things that don't have to happen on the same frame should be split up if they're likely to be slow. Remember, the game watcher runs only a few times per second. Extra function calls on the client aren't that big of a deal; the player will not notice if your game_watcher is slow. But the emulator has to be done with any given set of commands in 1/60th of a second to avoid hiccups (faster still if your players use speedup). Too many reads of too much data at the same time is more likely to cause a bad user experience.
• Your game_watcher will be called regardless of the status of the client's connection to the server. Double-check the server connection before trying to interact with it.
• By default, the player will be asked to provide their slot name after connecting to the server and validating, and that input will be used to authenticate with the Connect command. You can override set_auth in your own client to set it automatically based on data in the ROM or on your client instance.
• You can override on_package in your client to watch raw packages, but don't forget you also have access to a subclass of CommonContext and its API.
• You can import BizHawkClientContext for type hints using typing.TYPE_CHECKING. Importing it without conditions at the top of the file will probably cause a circular dependency.
• Your game's system may have multiple usable cores in BizHawk. You can use get_cores to try to determine which one is currently loaded (it's the best we can do). Some cores may differ in the names of memory domains. It's good to check all the available cores to find differences before your users do.
• The connector script includes a DEBUG variable that you can use to log requests/responses. (Be aware that as the log grows in size in BizHawk, it begins to stutter while trying to print it.)