2023-10-03 00:44:19 +00:00
|
|
|
"""
|
|
|
|
A module for interacting with BizHawk through `connector_bizhawk_generic.lua`.
|
|
|
|
|
|
|
|
Any mention of `domain` in this module refers to the names BizHawk gives to memory domains in its own lua api. They are
|
|
|
|
naively passed to BizHawk without validation or modification.
|
|
|
|
"""
|
|
|
|
|
|
|
|
import asyncio
|
|
|
|
import base64
|
|
|
|
import enum
|
|
|
|
import json
|
2023-11-23 19:51:53 +00:00
|
|
|
import sys
|
2023-10-03 00:44:19 +00:00
|
|
|
import typing
|
|
|
|
|
|
|
|
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
BIZHAWK_SOCKET_PORT_RANGE_START = 43055
|
|
|
|
BIZHAWK_SOCKET_PORT_RANGE_SIZE = 5
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
|
|
|
|
class ConnectionStatus(enum.IntEnum):
|
|
|
|
NOT_CONNECTED = 1
|
|
|
|
TENTATIVE = 2
|
|
|
|
CONNECTED = 3
|
|
|
|
|
|
|
|
|
|
|
|
class NotConnectedError(Exception):
|
|
|
|
"""Raised when something tries to make a request to the connector script before a connection has been established"""
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
|
|
class RequestFailedError(Exception):
|
|
|
|
"""Raised when the connector script did not respond to a request"""
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
|
|
class ConnectorError(Exception):
|
|
|
|
"""Raised when the connector script encounters an error while processing a request"""
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
|
|
class SyncError(Exception):
|
|
|
|
"""Raised when the connector script responded with a mismatched response type"""
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
2023-10-27 01:55:46 +00:00
|
|
|
class BizHawkContext:
|
|
|
|
streams: typing.Optional[typing.Tuple[asyncio.StreamReader, asyncio.StreamWriter]]
|
|
|
|
connection_status: ConnectionStatus
|
|
|
|
_lock: asyncio.Lock
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
_port: typing.Optional[int]
|
2023-10-27 01:55:46 +00:00
|
|
|
|
|
|
|
def __init__(self) -> None:
|
|
|
|
self.streams = None
|
|
|
|
self.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
self._lock = asyncio.Lock()
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
self._port = None
|
2023-10-27 01:55:46 +00:00
|
|
|
|
|
|
|
async def _send_message(self, message: str):
|
|
|
|
async with self._lock:
|
|
|
|
if self.streams is None:
|
|
|
|
raise NotConnectedError("You tried to send a request before a connection to BizHawk was made")
|
|
|
|
|
|
|
|
try:
|
|
|
|
reader, writer = self.streams
|
|
|
|
writer.write(message.encode("utf-8") + b"\n")
|
|
|
|
await asyncio.wait_for(writer.drain(), timeout=5)
|
|
|
|
|
|
|
|
res = await asyncio.wait_for(reader.readline(), timeout=5)
|
|
|
|
|
|
|
|
if res == b"":
|
|
|
|
writer.close()
|
|
|
|
self.streams = None
|
|
|
|
self.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
raise RequestFailedError("Connection closed")
|
|
|
|
|
|
|
|
if self.connection_status == ConnectionStatus.TENTATIVE:
|
|
|
|
self.connection_status = ConnectionStatus.CONNECTED
|
|
|
|
|
|
|
|
return res.decode("utf-8")
|
|
|
|
except asyncio.TimeoutError as exc:
|
|
|
|
writer.close()
|
|
|
|
self.streams = None
|
|
|
|
self.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
raise RequestFailedError("Connection timed out") from exc
|
|
|
|
except ConnectionResetError as exc:
|
|
|
|
writer.close()
|
|
|
|
self.streams = None
|
|
|
|
self.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
raise RequestFailedError("Connection reset") from exc
|
|
|
|
|
|
|
|
|
2023-10-03 00:44:19 +00:00
|
|
|
async def connect(ctx: BizHawkContext) -> bool:
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
"""Attempts to establish a connection with a connector script. Returns True if successful."""
|
|
|
|
rotation_steps = 0 if ctx._port is None else ctx._port - BIZHAWK_SOCKET_PORT_RANGE_START
|
|
|
|
ports = [*range(BIZHAWK_SOCKET_PORT_RANGE_START, BIZHAWK_SOCKET_PORT_RANGE_START + BIZHAWK_SOCKET_PORT_RANGE_SIZE)]
|
|
|
|
ports = ports[rotation_steps:] + ports[:rotation_steps]
|
|
|
|
|
|
|
|
for port in ports:
|
|
|
|
try:
|
2023-11-28 21:50:12 +00:00
|
|
|
ctx.streams = await asyncio.open_connection("127.0.0.1", port)
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
ctx.connection_status = ConnectionStatus.TENTATIVE
|
|
|
|
ctx._port = port
|
|
|
|
return True
|
|
|
|
except (TimeoutError, ConnectionRefusedError):
|
|
|
|
continue
|
2024-05-23 00:03:42 +00:00
|
|
|
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
# No ports worked
|
|
|
|
ctx.streams = None
|
|
|
|
ctx.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
return False
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
|
|
|
|
def disconnect(ctx: BizHawkContext) -> None:
|
|
|
|
"""Closes the connection to the connector script."""
|
|
|
|
if ctx.streams is not None:
|
|
|
|
ctx.streams[1].close()
|
|
|
|
ctx.streams = None
|
|
|
|
ctx.connection_status = ConnectionStatus.NOT_CONNECTED
|
|
|
|
|
|
|
|
|
|
|
|
async def get_script_version(ctx: BizHawkContext) -> int:
|
2023-10-27 01:55:46 +00:00
|
|
|
return int(await ctx._send_message("VERSION"))
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
|
|
|
|
async def send_requests(ctx: BizHawkContext, req_list: typing.List[typing.Dict[str, typing.Any]]) -> typing.List[typing.Dict[str, typing.Any]]:
|
|
|
|
"""Sends a list of requests to the BizHawk connector and returns their responses.
|
|
|
|
|
|
|
|
It's likely you want to use the wrapper functions instead of this."""
|
2023-11-23 19:51:53 +00:00
|
|
|
responses = json.loads(await ctx._send_message(json.dumps(req_list)))
|
|
|
|
errors: typing.List[ConnectorError] = []
|
|
|
|
|
|
|
|
for response in responses:
|
|
|
|
if response["type"] == "ERROR":
|
|
|
|
errors.append(ConnectorError(response["err"]))
|
|
|
|
|
|
|
|
if errors:
|
|
|
|
if sys.version_info >= (3, 11, 0):
|
|
|
|
raise ExceptionGroup("Connector script returned errors", errors) # noqa
|
|
|
|
else:
|
|
|
|
raise errors[0]
|
|
|
|
|
|
|
|
return responses
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
|
|
|
|
async def ping(ctx: BizHawkContext) -> None:
|
|
|
|
"""Sends a PING request and receives a PONG response."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "PING"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "PONG":
|
|
|
|
raise SyncError(f"Expected response of type PONG but got {res['type']}")
|
|
|
|
|
|
|
|
|
|
|
|
async def get_hash(ctx: BizHawkContext) -> str:
|
2025-01-12 07:03:31 +00:00
|
|
|
"""Gets the hash value of the currently loaded ROM"""
|
2023-10-03 00:44:19 +00:00
|
|
|
res = (await send_requests(ctx, [{"type": "HASH"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "HASH_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type HASH_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
return res["value"]
|
|
|
|
|
|
|
|
|
2025-01-12 07:03:31 +00:00
|
|
|
async def get_memory_size(ctx: BizHawkContext, domain: str) -> int:
|
|
|
|
"""Gets the size in bytes of the specified memory domain"""
|
|
|
|
res = (await send_requests(ctx, [{"type": "MEMORY_SIZE", "domain": domain}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "MEMORY_SIZE_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type MEMORY_SIZE_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
return res["value"]
|
|
|
|
|
|
|
|
|
2023-10-03 00:44:19 +00:00
|
|
|
async def get_system(ctx: BizHawkContext) -> str:
|
|
|
|
"""Gets the system name for the currently loaded ROM"""
|
|
|
|
res = (await send_requests(ctx, [{"type": "SYSTEM"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "SYSTEM_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type SYSTEM_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
return res["value"]
|
|
|
|
|
|
|
|
|
|
|
|
async def get_cores(ctx: BizHawkContext) -> typing.Dict[str, str]:
|
|
|
|
"""Gets the preferred cores for systems with multiple cores. Only systems with multiple available cores have
|
|
|
|
entries."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "PREFERRED_CORES"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "PREFERRED_CORES_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type PREFERRED_CORES_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
return res["value"]
|
|
|
|
|
|
|
|
|
|
|
|
async def lock(ctx: BizHawkContext) -> None:
|
|
|
|
"""Locks BizHawk in anticipation of receiving more requests this frame.
|
|
|
|
|
|
|
|
Consider using guarded reads and writes instead of locks if possible.
|
|
|
|
|
|
|
|
While locked, emulation will halt and the connector will block on incoming requests until an `UNLOCK` request is
|
|
|
|
sent. Remember to unlock when you're done, or the emulator will appear to freeze.
|
|
|
|
|
|
|
|
Sending multiple lock commands is the same as sending one."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "LOCK"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "LOCKED":
|
|
|
|
raise SyncError(f"Expected response of type LOCKED but got {res['type']}")
|
|
|
|
|
|
|
|
|
|
|
|
async def unlock(ctx: BizHawkContext) -> None:
|
|
|
|
"""Unlocks BizHawk to allow it to resume emulation. See `lock` for more info.
|
|
|
|
|
|
|
|
Sending multiple unlock commands is the same as sending one."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "UNLOCK"}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "UNLOCKED":
|
|
|
|
raise SyncError(f"Expected response of type UNLOCKED but got {res['type']}")
|
|
|
|
|
|
|
|
|
|
|
|
async def display_message(ctx: BizHawkContext, message: str) -> None:
|
|
|
|
"""Displays the provided message in BizHawk's message queue."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "DISPLAY_MESSAGE", "message": message}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "DISPLAY_MESSAGE_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type DISPLAY_MESSAGE_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
|
|
|
|
async def set_message_interval(ctx: BizHawkContext, value: float) -> None:
|
|
|
|
"""Sets the minimum amount of time in seconds to wait between queued messages. The default value of 0 will allow one
|
|
|
|
new message to display per frame."""
|
|
|
|
res = (await send_requests(ctx, [{"type": "SET_MESSAGE_INTERVAL", "value": value}]))[0]
|
|
|
|
|
|
|
|
if res["type"] != "SET_MESSAGE_INTERVAL_RESPONSE":
|
|
|
|
raise SyncError(f"Expected response of type SET_MESSAGE_INTERVAL_RESPONSE but got {res['type']}")
|
|
|
|
|
|
|
|
|
2024-09-28 20:49:11 +00:00
|
|
|
async def guarded_read(ctx: BizHawkContext, read_list: typing.Sequence[typing.Tuple[int, int, str]],
|
|
|
|
guard_list: typing.Sequence[typing.Tuple[int, typing.Sequence[int], str]]) -> typing.Optional[typing.List[bytes]]:
|
2023-10-03 00:44:19 +00:00
|
|
|
"""Reads an array of bytes at 1 or more addresses if and only if every byte in guard_list matches its expected
|
|
|
|
value.
|
|
|
|
|
|
|
|
Items in read_list should be organized (address, size, domain) where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `size` is the number of bytes to read
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to
|
|
|
|
|
|
|
|
Items in `guard_list` should be organized `(address, expected_data, domain)` where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `expected_data` is the bytes that the data starting at this address is expected to match
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to
|
|
|
|
|
|
|
|
Returns None if any item in guard_list failed to validate. Otherwise returns a list of bytes in the order they
|
|
|
|
were requested."""
|
|
|
|
res = await send_requests(ctx, [{
|
|
|
|
"type": "GUARD",
|
|
|
|
"address": address,
|
|
|
|
"expected_data": base64.b64encode(bytes(expected_data)).decode("ascii"),
|
|
|
|
"domain": domain
|
|
|
|
} for address, expected_data, domain in guard_list] + [{
|
|
|
|
"type": "READ",
|
|
|
|
"address": address,
|
|
|
|
"size": size,
|
|
|
|
"domain": domain
|
|
|
|
} for address, size, domain in read_list])
|
|
|
|
|
|
|
|
ret: typing.List[bytes] = []
|
|
|
|
for item in res:
|
|
|
|
if item["type"] == "GUARD_RESPONSE":
|
|
|
|
if not item["value"]:
|
|
|
|
return None
|
|
|
|
else:
|
|
|
|
if item["type"] != "READ_RESPONSE":
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
raise SyncError(f"Expected response of type READ_RESPONSE or GUARD_RESPONSE but got {item['type']}")
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
ret.append(base64.b64decode(item["value"]))
|
|
|
|
|
|
|
|
return ret
|
|
|
|
|
|
|
|
|
2024-09-28 20:49:11 +00:00
|
|
|
async def read(ctx: BizHawkContext, read_list: typing.Sequence[typing.Tuple[int, int, str]]) -> typing.List[bytes]:
|
2023-10-03 00:44:19 +00:00
|
|
|
"""Reads data at 1 or more addresses.
|
|
|
|
|
|
|
|
Items in `read_list` should be organized `(address, size, domain)` where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `size` is the number of bytes to read
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to
|
|
|
|
|
|
|
|
Returns a list of bytes in the order they were requested."""
|
|
|
|
return await guarded_read(ctx, read_list, [])
|
|
|
|
|
|
|
|
|
2024-09-28 20:49:11 +00:00
|
|
|
async def guarded_write(ctx: BizHawkContext, write_list: typing.Sequence[typing.Tuple[int, typing.Sequence[int], str]],
|
|
|
|
guard_list: typing.Sequence[typing.Tuple[int, typing.Sequence[int], str]]) -> bool:
|
2023-10-03 00:44:19 +00:00
|
|
|
"""Writes data to 1 or more addresses if and only if every byte in guard_list matches its expected value.
|
|
|
|
|
|
|
|
Items in `write_list` should be organized `(address, value, domain)` where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `value` is a list of bytes to write, in order, starting at `address`
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to
|
|
|
|
|
|
|
|
Items in `guard_list` should be organized `(address, expected_data, domain)` where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `expected_data` is the bytes that the data starting at this address is expected to match
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to
|
|
|
|
|
|
|
|
Returns False if any item in guard_list failed to validate. Otherwise returns True."""
|
|
|
|
res = await send_requests(ctx, [{
|
|
|
|
"type": "GUARD",
|
|
|
|
"address": address,
|
|
|
|
"expected_data": base64.b64encode(bytes(expected_data)).decode("ascii"),
|
|
|
|
"domain": domain
|
|
|
|
} for address, expected_data, domain in guard_list] + [{
|
|
|
|
"type": "WRITE",
|
|
|
|
"address": address,
|
|
|
|
"value": base64.b64encode(bytes(value)).decode("ascii"),
|
|
|
|
"domain": domain
|
|
|
|
} for address, value, domain in write_list])
|
|
|
|
|
|
|
|
for item in res:
|
|
|
|
if item["type"] == "GUARD_RESPONSE":
|
|
|
|
if not item["value"]:
|
|
|
|
return False
|
|
|
|
else:
|
|
|
|
if item["type"] != "WRITE_RESPONSE":
|
BizHawkClient: Add support for multiple concurrent instances (#2475)
This allows multiple client/connector pairs to run at the same time. It also includes a few other miscellaneous small changes that accumulated as I went. They can be split if desired
- Whatever the `client_socket:send` line (~440) was doing with that missing operator, it's no longer doing. Don't ask me how it was working before. Lua is witchcraft.
- Removed the `settimeout(2)` which causes the infamous emulator freeze (and replaced it with a `settimeout(0)` when the server socket is created). It appears to be unnecessary to set a timeout for discovering a client. Maybe at some point in time it was useful to keep the success rate for connecting high, but it seems to not be a problem if the timeout is 0 instead.
- Also updated the Emerald setup to remove mention of the freezing.
- Connector script now picks the first port that's not in use in a range of 5 ports.
- To summarize why I was previously under the impression that multiple running scripts would not detect when a port was in use:
1. Calling `socket.bind` in the existing script will first create an ipv6 socket.
2. A second concurrent script trying to bind to the same port would I think fail to create an ipv6 socket but then succeed in creating an ipv4 socket on the same port.
3. That second socket could never communicate with a client; extra clients would just bounce off the first script.
4. The third concurrent script will then fail on both and actually give an `address already in use` error.
- I'm not _really_ sure what's going on there. But forcing one or the other by calling `socket.tcp4()` or `socket.tcp6()` means that only one script will believe it has the port while any others will give `address already in use` as you'd expect.
- As a side note, our `socket.lua` is much wonkier than I had previously thought. I understand some parts were added for LADX and when BizHawk 2.9 came out, but as far back as the file's history in this repo, it has provided a strange, modified interface as compared to the file it was originally derived from, to no benefit as far as I can tell.
- The connector script closes `server` once it finds a client and opens a new one if the connection drops. I'm not sure this ultimately has an effect, but it seems more proper.
- If the connector script's main function returns because of some error or refusal to proceed, the script no longer tries to resume the coroutine it was part of, which would flood the log with irrelevant errors.
- Creating `SyncError`s in `guarded_read` and `guarded_write` would raise its own error because the wrong variable was being used in its message.
- A call to `_bizhawk.connect` can take a while as the client tries the possible ports. There's a modification that will wait on either the `connect` or the exit event. And if the exit event fires while still looking for a connector script, this cancels the `connect` so the window can close.
- Related: It takes 2-3 seconds for a call to `asyncio.open_connection` to come back with any sort of response on my machine, which can be significant now that we're trying multiple ports in sequence. I guess it could fire off 5 tasks at once. Might cause some weirdness if there exist multiple scripts and multiple clients looking for each other at the same time.
- Also related: The first time a client attempts to connect to a script, they accept each other and start communicating as expected. The second client to try that port seems to believe it connects and will then time out on the first message. And then all subsequent attempts to connect to that port by any client will be refused (as expected) until the script shuts down or restarts. I haven't been able to explain this behavior. It adds more time to a client's search for a script, but doesn't ultimately cause problems.
2023-11-23 14:00:46 +00:00
|
|
|
raise SyncError(f"Expected response of type WRITE_RESPONSE or GUARD_RESPONSE but got {item['type']}")
|
2023-10-03 00:44:19 +00:00
|
|
|
|
|
|
|
return True
|
|
|
|
|
|
|
|
|
2024-09-28 20:49:11 +00:00
|
|
|
async def write(ctx: BizHawkContext, write_list: typing.Sequence[typing.Tuple[int, typing.Sequence[int], str]]) -> None:
|
2023-10-03 00:44:19 +00:00
|
|
|
"""Writes data to 1 or more addresses.
|
|
|
|
|
|
|
|
Items in write_list should be organized `(address, value, domain)` where
|
|
|
|
- `address` is the address of the first byte of data
|
|
|
|
- `value` is a list of bytes to write, in order, starting at `address`
|
|
|
|
- `domain` is the name of the region of memory the address corresponds to"""
|
|
|
|
await guarded_write(ctx, write_list, [])
|