Options: implement progression balancing and accessibility on new system
Options: implement the notion of "common" and "per_game_common" options in various systems
Options: centralize item and location name checking
Spoiler: prettier print some lists, sets and dicts
WebHost: add common options into /templates
* first commit (not including OoT data files yet)
* added some basic options
* rule parser works now at least
* make sure to commit everything this time
* temporary change to BaseClasses for oot
* overworld location graph builds mostly correctly
* adding oot data files
* commenting out world options until later since they only existed to make the RuleParser work
* conversion functions between AP ids and OOT ids
* world graph outputs
* set scrub prices
* itempool generates, entrances connected, way too many options added
* fixed set_rules and set_shop_rules
* temp baseclasses changes
* Reaches the fill step now, old event-based system retained in case the new way breaks
* Song placements and misc fixes everywhere
* temporary changes to make oot work
* changed root exits for AP fill framework
* prevent infinite recursion due to OoT sharing usage of the address field
* age reachability works hopefully, songs are broken again
* working spoiler log generation on beatable-only
* Logic tricks implemented
* need this for logic tricks
* fixed map/compass being placed on Serenade location
* kill unreachable events before filling the world
* add a bunch of utility functions to prepare for rom patching
* move OptionList into generic options
* fixed some silly bugs with OptionList
* properly seed all random behavior (so far)
* ROM generation working
* fix hints trying to get alttp dungeon hint texts
* continue fixing hints
* add oot to network data package
* change item and location IDs to 66000 and 67000 range respectively
* push removed items to precollected items
* fixed various issues with cross-contamination with multiple world generation
* reenable glitched logic (hopefully)
* glitched world files age-check fix
* cleaned up some get_locations calls
* added token shuffle and scrub shuffle, modified some options slightly to make the parsing work
* reenable MQ dungeons
* fix forest mq exception
* made targeting style an option for now, will be cosmetic later
* reminder to move targeting to cosmetics
* some oot option maintenance
* enabled starting time of day
* fixed issue breaking shop slots in multiworld generation
* added "off" option for text shuffle and hints
* shopsanity functionality restored
* change patch file extension
* remove unnecessary utility functions + imports
* update MIT license
* change option to "patch_uncompressed_rom" instead of "compress_rom"
* compliance with new AutoWorld systems
* Kill only internal events, remove non-internal big poe event in code
* re-add the big poe event and handle it correctly
* remove extra method in Range option
* fix typo
* Starting items, starting with consumables option
* do not remove nonexistent item
* move set_shop_rules to after shop items are placed
* some cleanup
* add retries for song placement
* flagged Skull Mask and Mask of Truth as advancement items
* update OoT to use LogicMixin
* Fixed trying to assign starting items from the wrong players
* fixed song retry step
* improved option handling, comments, and starting item replacements
* DefaultOnToggle writes Yes or No to spoiler
* enable compression of output if Compress executable is present
* clean up compression
* check whether (de)compressor exists before running the process
* allow specification of rom path in host.yaml
* check if decompressed file already exists before decompressing again
* fix triforce hunt generation
* rename all the oot state functions with prefix
* OoT: mark triforce pieces as completion goal for triforce hunt
* added overworld and any-dungeon shuffle for dungeon items
* Hide most unshuffled locations and events from the list of locations in spoiler
* build oot option ranges with a generic function instead of defining each separately
* move oot output-type control to host.yaml instead of individual yamls
* implement dungeon song shuffle
* minor improvements to overworld dungeon item shuffle
* remove random ice trap names in shops, mostly to avoid maintaining a massive censor list
* always output patch file to folder, remove option to generate ROM in preparation for removal
* re-add the fix for infinite recursion due to not being light or dark world
* change AP-sendable to Ocarina of Time model, since the triforce piece has some extra code apparently
* oot: remove item_names and location_names
* oot: minor fixes
* oot: comment out ROM patching
* oot: only add CollectionState objects on creation if actually needed
* main entrance shuffle method and entrances-based rules
* fix entrances based rules
* disable master quest and big poe count options for client compatibility
* use get_player_name instead of get_player_names
* fix OptionList
* fix oot options for new option system
* new coop section in oot rom: expand player names to 16 bytes, write AP_PLAYER_NAME at end of PLAYER_NAMES
* fill AP player name in oot rom with 0 instead of 0xDF
* encode player name with ASCII for fixed-width
* revert oot player name array to 8 bytes per name
* remove Pierre location if fast scarecrow is on
* check player name length
* "free_scarecrow" not "fast_scarecrow"
* OoT locations now properly store the AP ID instead of the oot internal ID
* oot __version__ updates in lockstep with AP version
* pull in unmodified oot cosmetic files
* also grab JSONDump since it's needed apparently
* gather extra needed methods, modify imports
* delete cosmetics log, replace all instances of SettingsList with OOTWorld
* cosmetic options working, except for sound effects (due to ear-safe issues)
* SFX, Music, and Fanfare randomization reenabled
* move OoT data files into the worlds folder
* move Compress and Decompress into oot data folder
* Replace get_all_state with custom method to avoid the cache
* OoT ROM: increment item counter before setting incoming item/player values to 0, preventing desync issues
* set data_version to 0
* make Kokiri Sword shuffle off by default
* reenable "Random Choice" for various cosmetic options
* kill Ruto's Letter turnin if open fountain
also fix for shopsanity
* place Buy Goron/Zora Tunic first in shop shuffle
* make ice traps appear as other items instead of breaking generation
* managed to break ice traps on non-major-only
* only handle ice traps if they are on
* fix shopsanity for non-oot games, and write player name instead of player number
* light arrows hint uses player name instead of player number
* Reenable "skip child zelda" option
* fix entrances_based_rules
* fix ganondorf hint if starting with light arrows
* fix dungeonitem shuffle and shopsanity interaction
* remove has_all_of, has_any_of, count_of in BaseClasses, replace usage with has_all, has_any, has_group
* force local giveable item on ZL if skip_child_zelda and shuffle_song_items is any
* keep bosses and bombchu bowling chus out of data package
* revert workaround for infinite recursion and fix it properly
* fix shared shop id caches during patching process
* fix shop text box overflows, as much as possible
* add default oot host.yaml option
* add .apz5, .n64, .z64 to gitignore
* Properly document and name all (functioning) OOT options
* clean up some imports
* remove unnecessary files from oot's data
* fix typo in gitignore
* readd the Compress and Decompress utilities, since they are needed for generation
* cleanup of imports and some minor optimizations
* increase shop offset for item IDs to 0xCB
* remove shop item AP ids entirely
* prevent triforce pieces for other players from being received by yourself
* add "excluded" property to Location
* Hint system adapted and reenabled; hints still unseeded
* make hints deterministic with lists instead of sets
* do not allow hints to point to Light Arrows on non-vanilla bridge
* foreign locations hint as their full name in OoT rather than their region
* checkedLocations now stores hint names by player ID, so that the same location in different worlds can have hints associated
* consolidate versioning in Utils
* ice traps appear as major items rather than any progression item
* set prescription and claim check as defaults for adult trade item settings
* add oot options to playerSettings
* allow case-insensitive logic tricks in yaml
* fix oot shopsanity option formatting
* Write OoT override info even if local item, enabling local checks to show up immediately in the client
* implement CollectionState.can_live_dmg for oot glitched logic
* filter item names for invalid characters when patching shops
* make ice traps appear according to the settings of the world they are shuffled into, rather than the original world
* set hidden-spoiler items and locations with Shop items to events
* make GF carpenters, Gerudo Card, Malon, ZL, and Impa events if the relevant settings are enabled, preventing them from appearing in the client on game start
* Fix oot Glitched and No Logic generation
* fix indenting
* Greatly reduce displayed cosmetic options
* Change oot data version to 1
* add apz5 distribution to webhost
* print player name if an ALttP dungeon contains a good item for OoT world
* delete unneeded commented code
* remove OcarinaSongs import to satisfy lint
Since rules have not been set for later worlds, the cache believes the completion condition is freely available if it had been placed previously, which breaks beatable-only key placement.
LttP: move glitch_boots to new options system
WebHost: options.yaml no longer lists aliases
General: remove region.can_fill, it was only used as a hack to make dungeon-specific items to work
General: Move generic IDs from LttP to new Generic World
Generate: ensure thread errors are collected before data from their completion may be referenced in playthrough/spoiler
AutoWorld: Now correctly automatically applies State.remove if collect_item is also correct
LttP: Make keys advancement items
This feels like it improved generation chance. Might not be the case.
Generator: Re-allow names with spaces (and see what breaks)
Generator: Removed teams (Note that teams are intended to move from a generation step feature to a server runtime feature, allowing dynamic creation of an already generated MW)
LttP: All Rom Options are now on the new system
LttP: palette option "random" is now called "good"
LttP: Roms are now created as part of the general output file creation step
LttP: disable Music is now Music, removing potential double negatives
LttP & Factorio: Progressive option random is now grouped_random
LttP: Enemy damage option random is now Enemy damage: chaos
* Implement excluded locations
* Update Minecraft to use exclusion_rules for its exclusion pools
* Flag the enchanted books as advancement so they don't go on excluded locations (particularly the Infinity book)
* update playerSettings for exclusion
* new items: 32 Arrows, Saddle, structure compasses for overworld structures
* move structure linking to create_regions instead of generate_basic
* Update Minecraft to use LogicMixin
* add separate can_exclude property, so non-progression items can be marked non-excluded
* separate fill step for nonadvancement nonexcluded items
* made Saddle not a progression item, but also nonexcluded
* fix missing player arg
* remove higher xp amounts from pool, leaving only 50 XP
* fix new Minecraft item IDs
* added shulker box item for starting inventory
* increment client and data version
* change client_version to int instead of tuple
* make saddle a progression item
* added structure compass option and appropriate logic for all compasses
* Update playerSettings.yaml with MC options
* update minecraft tests
* update exclusion procedure for clarity
Other changes:
host.yaml Multi Mystery options were moved and changed
generate_output now has an output_directory argument
MultiWorld.get_game_players(<game>) now replaces <game>_player_ids
Python venv should now work properly
(There's still a lot of work ahead, such as:
registering locations and items to the World, as well as methods to create_item_from_name()
many more method names for various stages
embedding Options into the world type
and many more...)
* Minecraft locations, items, and generation without logic
* added id lookup for minecraft
* typing import fix in minecraft/Items.py
* fix 2
* implementing Minecraft options and hard/postgame advancement exclusion
* first logic pass (75/80)
* logic pass 2 and proper completion conditions
* added insane difficulty pool, modified method of excluding item pools for easier extension
* bump network_data_package version
* minecraft testing framework
* switch Ancient Debris to Netherite Scrap to avoid advancement triggering on receiving that item
* Testing now functions, split tests up by advancement pane, added some story tests
* Newer testing framework: every advancement gets its own function, for ease of testing
* fixed logic for The End... Again...
* changed option names to "include_hard_advancements" etc.
* village/pillager-related advancements now require can_adventure: weapon + food
* a few minecraft tests
* rename "Flint & Steel" to "Flint and Steel" for parity with in-game name
* additional MC tests
* more tests, mostly nether-related tests
* more tests, removed anvil path for Two Birds One Arrow
* include Minecraft slot data, and a world seed for each Minecraft player slot
* Added new items: ender pearls, lapis, porkchops
* All remaining Minecraft tests
* formatting of Minecraft tests and logic for better readability
* require Wither kill for Monsters Hunted
* properly removed 8 Emeralds item from item pool
* enchanting required for wither; fishing rod required for water breathing; water breathing required for elder guardian kill
* Added 12 new advancements (ported from old achievement system)
* renamed "On a Rail" for consistency with modern advancements
* tests for the new advancements
* moved slot_data generation for minecraft into worlds/minecraft/__init__.py, added logic_version to slot_data
* output minecraft options in the spoiler log
* modified advancement goal values for new advancements
* make non-native Minecraft items appear as Shovel in ALttP, and unknown-game items as Power Stars
* fixed glowstone block logic for Not Quite Nine Lives
* setup for shuffling MC structures: building ER world and shuffling regions/entrances
* ensured Nether Fortresses can't be placed in the End
* finished logic for structure randomization
* fixed nonnative items always showing up as Hammers in ALttP shops
* output minecraft structure info in the spoiler
* generate .apmc file for communication with MC client
* fixed structure rando always using the same seed
* move stuff to worlds/minecraft/Regions.py
* make output apmc file have consistent name with other files
* added minecraft bottle macro; fixed tests imports
* generalizing MC region generation
* restructured structure shuffling in preparation for structure plando
* only output structure rando info in spoiler if they are shuffled
* Force structure rando to always be off, for the stable release
* added Minecraft options to player settings
* formally added combat_difficulty as an option
* Added Ender Dragon into playthrough, cleaned up goal map
* Added new difficulties: Easy, Normal, Hard combat
* moved .apmc generation time to prevent outputs on failed generation
* updated tests for new combat logic
* Fixed bug causing generation to fail; removed Nether Fortress event since it should no longer be needed with the fix
* moved all MC-specific functions into gen_minecraft
* renamed "logic_version" to "client_version"
* bug fixes
properly flagged event locations/items with id None
moved generation back to Main.py to fix mysterious generation failures
* moved link_minecraft_regions into minecraft init, left create_regions in Main for caching
* added seed_name, player_name, client_version to apmc file
* reenabled structure shuffle
* added entrance tests for minecraft
* Minecraft logic updates
Wither kill now considers nether fortresses as a valid source of soul sand
A Furious Cocktail now requires beacons for resistance and village access for carrots
Uneasy Alliance now requires fishing rod to pull the ghast through the portal
On a Rail now requires iron pickaxe to make powered rails
Overkill now may require strength II without stone axe, which needs nether access
* embed all apmc info into slot_data
* updated MC tests for logic changes
* put apmc into zipfile
Co-authored-by: achuang <alexander.w.chuang@gmail.com>
Squash merge, original Commits:
* Minecraft locations, items, and generation without logic
* added id lookup for minecraft
* typing import fix in minecraft/Items.py
* fix 2
* implementing Minecraft options and hard/postgame advancement exclusion
* first logic pass (75/80)
* logic pass 2 and proper completion conditions
* added insane difficulty pool, modified method of excluding item pools for easier extension
* bump network_data_package version
* minecraft testing framework
* switch Ancient Debris to Netherite Scrap to avoid advancement triggering on receiving that item
* Testing now functions, split tests up by advancement pane, added some story tests
* Newer testing framework: every advancement gets its own function, for ease of testing
* fixed logic for The End... Again...
* changed option names to "include_hard_advancements" etc.
* village/pillager-related advancements now require can_adventure: weapon + food
* a few minecraft tests
* rename "Flint & Steel" to "Flint and Steel" for parity with in-game name
* additional MC tests
* more tests, mostly nether-related tests
* more tests, removed anvil path for Two Birds One Arrow
* include Minecraft slot data, and a world seed for each Minecraft player slot
* Added new items: ender pearls, lapis, porkchops
* All remaining Minecraft tests
* formatting of Minecraft tests and logic for better readability
* require Wither kill for Monsters Hunted
* properly removed 8 Emeralds item from item pool
* enchanting required for wither; fishing rod required for water breathing; water breathing required for elder guardian kill
* Added 12 new advancements (ported from old achievement system)
* renamed "On a Rail" for consistency with modern advancements
* tests for the new advancements
* moved slot_data generation for minecraft into worlds/minecraft/__init__.py, added logic_version to slot_data
* output minecraft options in the spoiler log
* modified advancement goal values for new advancements
* make non-native Minecraft items appear as Shovel in ALttP, and unknown-game items as Power Stars
* fixed glowstone block logic for Not Quite Nine Lives
* setup for shuffling MC structures: building ER world and shuffling regions/entrances
* ensured Nether Fortresses can't be placed in the End
* finished logic for structure randomization
* fixed nonnative items always showing up as Hammers in ALttP shops
* output minecraft structure info in the spoiler
* generate .apmc file for communication with MC client
* fixed structure rando always using the same seed
* move stuff to worlds/minecraft/Regions.py
* make output apmc file have consistent name with other files
* added minecraft bottle macro; fixed tests imports
* generalizing MC region generation
* restructured structure shuffling in preparation for structure plando
* only output structure rando info in spoiler if they are shuffled
* Force structure rando to always be off, for the stable release
* added Minecraft options to player settings
* formally added combat_difficulty as an option
* Added Ender Dragon into playthrough, cleaned up goal map
* Added new difficulties: Easy, Normal, Hard combat
* moved .apmc generation time to prevent outputs on failed generation
* updated tests for new combat logic
* Fixed bug causing generation to fail; removed Nether Fortress event since it should no longer be needed with the fix
* moved all MC-specific functions into gen_minecraft
* renamed "logic_version" to "client_version"
* bug fixes
properly flagged event locations/items with id None
moved generation back to Main.py to fix mysterious generation failures
* moved link_minecraft_regions into minecraft init, left create_regions in Main for caching
* added seed_name, player_name, client_version to apmc file
* reenabled structure shuffle
* added entrance tests for minecraft
Co-authored-by: achuang <alexander.w.chuang@gmail.com>
In some cases multi-thousand element lists were created in-memory with near identical contents, per player, then discarded and reassembled.
Was testing against a case with 3 GB of additional memory use (50 players) which appeared to get stuck, but really was just very slow. This example case is fixed with these changes.
Additionally, progression balancing is now run after ShopSlotFill, so it is now "aware" of the changed progression shops can produce.
As well, special handling for keys was removed, as not all games will have the notion of keys.
Make sure that dark lake hylia shop in inverted retains the blue potion, while allowing shop slots on top (potion will always be the leftmost slot, ignoring "i"/"f"/"g")
Cull Shop swap Candidates before generating weights, then keep track of updated sphere sizes during swaps. This significantly reduces the chance to run out of candidates, as large clumps of false candidates do not get included in the weight
Shop fill is slower with this, as all candidates need to be swept once, instead of on-demand; but this seemed the best way to address the remaining issues.
Fabian Dill
Brad Humphrey
Alchav
Adam Ziegler
pepperpow
espeon65536
espeon65536
CaitSith2
Hussein Farran