This is the third of a series of savefile-related changes.
This adds early-days experimental support for a completely optional
'sfctool' utility (savefile conversion tool), to be able to export
a savefile's contents into a more portable format. There are likely
to be bugs at this stage. In this initial first-attempt, the export
format is a very simple ascii output.
NetHack can be built entirely, without also building this tool.
NetHack has no dependencies on the tool.
Attempts were made to minimize duplication of existing NetHack code.
To achieve that, unfortunately, #ifdef SFCTOOL and #ifndef SFCTOOL
had to be sprinkled around through some of the existing NetHack
source code, so that it could be re-used for building the utility.
The process for building the sfctool typically recompiles the source
files with #define SFCTOOL and a distinct object file with SF- is
produced.
sfctool notes:
Universal ctags is used and required to produce the sfctool utility.
Some targets were added to the Unix and Windows Makefiles to
facilitate the build process.
make sfctool
That should build a copy in util.
Note: At present, the Unix Makefiles do not copy sfctool over to the
NetHack playground during 'make install' or 'make update'.
Until that gets resolved by someone, The tool will
have to be manually copied there by the builder/admin if
desired.
cp util/sfctool ~/nh/install/games/lib/nethackdir/sfctool
Also, a separate Visual Studio sfctool.sln solution was written and
placed in sys/windows/vs. That has has only very limited testing.
Usage:
i) To convert an existing savefile to an exportascii format
that co-resides with the savefile:
sfctool -c savefile
That *must* be executed on the same platform / architecture /
data model that produced the save file in the first place.
ii) To unconvert an existing exportascii format export file to a
historical format savefile that can then be used by NetHack:
sfctool -u savefile
That must be executed on the same target platform / architecture /
data model that was used to build the NetHack that will
utilize the save file that results.
A Windows example:
sfctool -c Fred.NetHack-saved-game
That should result in creation of Fred.NetHack-saved-game.exportascii
from existing savefile:
%USERPROFILE%\AppData\Local\NetHack\3.7\Fred.NetHack-saved-game
A Unix example:
sfctool -c 1000wizard
That should result in creation of 1000wizard.exportascii.gz
from existing savefile in the playground save directory:
1000wizard.gz
Current Mechanics:
1. Makefile recipe, or script uses universal ctags to produce
util/sf.tags.
2. util/sftags is built and executed to read util/sf.tags and
generate: include/sfproto.h and src/sfdata.c.
3. util/sfctool is built from the following:
generated file compiled with -DSFCTOOL:
src/sfdata.c -> sfdata.o
existing files compiled with -DSFCTOOL:
util/sfctool.c -> sfctool.o
util/sfexpasc.c -> sfexpasc.o
src/alloc.c -> sf-alloc.o
src/monst.c -> sf-monst.o
src/objects.c -> sf-objects.o
src/sfbase.c -> sfbase.o
src/sfstruct.c -> sfstruct.o
src/nhlua.c -> sf-nhlua.o
util/panic.c -> panic.o
src/date.c -> sf-date.o
src/decl.c -> sf-decl.o
src/artifact.c -> sf-artifact.o
src/dungeon.c -> sf-dungeon.o
src/end.c -> sf-end.o
src/engrave.c -> sf-engrave.o
src/cfgfiles.c -> sf-cfgfiles.o
src/files.c -> sf-files.o
src/light.c -> sf-light.o
src/mdlib.c -> sf-mdlib.o
src/mkmaze.c -> sf-mkmaze.o
src/mkroom.c -> sf-mkroom.o
src/o_init.c -> sf-o_init.o
src/region.c -> sf-region.o
src/restore.c -> sf-restore.o
src/rumors.c -> sf-rumors.o
src/sys.c -> sf-sys.o
src/timeout.c -> sf-timeout.o
src/track.c -> sf-track.o
src/version.c -> sf-version.o
src/worm.c -> sf-worm.o
src/strutil.c -> strutil.o
This is the second of a series of changes related to save/restore.
No EDITLEVEL bump has been included, because although the code
is changed extensively by this, the content of the savefiles have
not been changed.
Push the use of the structlevel bwrite() and mread() function use
out of the core and into sfstruct.c. This is groundwork for upcoming
changes.
In the core, replace the bwrite() and mread() calls with the
use of type-specific savefile output (Sfo) and savefile
input (Sfi) macros. The macros are defined in a new header file
savefile.h, which also contains the prototypes for the sfo_* and
sfi_* functions that the macros ultimately expand to. The functions
themselves are in src/sfbase.c.
On C99, each Sfo or Sfi macro expansion refers directly to the
corresponding type-specific sfo_* or sfi_* function.
If C23 or later is is use, the majority (all but 3 types) of the
macros refer to a single _Generic output routine sfo(nhfp, dt, tag),
and a single _Generic input routine sfi(nhfp, dt, tag), which handles
the dispatch of the type-specific underlying functions. This was
somewhat experimental, but turned out to be practical because the
compiler would gripe if the type for a variable was not included in
the _Generic when passed as an argument, so it could be fixed.
This alters the savefile verication process by having a common set
return values for the related functions such as uptodate(),
check_version(), etc. The new return values return more information
about savefile incompatibilities, beyond failure/sucess. The
additional information will be useful for an upcoming addition.
The expanded return values are:
SF_UPTODATE (0) everything matched and looks good
SF_OUTDATED (1) savefile is outdated
SF_CRITICAL_BYTE_COUNT_MISMATCH (2) critical size count mismatch
SF_DM_IL32LLP64_ON_ILP32LL64 (3) Windows x64 savefile on x86
SF_DM_I32LP64_ON_ILP32LL64 (4) Unix 64 savefile on x86
SF_DM_ILP32LL64_ON_I32LP64 (5) x86 savefile on Unix 64
SF_DM_ILP32LL64_ON_IL32LLP64 (6) x86 savefile on Windows x64
SF_DM_I32LP64_ON_IL32LLP64 (7) Unix 64 savefile on Windows x64
SF_DM_IL32LLP64_ON_I32LP64 (8) Windows x64 savefile on Unix 64
SF_DM_MISMATCH (9) some other mismatch
The callers in the core have been adjusted to deal with the expanded
return values.
Other miscellaneous inclusions:
- go.oracle_loc -> svo.oracle_loc.
- add a bit (1UL << 30) to called SFCTOOL_BIT as groundwork
for changes to follow.
This is the first of several savefile-related changes to
follow later. This one is groundwork for those later changes.
Remove internal compression schemes (RLECOMP and ZEROCOMP)
and discard the savefile_info struct that was primarily used to
convey which internal compression schemes had been in use.
Relocate some struct definitions into appropriate header files
for use by code to come in later changes.
Remove the two struct size-related fields from version_info and
from the nmakedefs_s. Instead, include a series of bytes near the
beginning of the savefile, representing the size of each
struct or base data type that impacts the historical savefile
content. Those are referred to as the "critical bytes".
(Related note: the "you" struct required two bytes, low and high,
due to its size).
Compare those critical bytes in a savefile against the NetHack
build that is reading the savefile. This allows mismatch detection
early in the savefile-reading process, and a clean exit, rather than
proceeding to read nonsensical values from the file. Include some
feedback on what the first mismatch was when encountering
one.
For arrays stored in the savefile, use loop-logic in the core
to write/read the array elements one at a time, rather than in
a single blob. This will be required for changes to follow later.
(impacts artiexist[], artidisco[], svd.dungeons[], svl.level_info[],
svl.level.locations[][], msrooms[] field of mapseen, svb.bases[],
svb.disco[] objects[], svm.mvitals[], svs.spl_book[], svd.doors[],
go.oracle_loc[], utrack[], wgrowtime[])
This also adds data model to the long version information.
This invalidates existing save and bones files due to the changes in
the information at the start of the file.
new .h files: hacklib.h selvar.h stairs.h
new .c files: calendar.c, getpos.c, report.c, selvar.c, stairs.c,
strutil.c, wizcmds.c
cleanup of hacklib.c and mdlib.c
hacklib contains functions that do not have to link with the core
relocate wiz commands from cmd.c to wizcmds.c
relocate CRASHREPORT stuff to report.c
relocate getpos stuff from do_name.c to getpos.c
remove temporary struct definition from extern.h
cross-compile PRE-section split into cross-pre1.370 and cross-pre2.370
Windows sys/windows/Makefile.nmake and sys/windows/Makefile.mingw32 and
visual studio project file updates
Unix sys/unix/Makefile.src, sys/unix/Makefile.utl
populate selvar.c and selvar.h
build on MS-DOS (not cross-compile) Makefile updates
for sys/msdos/Makefile.GCC (untested)
vms updates for above (untested)
add CRASHREPORT for Windows
add ^P info to report (via DUMPLOG)
new options: crash_email, crash_name, crash_urlmax
new game command: #bugreport
new config option: CRASHREPORT_EXEC_NOSTDERR
new command line option: --bidshow
deleted helper scripts:
NetHackCrashReport.Javascript
nhcrashreport.lua
misc:
update CRASHREPORTURL (will need to be updated before release)
update bitrot in winchain
winchain for Windows
add missing synch_wait for NetHackW --showpaths
add PANICTRACE (and CRASHREPORT) in mdlib.c:build_opts
missing:
packaging (Windows needs the pdb file)
no testing with MSVC command line build
port status:
linux: working, but glibc's backtrace doesn't show static functions
Windows VS: working. pdb file is large - looking into options
MacOS: working
msdos: not supported
VMS: not supported
MSVC: planned, but not attempted
MSYS2: working, but libbacktrace not showing symbols (yet?)
There is a slight change in behavior here when collecting build-time
option information. If MAXOPT is too small, collecting will now save
MAXOPT lines and then skip anything that should come after. It used
to always keep the very last line, each time replacing whatever the
previous last line was, but would leak memory if that ever happened.
And after yesterday's "fix" it would try to keep that very last line
beyond the array bounds.
opttext[] was given 120 elements but the MAXOPT macro used to limit
populating it was only 40. There happen to be exactly 40 lines of
runtime info--at least for my config--and the last slot was going to
waste, so the final dupstr("") at the end of build_options() wasn't
being tracked and freed.
It's interesting the whatever other memory checker is being used
failed to find something caught by old heaputil. Maybe the configs
running whatever it is are generating smaller options text?
There's no need to process windowtype a second time after options
parsing. The sequence set windowtype, process options, set
windowtype again was intended to deal with the options setting it
to some other value, but there's a flag available to disable that
from happening.
author Ray Chason <ray.chason@protonmail.com> 1684372172 -0400
committer nhmall <nhmall@nethack.org> 1685414340 -0400
Add configuration to support Curses on WinGUI
and enable support for Unicode on Curses.
Allow NetHackW to select the Curses interface
Reorder drawing of extended command prompt
Curses on WinGUI needs this change. This may be a bug in PDCursesMod,
but it seems to be harmless to the other ports.
Avoid calling Curses after the windows are closed
Provide erase_char and kill_char for WinGUI Curses
Set Lua version to 5.4.6
Avoid #ifdef/#else/#endif inside expansion of Strcat() macro.
Also, change constructed
|soundlib_nosound, soundlib_macsound, and soundlib_qtsound, user sounds.
to be
|nosound, macsound, qtsound, and user sounds.
OPTIONS=soundlib:somelib doesn't--or won't, when finished being
implemented--include the "soundlib_" prefix in the user-visible
value, and placement of "and" vs "user sounds" was odd.
The code should probably be reorganized so that makedefs can put that
into dat/options or at least have it put a placeholder for the missing
paragraph.
In file included from makedefs.c:180:
./../src/mdlib.c:92:12: warning: unused function 'count_and_validate_soundlibopts' [-Wunused-function]
static int count_and_validate_soundlibopts(void);
^
1 warning generated.
Function definition is in the #ifndef MAKEDEFS_C section, so move the prototype there too.
Also includes support by paxed for polearm targeting using the
frame color.
Also renames USE_TILES to TILES_IN_GLYPHMAP which is a more
accurate description.
Not all window interfaces have full support for the color framing
of the background square yet.
MS-DOS needs further work (to bring it to both VESA and VGA, with
and without tiles.
Windows GUI is missing support.
X11 and Qt have been started, but may require further refinement.
When dist2() got changed to use coordxy parameters, a macro that uses
it in its definition was overlooked and it had (int) casts in it.
That caused a warning about possible data loss when the int
then got converted to coordxy for the dist2() call.
Give online2() coordxy parameters instead of int, like its bretheren.
Avoid a couple of implicit conversion warnings where ints were being assigned
to smaller uchar or ints being assigned to smaller short.
A couple of signed vs unsigned warnings on some rumor processing.
Avoid some signed vs unsigned warnings in mdlib/makedefs where a signed int
param eventually got used in an external call that took size_t.
Eliminate all of it by just having the outer NetHack routine also take
a size_t.
Lastly, insert some default C99 alternative time-related code
in mdlib/makedefs since asctime() and ctime() are being flagged as
deprecated in the upcoming C23 standard and will now start to trigger
warnings for anyone using a C23-compliant compiler.
Instead of using index() macro defined to strchr, use C99 strchr.
Instead of using rindex() macro defined to strrchr, use C99 strrchr.
If you want to try building on a platform that doesn't offer those
two functions, these are available:
define NOT_C99 /* to make some non-C99 code available */
define NEED_INDEX /* to define a macro for index() */
define NEED_RINDX /* to define a macro for rindex() */
Change the inner workings of the experimental TTY_PERM_INVENT.
Switch to delivering the content to tty for the experimental perm_invent
via the existing window port interface (start_menu(), add_menu(), end_menu).
This also adds a new window port interface call ctrl_nhwindow() for
delivering information to the window port, and/or obtaining specific
information from the window port. The information and requests can
be extended as required. To be documented later once the changes settle
down.
Due to the intrusive nature of these changes and the possibility of
some bugs in the new code, I'm going to leave TTY_PERM_INVENT commented
out in the repository for a day or two. Anyone wishing to test it out
can do so by uncommenting TTY_PERM_INVENT in config.h.
The nomakedefs stuff for cross-compiling support broke the code to
treat enabling or disabling some optional features as not breaking
save and bones file compatibility. It was relying on a macro whose
definition was local to mdlib.c rather than propagated among files.
makedefs still constructs date.h with a value indicating the ignored
features but the actual compatability check doesn't use that anymore.
Toggling SCORE_ON_BOTL shouldn't have caused existing files to be
rejected but they were.
A new feature, enabled by default to maximize testing, but one which can
be disabled by commenting it out in config.h
With this, some additional information is added to the glyphmap entries
in a new optional substructure called u with these fields:
ucolor RGB color for use with truecolor terminals/platforms.
A ucolor value of zero means "not set." The actual
rgb value of 0 has the 0x1000000 bit set.
u256coloridx 256 color index value for use with 256 color
terminals, the closest color match to ucolor.
utf8str Custom representation via utf-8 string (can be null).
There is a new symset included in the symbols file, called enhanced1.
Some initial code has been added to parse individual
OPTIONS=glyph:glyphid/R-G-B entries in the config file.
The glyphid can, in theory, either be an individual glyph (G_* glyphid)
for a single glyph, or it can be an existing symbol S_ value
(monster, object, or cmap symbol) to store the custom representation for
all the glyphs that match that symbol.
Examples:
OPTIONS=glyph:G_fountain/U+03A8/0-150-255
(Your platform/terminal font needs to be able to include/display the
character, of course.)
The NetHack core code does parsing and storing the customized
entries, and adding them to the glyphmap data structure.
Any window port can utilize the additional information in the glyphinfo
that is passed to them, once code is added to do so.
Also, consolidate some symbol-related code into symbols.c, and remove it from
files.c and options.c
MONITOR_HEAP+heaputil pointed out some unreleased memory. The livelog
stuff wasn't being freed. Not surpringly the data used for collecting
and formatting build-options that just got changed from strdup() to
dupstr() wasn't being freed. And a couple of date/version bits.
mdlib.c was avoiding alloc() and dupstr() because mdlib.o gets linked
with makedefs and makedefs used to need to avoid those. But makedefs
doesn't avoid those anymore, so mdlib.c doesn't need to either.
Replace a couple of other strdup() calls in other files too.
error 28 in line 4090 of "invent.c": redeclaration of var <adjust_ok> with new storage-class
error 28 in line 4100 of "invent.c": redeclaration of var <adjust_gold_ok> with new storage-class
error 28 in line 610 of "mdlib.c": redeclaration of var <count_and_validate_winopts> with new storage-class
error 28 in line 3846 of "options.c": redeclaration of var <pfxfn_cond_> with new storage-class
error 28 in line 3886 of "options.c": redeclaration of var <pfxfn_font> with new storage-class
error 28 in line 5307 of "options.c": redeclaration of var <determine_ambiguities> with new storage-class
error 28 in line 5343 of "options.c": redeclaration of var <length_without_val> with new storage-class
error 28 in line 6853 of "options.c": redeclaration of var <illegal_menu_cmd_key> with new storage-class
error 28 in line 7708 of "options.c": redeclaration of var <count_apes> with new storage-class
error 28 in line 2686 of "pickup.c": redeclaration of var <stash_ok> with new storage-class
error 28 in line 1008 of "read.c": redeclaration of var <can_center_cloud> with new storage-class
error 28 in line 31 of "rnd.c": redeclaration of var <whichrng> with new storage-class
If you want to declare a pointer which the address pointed to is constant,
you should declare it as like `static const char *const var = "...";`.
This commit supplies missing `const` and prevents some programming
error in the future.
For Unix, add internal vs external pager choice to #version.
Others always use internal pagination so don't need to see that.
Also remove obsolete "command completion" option. I don't know
when it became unconditional but that was long enough ago to be
absent from the git log and from second cvs log included in that.
And streamline the RNG seed stuff a bit.
This evolves and hopefully eases the game-build requirements by
removing game-compile dependencies on any header files generated
by the makedefs utility, including:
date.h dependency and its inclusion is removed and comparable functionality
is produced at runtime via new file src/date.c.
pm.h dependency and its inclusion is removed and comparable functionality is
produced by moving the monster definitions from monst.c into new header
file called monsters.h and altering them slightly. The former pm.h header
file #define PM_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
onames.h dependency and its inclusion is removed and comparable functionality
is produced by moving the object definitions from objects.c into new header
file called objects.h and altering them slightly. The former onames.h header
file #define values are now replaced with appropriate emitted enum entries
during the compiler preprocessing.
artilist.h has been slightly altered, and the former onames.h artifact-related
header file #define ART_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
makedefs can still produce date.h (makedefs -v), pm.h (makedefs -p), and
onames.h (makedefs -o) for reference purposes. They won't be used during
the compiler.
The other uses for makedefs remain. They are used to prepare external
file content that the game utilizes, not prerequisite code for the
compile:
makedefs -d (database)
makedefs -r (rumors)
makedefs -h (oracles)
makedefs -s (epitaphs, engravings, bogusmons)
date.c
Pull the code for date/time stamping from mdlib.c into date.c.
Set date.o to be dependent on source files, header files, and .o files
so that date.o is rebuilt from date.c when any of those changes, thus
ensuring an accurate date/time stamp. It also includes git sha
functionality formerly done by makedefs writing #define directives
into include/date.h. For unix it passes the git info on
the compile line for date.c (via sys/unix/hints/linux.2020, macOS.2020)
nethack --dumpenums (optional, but on by default)
Allow developer to obtain some internal enum values from NetHack
without having to resort to an external utility such as
makedefs.
Uncomment #define NODUMPENUMS in config.h to disable this.
The updates to sys/windows/Makefile.gcc have not been tested yet.
