This is the third of a series of savefile-related changes.
This adds early-days experimental support for a completely optional
'sfctool' utility (savefile conversion tool), to be able to export
a savefile's contents into a more portable format. There are likely
to be bugs at this stage. In this initial first-attempt, the export
format is a very simple ascii output.
NetHack can be built entirely, without also building this tool.
NetHack has no dependencies on the tool.
Attempts were made to minimize duplication of existing NetHack code.
To achieve that, unfortunately, #ifdef SFCTOOL and #ifndef SFCTOOL
had to be sprinkled around through some of the existing NetHack
source code, so that it could be re-used for building the utility.
The process for building the sfctool typically recompiles the source
files with #define SFCTOOL and a distinct object file with SF- is
produced.
sfctool notes:
Universal ctags is used and required to produce the sfctool utility.
Some targets were added to the Unix and Windows Makefiles to
facilitate the build process.
make sfctool
That should build a copy in util.
Note: At present, the Unix Makefiles do not copy sfctool over to the
NetHack playground during 'make install' or 'make update'.
Until that gets resolved by someone, The tool will
have to be manually copied there by the builder/admin if
desired.
cp util/sfctool ~/nh/install/games/lib/nethackdir/sfctool
Also, a separate Visual Studio sfctool.sln solution was written and
placed in sys/windows/vs. That has has only very limited testing.
Usage:
i) To convert an existing savefile to an exportascii format
that co-resides with the savefile:
sfctool -c savefile
That *must* be executed on the same platform / architecture /
data model that produced the save file in the first place.
ii) To unconvert an existing exportascii format export file to a
historical format savefile that can then be used by NetHack:
sfctool -u savefile
That must be executed on the same target platform / architecture /
data model that was used to build the NetHack that will
utilize the save file that results.
A Windows example:
sfctool -c Fred.NetHack-saved-game
That should result in creation of Fred.NetHack-saved-game.exportascii
from existing savefile:
%USERPROFILE%\AppData\Local\NetHack\3.7\Fred.NetHack-saved-game
A Unix example:
sfctool -c 1000wizard
That should result in creation of 1000wizard.exportascii.gz
from existing savefile in the playground save directory:
1000wizard.gz
Current Mechanics:
1. Makefile recipe, or script uses universal ctags to produce
util/sf.tags.
2. util/sftags is built and executed to read util/sf.tags and
generate: include/sfproto.h and src/sfdata.c.
3. util/sfctool is built from the following:
generated file compiled with -DSFCTOOL:
src/sfdata.c -> sfdata.o
existing files compiled with -DSFCTOOL:
util/sfctool.c -> sfctool.o
util/sfexpasc.c -> sfexpasc.o
src/alloc.c -> sf-alloc.o
src/monst.c -> sf-monst.o
src/objects.c -> sf-objects.o
src/sfbase.c -> sfbase.o
src/sfstruct.c -> sfstruct.o
src/nhlua.c -> sf-nhlua.o
util/panic.c -> panic.o
src/date.c -> sf-date.o
src/decl.c -> sf-decl.o
src/artifact.c -> sf-artifact.o
src/dungeon.c -> sf-dungeon.o
src/end.c -> sf-end.o
src/engrave.c -> sf-engrave.o
src/cfgfiles.c -> sf-cfgfiles.o
src/files.c -> sf-files.o
src/light.c -> sf-light.o
src/mdlib.c -> sf-mdlib.o
src/mkmaze.c -> sf-mkmaze.o
src/mkroom.c -> sf-mkroom.o
src/o_init.c -> sf-o_init.o
src/region.c -> sf-region.o
src/restore.c -> sf-restore.o
src/rumors.c -> sf-rumors.o
src/sys.c -> sf-sys.o
src/timeout.c -> sf-timeout.o
src/track.c -> sf-track.o
src/version.c -> sf-version.o
src/worm.c -> sf-worm.o
src/strutil.c -> strutil.o
This is the second of a series of changes related to save/restore.
No EDITLEVEL bump has been included, because although the code
is changed extensively by this, the content of the savefiles have
not been changed.
Push the use of the structlevel bwrite() and mread() function use
out of the core and into sfstruct.c. This is groundwork for upcoming
changes.
In the core, replace the bwrite() and mread() calls with the
use of type-specific savefile output (Sfo) and savefile
input (Sfi) macros. The macros are defined in a new header file
savefile.h, which also contains the prototypes for the sfo_* and
sfi_* functions that the macros ultimately expand to. The functions
themselves are in src/sfbase.c.
On C99, each Sfo or Sfi macro expansion refers directly to the
corresponding type-specific sfo_* or sfi_* function.
If C23 or later is is use, the majority (all but 3 types) of the
macros refer to a single _Generic output routine sfo(nhfp, dt, tag),
and a single _Generic input routine sfi(nhfp, dt, tag), which handles
the dispatch of the type-specific underlying functions. This was
somewhat experimental, but turned out to be practical because the
compiler would gripe if the type for a variable was not included in
the _Generic when passed as an argument, so it could be fixed.
This alters the savefile verication process by having a common set
return values for the related functions such as uptodate(),
check_version(), etc. The new return values return more information
about savefile incompatibilities, beyond failure/sucess. The
additional information will be useful for an upcoming addition.
The expanded return values are:
SF_UPTODATE (0) everything matched and looks good
SF_OUTDATED (1) savefile is outdated
SF_CRITICAL_BYTE_COUNT_MISMATCH (2) critical size count mismatch
SF_DM_IL32LLP64_ON_ILP32LL64 (3) Windows x64 savefile on x86
SF_DM_I32LP64_ON_ILP32LL64 (4) Unix 64 savefile on x86
SF_DM_ILP32LL64_ON_I32LP64 (5) x86 savefile on Unix 64
SF_DM_ILP32LL64_ON_IL32LLP64 (6) x86 savefile on Windows x64
SF_DM_I32LP64_ON_IL32LLP64 (7) Unix 64 savefile on Windows x64
SF_DM_IL32LLP64_ON_I32LP64 (8) Windows x64 savefile on Unix 64
SF_DM_MISMATCH (9) some other mismatch
The callers in the core have been adjusted to deal with the expanded
return values.
Other miscellaneous inclusions:
- go.oracle_loc -> svo.oracle_loc.
- add a bit (1UL << 30) to called SFCTOOL_BIT as groundwork
for changes to follow.
add CRASHREPORT for Windows
add ^P info to report (via DUMPLOG)
new options: crash_email, crash_name, crash_urlmax
new game command: #bugreport
new config option: CRASHREPORT_EXEC_NOSTDERR
new command line option: --bidshow
deleted helper scripts:
NetHackCrashReport.Javascript
nhcrashreport.lua
misc:
update CRASHREPORTURL (will need to be updated before release)
update bitrot in winchain
winchain for Windows
add missing synch_wait for NetHackW --showpaths
add PANICTRACE (and CRASHREPORT) in mdlib.c:build_opts
missing:
packaging (Windows needs the pdb file)
no testing with MSVC command line build
port status:
linux: working, but glibc's backtrace doesn't show static functions
Windows VS: working. pdb file is large - looking into options
MacOS: working
msdos: not supported
VMS: not supported
MSVC: planned, but not attempted
MSYS2: working, but libbacktrace not showing symbols (yet?)
- Move secondary preprocessor defines down further in config.h
so that they can be overridden via [platform]conf.h which is
included from global.h, specifically:
LIVELOGFILE when LIVELOG is defined
DUMPLOG_FILE when DUMPLOG is defined
- Minimize platform-specific, or compiler-specific code in hack.h and decl.h.
- reorganize src/decl.c to align with include/decl.h.
- a new header file cstd.h added, containing calls to C99
standard header files.
- hack.h, decl.h, and decl.c have been cleaned up and had code
moved so that things line up as follows:
hack.h defines values that are available to all
NetHack source files, contains enums for use in all
NetHack source files, and contains a number of
struct definitions for use in all NetHack source files.
It does not contain variable declarations or variable
definitions.
decl.h contains the extern declarations for variables that
are defined in decl.c. These variables are global and
available to all NetHack source files. The location of
the variables within decl.h was random, so give it some
order for now.
decl.c contains the definition of the variables declared in
decl.h, and initializes them where appropriate. The
variable definitions are laid out in much the
same order as their declarations in decl.h.
- wintty.h: There were some varying terminal-related prototypes in
system.h, and that was the only thing left that demanded that
system.h be included. Those have been replaced by an #include
<term.h> in include/wintty.h to get the more current (and hopefully
more correct) prototypes, rather than hardcoding them in NetHack
sources.
For edge-case platform compatiblity, there is no #include <term.h>
if the build defines NO_TERMCAP_HEADERS. In that case one set of
hardcoded prototypes is still used in include/wintty.h.
The added #include "term.h" is also bypassed for NO_TERMS builds (builds
that don't link to terminfo/termcap at all, but still present a tty
interface using platform or window-port specific functions to fulfill
the same role as that of terminfo/termcap).
- some scattered, unnecessary #include "integer.h" were removed from
various files, since that's always included in current NetHack-3.7
sources, either directly from config.h or indirectly from #include
"hack.h".
- system.h references removed.
- new cstd.h added; the #include "system.h" references in Makefiles
and project files (Xcode, visual studio), were replaced
with #include "cstd.h" references. A "make depends" is probably
warranted.
Also:
- Use of <term.h>, which defines clear_screen() as a macro, conflicts
with an actual function with that name in win/tty/termcap.c. The most
straight-forward course of action was to rename the NetHack function,
and change the references to it, from clear_screen() to
term_clear_screen(), so that was done.
remove a couple of extraneous files from visual studio project
remove several trailing whitespace at end of line
remove an address sanitizer bounds error during player dialog init
rework fix for darkened lit tiles (fix issue #997)
Having a hardcoded relative path worked okay with the nmake Makefile
build but cause problems on a visual studio build. The CI caught that
right away.
Remove the relative paths, and add additional include directories for
the resouce compiler to search in the Makefile and in visual studio.
This evolves and hopefully eases the game-build requirements by
removing game-compile dependencies on any header files generated
by the makedefs utility, including:
date.h dependency and its inclusion is removed and comparable functionality
is produced at runtime via new file src/date.c.
pm.h dependency and its inclusion is removed and comparable functionality is
produced by moving the monster definitions from monst.c into new header
file called monsters.h and altering them slightly. The former pm.h header
file #define PM_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
onames.h dependency and its inclusion is removed and comparable functionality
is produced by moving the object definitions from objects.c into new header
file called objects.h and altering them slightly. The former onames.h header
file #define values are now replaced with appropriate emitted enum entries
during the compiler preprocessing.
artilist.h has been slightly altered, and the former onames.h artifact-related
header file #define ART_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
makedefs can still produce date.h (makedefs -v), pm.h (makedefs -p), and
onames.h (makedefs -o) for reference purposes. They won't be used during
the compiler.
The other uses for makedefs remain. They are used to prepare external
file content that the game utilizes, not prerequisite code for the
compile:
makedefs -d (database)
makedefs -r (rumors)
makedefs -h (oracles)
makedefs -s (epitaphs, engravings, bogusmons)
date.c
Pull the code for date/time stamping from mdlib.c into date.c.
Set date.o to be dependent on source files, header files, and .o files
so that date.o is rebuilt from date.c when any of those changes, thus
ensuring an accurate date/time stamp. It also includes git sha
functionality formerly done by makedefs writing #define directives
into include/date.h. For unix it passes the git info on
the compile line for date.c (via sys/unix/hints/linux.2020, macOS.2020)
nethack --dumpenums (optional, but on by default)
Allow developer to obtain some internal enum values from NetHack
without having to resort to an external utility such as
makedefs.
Uncomment #define NODUMPENUMS in config.h to disable this.
The updates to sys/windows/Makefile.gcc have not been tested yet.
There were multiple symbol-related lists that had to be kept
in sync in various places.
Consolidate some of that into a single new file
defsym.h
with a set of morphing macros that can be custom-called from
the various places that use the sym info without maintaining
multiple occurrences. Most maintenance can be done there.
Rename monsym.h to sym.h since it looks after some
symbols not related to monsters now too.
The defsym.h header file is included in multiple places to
produce different code depending on its use and the controlling
macro definitions in place prior to including it.
Its purpose is to have a definitive source for
pchar, objclass and mon symbol maintenance.
The controlling macros used to morph the resulting code are
used in these places:
- in include/sym.h for enums of some S_ symbol values
(define PCHAR_ENUM, MONSYMS_ENUM prior to #include defsym.h)
- in include/objclass.h for enums of some S_ symbol values
(define OBJCLASS_ENUM prior to #include defsym.h)
- in src/symbols.c for parsing S_ entries in config files
(define PCHAR_PARSE, MONSYMS_PARSE, OBJCLASS_PARSE prior
to #include defsym.h)
- in src/drawing.c for initializing some data structures/arrays
(define PCHAR_DRAWING, MONSYMS_DRAWING, OBJCLASS_DRAWING prior
to #include defsym.h)
- in win/share/tilemap.c for processing a tile file
(define PCHAR_TILES prior to #include defsym.h).
rename sys/winnt to sys/windows
move vs (visual studio) folder out of win/win32 and into sys/windows
rename include/ntconf.h to include/windconf.h
rename winnt.c to windsys.c
place visual studio projects into individual subfolders.
This will hopefully resolve GitHub issue #484 as well.
