This is the first of several savefile-related changes to
follow later. This one is groundwork for those later changes.
Remove internal compression schemes (RLECOMP and ZEROCOMP)
and discard the savefile_info struct that was primarily used to
convey which internal compression schemes had been in use.
Relocate some struct definitions into appropriate header files
for use by code to come in later changes.
Remove the two struct size-related fields from version_info and
from the nmakedefs_s. Instead, include a series of bytes near the
beginning of the savefile, representing the size of each
struct or base data type that impacts the historical savefile
content. Those are referred to as the "critical bytes".
(Related note: the "you" struct required two bytes, low and high,
due to its size).
Compare those critical bytes in a savefile against the NetHack
build that is reading the savefile. This allows mismatch detection
early in the savefile-reading process, and a clean exit, rather than
proceeding to read nonsensical values from the file. Include some
feedback on what the first mismatch was when encountering
one.
For arrays stored in the savefile, use loop-logic in the core
to write/read the array elements one at a time, rather than in
a single blob. This will be required for changes to follow later.
(impacts artiexist[], artidisco[], svd.dungeons[], svl.level_info[],
svl.level.locations[][], msrooms[] field of mapseen, svb.bases[],
svb.disco[] objects[], svm.mvitals[], svs.spl_book[], svd.doors[],
go.oracle_loc[], utrack[], wgrowtime[])
This also adds data model to the long version information.
This invalidates existing save and bones files due to the changes in
the information at the start of the file.
gcc has recognized various "magic comments" for white-listing
occurrences of implicit fallthrough in switch statements for
a long time:
The range and shape of "falls through" comments accepted are
contingent upon the level of the warning. (The default level is =3.)
-Wimplicit-fallthrough=0 disables the warning altogether.
-Wimplicit-fallthrough=1 treats any kind of comment as a "falls through" comment.
-Wimplicit-fallthrough=2 essentially accepts any comment that contains something
that matches (case insensitively) "falls?[ \t-]*thr(ough|u)" regular expression.
-Wimplicit-fallthrough=3 case sensitively matches a wide range of regular
expressions, listed in the GCC manual. E.g., all of these are accepted:
/* Falls through. */
/* fall-thru */
/* Else falls through. */
/* FALLTHRU */
/* ... falls through ... */
etc.
-Wimplicit-fallthrough=4 also, case sensitively matches a range of regular
expressions but is much more strict than level =3.
-Wimplicit-fallthrough=5 doesn't recognize any comments.
Plenty of other compilers did not recognize the gcc comment convention,
and up until now the compiler warning for detecting unintended
fallthrough had to be suppressed on other compilers. That's because the code
in NetHack has been relying on the gcc approach, and only the gcc approach.
The C23 standard introduces an attribute [[fallthrough]] for the
functionality, when implicit fallthrough warnings have been enabled.
Several popular compilers already support that, or a very similar attribute
style approach, today, even ahead of their C23 support:
C compiler whitelist approach
--------------------------- -------------------------------------
C23 conforming compilers [[fallthrough]]
clang versions supporting
standards prior to
C23 __attribute__((__fallthrough__))
Microsoft Visual Studio
since VS 2022 17.4.
The warning C5262 controls
whether the implict
fallthrough is detected and
warned about with
/std:clatest. [[fallthrough]]
This adds support to NetHack for the attribute approach by inserting a
macro FALLTHROUGH to the existing cases that require white-listing, so
other compilers can analyze things too.
The definition of the FALLTHROUGH macro is controlled in include/tradstdc.h.
The gcc comment approach has also been left in place at this time.
This may be useful for some build environments to avoid parallel make
issues, and artificially-concocted order dependencies, leaving the ordering
up to that specified in the Makefile.
The related makedefs options are now:
‐s Generate the bogusmon , engrave and epitaph files.
‐1 Generate the epitaph file.
‐2 Generate the engrave file.
‐3 Generate the bogusmon file.
Also resolves an existing issue encountered in doc/makedefs.6 where "and epitaphfiles"
was being produced in the result.
new .h files: hacklib.h selvar.h stairs.h
new .c files: calendar.c, getpos.c, report.c, selvar.c, stairs.c,
strutil.c, wizcmds.c
cleanup of hacklib.c and mdlib.c
hacklib contains functions that do not have to link with the core
relocate wiz commands from cmd.c to wizcmds.c
relocate CRASHREPORT stuff to report.c
relocate getpos stuff from do_name.c to getpos.c
remove temporary struct definition from extern.h
cross-compile PRE-section split into cross-pre1.370 and cross-pre2.370
Windows sys/windows/Makefile.nmake and sys/windows/Makefile.mingw32 and
visual studio project file updates
Unix sys/unix/Makefile.src, sys/unix/Makefile.utl
populate selvar.c and selvar.h
build on MS-DOS (not cross-compile) Makefile updates
for sys/msdos/Makefile.GCC (untested)
vms updates for above (untested)
If MAKEDEFS_FILTER_NONASCII is defined (which config.h now does by
default), it will check data.base, rumors.*, and {various}.txt for
characters outside the range of ' ' through '~'. If it finds any, it
will warn about them and change them to '#'.
Tab handling is incomplete; the files that use tabs for indentation
will allow tabs anywhere, even though that's not wanted. That could
be fixed but doesn't seem particularly urgent. This is more about
spotting and repairing the special 3-char punctuation characters that
crept into data.base fairly recently.
This was a part of an earlier change that was reverted.
Get all the obsolete parts of makedefs arranged so that
they can be disabled simply by not defining
OLD_MAKEDEFS_OPTIONS.
Right now OLD_MAKEDEFS_OPTIONS is defined.
When calling panic() or impossible(), create the option
of opening a browser window with most of the fields
already populated. Code for MacOS and linux is included;
other ports are affected by argument change to early_init
which are done but not tested.
To enable, define CRASHREPORT in config.h and set
CRASHREPORTURL in sysconf to (for the moment at least)
http[s]://www.nethack.org/common/contactcr.html
Adds --grep-defined option to makedefs for Makefiles.
Adds "bid" (binary identifier), an MD4 of the main nethack
binary. This is ONLY for helping (in the future) contact.html
to set the "NetHack from" field automatically for our own
binaries. This can be faked, but the user can lie so nothing
lost. There's nothing magic about MD4; other ports can use
anything that prodcues a long apparently random string we can
match against.
- new option --bidshow for us to get the MD4 of a
released binary so I can add it to the website.
Only available in wizard mode and not in nethack.6.
- typo macos -> macosx in hints file
No support for packaging builds as I'm not sure what that
would look like.
Adds a javascript helper for MacOS.
Adds a lua helper for linux (and builds and installs
nhlua).
A recent commit to alloc.c by Keni drew attention to the fact that
there are extern prototypes scattered around in various .c files.
Those can make use of ATTRNORETURN (non-gcc compilers and C23) the
same way the prototypes in extern.h can, and they were overlooked
when ATTRNORETURN was first added.
The unlink call wasn't operating the same on a makedefs built
on mingw, and dat/mdXXXX files were being left behind post-build.
Provide an alternative way of doing the temporary files if
MD_USE_TMPFILE_S is defined during the compile of makedefs.c
When dist2() got changed to use coordxy parameters, a macro that uses
it in its definition was overlooked and it had (int) casts in it.
That caused a warning about possible data loss when the int
then got converted to coordxy for the dist2() call.
Give online2() coordxy parameters instead of int, like its bretheren.
Avoid a couple of implicit conversion warnings where ints were being assigned
to smaller uchar or ints being assigned to smaller short.
A couple of signed vs unsigned warnings on some rumor processing.
Avoid some signed vs unsigned warnings in mdlib/makedefs where a signed int
param eventually got used in an external call that took size_t.
Eliminate all of it by just having the outer NetHack routine also take
a size_t.
Lastly, insert some default C99 alternative time-related code
in mdlib/makedefs since asctime() and ctime() are being flagged as
deprecated in the upcoming C23 standard and will now start to trigger
warnings for anyone using a C23-compliant compiler.
Instead of using index() macro defined to strchr, use C99 strchr.
Instead of using rindex() macro defined to strrchr, use C99 strrchr.
If you want to try building on a platform that doesn't offer those
two functions, these are available:
define NOT_C99 /* to make some non-C99 code available */
define NEED_INDEX /* to define a macro for index() */
define NEED_RINDX /* to define a macro for rindex() */
The nomakedefs stuff for cross-compiling support broke the code to
treat enabling or disabling some optional features as not breaking
save and bones file compatibility. It was relying on a macro whose
definition was local to mdlib.c rather than propagated among files.
makedefs still constructs date.h with a value indicating the ignored
features but the actual compatability check doesn't use that anymore.
Toggling SCORE_ON_BOTL shouldn't have caused existing files to be
rejected but they were.
Add new routine 're_alloc()' that functions as MONITOR_HEAP-aware
libc realloc(). 'nhrealloc()' is the version that passes source
file and line info if built with MONITOR_HEAP enabled. The heaplog
data might now contain '<' (freed by realloc), '>' (replacement
allocation by realloc), and '*' (resized by realloc) entries in
addition to the previous '+' (allocated) and '-' (freed) entries.
heaputil has already been updated in the NHinternal repository.
Move FITSint_() and FITSuint_() from hacklib.c to alloc.c so that
they can be accessed by miscellaneous utility programs.
Remove three or four copies of FITSint_() that were duplicated in
utility programs like dlb and tile2bmp due to those not having
access to src/hacklib.o. They do have access to src/alloc.o (and
util/panic.o).
Have makedefs do through a common point when exiting, in case it ever
needs extra memory or scratch file cleanup.
While testing, I discovered that the reference use of 'makedefs -o' to
build obsolete onames.h didn't work anymore because of the change to
not require object probabilities to add up to 1000 within classes.
I think fixing that is the only change besides new 'makedefs_exit()'.
This started out as fix for a comment but ended up redoing 'limit()'
and its 'temp[]' because they seemed likely to someday interfere with
other uses of such generic names.
No change to observable behavior.
Avoids any need for MONITOR_HEAP hackery. Link src/alloc.o and
util/panic.o into util/makedefs.
When replacing the realloc() call in fgetline(), I noticed that
fgetline() would miss the last line of an input file if it lacked a
terminating newline. This was hard to test because OSX seems to be
supplying one when it is missing (VMS would do that too). I had to
modify epitaph (my test bed) to take off the final character, run
'makedefs -s' under debugger control and strip away final newline
that stdio added back, build new nhdat and move it into place, then
run nethack and execute #wizrumorcheck all multiple times before
the fix and once more after it. Much effort for little gain...
Plus some of the recent reformatting: indent labels one space,
replace tabs with spaces, shorten or split wide lines.
I tried building with MONITOR_HEAP defined for the first time in a
while. It wasn't pretty.
date.c was calling libc's strdup() instead of our dupstr() so alloc.c
wasn't tracking those allocations when/if MONITOR_HEAP is enabled.
Then it called free() which is actually a call to nhfree() in that
situation. If the file of allocations and releases was subsequently
fed to heaputil, it would complain about freeing pointers that hadn't
been allocated.
Worse, makedefs and tilemap wouldn't link. For MONITOR_HEAP,
makedefs was undefining free() in order to avoid nhfree() but it now
links with date.o so got nhfree() calls anyway. And it wouldn't link
because that routine isn't available without alloc.o. tilemap
doesn't link with date.o but it does call malloc() and free() and it
wasn't undefining free(), so looked for nhfree() when linking and got
the same no-such-routine failure.
Simplify a glob handling bit in a recent shrink_glob change used when
catching for lost time upon returning to a level.
Revise a clumsily worded fixes entry.
Fix a comment typo in makedefs that's been there for a bunch of years
now. It's been within the diff context for several recent patches
and I still hadn't noticed it until just now.
Solve the uneven distribution situation that has been present for
picking random rumors for a long time and for random engravings,
epitaphs, and hallucinatory monster names since 3.6.0. This relies
on the previous partial solution where short lines have been padded
to a longer length. When that length is N and random seek lands in
a long line of length L, retry if the position is in the first L-N
characters. Put differently, it if takes more than N characters to
reach the next newline, reject that random seek and try again. This
effectively makes long lines behave as if they had the same length
of N as the short lines have been padded to and when all lines are
the same length, all entries have the same chance to be chosen.
While testing 'the("Capitalized Hallucinatory Monster")' I noticed that
some hallucinatory monsters showed up more often than others. When
the random engravings, epitaphs, and bogus monsters were converted from
hard-coded arrays to data files accessed by random seek (3.6.0), they
became subject to the same distribution irregularites that rumors suffer
from. The chance that an entry will be chosen depends upon the chance
that a random seek will hit somewhere in the line which precedes it, so
entries that follow long lines are more likely to be chosen and entries
that follow short lines are less likely. We improved that for rumors
by having makedefs pad the shortest lines. Distribution still isn't
uniform but is much better than it was (and could be further improved
with a longer padding length at the cost of making data files bigger so
possibly slower to access; both overall size and access speed mattered
back when floppy discs were supported but are probably irrelevant now).
Start doing the same thing for the newer files: pad the shortest lines
to increase the chance that seek will find them. The tradeoff is that
the data files become bigger. Rumors, engravings, and epitaphs lines
are all at least 60 characters now; bogus monsters are at least 20.
These are the data file sizes I see (in bytes: old, new; padding for
rumors was already in use so its size hasn't changed):
bogusmon 4449 7211
engrave 1326 2894
epitaph 14159 24075
rumors 49173 49173
The only place that padding is noticeable in-game is #wizrumorcheck.
makedefs.c:1560:5: warning: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Wstring-concatenation]
" ",
^
makedefs.c:1559:5: note: place parentheses around the string literal to silence warning
"currently available will not suffice for proving that P != NP or "
^
1 warning generated.
This evolves and hopefully eases the game-build requirements by
removing game-compile dependencies on any header files generated
by the makedefs utility, including:
date.h dependency and its inclusion is removed and comparable functionality
is produced at runtime via new file src/date.c.
pm.h dependency and its inclusion is removed and comparable functionality is
produced by moving the monster definitions from monst.c into new header
file called monsters.h and altering them slightly. The former pm.h header
file #define PM_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
onames.h dependency and its inclusion is removed and comparable functionality
is produced by moving the object definitions from objects.c into new header
file called objects.h and altering them slightly. The former onames.h header
file #define values are now replaced with appropriate emitted enum entries
during the compiler preprocessing.
artilist.h has been slightly altered, and the former onames.h artifact-related
header file #define ART_ values are now replaced with appropriate emitted enum
entries during the compiler preprocessing.
makedefs can still produce date.h (makedefs -v), pm.h (makedefs -p), and
onames.h (makedefs -o) for reference purposes. They won't be used during
the compiler.
The other uses for makedefs remain. They are used to prepare external
file content that the game utilizes, not prerequisite code for the
compile:
makedefs -d (database)
makedefs -r (rumors)
makedefs -h (oracles)
makedefs -s (epitaphs, engravings, bogusmons)
date.c
Pull the code for date/time stamping from mdlib.c into date.c.
Set date.o to be dependent on source files, header files, and .o files
so that date.o is rebuilt from date.c when any of those changes, thus
ensuring an accurate date/time stamp. It also includes git sha
functionality formerly done by makedefs writing #define directives
into include/date.h. For unix it passes the git info on
the compile line for date.c (via sys/unix/hints/linux.2020, macOS.2020)
nethack --dumpenums (optional, but on by default)
Allow developer to obtain some internal enum values from NetHack
without having to resort to an external utility such as
makedefs.
Uncomment #define NODUMPENUMS in config.h to disable this.
The updates to sys/windows/Makefile.gcc have not been tested yet.
There were multiple symbol-related lists that had to be kept
in sync in various places.
Consolidate some of that into a single new file
defsym.h
with a set of morphing macros that can be custom-called from
the various places that use the sym info without maintaining
multiple occurrences. Most maintenance can be done there.
Rename monsym.h to sym.h since it looks after some
symbols not related to monsters now too.
The defsym.h header file is included in multiple places to
produce different code depending on its use and the controlling
macro definitions in place prior to including it.
Its purpose is to have a definitive source for
pchar, objclass and mon symbol maintenance.
The controlling macros used to morph the resulting code are
used in these places:
- in include/sym.h for enums of some S_ symbol values
(define PCHAR_ENUM, MONSYMS_ENUM prior to #include defsym.h)
- in include/objclass.h for enums of some S_ symbol values
(define OBJCLASS_ENUM prior to #include defsym.h)
- in src/symbols.c for parsing S_ entries in config files
(define PCHAR_PARSE, MONSYMS_PARSE, OBJCLASS_PARSE prior
to #include defsym.h)
- in src/drawing.c for initializing some data structures/arrays
(define PCHAR_DRAWING, MONSYMS_DRAWING, OBJCLASS_DRAWING prior
to #include defsym.h)
- in win/share/tilemap.c for processing a tile file
(define PCHAR_TILES prior to #include defsym.h).
Whitelist all the verified existing triggers:
makedefs.c: In function ‘name_file’
attrib.c: one compiler balks at a ? b : c for fmtstring
cmd.c: In function ‘extcmd_via_menu’
cmd.c: In function ‘wiz_levltyp_legend’
do.c: In function ‘goto_level’
do_name.c: In function ‘coord_desc’
dungeon.c: In function ‘overview_stats’
eat.c: one compiler balks at a ? b : c for fmtstring
end.c: one compiler balks at a ? b : c for fmtstring
engrave.c: In function ‘engr_stats’
hack:c one compiler balks at a ? b : c for fmtstring
hacklib.c: one compiler balks at a ? b : c for fmtstring
insight.c: one compiler balks at a ? b : c for fmtstring
invent.c: In function ‘let_to_name’
light.c: In function ‘light_stats’
mhitm.c: In function ‘missmm’
options.c: In function ‘handler_symset’
options.c: In function ‘basic_menu_colors’
options.c: In function ‘optfn_o_autopickup_exceptions’
options.c: In function ‘optfn_o_menu_colors’
options.c: In function ‘optfn_o_message_types’
options.c: In function ‘optfn_o_status_cond’
options.c: In function ‘optfn_o_status_hilites’
options.c: In function ‘doset’
options.c: In function ‘doset_add_menu’
options.c: In function ‘show_menu_controls’
options.c: In function ‘handle_add_list_remove’
pager.c: In function ‘do_supplemental_info’
pager.c: In function ‘dohelp’
region.c: In function ‘region_stats’
rumors.c: sscanf usage
sounds.c: In function ‘domonnoise’
spell.c: In function ‘dospellmenu’
timeout.c: In function ‘timer_stats’
topten.c: In function ‘outentry’, fscanf, sscanf, fprintf usage
windows.c: In function ‘genl_status_update’
zap.c: one compiler balks at a ? b : c for fmtstring
win/curses/cursstat.c: In function ‘curses_status_update’
win/tty/wintty.c: In function ‘tty_status_update’
win/win32/mswproc.c: In function ‘mswin_status_update’
Microsoft and other non-GNU compilers don't recognize gcc tricks
like /*NOTREACHED*/ to suppress individual warnings. clang recognizes most
of them because it tries to be gcc-compatible. Because of that, a lot of
potentially useful warnings have had to be completely suppressed in the
past in all source files when using the non-gcc compatible compilers.
Now that the code is C99, take advantage of a way to suppress warnings for
individual functions, a big step up from suppressing the warnings
altogether.
Unfortunately, it does require a bit of ugliness caused by the
insertion of some macros in a few spots, but I'm not aware of
a cleaner alternative that still allows warnings to be enabled
in general, while suppressing a warning for known white-listed
instances.
Prior to the warning-tiggering function, place whichever one of
the following is needed to suppress the warning being encountered:
DISABLE_WARNING_UNREACHABLE_CODE
DISABLE_WARNING_CONDEXPR_IS_CONSTANT
After the warning-triggering function, place this:
RESTORE_WARNINGS
Under the hood, the compiler-appropriate warning-disabling
mechanics involve the use of C99 _Pragma, which can be used
in macros.
For unrecognized or inappropriate compilers, or if
DISABLE_WARNING_PRAGMAS is defined, the macros expand
to nothing.
Update makdefs source and its man page.
Remove all mentions of the vision table files from:
o .gitattributes
o .gitignore
o Files
o Cross-compiling
Add a brief note in the fixes file.
