roll parts of pr385 into source tree
This does not take the PR as is.
Unlike the PR, this streamlines and minimizes the integration somewhat:
- use hints/include mechanism instead of creating alternative
Makefile.dat, Makefile.src, Makefile.top, Makefile.utl in sys/lib;
those would have been a maintenance nightmare.
- don't have alternative mkmkfile.sh and setup.sh in sys/lib.
- sys/lib/libnethackmain.c differed from sys/unix/unixmain.c by
very little, so just place a small bit of conditional code at the
top of sys/unix/unixmain.c instead.
- changed the conditional code bits from __EMSCRIPTEN__ to
CROSS_TO_WASM.
- You should be able to build the wasm result by:
cd sys/unix ; sh setup.sh hints/linux.2020 ; cd ../..
make fetch-lua (<-one time)
make WANT_LIBNH all
- You should be able to build LIBNBH by:
cd sys/unix ; sh setup.sh hints/linux.2020 ; cd ../..
make fetch-lua (<-one time)
make CROSS_TO_WASM=1 all
As it is currently coded, winshim.c requires C99.
Update the cross-compiling doc at the top.
Remove sys/msdos/Makefile1.cross, sys/msdos/Makefile2.cross, and
sys/msdos/msdos-cross-compile.sh as they are no longer required.
Remove occurrences of CROSSCOMPILE_HOST as the host-side of a
cross-compile can be determined from:
defined(CROSSCOMPILE) && !defined(CROSSCOMPILE_TARGET)
without the additional macro.
Eliminate a couple of warnings about unused static routines.
That led to a couple of other things.
I hope I got host vs target right in the mdlib.c '#if's.
Instead of the hardcoded value that's in this right at the moment,
the intention is to get the Lua version information directly from
Lua itself for the insertion. For now, this will have to do.
Some support of new code #defines to faciliate cross-compiling:
OPTIONS_AT_RUNTIME If this is defined, code to support obtaining
the compile time options and features is
included. If you define this, you'll also have
to compile sys/mdlib.c and link the resulting
object file into your game binary/executable.
CROSSCOMPILE Flags that this is a cross-compiled NetHack build,
where there are two stages:
1. makedefs and some other utilities are compiled
on the host platform and executed there to generate
some output files and header files needed by the
game.
2. the NetHack game files are compiled by a
cross-compiler to generate binary/executables for
a different platform than the one the build is
being run on. The executables produced for the
target platform may not be able to execute on the
build platform, except perhaps via a software
emulator.
The 2-stage process (1. host, 2.target) can be done
on the same platform to test the cross-compile
process. In that case, the host and target platforms
would be the same.
CROSSCOMPILE_HOST Separates/identifies code paths that should only be
be included in the compile on the host side, for
utilities that will be run on the host as part of
stage 1 to produce output files needed to build the
game. Examples are the code for makedefs, tile
conversion utilities, uudecode, dlb, etc.
CROSSCOMPILE_TARGET Separates/identifies code paths that should be
included on the build for the target platform
during stage 2, the cross-compiler stage. That
includes most of the pieces of the game itself
but the code is only flagged as such if it must
not execute on the host.
If you don't define any of those, things should build as before.
One follow-on change that is likely required is setting the new dependency
makedefs has on src/mdlib.c in Makefiles etc.
More information about the changes:
makedefs
- splinter off some of makedefs functionality into a separate file
called src/mdlib.c.
- src/mdlib.c, while included during the compile of makedefs.c
for producing the makedefs utility, can also be compiled
as a stand-alone object file for inclusion in the link step
of your NetHack game build. The src/mdlib.c code can then
deliver the same functionality that it provided to makedefs
right to your NetHack game code at run-time.
For example, do_runtime_info() will provide the caller with
the features and options that were built into the game.
Previously, that information was produced at build time on the
host and stored in a dat file. Under a cross-compile situation,
those values are highly suspect and might not even reflect the
correct options and setting for the cross-compiled target
platform's binary/executable. The compile of those values and
the functionality to obtain them needs to move to the target
cross-compiler stage of the build (stage 2).
- date information on the target-side binary is produced from
the cross-compiler preprocessor pre-defined macros __DATE__
and __TIME__, as they reflect the actual compile time of the
cross-compiled target and not host-side execution of a utility
to produce them. The cross-compiler itself, through those
pre-defined preprocessor macros, provides them to the target
platform binary/executable. They reflect the actual build
time of the target binary/executable (not values produced
at the time the makefiles utility was built and the
appropriate option selected to store them in a text file.)
- most Makefiles should not require adding the new file
src/mdlib.c because util/makedefs.c has a preprocessor
include "../src/mdlib.c" to draw in its contents. As previously
stated though, the Makefile dependency may be required:
makedefs.o: ../util/makedefs.c ../src/mdlib.c
^^^^^^^^^^^^^^^
Make some progress on a couple of next minor release checklist
items, hopefully without introducing too many new bugs. This
is just the initial commit, and work continues.
Checklist items:
Savefiles compatible between Windows versions, whether 64-bit
or 32-bit in little-endian field format.
Selection of file formats:
historical (structlevel saves),
lendian (little-endian, fieldlevel saves),
and just for proof-of-concept, ascii fieldlevel saves
(the ascii is huge! 10x bigger than little-endian).
For the fieldlevel save, all complex data structures recursively
get broken down until until it is one of the simple types that
can't be broken down any further, and that gets when it gets
written to the output file.
New files needed for this build:
hand-coded:
include/sfprocs.h
src/sfbase.c - really a dispatcher to one of the
output/input format routines.
src/sflendian.c - little-endian output writer/reader.
src/sfascii.c - ascii text output writer/reader.
auto-coded (generated):
include/sfproto.h
src/sfdata.c
This is just one approach. I'm sure there are countless others
and they have different pros and cons.
For producing the auto-coded files a utility called
universal-ctags, that is actively maintained and evolving,
was used to do all the heavy-lifting of parsing the
NetHack C sources to tabulate the data fields, and store
them in an intermediate file called util/nethack.tags
(not required for building NetHack if you already have a
generated include/sfproto.h and src/sfdata.c)
util/readtags (also not required for building NetHack
itself) will decipher the nethack.tags file and produce
the functions that can deal with the NetHack struct data
fields.
You can obtain the source for universal-ctags by cloning it
from here:
https://github.com/universal-ctags/ctags.git
The combination universal-ctags + util/readtags has been
tried and tested under both Windows and Linux, so it is
not tied to a particular platform.
Note: util/readtags will work only with universal-ctags
output, so other ctags are unlikely to work as-is.
Universal-ctags can be build from source very easily
under Linux, or under Windows using visual studio.
Hide a bug in the X11 interface by manually wrapping 'short' version
line (the first line of #version output) when runtime ID or git hash
and/or branch is present. Otherwise the very wide 'short' line causes
a horizontal scrollbar to be present at the bottom of the popup window,
but the window size doesn't take that into account so ends up being
one line too short. Worse, there's no vertical scrollbar to bring the
last line into view.
Even if somebody figures out how to fix that properly, the really wide
line above a couple of paragraphs which have been carefully wrapped
for a much narrower width looked strange.
Also, move the 'short' version's period to the end of the text.
old: basic version info. (extra info)
new: basic version info (extra info).
Back out '#include "date.h"' so that cursinit.c won't be recompiled
every time any other file(s) need to be compiled. It doesn't need
patchlevel.h either. There is already a straightforward way to fetch
the copyright banner lines from version.c.
The splash screen (ascii art spelling "NetHack" preceding the normal
copyright lines) was invisible when showing white text on white-ish
background. Make it honor !guicolor.
"Shall I pick a character's role, race, gender and alignment for you?
[ynaq] (y) " was too wide to accept the answer on the same line on
an 80-column display so "(y) " was placed on the second line. That's
constructed in the core; change the construction to omit " a" when
using "character" rather than a role name. (tty shortens it by omitting
the default " (y)"; with " a" gone, it could revert to normal prompt.)
Also a bit of lint cleanup and some reformatting of cursinit.c....
Remove trailing spaces, and remove tabs from the files that had
trailing spaces.
Also, rndorcname() was using a random value to terminate a loop
and was recalculating a new one each iteration.
I think this is needed to avoid inevitable questions/confusion.
Having the git hash in the version string of official binaries
is a very good thing, however.
installing the hooks first
NHgithook.pm: add some warnings if nhversioning can't open files
make sure nhversioning fails before opening gitinfo.txt if it can't get valid
data
Incorporate some git information into NetHack so that it
is potentially visible to a player. That's useful when
collecting details about the version that they are
running and, if the gitinfo is present, it can tie the
code to a specific git commit in the repository.
This modifies 'makedefs -v' to check for the presence of a data file
called dat/gitinfo.txt and if it is there, parse out its
contents, then write additional lines to include/date.h beyond
what 'makedefs -v' was previously putting in there, similar to
this sample:
#define NETHACK_GIT_SHA "0c84e564c78e2024e562d39539376ce2e21eec8e"
#define NETHACK_GIT_BRANCH "NetHack-3.6.0"
The contents of an appropriate dat/gitinfo.txt are as follows,
and trailing/leading whitespace is not significant:
githash = 0c84e564c78e2024e562d39539376ce2e21eec8e
gitbranch = NetHack-3.6.0
It also adjusts the contents of the 'v' version information to
include the additional git info when available.
Also adds some hooks DEVEL/hooksdir and a perl file to DEVEL
for simplifying and automating the deposit of dat/gitinfo.txt
so that it generally reflects the most current git commit.
DEVEL/gitinfo.pl can be used to build dat/gitinfo.txt at any
time without doing a commit, merge, or checkout.
perl DEVEL/gitinfo.pl
command line --version and -version support
To complement the extra information being provided in the
version by the 'v' command, this also adds support for the
following new command line arguments:
--version
-version Output the NetHack version string then exit.
--version:paste Output the NetHack version string and also copy it to
-version:paste the platform's paste buffer for insertion somewhere,
then exit.
If the paste variation of -version is requested on a platform that
hasn't incorporated any support for the capability, it will deliver
the version info then an error message, prior to exiting.
To support the extended -version:paste variation, a port needs to:
- provide a port-specific routine to perform
the paste buffer copy in a port code file.
- #define RUNTIME_PASTEBUF_SUPPORT in the include/portconf.h header file.
--skeleton--
void port_insert_pastebuf(buf)
char *buf;
{
/* insert code to copy the version info from buf into
platform's paste buffer in a supported way */
}
macosx and Windows have both added support for RUNTIME_PASTEBUF_SUPPORT
I've sometimes seen
, and basic NetHack features.
as the last line of the features section from '#version'. I thought
it was due to the way feature phrases were split into individual words
by makedefs, but it turned out to be due to inserting pattern matching
method at run-time. That dynamic options update had a second problem:
if the final phrase "and basic NetHack features" was split across two
lines, the run-time substitution didn't find it and the pattern matching
entry ended up being left out. This fixes both problems, but if future
dynamic entries become more complex, the phrase-splitting/word-wrapping
being done by makedefs may need to be moved into nethack.
Also, add entries for XLOGFILE and PANICLOG to makedefs' options and
re-order a couple of existing ones alphabetically (a failry hopeless
endeavor).
Take the 4-5 line Debian patch and turn it into six dozen lines of
new code. The submitted patch introduces use of several C library
routines that aren't presently in use, so would need testing by all
functional or nearly-functional ports to verify that it wouldn't
break anything. It also switched the formatted build date+time
from localtime to UTC. This makes the code conditional so it can
be ignored by anybody and avoid the risk of breakage. And a lot of
the increase in size is comments attempting to explain what the new
conditional is for: when REPRODUCIBLE_BUILD is defined, makedefs
will use getenv("SOURCE_DATE_EPOCH") (whose value is an integer
representing seconds since 1-Jan-1970) instead of current date+time
when generating date.h. The purpose is to be able to rebuild at a
later date and produce an identical program, which doesn't happen
when compile time gets incorporated into the binary.
I've added some sanity checking to try to make sure the getenv()
value obtained isn't bogus. And the version string put into date.h
will be slightly different, allowing someone who sees date.h or 'v'
output to tell whether SOURCE_DATE_EPOCH was involved: showing
"<port> NetHack <version> last revision <date>" instead of the
usual "... last build <date>".
To test, checkout a new branch for building, make any local edits
to unixconf.h and config.h, including enabling REPRODUCIBLE_BUILD,
git add+commit them, then use
SOURCE_DATE_EPOCH=`git log -1 --pretty=%ct` make install
Other ports will need a bit more work to set up the environment,
but can still use git to track file dates and supply the latest.
Building with alternate configurations could be accomplished by
using tags instead of 'log -1' or by using distinct build branches
where nothing is commited/merged/rebased after completed build.
Unresolved issue: BUILD_DATE, VERSION_ID, and COPYRIGHT_BANNER_C
contain formatted date+time but omit timezone. SOURCE_DATE_EPOCH
is assumed to be UTC but the formatted values don't say so, so it
might appear to be incorrect when compared with local time. We
definitely don't want to start mucking about with timezones within
nethack, so I think we just live with this. It's not an issue for
default configruation where REPRODUCIBLE_BUILD is left disabled.
In light of the recent 'bad options' feedback issue where \r messed
up message display, try to to make newline handling be more consistent.
I'm sure there are lots of places that still handle \n manually, but
it's a start.
The automated reformatting put a space in casts of the form
'(type)(expression)', yielding '(type) (expression)', but it didn't
do that for '(typedef)(expression)'. There are lots of instances of
'(boolean)(expression)'; (uchar) and (xchar) also occur. I haven't
noticed other types, but I haven't looked in very many files yet.
Guard against the possibility of BETA_INFO (or other fields used to
construct the one-line version string) having a percent sign that would
be misinterpreted during pline processing.
I started out cleaning up a bit of lint in the recent run-time options
handling and discovered that pmatchregex wasn't finished. Finish it and
also deal with the version lint. Argument declarations for function
definitions in pmatchregex.c have been switched to K&R style. (The ones
in posixregex.c have been left in ANSI style.)
There wasn't any build rule for pmatchregex.o; now there is (for Unix).
posixregex.o is still the default.
There isn't any build rule for cppregex.o (again, for Unix); the change
to cppregex.cpp is untested.
Changes to be committed:
modified: src/version.c
modified: sys/share/cppregex.cpp
modified: sys/share/pmatchregex.c
modified: sys/share/posixregex.c
modified: util/makedefs.c
Some options in 3.6.0 are determined by what you link with.
The choice of regex support is one.
Let #version show that linked option along with the compile-time options.
I'll push a formatting guide at some point. There may still be
outstanding changes, but please feel free to resolve those as you arrive
a them.
To the best of my knowledge, there is no changes to the actual code
content, but the formatter does have the occasional bug. If you run into
an issue, please fix it!
