511 lines
26 KiB
Plaintext
511 lines
26 KiB
Plaintext
Cross-compiling NetHack 3.7 Last edit: December 1, 2019
|
|
|
|
The NetHack 3.7 build process differs from the build process of previous
|
|
versions in some important ways that make it possible to use a cross-compiler
|
|
running on one platform (the "host" platform of the build) to produce a binary
|
|
NetHack package that can execute on an entirely different platform.
|
|
|
|
Part A Contents:
|
|
A1. Why cross-compile?
|
|
A2. Building NetHack 3.6 (before)
|
|
A3. Building NetHack 3.7 (going forward)
|
|
A4. How was the build procedure reduced to 5 steps?
|
|
A5. How can I help with the cross-compiling initiative?
|
|
|
|
Part B Contents:
|
|
B1. Two sets of compiles and procedures
|
|
B2. What needs to be built and executed on the HOST?
|
|
B3. What needs to be built for the TARGET?
|
|
B4. Case sample: msdos
|
|
|
|
--------------------------------------------------------------------------------
|
|
Part A - Cross-compiling NetHack
|
|
--------------------------------------------------------------------------------
|
|
|
|
+--------------------------+
|
|
| A1. Why cross-compile? |
|
|
+--------------------------+
|
|
|
|
By using cross-compilers on host platforms with fast processors, plenty of RAM
|
|
and storage resources, and an available cross-compiler, it may be possible to
|
|
keep or resurrect a working version of NetHack on platforms that are now too
|
|
constrained to carry out the build process natively on the platform any more.
|
|
|
|
Some of the constraints in carrying out a native build on the desired target
|
|
may include, but not necessarily be limited to, any of the following:
|
|
|
|
o Access: Somebody with a working knowledge of the NetHack build process may
|
|
not have the desired target build platform available to them. Conversly,
|
|
somebody with a keen knowledge of the target platform, and access to it,
|
|
may not be all that familiar with the NetHack build process.
|
|
|
|
o Resources: Address space limitations, insufficient RAM, low amounts of disk
|
|
storage, slow processor performance, may impede the ability to execute the
|
|
compile process on the target platform.
|
|
|
|
o Compilers: Some of the native compilers on historical platforms may only
|
|
support the particular dialect of C that was popular when the platform and
|
|
compiler were in their prime.
|
|
|
|
Another useful potential result of cross-compiling, is that it paves the way
|
|
for carrying out test and production builds of NetHack for multiple target
|
|
platforms through automated steps carried out on the host platform(s).
|
|
|
|
|
|
+-------------------------------------+
|
|
| A2. Building NetHack 3.6 (before) |
|
|
+-------------------------------------+
|
|
|
|
Very generally, the build of NetHack in past versions required the following
|
|
steps to be carried out:
|
|
|
|
1. Compile and link util/makedefs.
|
|
2. Run makedefs repeatedly with different command line options to produce
|
|
several output files that are required for:
|
|
(a) additional build steps to follow, including some header
|
|
files: pm.h, onames.h, date.h.
|
|
(b) creation of files, containing information required by,
|
|
or about the game during its execution, that are stored in a
|
|
portable, platform-independent way, that need to be inserted
|
|
into the game package.
|
|
(c) creation of files containing information required by, or about
|
|
the game during its execution, that are stored in an architecture
|
|
and/or platform and/or operating system dependent way, that need
|
|
to be inserted into the game package (the quest text format is
|
|
one example).
|
|
3. Compile and link the level compiler. This step needs to execute
|
|
work-alike tools to lex and yacc, or needs to build pre-built lex and
|
|
yacc output (.c, .h files) that are provided in the sys/share part of
|
|
the NetHack source code tree.
|
|
4. Execute the level compiler to read dat/*.des files and create
|
|
a set of binary output files that are architecture and/or operating
|
|
system dependent on the build platform, for use by the game during
|
|
its execution.
|
|
5. Compile and link the dungeon compiler. Like the level compiler, this
|
|
step needs to execute work-alike tools to lex and yacc, or needs to
|
|
build pre-built lex and yacc output (.c, .h files) that are provided
|
|
in the sys/share part of the NetHack source code tree.
|
|
6. Execute the dungeon compiler to read dat/dungeon.def and create
|
|
a set of binary output files that are architecture and/or operating
|
|
system dependent on the build platform, for use by the game during
|
|
its execution.
|
|
7. Compile and link several less critical utilities such as uudecode,
|
|
tile-generation utilities, and so forth, all of which need to execute
|
|
on the build platform during the build process to produce output files
|
|
for use during the game, that are reasonably portable (not architecture
|
|
and/or operating system dependent; only the output of the utilities
|
|
becomes part of the game package, not the executable utilities
|
|
themselves.
|
|
8. Compile and link the game itself.
|
|
9. Package the game and its required files including the output from
|
|
previous steps 2b, 2c, 4, 6, 7 and 8 above.
|
|
|
|
Steps 1, 2a, 2b, 7, and 9 above are not impediments to cross-compiling NetHack.
|
|
|
|
Steps 2c, 3, 4, 5, 6 and 8 above are impediments to cross-compiling NetHack.
|
|
|
|
That's because the files that those steps produce are very much tied to the
|
|
platform where the build of NetHack is being carried out. Variations between
|
|
platforms (such as 32-bit vs 64-bit, integer sizes, pointer sizes, processor
|
|
byte order, data alignment requirements, struct padding and the way bitfields
|
|
are stored) impact the portability of those data files between different
|
|
platforms and operating systems. If all those things happen to match, the files
|
|
might, just might, be usable across platforms, but the chances are against it,
|
|
and that certainly cannot be counted on.
|
|
|
|
+------------------------------------------+
|
|
| A3. Building NetHack 3.7 (going forward) |
|
|
+------------------------------------------+
|
|
|
|
Again, very generally, the build of NetHack in 3.7 requires the following
|
|
steps to be carried out:
|
|
|
|
1. Compile and link util/makedefs.
|
|
2. Run makedefs repeatedly with different command line options to produce
|
|
several output files that are required for:
|
|
(a) additional build steps to follow, including some header
|
|
files: pm.h, onames.h, date.h.
|
|
(b) creation of files, containing information required by,
|
|
or about the game during its execution, that are stored in a
|
|
portable, platform-independent way, that need to be inserted
|
|
into the game package.
|
|
3. Compile and link several less critical utilities such as uudecode,
|
|
tile-generation utilities, and so forth, all of which need to execute
|
|
on the build platform during the build process to produce output files
|
|
for use during the game, that are reasonably portable (not architecture
|
|
and/or operating system dependent; only the output of the utilities
|
|
becomes part of the game package, not the executable utilities
|
|
themselves.
|
|
4. Compile and link the game components for the TARGET; that includes
|
|
NetHack itself, Lua, and any optional regular-expression or window port
|
|
libraries that you plan to link into the NetHack game executable.
|
|
5. Package the game and its required files including the output from
|
|
previous steps 2b, 3 and 4 above.
|
|
|
|
Step 4 is now the only impediment to cross-compiling NetHack, and is resolved
|
|
by executing step 4 using a cross-compiler that runs on the build (host)
|
|
platform to produce a resulting binary for the target platform, instead of
|
|
executing the native compiler.
|
|
|
|
+-----------------------------------------------------+
|
|
| A4. How was the build procedure reduced to 5 steps? |
|
|
+-----------------------------------------------------+
|
|
|
|
The following are among several design changes planned in NetHack 3.7,
|
|
and these specific changes are what altered the build process to make
|
|
cross-compiling possible:
|
|
|
|
o There is no creation of platform-dependent files, such as the quest
|
|
text files, by makedefs during the build process. Instead, the quest
|
|
text files have been converted to Lua and are inserted into the game
|
|
package for processing by the embedded Lua <fixme: interpreter?>
|
|
during execution of NetHack.
|
|
|
|
o There is no build-time level compiler involved. Instead, the level
|
|
descriptions have been converted to Lua and are inserted into the game
|
|
package for processing by the embeded Lua <fixme: interpreter?>
|
|
during execution of NetHack.
|
|
|
|
o There is no build-time dungeon compiler involved. Instead, the dungeon
|
|
description has been converted to Lua and is inserted into the game
|
|
package for processing by the embeded Lua <fixme: interpreter?>
|
|
during execution of NetHack.
|
|
|
|
o Some of the build and option information that was formerly produced
|
|
during build time by makedefs, and contained information about the
|
|
build-platform specifically, is now produced at runtime within the
|
|
game under a cross-compiled build. As such, it now produces information
|
|
applicable to the target NetHack environment, not the build environment.
|
|
|
|
+------------------------------------------------------------+
|
|
| A5. How can I help with the cross-compiling initiative? |
|
|
+------------------------------------------------------------+
|
|
|
|
o If you have a favourite target platform (let's call it XX-Platform for
|
|
example purposes) that you'd like to see NetHack be able to run on, do
|
|
some research to find out if a cross-compiler exists that:
|
|
- produces output for XX-Platform.
|
|
- executes on a platform that you use and love (Linux, Windows,
|
|
Mac OS X are some examples of platforms that have cross-compilers
|
|
for other targets available)
|
|
|
|
o Then, make the community, devteam, and others aware that you're starting
|
|
a cross-compile of NetHack for XX-Platform. You might need to ask some
|
|
"starting out" questions initially, and as you get deeper into it, you
|
|
might need to ask some tougher questions.
|
|
|
|
o Perhaps consider forking from NetHack on GitHub, and do the
|
|
cross-compiler work there in your fork. Strive to get it to a point where
|
|
it's ready to play-test on XX-Platform, or perhaps even use an emulator
|
|
of XX-Platform if one is available. We live in a time where plenty do.
|
|
|
|
Doing your work on a GitHub fork has the following advantages:
|
|
- It will make it really simple to integrate your work back into
|
|
the NetHack source tree if that's one of your goals.
|
|
- It will make it possible and straightforward to merge upstream
|
|
NetHack changes into your work for the XX-Platform cross-compile
|
|
so that it stays current with the game as it evolves.
|
|
- You may get help from others in the form of suggestions, or
|
|
pull-requests, or offers to join the development. Chances are,
|
|
you aren't the only person out there that would like to
|
|
establish/resurrect/maintain NetHack on XX-Platform.
|
|
|
|
Have fun!
|
|
|
|
-----------------------------------------------------------------------------
|
|
Part B - Cross-compiling details
|
|
-----------------------------------------------------------------------------
|
|
|
|
Part B Contents:
|
|
B1. Two sets of compiles and procedures
|
|
B2. What needs to be built and executed on the HOST?
|
|
B3. What needs to be built for the TARGET?
|
|
B4. Case sample: msdos
|
|
|
|
+-----------------------------------------+
|
|
| B1. Two sets of compiles and procedures |
|
|
+-----------------------------------------+
|
|
|
|
The HOST is the platform/place that you're running the build procedures using
|
|
the native compiler/linker, and the cross-compiler/linker that runs on the HOST
|
|
to build the game for a TARGET platform.
|
|
|
|
You have to:
|
|
1. Build mandatory utilities on the HOST.
|
|
2. Execute the mandatory utilities to generate components that will be used
|
|
during step 4 to build the game itself, or that will generate components
|
|
that will become part of the TARGET game package.
|
|
3. Build optional or enhancing utilities on the HOST, execute those
|
|
optional or enhancing utilities on the HOST to generate components that
|
|
will become part of the TARGET game package.
|
|
4. Execute a supported cross-compiler to compile the rest of the game
|
|
components like NetHack, Lua, and any optional libraries.
|
|
(the word "supported", in this sense, means a compiler that runs on your
|
|
HOST build platform, and generates output executable files for the TARGET
|
|
platform.
|
|
|
|
It should be mentioned that you can execute the cross-compile build approach
|
|
even to generate binaries for the same platform as the host, where the HOST
|
|
compilerand the TARGET compiler are actually one and the same.
|
|
|
|
|
|
+------------------------------------------------------+
|
|
| B2. What needs to be built and executed on the HOST? |
|
|
+------------------------------------------------------+
|
|
|
|
On the HOST, here are the mandatory things that have to be built.
|
|
|
|
a) Using the HOST native compiler, build HOST native utility makedefs
|
|
|
|
Compile and link the following with these compiler switches:
|
|
-DCROSSCOMPILE and -DCROSSCOMPILE_HOST
|
|
from sources: util/makedefs.c, src/mdlib.c, src/monst.c, src/objects.c
|
|
|
|
b) Execute HOST native makedefs utility, util/makedefs, as follows:
|
|
util/makedefs -v
|
|
util/makedefs -o
|
|
util/makedefs -p
|
|
util/makedefs -z
|
|
util/makedefs -d
|
|
util/makedefs -r
|
|
util/makedefs -h
|
|
util/makedefs -s
|
|
|
|
c) Using the HOST native compiler, build these additional utilities if your
|
|
target platform requires components that they produce. It is important
|
|
to note that all of the required source files need to be compiled to
|
|
native obj files for linking into the HOST-side utility. Some of the
|
|
source files (src/monst.c, src/objects.c) were likely already compiled
|
|
as native HOST-side obj files in order to build the native HOST utility
|
|
'makedefs' HOST utility above, and you don't need to compile them again
|
|
for use in linking other HOST utilities if the HOST-native obj files
|
|
produced from them are still around.
|
|
|
|
NOTE: A few other source files (src/drawing.c, src/decl.c) that need to
|
|
be compiled for native HOST utilities, also need to be compiled again
|
|
later as TARGET obj files for linking into the TARGET executable. It is
|
|
important to keep the compiled HOST-side obj files produced by the HOST
|
|
native compiler, and the TARGET-side obj files produced by the
|
|
cross-compiler separate and distinct from one another. That can be done
|
|
either by naming the differing object files a little differently
|
|
(perhaps with a suffix), or by placing the HOST-side obj files and the
|
|
TARGET-side obj files into different folders during the build process.
|
|
Whatever works best for your cross-compile.
|
|
|
|
util/dlb
|
|
|
|
from sources: src/dlb.c, src/dlb_main.c, src/alloc.c,
|
|
src/panic.c
|
|
purpose: For packaging up many files that are
|
|
required components of the TARGET game
|
|
into a single nhdat or nhdat370 combined
|
|
file
|
|
|
|
util/uudecode
|
|
|
|
from sources: sys/share/uudecode.c
|
|
purpose: convert some binary files, that are
|
|
distributed in the NetHack sources in
|
|
uuencoded format, back into their
|
|
original binary state
|
|
util/tilemap
|
|
|
|
from sources: win/share/tilemap.c
|
|
purpose: produce output file src/tile.c that is
|
|
required for building TARGET packages with
|
|
tile support
|
|
|
|
util/tile2bmp
|
|
|
|
from sources: win/share/tile2bmp.c, win/share/tiletext.c,
|
|
win/share/tilemap.c, src/drawing.c,
|
|
src/decl.c, src/monst.c, src/objects.c
|
|
purpose: <TBD>
|
|
|
|
util/gif2txt
|
|
|
|
from sources: win/share/gifread.c, win/share/tiletext.c,
|
|
win/share/tilemap.c, src/drawing.c,
|
|
src/decl.c, src/monst.c, src/objects.c,
|
|
src/alloc.c, src/panic.c
|
|
purpose: <TBD>
|
|
|
|
util/ppmwrite
|
|
|
|
from sources: win/share/ppmwrite.c, win/share/tiletext.c,
|
|
win/share/tilemap.c, src/drawing.c,
|
|
src/decl.c, src/monst.c, src/objects.c,
|
|
src/alloc.c, src/panic.c
|
|
purpose: <TBD>
|
|
|
|
|
|
+--------------------------------------------+
|
|
| B3. What needs to be built for the TARGET? |
|
|
+--------------------------------------------+
|
|
|
|
|
|
For the TARGET side, here are the mandatory things that have to be built via
|
|
the HOST-executed cross-compiler that generates code for the TARGET platform.
|
|
|
|
Using the cross-compiler, build the following targets:
|
|
|
|
a) NetHack sources (core is mandatory)
|
|
|
|
With the cross-compiler and linker for the TARGET platform,
|
|
cross-compile and link with these compiler switches:
|
|
-DCROSSCOMPILE and -DCROSSCOMPILE_TARGET
|
|
|
|
core sources (2019): src/allmain.c, src/apply.c, src/artifact.c,
|
|
src/attrib.c, src/ball.c, src/bones.c,
|
|
src/botl.c, src/cmd.c, src/dbridge.c, src/decl.c,
|
|
src/detect.c, src/dig.c, src/display.c, src/do.c,
|
|
src/do_name.c, src/do_wear.c, src/dog.c,
|
|
src/dogmove.c, src/dokick.c, src/dothrow.c,
|
|
src/drawing.c, src/dungeon.c, src/eat.c, src/end.c,
|
|
src/engrave.c, src/exper.c, src/explode.c,
|
|
src/extralev.c, src/files.c, src/fountain.c,
|
|
src/hack.c, src/hacklib.c, src/insight.c, src/invent.c,
|
|
src/isaac64.c, src/light.c, src/lock.c, src/mail.c,
|
|
src/makemon.c, src/mapglyph.c, src/mcastu.c,
|
|
src/mdlib.c, src/mhitm.c, src/mhitu.c, src/minion.c,
|
|
src/mklev.c, src/mkmap.c, src/mkmaze.c, src/mkobj.c,
|
|
src/mkroom.c, src/mon.c, src/mondata.c, src/monmove.c,
|
|
src/monst.c, src/mplayer.c, src/mthrowu.c, src/muse.c,
|
|
src/music.c, src/nhlsel.c, src/nhlua.c, src/nhlobj.c,
|
|
src/o_init.c, src/objects.c, src/objnam.c,
|
|
src/options.c, src/pager.c, src/pickup.c, src/pline.c,
|
|
src/polyself.c, src/potion.c, src/pray.c,
|
|
src/priest.c, src/quest.c, src/questpgr.c,
|
|
src/random.c, src/read.c, src/rect.c, src/region.c,
|
|
src/restore.c, src/rip.c, src/rnd.c, src/role.c,
|
|
src/rumors.c, src/save.c, src/sfstruct.c,
|
|
src/shk.c, src/shknam.c, src/sit.c, src/sounds.c,
|
|
src/sp_lev.c, src/spell.c, src/steal.c, src/steed.c,
|
|
src/sys.c, src/teleport.c, src/tile.c,
|
|
src/timeout.c, src/topten.c, src/track.c,
|
|
src/trap.c, src/u_init.c, src/uhitm.c, src/vault.c,
|
|
src/version.c, src/vis_tab.c, src/vision.c,
|
|
src/weapon.c, src/were.c, src/wield.c, src/windows.c,
|
|
src/wizard.c, src/worm.c, src/worn.c, src/write.c,
|
|
src/zap.c, sys/share/cppregex.cpp
|
|
|
|
tty sources: win/tty/getline.c, win/tty/termcap.c,
|
|
win/tty/topl.c, win/tty/wintty.c
|
|
|
|
plus your platform-specific source files that contain main, typically
|
|
*main.c, and unix support in *unix.c, tty support in *tty.c, and other
|
|
system support in *sys.c as well as others sources pertaining to your
|
|
specific target platform(s).
|
|
|
|
b) Lua (mandatory in 3.7)
|
|
|
|
lib/lua-5.3.5/src
|
|
|
|
from sources: lua.c, lapi.c, lauxlib.c, lbaselib.c, lcode.c,
|
|
lcorolib.c, lctype.c, ldblib.c, ldebug.c,
|
|
ldo.c, ldump.c, lfunc.c, lgc.c, linit.c,
|
|
liolib.c, llex.c, lmathlib.c, lmem.c,
|
|
loadlib.c, lobject.c, lopcodes.c,
|
|
loslib.c, lparser.c, lstate.c, lstring.c,
|
|
lstrlib.c, ltable.c, ltablib.c, ltm.c,
|
|
lundump.c, lutf8lib.c, lvm.c, lzio.c,
|
|
lbitlib.c
|
|
purpose: links into the game executable to interpret
|
|
lua level description files, lua dungeon
|
|
description files, and a lua quest text file.
|
|
|
|
d) recover (optional if desired/required; some targets have recover
|
|
functionality built into NetHack itself)
|
|
|
|
c) Additional optional library packages/obj files as required
|
|
|
|
lib/pdcurses/...
|
|
|
|
from sources: addch.c, addchstr.c, addstr.c, attr.c, beep.c,
|
|
bkgd.c, border.c, clear.c, color.c, delch.c,
|
|
deleteln.c, getch.c, getstr.c, getyx.c,
|
|
inch.c, inchstr.c, initscr.c, inopts.c,
|
|
insch.c, insstr.c, instr.c, kernel.c,
|
|
keyname.c, mouse.c, move.c, outopts.c,
|
|
overlay.c, pad.c, panel.c, printw.c,
|
|
refresh.c, scanw.c, scr_dump.c, scroll.c,
|
|
slk.c, termattr.c, touch.c, util.c, window.c,
|
|
debug.c, pdcclip.c, pdcdisp.c, pdcgetsc.c,
|
|
pdckbd.c, pdcutil.c
|
|
purpose: underlying curses platform support for some
|
|
target platforms where inclusion of the
|
|
NetHack curses window port in win/curses is
|
|
desired
|
|
|
|
+-------------------------+
|
|
| B4. Case sample: msdos |
|
|
+-------------------------+
|
|
|
|
Cross-compiler used: Andrew Wu's djgpp cross-compiler
|
|
Cross-compiler url: https://github.com/andrewwutw/build-djgpp
|
|
Cross-compiler pre-built binary downloads:
|
|
https://github.com/andrewwutw/build-djgpp/releases/download/v2.9/
|
|
Mac OS X pre-built binary: djgpp-osx-gcc550.tar.bz2 (tested)
|
|
Linux pre-built binary : djgpp-linux64-gcc550.tar.bz2 (tested)
|
|
mingw pre-built binary : djgpp-mingw-gcc550-standalone.zip (untested)
|
|
|
|
The msdos cross-compile for NetHack 3.7 uses two phases of compiles:
|
|
Phase1 is the host-side prerequisite stuff that needs to be done first.
|
|
Phase2 is the cross-compile pieces using the djgpp cross-compiler hosted on
|
|
Linux, Mac OS X, or Windows mingw.
|
|
|
|
First, on the host platform, you need to set up for a native Unix NetHack
|
|
build in the usual way. For example, on linux:
|
|
cd sys/unix
|
|
sh setup.sh hints/linux
|
|
cd ../..
|
|
make fetch-lua
|
|
|
|
Now, you could proceed to go ahead and issue
|
|
make all
|
|
to build a native NetHack at that point if you wish, but it is not needed
|
|
for the msdos cross-compile.
|
|
|
|
Instead, a test shell script has been put together that will next accomplish
|
|
each of the following tasks when it is executed. The shell script can be
|
|
invoked by:
|
|
sh sys/msdos/msdos-cross-compile.sh
|
|
but before you do that, please read the paragraphs below.
|
|
|
|
The shell script is meant to accomplish the following things:
|
|
|
|
Prep : the script downloads the djgpp cross-compiler for the host
|
|
platform into lib/djgpp (it doesn't install anything on the
|
|
system, nor does it need to, it just downloads them into the
|
|
identified directories), it downloads a copy of the msdos
|
|
dos-extender into lib/djgpp/cwsdpmi for later packaging up with
|
|
the msdos game, and it downloads pdcurses into lib/pdcurses
|
|
for cross-compiling during the TARGET build.
|
|
|
|
Be certain to ensure the right products are at the url's
|
|
identified above *before* you execute the Case sample msdos
|
|
cross-compile script. The correct products were at those url's
|
|
at the time this was written in Dec 2019, but we don't assume
|
|
any responsibility for what is at those url's now or in the
|
|
future. You need to check before executing the script.
|
|
|
|
Phase1 : the script uses the Makefile sys/msdos/Makefile1.cross
|
|
to complete the host-side build steps using the native gcc
|
|
compiler for the host platform. During phase1 the host obj
|
|
files are put in subfolder src/host_o to keep them separated
|
|
and distinguishable from the target obj files that will be
|
|
built in phase2.
|
|
|
|
Phase2 : the script uses the Makefile sys/msdos/Makefile2.cross
|
|
to complete the target-side build steps using the
|
|
cross-compiler that was obtained during the prep step of the
|
|
script described above. During phase2 the target obj files
|
|
are put in src/msdos_o to keep them separated and
|
|
distinguishable from the host obj files
|
|
|
|
Package: the script then packages up the results that reside in
|
|
msdos-binary into a zip file which it places in lib called
|
|
nh370dos.zip.
|
|
|
|
|