User Tools

Site Tools


server:running_the_server:compressed_map_files

This is an old revision of the document!


Compressed maps files

When operating the server , the Bigworld map files in CROSSFIRE_LIBDIR/maps can take up to 400 MB of space on the hard-disk.

Until version 1.60.0 and below, the server is able to operate on compressed map files, which would then just take up the space of the compressed tarball they came in.
Since around SVN revisions r14859 to r14871 in July 2011 the compression related code has been removed from the source code;
saying that since 1.70.0 and beyond the server does not handle compressed maps files anymore.

The compression related code is found in server/common/porting.c :

/**
 * This is a list of the suffix, uncompress and compress functions.  Thus,
 * if you have some other compress program you want to use, the only thing
 * that needs to be done is to extended this.
 * The first entry must be NULL - this is what is used for non
 * compressed files.
 */
const char *uncomp[NROF_COMPRESS_METHODS][3] = {
    { NULL, NULL, NULL },
    { ".Z", UNCOMPRESS, COMPRESS },
    { ".gz", GUNZIP, GZIP },
    { ".bz2", BUNZIP, BZIP }
};

And the functions are :

  • static FILE *open_and_uncompress_file(const char *ext, const char *uncompressor, const char *name, int flag, int *compressed, const char *mode)
  • FILE *open_and_uncompress(const char *name, int flag, int *compressed, const char *mode)
  • void close_and_delete(FILE *fp, int compressed)

To find out, if the server actually is able to open compressed map files,
would be to compress the early map file HallOfSelection
and probably the files in the start folder.
On Linux it would be

  • compress HallOfSelection to create a .Z file
  • bzip2 HallOfSelection to create a .bz2 file
  • gzip HallOfSelection to create a .gz file

Either the server keeps going, or would abort and send a message to the logfile as

[Error] Can't open SRC_PATH/crossfire-1.70.0/_install/share/crossfire/maps/HallOfSelection: No such file or directory
[Error] Initial map /HallOfSelection can't be found! Please ensure maps are correctly installed.
[Error] Unable to continue without initial map.

and when creating a link HallOfSelection to point to the HallOfSelection.gz writing to the logfile

[Error] Error loading map header - did not find a newline - perhaps file is truncated? Buf=BZh91AY&SY[GARBLED_STREAM]
[Error] Error loading map header for /HallOfSelection, flags=0
[Error] Initial map /HallOfSelection can't be found! Please ensure maps are correctly installed.
[Error] Unable to continue without initial map.
[Error] Can't open SRC_PATH/crossfire-1.70.0/_install/share/crossfire/maps/HallOfSelection: No such file or directory
[Error] Initial map /HallOfSelection can't be found! Please ensure maps are correctly installed.
[Error] Unable to continue without initial map.

Compression

When the server supports compressed map files, and it is desired to use compressed map files,
then some code has to filter out files inside the maps directory,
that are better left uncompressed.
:!: By all means, any .sh, .pl and .py script files need to be untouched.

The below bash code assumes that the script needs to run from a controlling terminal,
and that it would be located in the top level of the maps directory among the
many folders and files like HallOfSelection , regions | regions.reg , HallOfDMs .
Furthermore it omits files, that are (permanent) apartment, private shop and guilds related.

Of course, both gzip commands can be replaced by the desired compressor; f.ex compress or bzip2 .

In the below code gzip had been selected, since it allows easy recursive decompression of directories by gunzip -r * .

Jump to End Compression .

A shell code script to compress the map file using gzip could look as

#!/bin/bash
 
which gzip || exit
tty || exit  # exit if clicked on it in the filer
 
find . -type f \( -not       -name README          \
               -a -not  -wholename "*.svn/*"       \
               -a -not  -wholename "*python/*"     \
               -a -not  -wholename "*templates/*"  \
               -a -not  -wholename "*editor/*"     \
               -a -not  -wholename "*test/*"       \
               -a -not  -wholename "*styles/*"     \
               -a -not  -wholename "*Info/*"       \
               -a -not  -wholename "*/*apartment*"   \
               -a -not  -wholename "*/*Apartment*"   \
               -a -not  -wholename "*/*APARTMENT*"   \
               -a -not  -wholename "*/*apart*"       \
               -a -not  -wholename "*/villa/*"       \
               -a -not  -wholename "*/guild/*"       \
               -a -not  -wholename "*/guilds/*"      \
               -a -not  -whonename "*/pshops/*"      \
               -a -not  -wholename "*/pshop*"        \
               -a -not  -wholename "*/privateshop*"  \    
               -a -not   -name "*.sh" \
               -a -not   -name "*.pl" \
               -a -not   -name "*.py" \
               -a -not   -name "*.gz" \
               -a -not  -name "*.bz2" \
               -a -not    -name "*.Z" \
               -a -not   -name "luxhouse"    \
               -a -not   -name "housebrxzl"  \
               -a -not   -name "keysale"     \
               -a -not   -name "cdcapart*"   \
               -a -not   -name "guild*"      \
               -a -not   -name "*_lounge"    \
               -a -not   -name "storage_hall*" \
               -a -not   -name "bigchest"    \
               -a -not   -name "basement"    \
               -a -not   -name "mainfloor"   \
               -a -not   -name "secondfloor" \
               -a -not   -name "hallofjoining" \
               -a -not   -name "privateshop*"  \
               -a -not   -name "ChangeLog"   \
               -a -not   -name "Copying"     \
               -a -not   -name "regions*" \) \
                -exec gzip {} \;
End Compression

Jump up to Compression .

server/running_the_server/compressed_map_files.1523276048.txt.gz · Last modified: 2018/04/09 07:14 by karl