PRC builds prc_data biowaredb even eith MySQL activated

[Halgroth]

Hi, I could really use some help on this one. Problem description is as follows:

Ubuntu 14 x64
nwnx2

Our test server runs a fully functional PRC module, the only issue is that it builds the prc_data using biowaredb, and every week or so the server instance crashes with the following error:

Code: [Select]

NWNX2lib: Init
NWNX2lib: org SetString() at 81f41b4, new SetString() at f774cd86
NWNX2lib: org GetObj() at 81f40bc, new GetObj() at f774cdad
* Parsing configuration...
NWN Extender v2.8-dev
(c) 2004 by the APS/NWNX Linux Conversion Group
(c) 2007-2010 by virusman and Acaos
Based on the Win32 version (c) 2003 by Ingmar Stieger (Papillon)
and Jeroen Broekhuizen
visit us at http://www.avlis.org

* Searching for signatures...
* Loading modules...
CONNECT plugin registered.
ODBC plugin registered.
FUNCS plugin registered.
FUNCSEXT plugin registered.
PATCH plugin registered.
* NWNX2 activated.
Neverwinter Nights Server
Build:8109
Copyright BioWare Corp 1998-2004

Server: Loading...
Server: Running...

Server: Loading module "test_server"...
Server: Module loaded
INIT(S): valid ret=0x082041ec
INIT(S): reset ignored ret=0x082041ec


Error Number  -120, 90619
Writing to File
file4write

/home/bastionofpeace/nwntestserver/database/prc_data.fpt

Content of database folder

Code: [Select]

/home/bastionofpeace/nwntestserver ~ $ ls -lh database
total 2,1G
3,0K feb 22 16:22 prc_data.cdx
1,1K feb 29 16:39 prc_data.dbf
2,0G feb 29 16:39 prc_data.fpt

Relevant switches (bop_switches.nss):

Code: [Select]

#include "x2_inc_switches"
#include "prc_inc_switch"

void main()
{
    SetPRCSwitch(PRC_CONVOCC_ENABLE, FALSE);
    SetPRCSwitch(PRC_CONVOCC_USE_XP_FOR_NEW_CHAR, FALSE);
    SetPRCSwitch(PRC_USE_DATABASE, TRUE);
    SetPRCSwitch(PRC_DB_MYSQL, TRUE);
    SetPRCSwitch(PRC_USE_LETOSCRIPT, FALSE);
    SetPRCSwitch(PRC_USE_BIOWARE_DATABASE, FALSE);
...
}

and on module load:

Code: [Select]

// prc_onmodload,m_on_mod_load
/////////////////////////////////////////////////////////////////////
//
// This script has been auto-generated by HakInstaller to call
// multiple handlers for the onmoduleload event.
//
/////////////////////////////////////////////////////////////////////
void main()
{
    ExecuteScript("bop_switches", OBJECT_SELF);
    ExecuteScript("prc_onmodload", OBJECT_SELF);
    ExecuteScript("m_on_mod_load", OBJECT_SELF);

}

Deleting the database on regular intervals is not an acceptable workaround as building the database drains significant resources. I need to get to the bottom of it. I've been digging through the PRC sources but cannot find anything obvious, and the configuration should be fine for all I know. My main problem is that I don't have enough time to familiarize myself with the PRC sources and I don't know the debugging strategy, so I could really use a hand here.

Please let me know if any additional information is required.

Thank you for reading, and for any help you may have to offer.

Cheers

[xwarren]

PRC scripts use bioware database to store cache object with data from various 2da files (Please check Get2DACache() function). To disable bio-db completly you would have to disable this function and recompile most of PRC scripts. The 'clean' prc database (prc_data.*) is several MB in size - yours is 2 GB?!? That's strange. Bioware database files get easly corrupted (especially when they are that large). Could you check what else is stored in the db, please?

[Halgroth]

Thank you for replying.

I have difficulties viewing the biowaredb content, but as far as I can tell it's PRC_2DA_Cache_FingerprintCacheChest.

Any suggestions for viewing the contents?

Edit1:
There seems to be a few alternatives for viewing xbase db files.
dbfdump is very simple and available from the shapelib package on Ubuntu

Edit2:
dbf_dump provides better output and functionalities

[Halgroth]

Code: [Select]

/nwntestserver/database $ dbf_dump --table --nomemo prc_data.dbf
 +---------------------------+----------+------------------+---------+-----+------+------+------+------+------+------+------+
 |          VARNAME          | PLAYERID |    TIMESTAMP     | VARTYPE | INT | DBL1 | DBL2 | DBL3 | DBL4 | DBL5 | DBL6 | MEMO |
 +---------------------------+----------+------------------+---------+-----+------+------+------+------+------+------+------+
 | version                   |          | 02/29/1616:08:12 | S       |     |      |      |      |      |      |      |      |
 | CacheChest                |          | 02/29/1616:08:11 | O       |     |      |      |      |      |      |      |      |
 | PRC_2DA_Cache_Fingerprint |          | 02/29/1616:08:12 | S       |     |      |      |      |      |      |      |      |
 +---------------------------+----------+------------------+---------+-----+------+------+------+------+------+------+------+


Printing with memo goes haywire. There's so much data (+2GB) and it doesn't seem to print any readable data. Not sure if that is due to corrupted data or not. Working on it..

Edit:
So I realize the data is just encoded and the tools I tried so far do not seem to handle decoding very well.

[Halgroth]

Either the data is encoded, or it's binary. At this point I'm going to go with binary. Still, the table names should give a pointer as to what's being put in them. Are these tables in accord with expected behavior for the above configuration?

[TemplarDrake]

*Eagerly awaits reply*

[xwarren]

Sorry for late replay. I confirm that your database is in whole made by prc scripts. As I wrote before - "clean" database is several MB in size (i think it was around 12 MB on my computer). It seems that the database object has been read and saved multiple times. NWN doesn't actually delete old entries from the database - it only makrs them as 'deleted'. That's why your database file is so large. There was a tool on Windows that could delete those entries - I am not sure if there is one for Linux. If there is you need to run it once in a while to keep your database in good condition. If there is not I only see two solutions:
1. create a copy of fresh database (I would suggest packing it with zip or something - it easier to check if the archive was corrupted), write a batch script that would copy the backup and overwrite files in database folder and then start the server.
2. modify the Get2DACashe() function and recompile all srcipts

[Halgroth]

Thank you for getting back to me, xwarren. Things are making more sense now.

Option 1 has already been applied, but I would like to have a cleaner solution in the long run so I'm going to have a shot at option 2. To begin with I need to know the purpose of re-reading and storing the database object in this way. The only reason I can think of is to update some data, but I have no idea what needs updating. Last night, Drake and I set up a stripped down PRC server with only one area and no extra content, and this morning the database had accumulated to +160MB, with absolutely no activity on the server. As far as I can tell, during the first 30 seconds after the module loads a database with 6.8MB of content is created, and then this content is appended every 30 min or so. For what purpose? Nothing happened, so nothing should need updating.

I'm digging through sources now and will try to recompile with some changes and see where that gets me.