[GLLUG] Reading/converting unknown database file

[Damion Yates]

On Thu, 26 Sep 2024, Henrik Morsing via GLLUG wrote:

> On Wed, Sep 25, 2024 at 09:51:31PM +0100, Damion Yates wrote: 
>> I'm not convinced this is compressed, especially based on the initial 
>> hex dump in the OP.  Continue the hd and look for more utf-16 strings 
>> later, or generally repetitive blocks that you'd expect to be 
>> compressed away.  Or gzip it and see if it shrinks more that a few 
>> percent, which should be unlikely if already compressed.  Of course, 
>> it still could be compressed later on, but this doesn't necessarily 
>> get you closer to recognising the uncompressed format, if it's 
>> inhouse/bespoke.  That said it's reminiscent to sqlite, which would 
>> make a lot of sense for this type of database.
>> 
>> There are a few approaches I'd try next:
>> 
>> Create records with very known data, like all 'A's or 0123456789 in 
>> various fields.  You can then try and determine where that ends up 
>> when expected.  Working out the offsets to the various data segments 
>> from the hex will be challenging, I used to crack 8bit game loaders 
>> in the late 1900s by starring at hex dumps for hours.  But as 
>> nostalgic, as that sounds, I would probably approach this using 
>> another technique.

Only slightly complicated and would fit with James Dutton's post with a
start of a breakdown of the format.

>> Ask for the format details from the company... Job done.  At they're 
>> moving to a web system, maybe they consider their old binary 
>> abandonware and don't care now.

Not complicated.

>> Next up, run a decompiler/disassembler over the binary.  It'll still 
>> be a nightmare but you might be able to trace through to how the data 
>> is dumped into the on disc format.  You could also maybe try running 
>> under gdb to step through the db population part.
>> 
>> Finally, I'd play around with xdotool, have it open each record, copy 
>> to clipboard, switch to another tool (e.g. a simple gedit window) and 
>> paste.  Run this for a few hours to extract the data.  You might not 
>> like the web app, but if it can import your data then a web-based 
>> manipulation tool could also scrape the data back out.
>
> Gosh, that's very complicated but I'll have a think.

Well that's like your opinion dude ;)  IMO only some of the suggestions 
to try were complicated.

> TBH, being a Windows app, I started looking at common file-backed DBs 
> Windows developers might use as an equivalent to SQLite and a quick 
> Google mentioned a handful. I might download them and try.

I think recorder.exe can do what xdotool can do, it's a 1990s win16 
binary that can probably still be found and should work, there may be a 
newer equivalent too.  Personally I'd run this in wine and stick with 
xdotool but you could try either.

  - Damion