[HLUG] Open Source Databases

[Mark Broadbent]

On Tuesday, 13 November 2012, Dr Chris Owens wrote:

> Hi All,
>
> Please can I pick the brains of the group for a bit of advice.
>
> I have been asked to re-start work from 15-20 years ago on Healthcare
> content and logistics, trouble is it was all done with SuperBase which has
> long gone bust and won't in any case run on modern machines.
>
> The key ingredient SB had was an integrated provision to write scripts
> easily (in a form of Basic) that by-passed GUI services (especially for
> Forms and Output) and gave easy and very tight control of cursor movement
> to rows and after a bit of fiddling to columns apart from customising Forms
> and Output. This meant the content of every cell was easily accessed which
> is my first requirement. (I could be wrong, but using SQL in e.g. MySql
> would be a hard way of managing this sort of access with or without content
> manipulation).
>
> Two other key requirements are a strong import and export facility that
> can handle as many  (esp proprietary) formats as possible with good
> filtering capability and the system needs a good capacity for volume. Speed
> is not critical and finally I would like it to be Open Source.
>
> So I suppose what is really needed is a dogged stable facility without too
> many bells and whistles that is reasonably easy to use and can act as a
> processing engine - any sophisticated data visualisation capability would
> just be a bonus.


Hi Chris,

Your right that using SQL might not be ideal for this type data, however
there are now a huge range of databases that offer different ways of
searching and mapping data.  I use Riak which allows data to be stored in
JSON format which can be indexed and searched using JavaScript (this being
an example of a NoSQL database where there isn't strict schemes in place by
default).  However there are plenty of other choices dependant on how you
need to read and write the data.

For your import/export requirement I'm not sure your going to find many
databases that can import many formats, this would generally be left to an
external program to map the input into the scheme of your database in my
experience.  There is plenty of choice out there in the open source world
for DBs, the hard bit is choosing the right one, particularly as you
mention volume of data to be stored as a requirement.  What sort of volumes
are we talking?  I'm guessing that if the project is 10-15 years old the
volume is not in the terabytes?

Thanks
Mark