Dark Data is a library that gives you the ability to create a database with keyed filing systems, and resuable direct access records. With Dark Data you can create invoicing systems, planetary systems for your space games, character data storage in RPG games, in fact whatever your imagination brings that requires fast record seeking and database storage.

• Keyed lookup via a custom built B-Tree system typical of what is used in most modern databases.
• Fast retrieval.
• Always sorted.
• Scroll backwards or forwards from any position and remain in order.
• Multiple keyed systems per file allowed.
• Records automatically resued after deletion so no packing of space is necessary.
• All commands are simple to use straight from your program.
• Both KFS and DFS can work independently from each other.

Commands are split between KFS for the Keyed Filing System and DFS for the Dynamic File System. KFS is the index or lookup file. DFS is your actual database. You can have a customer database called Customer.DAT and three indexes for easy search and sorts like CustName.KFS, CustID.KFS, CustPostal.KFS for example. Mix and match. If you only need to report by the customer name, you only need to open the customer name index and the customer file. The overhead of having multiple files open is unnecessary.

Lastly, let's say you delete ten records. What happens to that empty space? With a built-in double linked list, it gets reclaimed so that there is no unnecessary waste of disk space! Efficient, fast, and easy to use.

Dark Data

Dark Data is a library that gives you the ability to create a database with keyed filing systems, and resuable direct access records.   $ Dollar € Euros £ Sterling Click to Buy Price: $19.99 €15.99 £13.99 Dark Data is distributed electronically. You must download this product. Download instructions are sent after purchase.

Adventure, Role Play and Strategy games all have one thing in common; an abundance of in-game items and objects that the player must interact with at some level. Traditionally these items are supported in your game code through the creation of your own file system, parsers and internal data structures, and often this code needs to change as your item and object data grows and changes structure. Additionally, when it comes time to populate your game with the remaining hundreds of item instances you will find yourself writing an editor to organize and store your item data. With Dark Data you can remove a whole chunk of coding by replacing your own file handler with one that has been tested for efficiency, speed, stability and flexibility. Whether you game has ten objects or a thousand complex items, your code remains small and flexible while the grunt work is handled by Dark Data.

As Dark Data is designed to a fast lightweight module, you can even employ as a real-time component of your game. With no external database dependencies to worry about, a few lines of code and your game is linked with a potentially massive archive of information without eating up megabytes of dynamic memory. Imagine a trading game or inventory component which needs to keep track of item descriptions, prices, quantities, strengths, abilities, experience points and game specific data for which you need instant access. Furthermore, because it is a database architecture, you can access the same data from other tools such as an item or level editor without having to copy and paste your entire parser to the other applications.

Think of a Keyed Filing System as the index in a back of a book. It's arranged alphabetically. When you look up a word, you find the corresponding page number. Then you go to that page to get your information.

The other half of this is called DFS for Data Filing System. This is where your "page" numbers will point to. Generally speaking, every index in the KFS will point to a single page in the DFS. Note that there can be multiple KFS records pointing to a single DFS. More on that later.

A simple checkbook system:

Let's use the check number as the main index key. Our data will look something like this:

Check: 0101
Date: 10/19/2008
Amount: 25.37
Payee: Acme Gas 'N' Go

We create a database first like this:

DFS Create 1, "Checkbook.dat"
DFS Add Field 1, "string 4 as number"
DFS Add Field 1, "string 10 as date"
DFS Add Field 1, "float as amount"
DFS Add Field 1, "string 20 as Payee"
DFS Finish

Now we add the indexes, the zero at the end is for case insensitive. If we wanted to make it case sensitive we'd put a one (1) in it's place.

KFS Create 2, "CheckNumber.kfs", 4, 0, 0  `0 - unique
KFS Create 3, "CheckDate.kfs", 10, 1, 0   `1 - non unique
KFS Create 4, "CheckPayee.kfs", 20, 1, 0

What we just did is create a database called "Checkbook.dat" with four fields. Each field has a type of "string", "integer", "float", or "byte". (There will be a date field in an upcoming version.) Strings need a maximum length such as 40 for the description. All others are stored in binary and take their defaults.

We've also created three indexes, one for the check number, one for the date and one for the payee. This is to show the power of the KFS in a little bit by being able to use multiple indexes on a single database. The first index is called "CheckNumber.kfs" and has a key length of four which is the same as the check number in the database. This is what we are going to key off of for now. The next is for the date and lastly for the payee. Now, we could have stripped off the slashes and reduce the date to eight but for simplicity we're going to leave the lengths the same.

Next let's add that record above.

record = DFS Add(1)

We just added a blank record and got the new record number from it. It's probably (1) but it doesn't matter to us. We just need it for the KFS.

DFS Put 1, "number", "0101"
DFS Put 1, "date", "2008/10/19" `Reversed the order for ease of sorting later.
DFS Put 1, "amount", "25.37"    `Even though it's numeric, 
we use a string. This will change in the future.
DFS Put 1, "payee", "Acme Gas 'N' Go"

DFS Save 1, record

We "PUT" the data into the buffer and when we are done we "SAVE" it. Simple enough. But what about the keys? Let's do that now.

KFS Add 2, "0101", record
KFS Add 3, "2008/10/19", record
KFS Add 4, "Acme Gas 'N' Go", record
    

Keys added. We'll add a few more records and keys to get some data to play with.

number$ = "0104"
date$ = "2008/10/12"
amount = 75.95
payee$ = "Joe's Plumbing"
GOSUB storerecord

number$ = "0103"
date$ = "2008/10/16" 
amount = 45.00
payee$ = "Mediocre Supply"
GOSUB storerecord

number$ = "0102"
date$ = "2008/10/17
amount = 14.60
payee$ = "Heather's Locks"
GOSUB storerecord

number$ = "0105"
date$ = "2008/10/17
amount = 57.33
payee$ = "Chains & Things"
GOSUB storerecord

DFS Close 1
KFS Close 2
KFS Close 3
KFS Close 4

end

storerecord:
  record = DFS Add(1)
  
  DFS Put 1, "number", number$
  DFS Put 1, "date", date$
  DFS Put 1, "amount", str$(amount)
  DFS Put 1, "payee", payee$
  
  DFS Save 1, record
  
  KFS Add 2, number$, record
  KFS Add 3, date$, record
  KFS Add 4, payee$, record
return

Five new records. We created a simple subroutine to make it a little easier. Also, we went ahead and closed the database and indexes so we can show how to reopen them as well.

Let's suppose you wanted to get a list of checks. They were entered in random order but you want them listed by number. No sorting is needed. This is one of the beautys of a keyed filing system. Each entry goes into something called a B-Tree (not to be confused with a binary tree which is totally different). The tree doesn't do any sorting beyond its leaves. It's all done with internal pointers which keep it balanced for optimum searching, adding, deleting, and listing. Let's get that list now:

`*** New program ***
`Open the database
DFS Open 1, "Checkbook.dat"

`Open the indexes
KFS Open 2, "CheckNumber.kfs"
KFS Open 3, "CheckDate.kfs"
KFS Open 4, "CheckPayee.kfs"

`Show off
print "List by number": print "==============": print
GetList(2)
print "List by date": print "============": print
GetList(3)
print "List by payee": print "=============": print
GetList(4)

`Close the database
DFS Close 1

`Close the indexes
KFS Close 2
KFS Close 3
KFS Close 4
end

function GetList(idx)
  KFS Reset idx   `Point to the start of the table.
  for i = 1 to KFS Count(idx)

    `Get the page number found in the index
    record = KFS Next(idx)  
    
    `Load the page
    DFS Load 1, record   
    
    `Get the information from each field
    number$ = DFS Get(1, "number")
    date$ = DFS Get(1, "date")
    amount = val(DFS Get(1, "amount")
    payee$ = DFS Get(1, "payee")
    
    `Display it
    print number$; " "; date$; " "; amount; " "; payee$; " "
  next i
  print: print: print "Press any key to continue..."
  wait key
endfunction

Simple. Start at the top of the table. Go through all the keys using KFS Count(). Get each record by the index you want and display it.

There are more commands such as KFS Find which of course, finds a page belonging to the thing you looked for in the index. It works similar to the KFS Next() and KFS Previous().

    
record = KFS Find(2, "0104")
DFS Load(1, record)

Deletes are the last thing on our list. Let's delete check number 0104.

record = KFS Find(2, "0104")
if record > 0
  DFS Delete 1, record
  KFS Delete 2
  KFS Delete 3
  KFS Delete 4
endif

Notice that we have something a little new here. "if record > 0" means "if it is a valid record". If it is less than zero, there is an error. These errors are listed at the end of this manual.

Let's get that list again.

KFS Reset 2   `Point to the start of the table.
for i = 1 to KFS Count(2)

  `Get the page number found in the index
  record = KFS Next(2)  
    
  `Load the page
  DFS Load 1, record   
   
  `Get the information from each field
  number$ = DFS Get(1, "number")
  date$ = DFS Get(1, "date")
  amount = val(DFS Get(1, "amount")
  payee$ = DFS Get(1, "payee")
  
  `Display it
  print number$; " "; date$; " "; amount; " "; payee$; " "
next i
print: print: print "Press any key to continue..."
wait key

Check 0104 is gone. Now if we were to add another check into the database you would find something interesting. If you looked at the record number before the delete, and wrote it down somewhere, then when you add a new entry into the database the same record number will be used when compared to what is written. DFS keeps track of it's deleted records so that it can reuse them cutting down on wasted space. It uses a dual linked list system for this. We'll not go into it here but the web is always a good place for information like this and B-Tree systems to see how they work.