A new version 0.54 has been released. That is more than 7 months since the previous one (version 0.53), a significantly long time compared with the previous release frequency of only 10 days. We have been still active and working hard on BSG. One of the reasons for that long is we have changed to a new working style: working a bit slower with longer release periods to have more time to think, implement, and test. The other reason we have created some big new things: two completely new designs and implementations for game databases and opening books. In this post, we discuss the first one.
The database was one of the weakest points of BSG. Since the first versions, it had its own database design in a binary format. However both the design and the implementation were not impressive: database functions were slow in general, especially with large numbers of games. It was also buggy, lacking many database functions/features, say, it couldn’t do position searching. Definitely, it was far behind top chess database programs such as SCID. BSG was widely well known as a chess GUI but not as a chess database program.
We decided that was the time to solve the problem about chess databases, regardless of the time and effort we should spend. Instead of going in an easy way by using some existing code or creating a similar design to some top chess database programs, we went on the much more challenging way by creating a new format. The main idea was to use some open, popular general database frameworks for chess and then test to find all problems and solutions. SQLite was picked up for the trial. Using SQL is actually not a completely new idea. Some programmers have tried to use SQL/SQLite for their chess databases already. However almost they all have given up or just created some designs and programs with a lot of limitations, so slow and badly missing chess features.
Chess position searching is another problem. It is not a simple matching of someone’s thought but it is a kind of approximately/arbitrarily searching, to find positions satisfying queries. If we can’t implement it, we archive only half of our targets.
Frankly speaking, when starting the project we weren’t sure if we could gain any reasonable achievement. We still started, trying to understand deeply the problems and managed to push hard all boundaries, to see how far we could go.
That took us a lot of effort, trials, and more than half a year of working hard-faced a series of problems and challenges. To solve the position searching task, we even created a total new artificial query language, named “Position Query Language” (PQL) as well as all techniques, and code to make it works. It is a small language, widely covering, high flexible, very fast, much easy to learn and use, and works perfectly as a pair with SQL to query almost everything about the chess database.
In the end, the result is much better than we expected: the program has full vital functions and features, and it could create the smallest, fastest, and most intelligent databases. They are comparable with any best chess database format/program on any parameter.
