As the main objective of the Scid++ project, we intend to develop a number of composition-oriented features on top of the excellent Scid vs PC chess database project. We are particularly interested in extending our capability of searching a database of orthodox chess problems (with solutions) for specific patterns and themes.
This project is not a fork of the Scid vs PC project. Rather, Scid++ provides feature branches (patches) off the Scid vs PC main trunk which may be selectively merged to add new application features or to extend existing functionality.
The svnhead branch of the project mirrors (via the git-svn bridge) the Scid vs PC SVN repository. The full commit history of the SVN repository is preserved by this project and that branch will always track the SVN repo up to the most recent Scid vs PC release. The other branches are outlined below.
CAUTION: We should emphasize, here, that the commit history of this project will occasionally be rewritten. Periodically, the svnhead branch will be updated to include the most recent commits from the Scid vs PC SVN repository. Feature branches from this project will then be rebased to the commit that has been tagged for the new release, thus tracking progress on that project with progress on this. Developers will understand the implications. Others may shrug it off. (All branches are presently based off the upstream v4.24 commit.)
There are several large online databases of chess problems, each with a fairly robust facility for searching through reams of metadata. Unfortunately, nowhere is there a problem database that can be searched for patterns and themes in the solutions -- something analogous to the Heijden endgame studies database. It is a capability that has been glaringly absent since the dawn of time. This project proposes to remedy that state of affairs.
Two essential components comprise such a remedy: firstly, a search engine capable of matching patterns in solution trees, and secondly, a solving engine which will generate solutions in a format that the search engine can consume. The Chess Query Language -- an exceptionally powerful language -- is quite capable of matching patterns in solutions, and the Popeye solving engine has a long history of generating those accurate and complete solutions. The missing link is that common exchange format allowing the two applications to cooperate.
This project integrates both the CQL search engine and the Popeye solving engine in a seamless fashion (as extensions to the Scid tool chest), translating and importing Popeye-generated batches of solutions into Scid's internal database format, then feeding those solutions to the CQL search engine in a format that it can understand. Problems (solutions) matching a CQL query are likewise streamed back to an Scid database.
It is fairly easy to assemble a sizable database of high quality orthodox chess problems off the net, and this project's support files include a few of those. Adding high quality solutions to a database of problems is now a matter of a single click of the mouse, and searching those solutions requires only a modicum of fluency in a query language that was designed just for the task. In order to ease the user into the use of that language, we have embarked on the development of a rather lengthy document suite intended to convey language techniques which may be useful in the problem domain.
The branches of this project may be merged without conflict, but each branch may also stand on its own. Pick and choose as you like.
This branch integrates the latest version of Costeff's & Stiller's CQL search engine. Version 6.0 implements parallel processing and the CQL language itself is much more powerful and expressive. Version 6.1 is a gift to the software developer types who aspire to construct exceptionally complex searches which -- until now -- required jumping through hoops.
The solver branch integrates the popular problem solving engine Popeye. Solutions may be generated for direct/help/self/reflex mates/stalemates, and the solution may be imported as an Scid game.
This branch extends existing features to improve the composing experience. Scid++ can now search for duplicate compositions with matching starting positions within a database of compositions. The board setup dialog has also been extended to be more composer-friendly, preserving PGN tags between setups and remaining open between sessions. Used in tandem with the integrated solving and analysis engines, the dialog allows for interactive problem composition with on-demand engine evaluation.
Adds a dockable game tree rendering window to Scid, intended for use with broad, shallow game trees such as those that one would find with chess problems.
Also included is the twins2series.py script for twin series database production. The twins.pgn database, which can be found with the project's support files, was created with this script.
First and foremost, back up your .scidvspc directory. This project has a number of windows that can be docked but cannot be docked with the upstream project. Saving a layout that includes one of those windows will leave the upstream executable somewhat bewildered. If you find that to be the case and you neglected to back up your .scidvspc directory, edit the options.dat file and delete every line beginning with set ::docking::layout. It's best to avoid saving the docking layouts, altogether, if you might return to using the upstream project.
The feature branches provided by this project have been tested on the Linux OS. For other platforms, we recommend running the application either in a Crostini container (Chrome OS) or in a WSLv2 VM (Windows). For the Mac, there should be no problem building and running the native executable, though we have no way of testing this and the Makefile is not current.
To build the executable: clone the repository, create a new branch in the locally cloned repo and merge multiple selected branches. The configure script will detect the installation and location of critical packages. One may build and install the entire suite of executables and resources, or one may save time and just build the scid and tkscid executables and launch the application from the root of the source tree. An install is not necessary.
For example:
cd /some/directory git clone https://haydoooke@git.code.sf.net/p/scidpp/code scidpp cd scidpp git checkout svnhead git branch merge git checkout merge git merge origin/solver git merge origin/composer git merge origin/cql-v6 git merge origin/utility ./configure make && ./scid
The build and run-time environments for this project require the installation of a number of packages. The easiest way to set up the run-time environment is simply to install an existing Scid package for one's distribution, which should pull in all dependencies. Setting up the build environment will vary from system to system. For a Debian based system using the Apt package manager, for example, the build-dep command will install the build dependencies for a package.
The CQL source must be acquired from the CQL web site, and should be CQL build 9.362 or later. Build the executable, name it cql6 and place it in a directory in your PATH.
The user is responsible for the installation of the Popeye executable. Clone the Popeye repository, check out the v4.85 commit, build the executable and name it popeye. Place it in a directory in your PATH.
The game tree rendering window launches a sub-process (pgn2tree) and that executable must reside in a directory in one's PATH. The install process will take care of copying the executable to {BINDIR} as determined by the configure script. For those who forego the installation process, copy the pgn2tree.py utility from the scripts directory to some directory in your PATH, renaming it to pgn2tree, and giving it execution privileges. The Python script requires the installation of the python-chess package.
The twin series production script (twins2series.py) also requires the installation of the python-chess package, with v0.29.0 strongly recommended.
The obligatory shots of the screen.
The cql-v6 and solver branches are documented as Scid Help pages. From the Help Index, select one of CQL or Problem Solving to pull up the respective page. The composer branch also has a Help page but it is not associated with any specific application window. From the Help Index, select Chess Composition for an explanation of how best to use the feature extensions in a composing context.
There is also a CQL by Example document suite under development.
For those who are new to the chess composition world, a very nice tutorial by Peter Wong can be found at the Australian Chess Problem Composition site. The British Chess Problem Society also has a couple of tutorials online: one for beginners and another that is slightly more advanced.
While this is not a collaborative project, patches for the feature branches will be happily considered.
Mailing lists have been set up for this project at SourceForge and are monitored by project management. Regular project status updates are posted on the -announce list, and may be reviewed from the list archives. A SourceForge account is not required.