The idea for rating warriors using a test suite seems to have been around
as long as Core War but the suites of programs used have always been
home-grown affairs compiled by individuals. When I was learing the game I
used a suite of about 30 warriors from one of the ICWS tournaments, of
course I was still learning then and I didn't realise that the rules were
slightly different for that particular tournament. I remember proposing
the idea of a standardised benchmark suite a few months ago and when
John K. Wilkinson published a suite of warriors for beginners to test
I thought I'd kickstart the whole concept and start publishing ratings
for some warriors, these ratings have now been named the Wilkies to credit
Now I am in possesion of a suite of more advanced warriors compiled by
Beppe Bezzi and JKW (what do we call them, the WilBez?) so it is now my
aim to rate every warrior from Planar's archive. Well at least those which
work in the standard rules.
The benchmark script is now automated but still a bit limited, it hasn't
been set up to handle errors from Pmars, so if your ;assert line is
wrong or missing then it'll have to wait a while to be tested. But at
the moment I've started it rating every warrior in Planar's archive,
but for administrative details (it won't remember if it's already rated a
warrior) I've sent it back to the start. You can see the progress on the
three Latest pages below.
If anybody has
any warriors they would like rated then they can be e-mailed to me and I will
test them. Even if you haven't published a warrior then I'll test it and keep
it secret, you can trust me.....
Thanks to John, Beppe and all the other individuals who offered advice on
which warriors should be included. And also thanks should go to the original
authors of the warriors used in the test suites.