Tableaux 2000 - Easy/medium benchmarks

February 2000


These benchmarks are described at the Tableaux 2000 competition homepage.

They consist of modal formulas obtained by translating randomly generated quantified boolean formulas. For each test:

  • C are the of 4-literal clauses in the original QBF
  • D are the alternations of quantifiers
  • V is the maximum number of variables under the scope of each quantifier
In these benchmarks, the very first translation is preprocessed by performing some resolution steps. Therefore, the resulting benchmarks are less difficult than the unpreprocessed version.

Here we evaluate *SAT+USB performance, i.e., *SAT plus bit-matrix based caching; its distinguishing features are:

  • it can store sets of formulas encoded as sets of integers
  • it can query for subsets (supersets)
  • it can be bounded in size and, if bounded, it allows to use a FIFO politic to discharge obsolete reults
We show the geometric mean of the Cpu time taken to solve some samples of each problem. A timelimit of 1200 seconds is enforced. -- in the table, means that all the samples exceded the timelimit. Also the geometric mean of the number of Consistency checks is shown. For the samples that exceeed the timelimit we take into account the number of checks performed so far.


Test *SAT+USB
S/U/T Cpu Cons Mem
C20-V16-D4 8/0/0 6.57 6684 4565
C10-V16-D6 8/0/0 0.73 1316 4370
C30-V16-D4 8/0/0 83.77 43580 5942
C20-V16-D6 8/0/0 8.93 6701 5847
C40-V16-D4 2/0/6 604.25 234336 8195
C30-V16-D6 8/0/0 176.68 72254 7918
C50-V16-D4 0/0/8 -- 340587 --
C40-V16-D6 4/0/4 561.58 168941 10445
C50-V16-D6 0/0/8 -- 248497 --
C10-V4-D4 8/0/0 0.08 246 1156
C20-V4-D4 6/2/0 0.48 945 1362
C10-V4-D6 8/0/0 0.16 410 1455
C30-V4-D4 2/6/0 0.66 1036 1434
C20-V4-D6 8/0/0 1.78 2437 1796
C40-V4-D4 1/7/0 0.96 1268 1492
C10-V8-D4 8/0/0 0.25 643 1784
C30-V4-D6 7/1/0 7.46 7435 2088
C50-V4-D4 0/8/0 0.66 726 1683
C40-V4-D6 2/6/0 5.83 5396 2492
C20-V8-D4 8/0/0 3.28 4037 2584
C50-V4-D6 0/8/0 4.66 3755 2502
C30-V8-D4 8/0/0 42.72 30507 3409
C10-V8-D6 8/0/0 0.29 679 2346
C20-V8-D6 8/0/0 4.81 4808 3488
C40-V8-D4 2/6/0 58.20 32698 3736
C30-V8-D6 8/0/0 90.65 52982 4262
C50-V8-D4 0/8/0 75.71 37776 4551
C40-V8-D6 4/1/3 396.80 163496 5776
C50-V8-D6 0/0/8 -- 393254 --
C10-V16-D4 8/0/0 0.46 906 3334



Back to the state of the art page