APARA: Deviations and Findings¶
This project's APARA model is its own implementation of the engine, written from the engine's documents and checked against APARA's reference model. Its sources, in order of authority:
- The microarchitecture document (APARA Engine: Microarchitecture and Timing), for the engine's structure and timing. Its latency table gives the timing model's default latencies. The timing model has been checked against this document, but not yet against the hardware's own (Aa_model) simulation traces.
- The ISA document (
engine_isp/isa/isa.txtin the APARA repository), for what each instruction does. - The reference model, McodeMachine, with its tools (
mcode_align,mcode_assemble,mcode_run). The narrow-float conversion rules (see Instruction Set) are exactly its rules. Its alignment (mcode_align) and assembly (mcode_assemble) steps are one step in this project's assembler, which uses the same file formats, so files move freely between the two sets of tools; the simulators acceptmcode_run's options.
For every program the reference tools handle correctly, the model gives the same results: the assembler writes the same object file, and a run reaches the same final state (PC, registers, DMEM). This was checked on every valid program of the APARA repository, and on this project's own tests (see APARA Validation Suite).
This page lists where the model deliberately differs from the reference
model, and why, followed by the open questions in the sources, which
are to be reviewed with APARA's designers. Every test that depends on a
deviation has a DEVIATION line in its .vprj, and the tests of the
deviations themselves are collected in validation/apara/deviations/.
Deviations from the reference model¶
Each row gives what the reference model does, what this model does instead, and why. Most are fixes of defects in the reference model, where it contradicts the ISA document or its own intent.
Semantics¶
| # | Reference model | This model | Why |
|---|---|---|---|
| 1 | $return reads the top of the RAS but never pops it: only one level of calls returns correctly |
$return pops |
a stack; nested calls (dev_control.mcode) |
| 2 | a 4-byte load picks the word's half by address bit 1 (its store uses bit 2) | bit 2 for both | a load at offset 4 read the wrong half (dev_arith.mcode) |
| 3 | $vreduce starts every reduction at 0, so * and & always give 0 |
each starts at its operation's identity (1 for *, all ones for & and ~^) |
correct reductions |
| 4 | $vreduce sign-extends unsigned elements |
zero-extends them | $vu8 255+1 gave 0 |
| 5 | $cmov writes $rs2 back with its sign-extended test value plus the increment, even without an increment; the 2-bit increment is not sign-extended (--1 adds 3) |
$rs2 written only with an increment, as $rs2 + increment, signed |
an ($i8) test clobbered a 64-bit $rs2 |
| 6 | an unsigned test (branch, $cmov) compares as a signed 64-bit value; a float test compares the sign-extended bit pattern |
unsigned compares unsigned; floats compare by value, NaN satisfies only != |
($u64) 2^63 > 0 was false |
| 7 | a float $dot $accumulate ignores the accumulator |
starts from $rd's f32 value |
the instruction's meaning |
| 8 | no $fsqrt (an error, result 0) |
implemented for f32 and f64 (as pow(x, 0.5), as the reference's helpers would) |
isa.txt |
| 9 | no float casts (an error) | implemented: integer to float converts the full source value; float to integer truncates toward zero and saturates; NaN gives 0 | isa.txt lists them |
| 10 | a scalar integer cast from a 64-bit source is not truncated to the destination width (narrower sources are) | always truncated | consistency |
| 11 | no float divide for f4, f8, f16 (assert) | every float width divides | |
| 12 | a load/store's negative increment is added as an unsigned 32-bit value (bytes * increment is evaluated as unsigned): --1 on an 8-byte load adds 0xFFFFFFF8 |
a signed 64-bit add | a C++ promotion bug |
| 13 | float ALU operations other than + - * /, and vector ones other than + - *, assert |
illegal instruction (the assembler rejects them) | |
| 14 | the branch decoder reads rs2 as 6 bits [5:0], including the offset's low bit |
bits [4:0] | an odd offset selected a register 32 higher |
| 31 | every element-wise $v operation on 4-bit elements ($vi4, $vu4) gives 0 |
computed, each element wrapping at 4 bits (dev_vector4.mcode) |
the ISA lists the 4-bit vector types; the reference's own $dot ($vi4) works |
| 32 | a non-negative operand of a narrow signed type keeps the bits above the type's width (($i8) 0x105 + 0 = 0x105), in the ALU, branches and $cmov |
only the type's width is read (5) (dev_narrow_signed.mcode) |
"r1 and r2 are i8"; negative values were already sign-extended |
Errors and limits¶
| # | Reference model | This model | Why |
|---|---|---|---|
| 15 | errors (divide by zero, unaligned address, ...) print a message and the thread goes on | the thread halts with the hardware's ERROR codes; the faulting bundle changes nothing | the hardware's behavior (microarch doc) |
| 16 | a push into a full RAS asserts; a return with an empty one asserts | the thread halts, ERROR bit 26 | the microarch doc |
| 17 | 32-byte transfers are checked for 2-byte alignment only | 32-byte alignment | |
| 18 | when the thread picked to run has no bundle at its PC, the whole run ends | that thread halts (exception 7, slot 0); the others go on | a thread's fault should not stop the others |
| 19 | no check of the slot rules | a branch outside slot 0, a divide outside slot 1, a load/store outside slots 4-7 raise exception 7 | the hardware's instruction_alignment_error |
| 20 | IMEM is 2048 words (8 KB) | 4096 words (16 KB) | the microarch doc and imem.aa |
Tools¶
| # | Reference tools | Ours | Why |
|---|---|---|---|
| 21 | the assembler silently truncates out-of-range registers ($r40), immediates and increments, and leaves an unknown label's offset 0 |
errors | |
| 22 | the assembler rejects $vreduce with | & ^ ~^, and * on elements other than 32 bits |
accepts them (element width below 64) | isa.txt permits all six |
| 23 | $vreduce ~^ is assembled as ^ |
as ~^ (XNOR, opcode 0x0D) |
|
| 24 | a blank line in an object file writes IMEM[0] = 0 | blank lines are ignored | |
| 25 | mcode_run's results check always exits 0 |
exit status 1 on a mismatch | scripts can use it |
The engine in a system (sparc_apara)¶
From the hardware's Aa sources (cpu/ccu/ccu.aa, dma/control.aa,
data_mem/*.aa), which the reference model does not cover:
| # | Source | This model | Why |
|---|---|---|---|
| 26 | dma/control.aa uses control-word bit 3 as interrupt enable, as the error flag on completion, and (written) to clear the interrupt; a command with it set never starts |
flags [0] scheduled, [1] started, [2] completed, [3] interrupt enable, [4] error | one meaning per bit |
| 27 | CCU.TXT calls command 7 "wait-on-complete"; ccu.aa polls (non-blocking), so its interrupt fires only if a poll finds the thread already stopped |
without interrupt enable, a poll; with it, the command waits for the stop, then interrupts | an interrupt a program can wait for |
| 28 | command 8 (step) executes the instruction in R2 at PC R1 | one bundle from IMEM at R1, with local data only; R7 := the next PC | debugging use |
| 29 | command 9 returns the RAS's 52-bit internal state | R6 := depth, R7 := top | |
| 30 | (reset leaves registers to the hardware's reset) | reset keeps the registers, clears PC, RAS and ERROR | registers set before a run survive a reset |
Which addresses are local also follows the hardware: in a system, only
the engine's own DMEM window is local, and every other address goes out
to the system. The reference model aliases every address into DMEM, and
the standalone apara configuration keeps that behavior.
The system's physical addresses (see The Engine in a System) follow the AJIT_APARA layout, with the addresses APARA's parameter template gives engine 0 of cluster 0.
Cross-checking against the reference simulator¶
validation/apara/run_tests.py --reference --mcode-run <path to
mcode_run> also cross-checks the model against the reference
simulator. For every test without a DEVIATION line, the model writes
its final state (the PC, every register, and every DMEM word that is or
was non-zero) as a results file, and the reference simulator, given the
same program and inputs, must reach exactly that state. This checks the
model and the test's own expectations at the same time.
Open questions in the sources¶
Where the sources disagree or are silent, the model makes an explicit choice, listed here. D1 to D7 come from the first review of the sources, the rest from the findings since.
- D1, opcode numbering from 0x16. The Aa_model's
parameters.aaandOperatorClassification.txtdisagree. The model uses the assembler's numbering (isa/parameters.aa) and keys its latencies by instruction class, not opcode. - D2, memory operations per bundle. The microarch doc says at most two
(one per DMEM port); the slot rules,
mcode_alignand the reference documentation allow four. The model allows four, two ports, same-bank accesses one after another. - D3, slot rules. The model enforces the microarch doc's (and the assembler produces them).
- D4, EXEC's structure. "Holds one bundle until the slowest finishes"
versus "internally pipelined, full rate". The model is pipelined, results
in order;
engine.exec.pipelined = falsegives the other. - D5, latencies. DIV and FSQRT 10 cycles (verbal), VREDUCE 2 and CMOV 1
(assumed): not in
OperatorClassification.txt. - D6, the parameter template. Its comments and values disagree on the DMEM and configuration base addresses; the model follows the values (they agree with the AFB demultiplexer's decode).
- D7, a push into a full RAS. Halts the thread (microarch doc) versus "blocks the engine" (ISA text): the model halts the thread.
- Same-bank order. The microarch doc serializes same-bank accesses in slot order; the Aa_model does not fix an order. The model uses slot order.
- Exception code 4, "integer square root of a negative number", is in the hardware's exception table, but isa.txt says there is no integer square root. The model has none (the code is unused).
- The BTB is shared by the four threads; two threads' branches that alias to one entry evict each other. Modeled as described.
- REGFILE and WRITEBACK. The microarch doc's channel table shows lock releases going from WRITEBACK to REGFILE, its text (from the Aa_model) has REGFILE reading EXEC's results and forwarding them to the retire daemon. The model follows the text; with two tokens it is the version that sustains one bundle per cycle.
Findings of the validation suites¶
The validation suites (validation/apara/, validation/sparc_apara/)
also turned up places where the engine's ISA and microarchitecture
documents are contradictory, ambiguous, or describe a design property
worth a decision: for example, whether $set inserts or replaces, the
limits of $pack and $dot, the branch predictor training only on
mispredicts, returns being predicted by the BTB alone, the throughput
limit of two REGFILE tokens, and the lack of an atomic read-modify-write.
They are listed, each with the test that shows it, in
docs/compliance/apara_specification_findings.md.