English
资源说明：RISC-V Torture Test
=========================================================================== RISC-V Torture Test Generator =========================================================================== # Author: Yunsup Lee and Henry Cook # Date: January 29th, 2012 # Version: (under version control) This is the RISC-V torture test generator and framework. This repository contains three sub-projects that build upon one another. The first, [generator], is used to create a single random torture test. The second, [testrun], is used to run a particular test on particular simulators, diffing the resulting signature with the ISA simulator and optionally creating a derivative test subset that pinpoints the divergence. The third, [overnight], wraps testrun, allowing tests to be run repeatedly for a given duration or until a failure count. --------------------------------------------------------------------------- Instructions --------------------------------------------------------------------------- Modify "config/default.config" to set the parameters desired for building tests (e.g., setting which instructions to use and in which ratio). Modify "Makefile" as desired to execute the C simulator or RTL simulator of your choice, and to set the other parameters as you require. To build a single test and test it on Spike: $ make igentest To build single test and run it on the C simulator or RTL simulator, use "make cgentest" or "make rgentest". To run overnight tests, you can use "make cnight" and "make rnight". --------------------------------------------------------------------------- Signatures --------------------------------------------------------------------------- Torture works by dumping the register state out to memory at the end of the test program execution. This output is then compared against the output from the Spike ISA simulator. The torture program writes the register state to the memory address specified by "xreg_output_data", which is located in the memory section ".global begin_signature". The Spike ISA simulator will write out the data found in the "begin_signature" section on exit if provided with the "+signature=" argument: $ spike +signature=my_spike_signature.txt test_binary The Rocket-chip infrastructure uses the "riscv-fesvr" program to control the execution of the C and RTL simulators. The "riscv-fesvr" also accepts the +signature argument too. $ ./csim-rocket-chip +signature=my_rocket_signature.txt test_binary A simple diff between the Spike and chip simulator signatures will tell you if any errors have occurred. $ diff my_spike_signature.txt my_rocket_signature.txt **PORTING TORTURE TO YOUR OWN RISC-V PROCESSOR:** If you would like to use riscv-torture with your own RISC-V processor, you will need to provide a way to dump the "begin_signature" section to a file. --------------------------------------------------------------------------- Low-level Usage --------------------------------------------------------------------------- Some basic use cases are illustrated here (note the Makefile abstracts this for you). Make a single test: % ./sbt generator/run % cd output % make % spike +signature=test.sig test Take an existing test and diff the signatures of ISA and C simulators: % ./sbt 'testrun/run -a output/test.S -c /path/to/reference-chip/emulator/emulator' *** Currently, due to the limiation of scala process library, you cannot torture the RTL simulator *** # Generate a random test and diff the signatures of ISA and RTL simulators: # % ./sbt 'testrun/run -r /path/to/reference-chip/vlsi/build/vcs-sim-rtl/simv' Run tests for 30 minutes, email hcook when done, and save failures to dir: % ./sbt 'overnight/run -m 30 -e hcook@eecs.berkeley.edu -p dir' --------------------------------------------------------------------------- Installing --------------------------------------------------------------------------- % git submodule update --init --------------------------------------------------------------------------- Overnight Overview --------------------------------------------------------------------------- This framework utilizes both the test runner and test generator to perform a long terms serach for failing test cases. It takes the following command line arguments: Usage: overnight/run [options] -C| --config config file -p | --permdir dir to store failing tests -c | --csim C simulator -r | --rtlsim RTL simulator -e | --email email to report to -t | --threshold number of failures to trigger email -m | --minutes number of minutes to run tests You can only generate tests with one instruction mix at a time (based on the setting in the config file). It doesn't matter what simulator you use with the -r and -c flags, they just determines the name used to describe whose diff failed. --------------------------------------------------------------------------- Testrun Overview --------------------------------------------------------------------------- This utility compares the signatures generated by passing the -testsig flag to the specified simulators. If it encounters a difference, it subdivides the test into many subtests and searches for which exact program segment reveals the failure. It takes the following command line arguments: Usage: testrun/run [options] -C | --config config file -a | --asm input ASM file -c | --csim C simulator -r | --rtlsim RTL simulator -s | --seek Seek for failing pseg -d | --dump Dump mismatched signatures If you don't specify a asm file, a random one will be generated for you. You can only generate tests with one instruction mix at a time (based on the setting in the config file). It doesn't matter what simulator you use with the -r and -c flags, they just determines the name used to describe whose diff failed. By default, a failed diff will result in the subtest sweep occuring, but this search can be diasbled. Note that the pseg ID reported is actually the pseg following the pseg containing the error. You can optionally dump mistmatched signatures to the dir containing the asm file under test. --------------------------------------------------------------------------- Generator Overview --------------------------------------------------------------------------- To generate a random test, the torture test generator randomly generates many test sequences from a set of test sequences that are written by hand, performs a random register allocation for all test sequences, and finally randomly interleaves instructions from these test sequences. To extend the set of tests or coverage, the programmer needs to write new test sequences. It takes the following command line arguments: Usage: generator/run [options] -o | --output output filename -C | --config config file The following sections describe adding new functionality to the generator. --------------------------------------------------------------------------- Test sequence example --------------------------------------------------------------------------- Before we talk about how to write a test sequence, let's look at a very simple example. The following example is a test sequence, which emits an add instruction. class SeqADD extends Seq { val src1 = reg_read_any() val src2 = reg_read_any() val dest = reg_write(src1, src2) insts += ADD(dest, src1, src2) } As I hinted in the overview that the test generator will do register allocation you don't write a string of instructions with architectural registers. You request a virtual registers (i.e., registers that are yet tied down to architectural registers) when you need them, save them in scala values, and use them when you need to (e.g., in an instruction). --------------------------------------------------------------------------- Types of virtual registers --------------------------------------------------------------------------- - Hidden (position dependent registers): Registers that will have different values when the code is positioned at a different address. An example is registers that hold addresses. Registers that are hidden should be excluded from the output signature. - Visible (position independent registers): Registers that are not hidden, therefore will have the same values when the code is positioned at a different address. These registers should be included as part of the output signature. --------------------------------------------------------------------------- How to write a sequence --------------------------------------------------------------------------- Use the following functions to request a register, and generate a string of instructions (look at Inst.scala to see what instructions are available) that uses these virtual registers, and add them to the insts array. - reg_read_zero(): returns register x0 - reg_read_any(): returns any type of register (hidden or visible) - reg_read_visible(): returns a visible register - reg_write_ra(): returns register ra for write - reg_write_visible(): returns a visible register for write - reg_write_hidden(): returns a hidden register for write - reg_write(regs: Reg*): returns a register that matches the type of regs (if any reg in regs are hidden, the output type is hidden) Note that the torture test framework is written in scala, you can use any scala functionality to generate instructions. Look at SeqALU.scala, SeqMem.scala, and SeqBranch.scala to get inspired. --------------------------------------------------------------------------- Future TODO --------------------------------------------------------------------------- - provide support for loops - generate statistics of a test to get a sense of coverage + statistics should include instruction count of each type + statistics should include register usage - complete floating point tests + add floating point memory move tests + improve floating point init randomization + add rounding modes tests - complete vector tests + better randomization + add SeqVOnly: Tests special vf-only instructions - code refactoring + consolidate RegPool logic + detect and suppress unallocatable sequences
本源码包内暂不包含可直接显示的源代码文件，请下载源码包。
