Tool: mp_c2mpt
Overview
C to MicroprobeTest (mpt) tool provides a command-line interface (CLI) to generate MPT files (Microprobe test (mpt) format) from C source code by following certain guidelines (C test format). Then, from the MPT format, the user can use existing tools (Command line tools) to produce test cases for different environments and formats .
Note
This tool relies on third-party tools (compilers and object code inspectors). The microbenchmark generation policy implemented in this tool reproduces exactly the code provided by the compiler and the object code inspector. The framework resolves any symbolic references (references to data, or branch targets) and adds necessary code to initialize the required registers so that it does not break the target Application Binary Interface.
Basic usage
> mp_objdump2mpt -T TARGET -i OBJDUMP_FILE -O MPT_OUTPUT_FILE
where:
Flag/Argument |
Description |
|---|---|
|
Target definition string. Check: Command line target definition scheme |
|
C test file following the format in C test format |
|
Output file name |
Tool work-flow
The figure below summarizes the high-level work-flow of the C to MPT command line tool:
Fig. 1 C to MPT high level work-flow
From a given C file that complies with the C test format, the process is two-fold (see top and bottom flows in the figure).
The C file is compiled using the local host compiler and executed in the local host. This process is necessary to obtain the variable values at run-time just before the test main function c2mpt_function (check C test format) . Also, the host execution with an exit code of 0 confirms the correctness of the code that is provided.
The C file is compiled using the target host compiler (which can be the same as the local) and then the generated instructions are obtained. The instruction sequence is extended to add the necessary initialization to ensure a correct execution of the code. We call this process as environment construction in the figure. For instance, the stack is defined and the stack pointer is initialized.
Finally, all the information (instructions, variables and their contents) is dumped into a MPT file which can then be used to generate test cases in different formats.
The user can control some features of this work-flow using the following flags:
Flag/Argument |
Description |
|---|---|
–no-data-initialization |
Do not run the compiled code locally to get the contents of registered variables before executing the c2mpt_function. Only statically specified variable contents will be dumped in the MPT generated |
|
Store the generated intermediate files permanently; place them in the input source file directory and name them based on the source file |
Note
It is up to the user to control the compiler behavior in order to generate a valid test case. For instance, the compiler might consider a function to be useless and therefore, remove it from the generated code.
C test format
The input file has to be compliant with the C test case format in order to ensure the generation of a correct test case. Check the section C test format for details. For your convenience, the tool provides a flag to dump an empty C file template, which you can use to start the specification of the C-based test:
Flag/Argument |
Description |
|---|---|
|
Dump a template C file, which can be used afterwards as an input file |
Third-party tool customization
As explained in the previous section, this tool relies on third-party tools to generate the final MPT. Three external tools are required: a local host compiler, a target host compiler and the objdump utility. By default, cc and objdump commands are assumed, but in case that the user needs to customize these commands, a set of customization flags are provided:
Flag/Argument |
Description |
|---|---|
|
Local C compiler (Default:’cc’) |
|
Local C compiler flags (Default:’-Wall -Werror -m64 -O3 -Wpedantic -pedantic -pedantic-errors -std=c99’) |
|
Target C compiler (Default:’cc’) |
|
Target objdump utility (Default:’objdump’) |
|
Target C compiler flags (Default:’-Wall -Werror -m64 -O3 -Wpedantic -pedantic -pedantic-errors -std=c99’) |
Note
Only GNU GCC tool-chains (gcc, objdump) are currently supported.
Customizing Application Binary Interface parameters
The ABI (Application Binary Interface) for a given environment (defined in Linux Foundation - Referenced Specifications ) specifies the semantics of registers and the calling conventions. For instance, usually a register is reserved to point to the stack. So, the code being extracted from the objdump output requires that certain registers contain appropriate values. This tool supports the automatic definition of the stack, the automatic initialization of the stack pointer and the definition of the start symbol.
In case the default values are not appropriate, the user can use the following flags to change the different options:
Flag/Argument |
Description |
|---|---|
|
Stack size in bytes (default: 4096) |
|
Stack name (Default: microprobe_stack) |
|
Stack address (Default: allocated in the data area) |
|
End the code with a branch to itself instruction |
Note
The necessary instructions required for initializing the stack pointer will be added at the beginning of the code (on lower addresses) and the start code address will be modified accordingly.
Current limitations
Code constraints: Do not call library functions in any of the functions to be included in the final MPT. Notice that this restriction does not affect the initialization functions that are not included in the MPT.
Variable initialization constraints: As explained in previous section, the program is executed locally to obtain the variable values. As a consequence, in the case that the variable type is not a basic type or a fixed width type as specified here and if the local host and the target host endianess and/or struct padding differ, the initial values of the variables will not be correct. A warning is triggered to alert the user about this issue.
Code/Data segment addresses: One should be careful when specifying the addresses for the code and data regions. Sometimes, the user may face some limitations for some ranges that are not valid in some environments. For instance, if you get a segmentation fault when running the test locally and you are sure that the code is correct and gdb is telling you that the fault is generated during the program loading phase, then it is very likely that the code/data segment addresses are conflicting with others or they are out of the valid range for that environment.
Full usage
mp_c2mpt.py: INFO: Processing input arguments...
usage: mp_c2mpt.py [-h] [-P SEARCH_PATH [SEARCH_PATH ...]] [-V] [-v] [-d]
[-c CONFIG_FILE [CONFIG_FILE ...]] [-C FORCE_CONFIG_FILE]
[--dump-configuration-file OUTPUT_CONFIG_FILE]
[--dump-full-configuration-file OUTPUT_CONFIG_FILE]
[-A ARCHITECTURE_PATHS] [-M MICROARCHITECTURE_PATHS]
[-E ENVIRONMENT_PATHS] -T TARGET [--list-architectures]
[--list-microarchitectures] [--list-environments]
[--traceback] [--profile PROFILE_OUTPUT]
[--host-c-compiler HOST_C_COMPILER]
[--host-cxx-compiler HOST_CXX_COMPILER]
[--target-c-compiler TARGET_C_COMPILER]
[--target-cxx-compiler TARGET_CXX_COMPILER]
[--target-objdump TARGET_OBJDUMP]
[--host-c-compiler-flags HOST_C_COMPILER_FLAGS]
[--host-cxx-compiler-flags HOST_CXX_COMPILER_FLAGS]
[--target-c-compiler-flags TARGET_C_COMPILER_FLAGS]
[--target-cxx-compiler-flags TARGET_CXX_COMPILER_FLAGS]
[-i INPUT_C_FILE] -O OUTPUT_MPT_FILE [-S]
[-X DEFAULT_CODE_ADDRESS] [-D DEFAULT_DATA_ADDRESS]
[--stack-size STACK_SIZE]
[--host-displacement HOST_DISPLACEMENT]
[--fix-displacement] [--stack-name STACK_NAME]
[--stack-address STACK_ADDRESS] [--no-data-initialization]
[--save-temps] [--dump-c2mpt-template]
[--end-branch-to-itself]
Microprobe C to MPT tool
optional arguments:
-h, --help show this help message and exit
-P SEARCH_PATH [SEARCH_PATH ...], --default_paths SEARCH_PATH [SEARCH_PATH ...]
Default search paths for microprobe target definitions
-V, --version Show Microprobe version and exit
-v, --verbosity Verbosity level (Values: [0,1,2,3,4]). Each time this
argument is specified the verbosity level is
increased. By default, no logging messages are shown.
These are the four levels available:
-v (1): critical messages
-v -v (2): critical and error messages
-v -v -v (3): critical, error and warning messages
-v -v -v -v (4): critical, error, warning and info messages
Specifying more than four verbosity flags, will
default to the maximum of four. If you need extra
information, enable the debug mode (--debug or -d
flags).
-d, --debug Enable debug mode in Microprobe framework. Lots of
output messages will be generated
Configuration arguments:
Command arguments related to configuration file handling
-c CONFIG_FILE [CONFIG_FILE ...], --configuration CONFIG_FILE [CONFIG_FILE ...]
Configuration file. The configuration files will be
readed in order of appearance. Values are reset by the
last configuration file in case of non-list values.
List values will be appended (not reset)
-C FORCE_CONFIG_FILE, --force-configuration FORCE_CONFIG_FILE
Force configuration file. Use this configuration file
as the default start configuration. This disables any
system-wide, or user-provided configuration.
--dump-configuration-file OUTPUT_CONFIG_FILE
Dump a configuration file with the actual
configuration used
--dump-full-configuration-file OUTPUT_CONFIG_FILE
Dump a configuration file with the actual
configuration used plus all the configuration options
not set
Target path arguments:
Command arguments related to target paths
-A ARCHITECTURE_PATHS, --architecture-paths ARCHITECTURE_PATHS
Search path for architecture definitions. Microprobe
will search in these paths for architecture
definitions
-M MICROARCHITECTURE_PATHS, --microarchitecture-paths MICROARCHITECTURE_PATHS
Search path for microarchitecture definitions.
Microprobe will search in these paths for
microarchitecture definitions
-E ENVIRONMENT_PATHS, --environment-paths ENVIRONMENT_PATHS
Search path for environment definitions. Microprobe
will search in these paths for environment definitions
Target arguments:
Command arguments related to target specification and queries
-T TARGET, --target TARGET
Target tuple. Microprobe follows a GCC-like target
definition scheme, where a target is defined by a
tuple as following:
<arch-name>-<uarch-name>-<env-name>
where:
<arch-name>: is the name of the architecture
<uarch-name>: is the name of the microarchitecture
<env-name>: is the name of the environment
One can use --list-* options to get the list of
definitions available in the default search paths or
the paths specified by the different --*-paths options
--list-architectures Generate a list of architectures available in the
defined search paths and exit
--list-microarchitectures
Generate a list of microarchitectures available in the
defined search paths and exit
--list-environments Generate a list of environments available in the
defined search paths and exit
Debug arguments:
Command arguments related to debugging facilities
--traceback show a traceback and starts a python debugger (pdb)
when an error occurs. 'pdb' is an interactive python
shell that facilitates the debugging of errors
--profile PROFILE_OUTPUT
dump profiling information into given file (see
'pstats' module)
Compilation arguments:
Command arguments related to compilation options
--host-c-compiler HOST_C_COMPILER
Local C compiler (Default:'cc')
--host-cxx-compiler HOST_CXX_COMPILER
Local C++ compiler (Default:'c++')
--target-c-compiler TARGET_C_COMPILER
Target C compiler (Default:'cc')
--target-cxx-compiler TARGET_CXX_COMPILER
Target C++ compiler (Default:'c++')
--target-objdump TARGET_OBJDUMP
Target objdump utility (Default:'objdump')
--host-c-compiler-flags HOST_C_COMPILER_FLAGS
Local C compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--host-cxx-compiler-flags HOST_CXX_COMPILER_FLAGS
Local C++ compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--target-c-compiler-flags TARGET_C_COMPILER_FLAGS
Target C compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
--target-cxx-compiler-flags TARGET_CXX_COMPILER_FLAGS
Target C++ compiler flags (Default:'-Wall -Werror -m64
-O3 -pedantic -pedantic-errors -std=c99')
C to MPT arguments:
Command arguments related to C to MPT tool
-i INPUT_C_FILE, --input-c-file INPUT_C_FILE
C file to process
-O OUTPUT_MPT_FILE, --output-mpt-file OUTPUT_MPT_FILE
Output file name
-S, --strict Be strict when parsing objdump input, if not set,
silently skip unparsed elements
-X DEFAULT_CODE_ADDRESS, --default-code-address DEFAULT_CODE_ADDRESS
Default code address (default: 0x10030000)
-D DEFAULT_DATA_ADDRESS, --default-data-address DEFAULT_DATA_ADDRESS
Default data address (default: 0x10040000)
--stack-size STACK_SIZE
Stack size in bytes (Default: 4096)
--host-displacement HOST_DISPLACEMENT
Displacement between static objdump code and loaded
image on the host. Default computed automatically
--fix-displacement If data contains addresses (such as pointers) to code
or data regions, the tool will try to fix them adding
the necessary displacement
--stack-name STACK_NAME
Stack name (Default: microprobe_stack)
--stack-address STACK_ADDRESS
Stack address (Default: allocated in the data area)
--no-data-initialization
Do not run the compiled code locally to get the
contents of registered variables before executing the
*c2mpt_function*. Only statically specified variable
contents will be dumped in the MPT generated
--save-temps Store the generated intermediate files permanently;
place them in the input source file directory and name
them based on the source file
--dump-c2mpt-template
Dump a template C file, which can be used afterwards
as an input file
--end-branch-to-itself
A branch to itself instruction will be added at the
end of the test
Environment variables:
MICROPROBETEMPLATES Default path for microprobe templates
MICROPROBEDEBUG If set, enable debug
MICROPROBEDEBUGPASSES If set, enable debug during passes
MICROPROBEASMHEXFMT Assembly hexadecimal format. Options:
'all' -> All immediates in hex format
'address' -> Address immediates in hex format (default)
'none' -> All immediate in integer format
CC Default C compiler
CPP Default C++ compiler
TARGET_OBJDUMP Default target objdump utility
CFLAGS Default C compiler flags
CXFLAGS Default C++ compiler flags
Example outputs
Example 1:
Command:
> mp_c2mpt.py -T z14-z14-z64_linux_gcc -i input.c -O output.mpt
Input file input.c:
1#include "c2mpt.h"
2
3/****************************************************************************/
4// Global variables
5//
6// Define in the section below the global variable to be used in the
7// "c2mpt_function" and its subroutines. Variable below are going to be
8// imported to the mpt format. The variables have to be declared first
9// and then, they have to be registered (see example below).
10//
11// The following macros are defined to declare variables:
12//
13// DECLARE_VARIABLE(type, name, alignment)
14// DECLARE_VARIABLE_WITH_VALUE(type, name, alignment, init_value)
15// DECLARE_VARIABLE_ARRAY(type, name, name+dimension, alignment)
16// DECLARE_VARIABLE_ARRAY_WITH_VALUE(type, name, name+dimension alignment, init_value)
17//
18// where:
19//
20// - type: is the variable type (e.g. char)
21// - name: is the variable name (e.g. myvar)
22// - name+dimension: is the name of the array and the dimensions of the array
23// (e.g. myvar[10][20] )
24// - alignment: is the minimum algnment for the variable
25// - init_value: is the initial value
26//
27// The following macros are defined to register the variables:
28//
29// BEGIN_VARIABLE_REGISTRATION
30// REGISTER_VARIABLE(name)
31// END_VARIABLE_REGISTRATION
32//
33// where:
34//
35// - name: is the variable name to register
36//
37/****************************************************************************/
38
39#define N 5
40
41struct node {
42 int64_t x;
43 struct node *next;
44};
45
46typedef struct node node_t;
47node_t array[N];
48
49DECLARE_VARIABLE_ARRAY(node_t,linkedlist,linkedlist[N],sizeof(node_t))
50DECLARE_VARIABLE_WITH_VALUE(int64_t,count,sizeof(int64_t), 0xCAFECAFE)
51
52BEGIN_VARIABLE_REGISTRATION
53REGISTER_VARIABLE(count)
54REGISTER_VARIABLE(linkedlist)
55END_VARIABLE_REGISTRATION
56
57/****************************************************************************/
58// Function declaration
59//
60// Declare the functions to be converted to the MPT format. It is mandatory
61// to define a "c2mpt_function", which will be the "main" of the test. Also
62// include any related subroutines. The related subroutines have to be
63// defined using the MPT_FUNCTION(signature) macro provided.
64//
65// The function signature for the main c2mpt_function should not be modified.
66// It does not have any parameter. One can use global variables to pass parameters
67// to the function (see the example).
68//
69/****************************************************************************/
70
71MPT_FUNCTION(void my_subroutine(int64_t count))
72
73/****************************************************************************/
74// Function implementation
75//
76// Include below the implementation of the routines defined above, which
77// should be included in the mpt. They should not call functions not defined
78// here because the test should be self-contained to be reproduced safely.
79//
80// Remember to defined the "c2mpt_function" main function.
81//
82/****************************************************************************/
83
84void c2mpt_function()
85{
86
87 node_t* node = & linkedlist[0];
88 while(node->next != NULL)
89 {
90 node = node->next;
91 count += node->x;
92 }
93
94 my_subroutine(count);
95
96}
97
98void my_subroutine(int64_t lcount)
99{
100 count=lcount+lcount;
101}
102
103/****************************************************************************/
104// Initialization function
105//
106// In case you need to initialize the global variables, you can do that in
107// the function below. This function can use any external function
108// including IO (e.g. reading values from disk) . This function will not
109// be included the mpt.
110//
111// If you initialize variable pointers, they should point to addresses of
112// variables defined to be included in the MPT.
113//
114/****************************************************************************/
115
116void c2mpt_init_global_vars()
117{
118 for(int i=0; i < N; i++){
119 linkedlist[i].x = 0x0102030405060708;
120
121 if(i<N-1)
122 {
123 linkedlist[i].next = &(linkedlist[i+1]);
124 }
125 else
126 {
127 linkedlist[i].next = NULL;
128 }
129 }
130}
131
Output file output.mpt:
1; Microprobe Test Definition File
2[MPT]
3mpt_version = 0.5 ; Format version of this MPT file.
4
5[REGISTERS] ; Section to specify the initial register values
6
7; Format: register = value. E.g.:
8
9; Set GR0, GR1 and GR2 register to 0, 1, 2 values respectively
10;GR0 = 0x0
11
12
13[DATA] ; Section to specify the variables
14
15; Data section default address. Variables will be placed from this address
16; if their address is not specified
17
18default_address = 0x0000000010040000
19
20; Variable Declaration
21; Format: var_name = [ "type", nelems, address, alignment, init_values ]
22; where:
23; - "type": is a string specifying the type of elements in the variable
24; - nelems: is the number of elements in the variable
25; - address : is the address of the variable, if set the address will be
26; fixed, otherwise, it will be computer by microprobe
27; - alignment : alignment requirements of the variable. It should not
28; conflict with address if specified. It can be set to None
29; - init_values : if it is a single value, all the elements will be
30; initialized to that value, if it is an array, elements
31; will be initialized to the values specified in a round-
32; robin fashion. Two special keywords can be specified:
33; RNDFP and RNDINT to initialize the elements to random FP
34; and random INT values
35;
36; Note that variable names ARE NOT case sensitive. I.e. VAR = Var = var
37
38count = ["int64_t", 00000001, 0x0000000010040050, 0x0008, 3405695742]
39linkedlist = ["uint8_t", 00000080, 0x0000000010040000, 0x0000, [8, 7, 6, 5, 4, 3, 2, 1, 16, 0, 4, 16, 0, 0, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 32, 0, 4, 16, 0, 0, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 48, 0, 4, 16, 0, 0, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 64, 0, 4, 16, 0, 0, 0, 0, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0]]
40microprobe_stack = ["uint8_t", 00004096, None, 0x0010, None]
41
42[CODE] ; Section to specify the code
43
44; Code section default address. Code will be placed from this address
45; if the instruction address is not specified
46
47default_address = 0x000000001002fff0
48
49; The code specified after 'instructions' entry (below) is the code that will be
50; processed by microprobe. The instruction format is similar to GNU assembler
51; format, it also allows the specification of labels (NOT case sensitive) and
52; references to the declared variables. It is also possible to specify instruction
53; addresses and to do code expansion by referencing other user
54; defined entries. Check the example below to see examples of these features.
55;
56; *****************************************************************************
57; ****** Although Microprobe performs some sanity checks, it is the ********
58; ****** responsibility of the user to define correct code. ********
59; ****** ********
60; *****************************************************************************
61
62instructions =
63 <ELF_ABI_START>:
64 AUIPC x2, %pcrel_hi(microprobe_stack)
65 ADDI x2, x2, %pcrel_lo(microprobe_stack_pcrel_1)
66 JAL x1, c2mpt_function
67 JAL x1, ELF_ABI_EXIT
68 0x0000000010030000 <C2MPT_FUNCTION>:
69 LUI x12, 65600
70 ADDI x12, x12, 0x0
71 C.LD x15, 0x8(x12)
72 C.LD x14, 0x50(x12)
73 C.BEQZ x15, C2MPT_FUNCTION+0X16
74 C.LD x13, 0x0(x15)
75 C.LD x15, 0x8(x15)
76 C.ADD x14, x13
77 C.BNEZ x15, C2MPT_FUNCTION+0XE
78 C.SLLI x14, 1
79 C.SD x14, 0x50(x12)
80 C.JR x1
81 0x000000001003001C <MY_SUBROUTINE>:
82 C.SLLI x10, 1
83 LUI x15, 65600
84 SD x10, 0x50(x15)
85 C.JR x1
86 <ELF_ABI_EXIT>:
87 ADDI x0, x0, 0x0