Examples on POWER
In the definitions/power/examples directory of the Microprobe distribution
(if you installed the microprobe_target_power package),
you will find different examples showing the usage of Microprobe
for the power architecture. Although we have split the examples by
architecture, the concepts we introduce in these examples are common in all
the architectures.
We recommend users to go through the code of these examples to understand specific details on how to use the framework.
Contents:
isa_power_v206_info.py
The first example we show is isa_power_v206_info.py. This example
shows how to search for architecture definitions (e.g. the ISA properties),
how to import the definitions and then how to dump the definition.
If you execute the following command:
> ./isa_power_v206_info.py
will generate the following output, which shows all the details of the POWER v2.06 architecture (first and last 20 lines for brevity):
--------------------------------------------------------------------------------
ISA Name: power_v206
ISA Description: power_v206
--------------------------------------------------------------------------------
Register Types:
GPR: General Register (bit size: 64)
VSCR: Vector Status and Control Register (bit size: 32)
FPR: Floating-Point Register (bit size: 64)
SPR: Special Purpose Register (64 bits) (bit size: 64)
VR: Vector Register (bit size: 128)
MSR: Machine State Register (bit size: 64)
SPR32: Special Purpose Register (32 bits) (bit size: 32)
VSR: Vector Scalar Register (bit size: 128)
FPSCR: Floating-Point Status and Control Register (bit size: 32)
CR: Condition Register (bit size: 4)
--------------------------------------------------------------------------------
Architected registers:
AESR : AESR Register (Type: SPR)
AMOR : AMOR Register (Type: SPR)
AMR : Authority Mask Register (Type: SPR)
...
access_storage : False (Boolean indicating if the instruction has storage operands )
access_storage_with_update : False (Boolean indicating if the instruction accesses to storage and updates the source register with the generated address)
algebraic : False (Boolean indicating if operation uses algebraic rules to keep values )
branch : False (Boolean indicating if the instruction is a branch )
branch_conditional : False (Boolean indicating if the instruction is a branch conditional )
branch_relative : False (Boolean indicating if the instruction is a relative branch )
category : VSX (String indicating if the instruction the instruction category )
decimal : False (Boolean indication if the instruction requires inputs in decimal format )
disable_asm : False (Boolean indicating if ASM generation is disabled for the instruction. If so, binary codification is used. )
hypervisor : False (Boolean indicating if the instruction need hypervisor mode )
privileged : False (Boolean indicating if the instruction is privileged )
privileged_optional : False (Boolean indicating the instrucion is priviledged or not depending on the input values )
switching : None (Input values required to maximize the computational switching )
syscall : False (Boolean indicating if the instruction is a syscall or return from one )
trap : False (Boolean indicating if the instruction is a trap )
Instructions defined: 938
Variants defined: 964
--------------------------------------------------------------------------------
The following code is what has been executed:
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16isa_power_v206_info.py
17
18Example module to show how to access to isa definitions.
19"""
20
21# Futures
22from __future__ import absolute_import, print_function
23
24# Built-in modules
25import os
26
27# Own modules
28from microprobe.target.isa import find_isa_definitions, import_isa_definition
29
30__author__ = "Ramon Bertran"
31__copyright__ = "Copyright 2011-2021 IBM Corporation"
32__credits__ = []
33__license__ = "IBM (c) 2011-2021 All rights reserved"
34__version__ = "0.5"
35__maintainer__ = "Ramon Bertran"
36__email__ = "rbertra@us.ibm.com"
37__status__ = "Development" # "Prototype", "Development", or "Production"
38
39# Constants
40ISANAME = "power_v206"
41
42# Functions
43
44# Classes
45
46# Main
47
48# Search and import definition
49ISADEF = import_isa_definition(
50 os.path.dirname([
51 isa for isa in find_isa_definitions() if isa.name == ISANAME
52 ][0].filename))
53
54# Print definition
55print((ISADEF.full_report()))
56exit(0)
In this simple code, first the find_isa_definitions,
import_isa_definition from the microprobe.target.isa module
are imported (line 14). Then, the first one is used to look for definitions of
architectures, a list returned and filtered and only the one with
name power_v206 is imported using the second method:
import_isa_definition (lines 34-37). Finally, the full report of
the ISADEF object is printed to standard output in line 40.
In the case, the full report is printed but the user can query any
information about the particular ISA that has been imported by using the
microprobe.target.isa.ISA API.
power_v206_power7_ppc64_linux_gcc_profile.py
The aim of this example is to show how the code generation works in Microprobe. In particular, this example shows how to generate, for each instruction of the ISA, an endless loop containing such instruction. The size of the loop and the dependency distance between the instructions of the loop can specified as a parameter. Using Microprobe you can generate thousands of microbenchmarks in few minutes. Let’s start with the command line interface. Executing:
> ./power_v206_power7_ppc64_linux_gcc_profile.py --help
will generate the following output:
power_v206_power7_ppc64_linux_gcc_profile.py: INFO: Processing input arguments...
usage: power_v206_power7_ppc64_linux_gcc_profile.py [-h]
[-P SEARCH_PATH [SEARCH_PATH ...]]
[-V] [-v] [-d]
[-i INSTRUCTION_NAME [INSTRUCTION_NAME ...]]
[--output_prefix PREFIX]
[-O PATH] [-p NUM_JOBS]
[-S BENCHMARK_SIZE]
[-D DEPENDECY_DISTANCE]
ISA power v206 profile example
optional arguments:
-h, --help show this help message and exit
-P SEARCH_PATH [SEARCH_PATH ...], --default_paths SEARCH_PATH [SEARCH_PATH ...]
Default search paths for microprobe target definitions
-V, --version Show Microprobe version and exit
-v, --verbosity Verbosity level (Values: [0,1,2,3,4]). Each time this
argument is specified the verbosity level is
increased. By default, no logging messages are shown.
These are the four levels available:
-v (1): critical messages
-v -v (2): critical and error messages
-v -v -v (3): critical, error and warning messages
-v -v -v -v (4): critical, error, warning and info messages
Specifying more than four verbosity flags, will
default to the maximum of four. If you need extra
information, enable the debug mode (--debug or -d
flags).
-d, --debug Enable debug mode in Microprobe framework. Lots of
output messages will be generated
-i INSTRUCTION_NAME [INSTRUCTION_NAME ...], --instruction INSTRUCTION_NAME [INSTRUCTION_NAME ...]
Instruction names to generate. Default: All
instructions
--output_prefix PREFIX
Output prefix of the generated files. Default:
POWER_V206_PROFILE
-O PATH, --output_path PATH
Output path. Default: current path
-p NUM_JOBS, --parallel NUM_JOBS
Number of parallel jobs. Default: number of CPUs
available (80). Valid values: 1, 2, 3, 4, 5, 6, 7, 8,
9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,
62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74,
75, 76, 77, 78, 79, 80
-S BENCHMARK_SIZE, --size BENCHMARK_SIZE
Benchmark size (number of instructions in the endless
loop). Default: 64 instructions
-D DEPENDECY_DISTANCE, --dependency_distance DEPENDECY_DISTANCE
Average dependency distance between the instructions.
Default: 1000 (no dependencies)
Environment variables:
MICROPROBETEMPLATES Default path for microprobe templates
MICROPROBEDEBUG If set, enable debug
MICROPROBEDEBUGPASSES If set, enable debug during passes
MICROPROBEASMHEXFMT Assembly hexadecimal format. Options:
'all' -> All immediates in hex format
'address' -> Address immediates in hex format (default)
'none' -> All immediate in integer format
Lets look at the code to see how this command line tool is implemented. This is the complete code of the script:
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_profile.py
17
18Example module to show how to generate a benchmark for each instruction
19of the ISA
20"""
21
22# Futures
23from __future__ import absolute_import
24
25# Built-in modules
26import multiprocessing as mp
27import os
28import sys
29import traceback
30
31# Third party modules
32
33# Own modules
34import microprobe.code.ins
35import microprobe.passes.address
36import microprobe.passes.branch
37import microprobe.passes.decimal
38import microprobe.passes.float
39import microprobe.passes.ilp
40import microprobe.passes.initialization
41import microprobe.passes.instruction
42import microprobe.passes.memory
43import microprobe.passes.register
44import microprobe.passes.structure
45import microprobe.utils.cmdline
46from microprobe import MICROPROBE_RC
47from microprobe.exceptions import MicroprobeException
48from microprobe.target import import_definition
49from microprobe.utils.cmdline import existing_dir, \
50 int_type, print_error, print_info, print_warning
51from microprobe.utils.logger import get_logger
52
53__author__ = "Ramon Bertran"
54__copyright__ = "Copyright 2011-2021 IBM Corporation"
55__credits__ = []
56__license__ = "IBM (c) 2011-2021 All rights reserved"
57__version__ = "0.5"
58__maintainer__ = "Ramon Bertran"
59__email__ = "rbertra@us.ibm.com"
60__status__ = "Development" # "Prototype", "Development", or "Production"
61
62# Constants
63LOG = get_logger(__name__) # Get the generic logging interface
64
65
66# Functions
67def main_setup():
68 """
69 Set up the command line interface (CLI) with the arguments required by
70 this command line tool.
71 """
72
73 args = sys.argv[1:]
74
75 # Create the CLI interface object
76 cmdline = microprobe.utils.cmdline.CLI("ISA power v206 profile example",
77 config_options=False,
78 target_options=False,
79 debug_options=False)
80
81 # Add the different parameters for this particular tool
82 cmdline.add_option(
83 "instruction",
84 "i",
85 None,
86 "Instruction names to generate. Default: All instructions",
87 required=False,
88 nargs="+",
89 metavar="INSTRUCTION_NAME")
90
91 cmdline.add_option(
92 "output_prefix",
93 None,
94 "POWER_V206_PROFILE",
95 "Output prefix of the generated files. Default: POWER_V206_PROFILE",
96 opt_type=str,
97 required=False,
98 metavar="PREFIX")
99
100 cmdline.add_option("output_path",
101 "O",
102 "./",
103 "Output path. Default: current path",
104 opt_type=existing_dir,
105 metavar="PATH")
106
107 cmdline.add_option(
108 "parallel",
109 "p",
110 MICROPROBE_RC['cpus'],
111 "Number of parallel jobs. Default: number of CPUs available (%s)" %
112 mp.cpu_count(),
113 opt_type=int,
114 choices=list(range(1, MICROPROBE_RC['cpus'] + 1)),
115 metavar="NUM_JOBS")
116
117 cmdline.add_option(
118 "size",
119 "S",
120 64, "Benchmark size (number of instructions in the endless loop). "
121 "Default: 64 instructions",
122 opt_type=int_type(1, 2**20),
123 metavar="BENCHMARK_SIZE")
124
125 cmdline.add_option("dependency_distance",
126 "D",
127 1000,
128 "Average dependency distance between the instructions. "
129 "Default: 1000 (no dependencies)",
130 opt_type=int_type(1, 1000),
131 metavar="DEPENDECY_DISTANCE")
132
133 # Start the main
134 print_info("Processing input arguments...")
135 cmdline.main(args, _main)
136
137
138def _main(arguments):
139 """
140 Main program. Called after the arguments from the CLI interface have
141 been processed.
142 """
143
144 print_info("Arguments processed!")
145
146 print_info("Importing target definition "
147 "'power_v206-power7-ppc64_linux_gcc'...")
148 target = import_definition("power_v206-power7-ppc64_linux_gcc")
149
150 # Get the arguments
151 instructions = arguments.get("instruction", None)
152 prefix = arguments["output_prefix"]
153 output_path = arguments["output_path"]
154 parallel_jobs = arguments["parallel"]
155 size = arguments["size"]
156 distance = arguments["dependency_distance"]
157
158 # Process the arguments
159 if instructions is not None:
160
161 # If the user has provided some instructions, make sure they
162 # exists and then we call the generation function
163
164 instructions = _validate_instructions(instructions, target)
165
166 if len(instructions) == 0:
167 print_error("No valid instructions defined.")
168 exit(-1)
169
170 # Set more verbose level
171 # set_log_level(10)
172 #
173 list(
174 map(_generate_benchmark,
175 [(instruction, prefix, output_path, target, size, distance)
176 for instruction in instructions]))
177
178 else:
179
180 # If the user has not provided any instruction, go for all of them
181 # and then call he generation function
182
183 instructions = _generate_instructions(target, output_path, prefix)
184
185 # Since several benchmark will be generated, reduce verbose level
186 # and call the generation function in parallel
187
188 # set_log_level(30)
189
190 if parallel_jobs > 1:
191 pool = mp.Pool(processes=parallel_jobs)
192 pool.map(
193 _generate_benchmark,
194 [(instruction, prefix, output_path, target, size, distance)
195 for instruction in instructions], 1)
196 else:
197 list(
198 map(_generate_benchmark,
199 [(instruction, prefix, output_path, target, size, distance)
200 for instruction in instructions]))
201
202
203def _validate_instructions(instructions, target):
204 """
205 Validate the provided instruction for a given target
206 """
207
208 nins = []
209 for instruction in instructions:
210
211 if instruction not in list(target.isa.instructions.keys()):
212 print_warning("'%s' not defined in the ISA. Skipping..." %
213 instruction)
214 continue
215 nins.append(instruction)
216 return nins
217
218
219def _generate_instructions(target, path, prefix):
220 """
221 Generate the list of instruction to be generated for a given target
222 """
223
224 instructions = []
225 for name, instr in target.instructions.items():
226
227 if instr.privileged or instr.hypervisor:
228 # Skip priv/hyper instructions
229 continue
230
231 if instr.branch and not instr.branch_relative:
232 # Skip branch absolute due to relocation problems
233 continue
234
235 if instr.category in ['LMA', 'LMV', 'DS', 'EC']:
236 # Skip some instruction categories
237 continue
238
239 if name in [
240 'LSWI_V0', 'LSWX_V0', 'LMW_V0', 'STSWX_V0', 'LD_V1', 'LWZ_V1',
241 'STW_V1'
242 ]:
243 # Some instructions are not completely supported yet
244 # String-related instructions and load multiple
245
246 continue
247
248 # Skip if the files already exists
249
250 fname = "%s/%s_%s.c" % (path, prefix, name)
251 ffname = "%s/%s_%s.c.fail" % (path, prefix, name)
252
253 if os.path.isfile(fname):
254 print_warning("Skip %s. '%s' already generated" % (name, fname))
255 continue
256
257 if os.path.isfile(ffname):
258 print_warning("Skip %s. '%s' already generated (failed)" %
259 (name, ffname))
260 continue
261
262 instructions.append(name)
263
264 return instructions
265
266
267def _generate_benchmark(args):
268 """
269 Actual benchmark generation policy. This is the function that defines
270 how the microbenchmark are going to be generated
271 """
272
273 instr_name, prefix, output_path, target, size, distance = args
274
275 try:
276
277 # Name of the output file
278 fname = "%s/%s_%s" % (output_path, prefix, instr_name)
279
280 # Name of the fail output file (generated in case of exception)
281 ffname = "%s.c.fail" % (fname)
282
283 print_info("Generating %s ..." % (fname))
284
285 instruction = microprobe.code.ins.Instruction()
286 instruction.set_arch_type(target.instructions[instr_name])
287 sequence = [target.instructions[instr_name]]
288
289 # Get the wrapper object. The wrapper object is in charge of
290 # translating the internal representation of the microbenchmark
291 # to the final output format.
292 #
293 # In this case, we obtain the 'CInfGen' wrapper, which embeds
294 # the generated code within an infinite loop using C plus
295 # in-line assembly statements.
296 cwrapper = microprobe.code.get_wrapper("CInfGen")
297
298 # Create the synthesizer object, which is in charge of driving the
299 # generation of the microbenchmark, given a set of passes
300 # (a.k.a. transformations) to apply to the an empty internal
301 # representation of the microbenchmark
302 synth = microprobe.code.Synthesizer(target,
303 cwrapper(),
304 value=0b01010101)
305
306 # Add the transformation passes
307
308 #######################################################################
309 # Pass 1: Init integer registers to a given value #
310 #######################################################################
311 synth.add_pass(
312 microprobe.passes.initialization.InitializeRegistersPass(
313 value=_init_value()))
314 floating = False
315 vector = False
316
317 for operand in instruction.operands():
318 if operand.type.immediate:
319 continue
320
321 if operand.type.float:
322 floating = True
323
324 if operand.type.vector:
325 vector = True
326
327 if vector and floating:
328 ###################################################################
329 # Pass 1.A: if instruction uses vector floats, init vector #
330 # registers to float values #
331 ###################################################################
332 synth.add_pass(
333 microprobe.passes.initialization.InitializeRegistersPass(
334 v_value=(1.000000000000001, 64)))
335 elif vector:
336 ###################################################################
337 # Pass 1.B: if instruction uses vector but not floats, init #
338 # vector registers to integer value #
339 ###################################################################
340 synth.add_pass(
341 microprobe.passes.initialization.InitializeRegistersPass(
342 v_value=(_init_value(), 64)))
343 elif floating:
344 ###################################################################
345 # Pass 1.C: if instruction uses floats, init float #
346 # registers to float values #
347 ###################################################################
348 synth.add_pass(
349 microprobe.passes.initialization.InitializeRegistersPass(
350 fp_value=1.000000000000001))
351
352 #######################################################################
353 # Pass 2: Add a building block of size 'size' #
354 #######################################################################
355 synth.add_pass(
356 microprobe.passes.structure.SimpleBuildingBlockPass(size))
357
358 #######################################################################
359 # Pass 3: Fill the building block with the instruction sequence #
360 #######################################################################
361 synth.add_pass(
362 microprobe.passes.instruction.SetInstructionTypeBySequencePass(
363 sequence))
364
365 #######################################################################
366 # Pass 4: Compute addresses of instructions (this pass is needed to #
367 # update the internal representation information so that in #
368 # case addresses are required, they are up to date). #
369 #######################################################################
370 synth.add_pass(
371 microprobe.passes.address.UpdateInstructionAddressesPass())
372
373 #######################################################################
374 # Pass 5: Set target of branches to be the next instruction in the #
375 # instruction stream #
376 #######################################################################
377 synth.add_pass(microprobe.passes.branch.BranchNextPass())
378
379 #######################################################################
380 # Pass 6: Set memory-related operands to access 16 storage locations #
381 # in a round-robin fashion in stride 256 bytes. #
382 # The pattern would be: 0, 256, 512, .... 3840, 0, 256, ... #
383 #######################################################################
384 synth.add_pass(microprobe.passes.memory.SingleMemoryStreamPass(
385 16, 256))
386
387 #######################################################################
388 # Pass 7.A: Initialize the storage locations accessed by floating #
389 # point instructions to have a valid floating point value #
390 #######################################################################
391 synth.add_pass(
392 microprobe.passes.float.InitializeMemoryFloatPass(
393 value=1.000000000000001))
394
395 #######################################################################
396 # Pass 7.B: Initialize the storage locations accessed by decimal #
397 # instructions to have a valid decimal value #
398 #######################################################################
399 synth.add_pass(
400 microprobe.passes.decimal.InitializeMemoryDecimalPass(value=1))
401
402 #######################################################################
403 # Pass 8: Set the remaining instructions operands (if not set) #
404 # (Required to set remaining immediate operands) #
405 #######################################################################
406 synth.add_pass(
407 microprobe.passes.register.DefaultRegisterAllocationPass(
408 dd=distance))
409
410 # Synthesize the microbenchmark.The synthesize applies the set of
411 # transformation passes added before and returns object representing
412 # the microbenchmark
413 bench = synth.synthesize()
414
415 # Save the microbenchmark to the file 'fname'
416 synth.save(fname, bench=bench)
417
418 print_info("%s generated!" % (fname))
419
420 # Remove fail file if exists
421 if os.path.isfile(ffname):
422 os.remove(ffname)
423
424 except MicroprobeException:
425
426 # In case of exception during the generation of the microbenchmark,
427 # print the error, write the fail file and exit
428 print_error(traceback.format_exc())
429 open(ffname, 'a').close()
430 exit(-1)
431
432
433def _init_value():
434 """ Return a init value """
435 return 0b0101010101010101010101010101010101010101010101010101010101010101
436
437
438# Main
439if __name__ == '__main__':
440 # run main if executed from the command line
441 # and the main method exists
442
443 if callable(locals().get('main_setup')):
444 main_setup()
445 exit(0)
The code is self-documented. You can take a look to understand the basic concepts of the code generation in Microprobe. In order to help the readers, let us summarize and elaborate the explanations in the code. The following are the suggested steps required to implement a command line tool to generate microbenchmarks using Microprobe:
Define the command line interface and parameters (
main_setup()function in the example). This includes:Create a command line interface object
Define parameters using the
add_optioninterfaceCall the actual main with the arguments
Define the function to process the input parameters (
_main()function in the example). This includes:Import target definition
Get processed arguments
Validate and use the arguments to call the actual microbenchmark generation function
Define the function to generate the microbenchmark (
_generate_benchmarkfunction in the example). The main elements are the following:Get the wrapper object. The wrapper object defines the general characteristics of code being generated (i.e. how the internal representation will be translated to the final file being generated). General characteristics are, for instance, code prologs such as
#include <header.h>directives, the main function declaration, epilogs, etc. In this case, the wrapper selected is theCInfGen. This wrapper generates C code with an infinite loop of instructions. This results in the following code:#include <stdio.h> #include <string.h> // <declaration of variables> int main(int argc, char** argv, char** envp) { // <initialization_code> while(1) { // <generated_code> } // end while }
The user can subclass or define their own wrappers to fulfill their needs. See
microprobe.code.wrapper.Wrapperfor more details.Instantiate synthesizer. The benchmark synthesizer object is in charge of driving the code generation object by applying the set of transformation passes defined by the user.
Define the transformation passes. The transformation passes will fill the
declaration of variables,<initialization_code>and<generated_code>sections of the previous code block. Depending on the order and the type of passes applied, the code generated will be different. The user has plenty of transformation passes to apply. Seemicroprobe.passesand all its submodules for further details. Also, the use can define its own passes by subclassing the classmicroprobe.passes.Pass.Finally, once the generation policy is defined, the user only has to synthesize the benchmark and save it to a file.
power_v206_power7_ppc64_linux_gcc_fu_stress.py
The following example shows how to generate microbenchmarks that stress a particular functional unit of the architecture. The code is self explanatory:
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_fu_stress.py
17
18Example module to show how to generate a benchmark stressing a particular
19functional unit of the microarchitecture at different rate using the
20average latency of instructions as well as the average dependency distance
21between the instructions
22"""
23
24# Futures
25from __future__ import absolute_import
26
27# Built-in modules
28import os
29import sys
30import traceback
31
32# Own modules
33import microprobe.code.ins
34import microprobe.passes.address
35import microprobe.passes.branch
36import microprobe.passes.decimal
37import microprobe.passes.float
38import microprobe.passes.ilp
39import microprobe.passes.initialization
40import microprobe.passes.instruction
41import microprobe.passes.memory
42import microprobe.passes.register
43import microprobe.passes.structure
44import microprobe.utils.cmdline
45from microprobe.exceptions import MicroprobeException, \
46 MicroprobeTargetDefinitionError
47from microprobe.target import import_definition
48from microprobe.utils.cmdline import dict_key, existing_dir, \
49 float_type, int_type, print_error, print_info
50from microprobe.utils.logger import get_logger
51
52__author__ = "Ramon Bertran"
53__copyright__ = "Copyright 2011-2021 IBM Corporation"
54__credits__ = []
55__license__ = "IBM (c) 2011-2021 All rights reserved"
56__version__ = "0.5"
57__maintainer__ = "Ramon Bertran"
58__email__ = "rbertra@us.ibm.com"
59__status__ = "Development" # "Prototype", "Development", or "Production"
60
61# Constants
62LOG = get_logger(__name__) # Get the generic logging interface
63
64
65# Functions
66def main_setup():
67 """
68 Set up the command line interface (CLI) with the arguments required by
69 this command line tool.
70 """
71
72 args = sys.argv[1:]
73
74 # Get the target definition
75 try:
76 target = import_definition("power_v206-power7-ppc64_linux_gcc")
77 except MicroprobeTargetDefinitionError as exc:
78 print_error("Unable to import target definition")
79 print_error("Exception message: %s" % str(exc))
80 exit(-1)
81
82 func_units = {}
83 valid_units = [elem.name for elem in target.elements.values()]
84
85 for instr in target.isa.instructions.values():
86 if instr.execution_units == "None":
87 LOG.debug("Execution units for: '%s' not defined", instr.name)
88 continue
89
90 for unit in instr.execution_units:
91 if unit not in valid_units:
92 continue
93
94 if unit not in func_units:
95 func_units[unit] = [
96 elem for elem in target.elements.values()
97 if elem.name == unit
98 ][0]
99
100 # Create the CLI interface object
101 cmdline = microprobe.utils.cmdline.CLI("ISA power v206 profile example",
102 config_options=False,
103 target_options=False,
104 debug_options=False)
105
106 # Add the different parameters for this particular tool
107 cmdline.add_option("functional_unit",
108 "f", [func_units['ALU']],
109 "Functional units to stress. Default: ALU",
110 required=False,
111 nargs="+",
112 choices=func_units,
113 opt_type=dict_key(func_units),
114 metavar="FUNCTIONAL_UNIT_NAME")
115
116 cmdline.add_option(
117 "output_prefix",
118 None,
119 "POWER_V206_FU_STRESS",
120 "Output prefix of the generated files. Default: POWER_V206_FU_STRESS",
121 opt_type=str,
122 required=False,
123 metavar="PREFIX")
124
125 cmdline.add_option("output_path",
126 "O",
127 "./",
128 "Output path. Default: current path",
129 opt_type=existing_dir,
130 metavar="PATH")
131
132 cmdline.add_option(
133 "size",
134 "S",
135 64, "Benchmark size (number of instructions in the endless loop). "
136 "Default: 64 instructions",
137 opt_type=int_type(1, 2**20),
138 metavar="BENCHMARK_SIZE")
139
140 cmdline.add_option("dependency_distance",
141 "D",
142 1000,
143 "Average dependency distance between the instructions. "
144 "Default: 1000 (no dependencies)",
145 opt_type=int_type(1, 1000),
146 metavar="DEPENDECY_DISTANCE")
147
148 cmdline.add_option("average_latency",
149 "L",
150 2, "Average latency of the selected instructins. "
151 "Default: 2 cycles",
152 opt_type=float_type(1, 1000),
153 metavar="AVERAGE_LATENCY")
154
155 # Start the main
156 print_info("Processing input arguments...")
157 cmdline.main(args, _main)
158
159
160def _main(arguments):
161 """
162 Main program. Called after the arguments from the CLI interface have
163 been processed.
164 """
165
166 print_info("Arguments processed!")
167
168 print_info("Importing target definition "
169 "'power_v206-power7-ppc64_linux_gcc'...")
170 target = import_definition("power_v206-power7-ppc64_linux_gcc")
171
172 # Get the arguments
173 functional_units = arguments["functional_unit"]
174 prefix = arguments["output_prefix"]
175 output_path = arguments["output_path"]
176 size = arguments["size"]
177 latency = arguments["average_latency"]
178 distance = arguments["dependency_distance"]
179
180 if functional_units is None:
181 functional_units = ["ALL"]
182
183 _generate_benchmark(target, "%s/%s_" % (output_path, prefix),
184 (functional_units, size, latency, distance))
185
186
187def _generate_benchmark(target, output_prefix, args):
188 """
189 Actual benchmark generation policy. This is the function that defines
190 how the microbenchmark are going to be generated
191 """
192
193 functional_units, size, latency, distance = args
194
195 try:
196
197 # Name of the output file
198 func_unit_names = [unit.name for unit in functional_units]
199 fname = "%s%s" % (output_prefix, "_".join(func_unit_names))
200 fname = "%s_LAT_%s" % (fname, latency)
201 fname = "%s_DEP_%s" % (fname, distance)
202
203 # Name of the fail output file (generated in case of exception)
204 ffname = "%s.c.fail" % (fname)
205
206 print_info("Generating %s ..." % (fname))
207
208 # Get the wrapper object. The wrapper object is in charge of
209 # translating the internal representation of the microbenchmark
210 # to the final output format.
211 #
212 # In this case, we obtain the 'CInfGen' wrapper, which embeds
213 # the generated code within an infinite loop using C plus
214 # in-line assembly statements.
215 cwrapper = microprobe.code.get_wrapper("CInfGen")
216
217 # Create the synthesizer object, which is in charge of driving the
218 # generation of the microbenchmark, given a set of passes
219 # (a.k.a. transformations) to apply to the an empty internal
220 # representation of the microbenchmark
221 synth = microprobe.code.Synthesizer(target,
222 cwrapper(),
223 value=0b01010101)
224
225 # Add the transformation passes
226
227 #######################################################################
228 # Pass 1: Init integer registers to a given value #
229 #######################################################################
230 synth.add_pass(
231 microprobe.passes.initialization.InitializeRegistersPass(
232 value=_init_value()))
233
234 #######################################################################
235 # Pass 2: Add a building block of size 'size' #
236 #######################################################################
237 synth.add_pass(
238 microprobe.passes.structure.SimpleBuildingBlockPass(size))
239
240 #######################################################################
241 # Pass 3: Fill the building block with the instruction sequence #
242 #######################################################################
243 synth.add_pass(
244 microprobe.passes.instruction.SetInstructionTypeByElementPass(
245 target, functional_units, {}))
246
247 #######################################################################
248 # Pass 4: Compute addresses of instructions (this pass is needed to #
249 # update the internal representation information so that in #
250 # case addresses are required, they are up to date). #
251 #######################################################################
252 synth.add_pass(
253 microprobe.passes.address.UpdateInstructionAddressesPass())
254
255 #######################################################################
256 # Pass 5: Set target of branches to be the next instruction in the #
257 # instruction stream #
258 #######################################################################
259 synth.add_pass(microprobe.passes.branch.BranchNextPass())
260
261 #######################################################################
262 # Pass 6: Set memory-related operands to access 16 storage locations #
263 # in a round-robin fashion in stride 256 bytes. #
264 # The pattern would be: 0, 256, 512, .... 3840, 0, 256, ... #
265 #######################################################################
266 synth.add_pass(microprobe.passes.memory.SingleMemoryStreamPass(
267 16, 256))
268
269 #######################################################################
270 # Pass 7.A: Initialize the storage locations accessed by floating #
271 # point instructions to have a valid floating point value #
272 #######################################################################
273 synth.add_pass(
274 microprobe.passes.float.InitializeMemoryFloatPass(
275 value=1.000000000000001))
276
277 #######################################################################
278 # Pass 7.B: Initialize the storage locations accessed by decimal #
279 # instructions to have a valid decimal value #
280 #######################################################################
281 synth.add_pass(
282 microprobe.passes.decimal.InitializeMemoryDecimalPass(value=1))
283
284 #######################################################################
285 # Pass 8: Set the remaining instructions operands (if not set) #
286 # (Required to set remaining immediate operands) #
287 #######################################################################
288 synth.add_pass(
289 microprobe.passes.register.DefaultRegisterAllocationPass(
290 dd=distance))
291
292 # Synthesize the microbenchmark.The synthesize applies the set of
293 # transformation passes added before and returns object representing
294 # the microbenchmark
295 bench = synth.synthesize()
296
297 # Save the microbenchmark to the file 'fname'
298 synth.save(fname, bench=bench)
299
300 print_info("%s generated!" % (fname))
301
302 # Remove fail file if exists
303 if os.path.isfile(ffname):
304 os.remove(ffname)
305
306 except MicroprobeException:
307
308 # In case of exception during the generation of the microbenchmark,
309 # print the error, write the fail file and exit
310 print_error(traceback.format_exc())
311 open(ffname, 'a').close()
312 exit(-1)
313
314
315def _init_value():
316 """ Return a init value """
317 return 0b0101010101010101010101010101010101010101010101010101010101010101
318
319
320# Main
321if __name__ == '__main__':
322 # run main if executed from the command line
323 # and the main method exists
324
325 if callable(locals().get('main_setup')):
326 main_setup()
327 exit(0)
power_v206_power7_ppc64_linux_gcc_memory.py
The following example shows how to create microbenchmarks with different activity (stress levels) on the different levels of the cache hierarchy. Note that it is not necessary to use the built-in command line interface provided by Microprobe, as the example shows.
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_memory.py
17
18Example python script to show how to generate microbenchmarks with particular
19levels of activity in the memory hierarchy.
20"""
21
22# Futures
23from __future__ import absolute_import
24
25# Built-in modules
26import multiprocessing as mp
27import os
28import random
29import sys
30from typing import List, Tuple
31
32# Own modules
33import microprobe.code
34import microprobe.passes.address
35import microprobe.passes.ilp
36import microprobe.passes.initialization
37import microprobe.passes.instruction
38import microprobe.passes.memory
39import microprobe.passes.register
40import microprobe.passes.structure
41from microprobe import MICROPROBE_RC
42from microprobe.exceptions import MicroprobeTargetDefinitionError
43from microprobe.model.memory import EndlessLoopDataMemoryModel
44from microprobe.target import import_definition
45from microprobe.target.isa.instruction import InstructionType
46from microprobe.target.uarch.cache import SetAssociativeCache
47from microprobe.utils.cmdline import print_error, print_info
48from microprobe.utils.typeguard_decorator import typeguard_testsuite
49
50__author__ = "Ramon Bertran"
51__copyright__ = "Copyright 2011-2021 IBM Corporation"
52__credits__ = []
53__license__ = "IBM (c) 2011-2021 All rights reserved"
54__version__ = "0.5"
55__maintainer__ = "Ramon Bertran"
56__email__ = "rbertra@us.ibm.com"
57__status__ = "Development" # "Prototype", "Development", or "Production"
58
59# Get the target definition
60try:
61 TARGET = import_definition("power_v206-power7-ppc64_linux_gcc")
62except MicroprobeTargetDefinitionError as exc:
63 print_error("Unable to import target definition")
64 print_error("Exception message: %s" % str(exc))
65 exit(-1)
66
67assert TARGET.microarchitecture is not None, \
68 "Target must have a defined microarchitecture"
69
70BASE_ELEMENT = [
71 element for element in TARGET.microarchitecture.elements.values()
72 if element.name == 'L1D'
73][0]
74CACHE_HIERARCHY: List[SetAssociativeCache] = \
75 TARGET.microarchitecture.cache_hierarchy.get_data_hierarchy_from_element(
76 BASE_ELEMENT)
77
78# Benchmark size
79BENCHMARK_SIZE = 8 * 1024
80
81# Fill a list of the models to be generated
82
83MEMORY_MODELS: List[Tuple[str, List[SetAssociativeCache], List[int]]] = []
84
85#
86# Due to performance issues (long exec. time) this
87# model is disabled
88#
89# MEMORY_MODELS.append(
90# (
91# "ALL", CACHE_HIERARCHY, [
92# 25, 25, 25, 25]))
93
94MEMORY_MODELS.append(("L1", CACHE_HIERARCHY, [100, 0, 0, 0]))
95MEMORY_MODELS.append(("L2", CACHE_HIERARCHY, [0, 100, 0, 0]))
96MEMORY_MODELS.append(("L3", CACHE_HIERARCHY, [0, 0, 100, 0]))
97MEMORY_MODELS.append(("L1L3", CACHE_HIERARCHY, [50, 0, 50, 0]))
98MEMORY_MODELS.append(("L1L2", CACHE_HIERARCHY, [50, 50, 0, 0]))
99MEMORY_MODELS.append(("L2L3", CACHE_HIERARCHY, [0, 50, 50, 0]))
100MEMORY_MODELS.append(("CACHES", CACHE_HIERARCHY, [33, 33, 34, 0]))
101MEMORY_MODELS.append(("MEM", CACHE_HIERARCHY, [0, 0, 0, 100]))
102
103# Enable parallel generation
104PARALLEL = False
105
106DIRECTORY = None
107
108
109@typeguard_testsuite
110def main():
111 """Main function. """
112 # call the generate method for each model in the memory model list
113
114 if PARALLEL:
115 print_info("Start parallel execution...")
116 pool = mp.Pool(processes=MICROPROBE_RC['cpus'])
117 pool.map(generate, MEMORY_MODELS, 1)
118 else:
119 print_info("Start sequential execution...")
120 list(map(generate, MEMORY_MODELS))
121
122 exit(0)
123
124
125@typeguard_testsuite
126def generate(model: Tuple[str, List[SetAssociativeCache], List[int]]):
127 """Benchmark generation policy function. """
128
129 assert DIRECTORY is not None, "DIRECTORY variable cannot be None"
130
131 print_info(f"Creating memory model '{model[0]}' ...")
132 memmodel = EndlessLoopDataMemoryModel(*model)
133
134 modelname = memmodel.name
135
136 print_info(f"Generating Benchmark mem-{modelname} ...")
137
138 # Get the architecture
139 garch = TARGET
140
141 # For all the supported instructions, get the memory operations,
142 sequence: List[InstructionType] = []
143 for instr_name in sorted(garch.isa.instructions.keys()):
144
145 instr = garch.isa.instructions[instr_name]
146
147 if not instr.access_storage:
148 continue
149 if instr.privileged: # Skip privileged
150 continue
151 if instr.hypervisor: # Skip hypervisor
152 continue
153 if instr.trap: # Skip traps
154 continue
155 if "String" in instr.description: # Skip unsupported string instr.
156 continue
157 if "Multiple" in instr.description: # Skip unsupported mult. ld/sts
158 continue
159 if instr.category in ['LMA', 'LMV', 'DS', 'EC',
160 'WT']: # Skip unsupported categories
161 continue
162 if instr.access_storage_with_update: # Not supported by mem. model
163 continue
164 if "Reserve Indexed" in instr.description: # Skip (illegal intr.)
165 continue
166 if "Conditional Indexed" in instr.description: # Skip (illegal intr.)
167 continue
168 if instr.name in ['LD_V1', 'LWZ_V1', 'STW_V1']:
169 continue
170
171 sequence.append(instr)
172
173 # Get the loop wrapper. In this case we take the 'CInfPpc', which
174 # generates an infinite loop in C using PowerPC embedded assembly.
175 cwrapper = microprobe.code.get_wrapper("CInfPpc")
176
177 # Define function to return random numbers (used afterwards)
178 def rnd():
179 """Return a random value. """
180 return random.randrange(0, (1 << 64) - 1)
181
182 # Create the benchmark synthesizer
183 synth = microprobe.code.Synthesizer(garch, cwrapper())
184
185 ##########################################################################
186 # Add the passes we want to apply to synthesize benchmarks #
187 ##########################################################################
188
189 # --> Init registers to random values
190 synth.add_pass(
191 microprobe.passes.initialization.InitializeRegistersPass(value=rnd))
192
193 # --> Add a single basic block of size 'size'
194 if memmodel.name in ['MEM']:
195 synth.add_pass(
196 microprobe.passes.structure.SimpleBuildingBlockPass(
197 BENCHMARK_SIZE * 4))
198 else:
199 synth.add_pass(
200 microprobe.passes.structure.SimpleBuildingBlockPass(
201 BENCHMARK_SIZE))
202
203 # --> Fill the basic block using the sequence of instructions provided
204 synth.add_pass(
205 microprobe.passes.instruction.SetInstructionTypeBySequencePass(
206 sequence))
207
208 # --> Set the memory operations parameters to fulfill the given model
209 synth.add_pass(microprobe.passes.memory.GenericMemoryModelPass(memmodel))
210
211 # --> Set the dependency distance and the default allocation. Sets the
212 # remaining undefined instruction operands (register allocation,...)
213 synth.add_pass(microprobe.passes.register.NoHazardsAllocationPass())
214 synth.add_pass(
215 microprobe.passes.register.DefaultRegisterAllocationPass(dd=0))
216
217 # Generate the benchmark (applies the passes).
218 bench = synth.synthesize()
219
220 print_info(f"Benchmark mem-{modelname} saving to disk...")
221
222 # Save the benchmark
223 synth.save(f"{DIRECTORY}/mem-{modelname}", bench=bench)
224
225 print_info(f"Benchmark mem-{modelname} generated")
226 return True
227
228
229if __name__ == '__main__':
230 # run main if executed from the command line
231 # and the main method exists
232
233 if len(sys.argv) != 2:
234 print_info("Usage:")
235 print_info("%s output_dir" % (sys.argv[0]))
236 exit(-1)
237
238 DIRECTORY = sys.argv[1]
239
240 if not os.path.isdir(DIRECTORY):
241 print_error(f"Output directory '{DIRECTORY}' does not exists")
242 exit(-1)
243
244 main()
power_v206_power7_ppc64_linux_gcc_random.py
The following example generates random microbenchmarks:
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_memory.py
17
18Example python script to show how to generate random microbenchmarks.
19"""
20
21# Futures
22from __future__ import absolute_import
23
24# Built-in modules
25import multiprocessing as mp
26import os
27import random
28import sys
29from typing import List
30
31# Own modules
32import microprobe.code
33import microprobe.passes.address
34import microprobe.passes.branch
35import microprobe.passes.ilp
36import microprobe.passes.initialization
37import microprobe.passes.instruction
38import microprobe.passes.memory
39import microprobe.passes.register
40import microprobe.passes.structure
41from microprobe import MICROPROBE_RC
42from microprobe.exceptions import MicroprobeError, \
43 MicroprobeTargetDefinitionError
44from microprobe.model.memory import EndlessLoopDataMemoryModel
45from microprobe.target import import_definition
46from microprobe.target.isa.instruction import InstructionType
47from microprobe.utils.cmdline import print_error, print_info
48from microprobe.utils.typeguard_decorator import typeguard_testsuite
49
50__author__ = "Ramon Bertran"
51__copyright__ = "Copyright 2011-2021 IBM Corporation"
52__credits__ = []
53__license__ = "IBM (c) 2011-2021 All rights reserved"
54__version__ = "0.5"
55__maintainer__ = "Ramon Bertran"
56__email__ = "rbertra@us.ibm.com"
57__status__ = "Development" # "Prototype", "Development", or "Production"
58
59# Benchmark size
60BENCHMARK_SIZE = 8 * 1024
61
62# Get the target definition
63try:
64 TARGET = import_definition("power_v206-power7-ppc64_linux_gcc")
65except MicroprobeTargetDefinitionError as exc:
66 print_error("Unable to import target definition")
67 print_error("Exception message: %s" % str(exc))
68 exit(-1)
69
70assert TARGET.microarchitecture is not None, \
71 "Target must have a defined microarchitecture"
72BASE_ELEMENT = [
73 element for element in TARGET.microarchitecture.elements.values()
74 if element.name == 'L1D'
75][0]
76CACHE_HIERARCHY = \
77 TARGET.microarchitecture.cache_hierarchy.get_data_hierarchy_from_element(
78 BASE_ELEMENT)
79
80PARALLEL = True
81
82DIRECTORY = None
83
84
85@typeguard_testsuite
86def main():
87 """ Main program. """
88 if PARALLEL:
89 pool = mp.Pool(processes=MICROPROBE_RC['cpus'])
90 pool.map(generate, list(range(0, 100)), 1)
91 else:
92 list(map(generate, list(range(0, 100))))
93
94
95@typeguard_testsuite
96def generate(name: str):
97 """ Benchmark generation policy. """
98
99 assert DIRECTORY is not None, "DIRECTORY variable cannot be None"
100
101 if os.path.isfile(f"{DIRECTORY}/random-{name}.c"):
102 print_info(f"Skip {name}")
103 return
104
105 print_info(f"Generating {name}...")
106
107 # Seed the randomness
108 rand = random.Random()
109 rand.seed(64) # My favorite number ;)
110
111 # Generate a random memory model (used afterwards)
112 model: List[int] = []
113 total = 100
114 for mcomp in CACHE_HIERARCHY[0:-1]:
115 weight = rand.randint(0, total)
116 model.append(weight)
117 print_info("%s: %d%%" % (mcomp, weight))
118 total = total - weight
119
120 # Fix remaining
121 level = rand.randint(0, len(CACHE_HIERARCHY[0:-1]) - 1)
122 model[level] += total
123
124 # Last level always zero
125 model.append(0)
126
127 # Sanity check
128 psum = 0
129 for elem in model:
130 psum += elem
131 assert psum == 100
132
133 modelobj = EndlessLoopDataMemoryModel("random-%s", CACHE_HIERARCHY, model)
134
135 # Get the loop wrapper. In this case we take the 'CInfPpc', which
136 # generates an infinite loop in C using PowerPC embedded assembly.
137 cwrapper = microprobe.code.get_wrapper("CInfPpc")
138
139 # Define function to return random numbers (used afterwards)
140 def rnd():
141 """Return a random value. """
142 return rand.randrange(0, (1 << 64) - 1)
143
144 # Create the benchmark synthesizer
145 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
146
147 ##########################################################################
148 # Add the passes we want to apply to synthesize benchmarks #
149 ##########################################################################
150
151 # --> Init registers to random values
152 synth.add_pass(
153 microprobe.passes.initialization.InitializeRegistersPass(value=rnd))
154
155 # --> Add a single basic block of size size
156 synth.add_pass(
157 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
158
159 # --> Fill the basic block with instructions picked randomly from the list
160 # provided
161
162 instructions: List[InstructionType] = []
163 for instr in TARGET.isa.instructions.values():
164
165 if instr.privileged: # Skip privileged
166 continue
167 if instr.hypervisor: # Skip hypervisor
168 continue
169 if instr.trap: # Skip traps
170 continue
171 if instr.syscall: # Skip syscall
172 continue
173 if "String" in instr.description: # Skip unsupported string instr.
174 continue
175 if "Multiple" in instr.description: # Skip unsupported mult. ld/sts
176 continue
177 if instr.category in ['LMA', 'LMV', 'DS', 'EC',
178 'WT']: # Skip unsupported categories
179 continue
180 if instr.access_storage_with_update: # Not supported by mem. model
181 continue
182 if instr.branch and not instr.branch_relative: # Skip branches
183 continue
184 if "Reserve Indexed" in instr.description: # Skip (illegal intr.)
185 continue
186 if "Conitional Indexed" in instr.description: # Skip (illegal intr.)
187 continue
188 if instr.name in [
189 'LD_V1',
190 'LWZ_V1',
191 'STW_V1',
192 ]:
193 continue
194
195 instructions.append(instr)
196
197 synth.add_pass(
198 microprobe.passes.instruction.SetRandomInstructionTypePass(
199 instructions, rand))
200
201 # --> Set the memory operations parameters to fulfill the given model
202 synth.add_pass(microprobe.passes.memory.GenericMemoryModelPass(modelobj))
203
204 # --> Set target of branches to next instruction (first compute addresses)
205 synth.add_pass(microprobe.passes.address.UpdateInstructionAddressesPass())
206 synth.add_pass(microprobe.passes.branch.BranchNextPass())
207
208 # --> Set the dependency distance and the default allocation. Dependency
209 # distance is randomly picked
210 synth.add_pass(
211 microprobe.passes.register.DefaultRegisterAllocationPass(
212 dd=rand.randint(1, 20)))
213
214 # Generate the benchmark (applies the passes)
215 # Since it is a randomly generated code, the generation might fail
216 # (e.g. not enough access to fulfill the requested memory model, etc.)
217 # Because of that, we handle the exception accordingly.
218 try:
219 print_info(f"Synthesizing {name}...")
220 bench = synth.synthesize()
221 print_info(f"Synthesized {name}!")
222 # Save the benchmark
223 synth.save(f"{DIRECTORY}/random-{name}", bench=bench)
224 except MicroprobeError:
225 print_info(f"Synthesizing error in '{name}'. This is Ok.")
226
227 return True
228
229
230if __name__ == '__main__':
231 # run main if executed from the command line
232 # and the main method exists
233
234 if len(sys.argv) != 2:
235 print_info("Usage:")
236 print_info("%s output_dir" % (sys.argv[0]))
237 exit(-1)
238
239 DIRECTORY = sys.argv[1]
240
241 if not os.path.isdir(DIRECTORY):
242 print_error(f"Output directory '{DIRECTORY}' does not exists")
243 exit(-1)
244
245 if callable(locals().get('main')):
246 main()
power_v206_power7_ppc64_linux_gcc_custom.py
The following example shows different examples on how to customize the generation of microbenchmarks:
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_custom.py
17
18Example python script to show how to generate random microbenchmarks.
19"""
20
21# Futures
22from __future__ import absolute_import
23
24# Built-in modules
25import os
26import sys
27
28# Own modules
29import microprobe.code
30import microprobe.passes.initialization
31import microprobe.passes.instruction
32import microprobe.passes.memory
33import microprobe.passes.register
34import microprobe.passes.structure
35from microprobe.exceptions import MicroprobeTargetDefinitionError
36from microprobe.model.memory import EndlessLoopDataMemoryModel
37from microprobe.target import import_definition
38from microprobe.utils.cmdline import print_error, print_info
39from microprobe.utils.misc import RNDINT
40
41__author__ = "Ramon Bertran"
42__copyright__ = "Copyright 2011-2021 IBM Corporation"
43__credits__ = []
44__license__ = "IBM (c) 2011-2021 All rights reserved"
45__version__ = "0.5"
46__maintainer__ = "Ramon Bertran"
47__email__ = "rbertra@us.ibm.com"
48__status__ = "Development" # "Prototype", "Development", or "Production"
49
50# Benchmark size
51BENCHMARK_SIZE = 8 * 1024
52
53if len(sys.argv) != 2:
54 print_info("Usage:")
55 print_info("%s output_dir" % (sys.argv[0]))
56 exit(-1)
57
58DIRECTORY = sys.argv[1]
59
60if not os.path.isdir(DIRECTORY):
61 print_info("Output DIRECTORY '%s' does not exists" % (DIRECTORY))
62 exit(-1)
63
64# Get the target definition
65try:
66 TARGET = import_definition("power_v206-power7-ppc64_linux_gcc")
67except MicroprobeTargetDefinitionError as exc:
68 print_error("Unable to import target definition")
69 print_error("Exception message: %s" % str(exc))
70 exit(-1)
71
72
73###############################################################################
74# Example 1: loop with instructions accessing storage , hitting the first #
75# level of cache and with dependency distance of 3 #
76###############################################################################
77def example_1():
78 """ Example 1 """
79 name = "L1-LOADS"
80
81 base_element = [
82 element for element in TARGET.elements.values()
83 if element.name == 'L1D'
84 ][0]
85 cache_hierarchy = TARGET.cache_hierarchy.get_data_hierarchy_from_element(
86 base_element)
87
88 model = [0] * len(cache_hierarchy)
89 model[0] = 100
90
91 mmodel = EndlessLoopDataMemoryModel("random-%s", cache_hierarchy, model)
92
93 profile = {}
94 for instr_name in sorted(TARGET.instructions.keys()):
95 instr = TARGET.instructions[instr_name]
96 if not instr.access_storage:
97 continue
98 if instr.privileged: # Skip privileged
99 continue
100 if instr.hypervisor: # Skip hypervisor
101 continue
102 if "String" in instr.description: # Skip unsupported string instr.
103 continue
104 if "ultiple" in instr.description: # Skip unsupported mult. ld/sts
105 continue
106 if instr.category in ['DS', 'LMA', 'LMV',
107 'EC']: # Skip unsupported categories
108 continue
109 if instr.access_storage_with_update: # Not supported
110 continue
111
112 if instr.name in [
113 'LD_V1',
114 'LWZ_V1',
115 'STW_V1',
116 ]:
117 continue
118
119 if (any([moper.is_load for moper in instr.memory_operand_descriptors])
120 and all([
121 not moper.is_store
122 for moper in instr.memory_operand_descriptors
123 ])):
124 profile[instr] = 1
125
126 cwrapper = microprobe.code.get_wrapper("CInfPpc")
127 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
128
129 synth.add_pass(
130 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
131 synth.add_pass(
132 microprobe.passes.initialization.InitializeRegistersPass(value=RNDINT))
133 synth.add_pass(
134 microprobe.passes.initialization.InitializeRegisterPass("GPR1",
135 0,
136 force=True,
137 reserve=True))
138 synth.add_pass(
139 microprobe.passes.instruction.SetInstructionTypeByProfilePass(profile))
140 synth.add_pass(microprobe.passes.memory.GenericMemoryModelPass(mmodel))
141 synth.add_pass(
142 microprobe.passes.register.DefaultRegisterAllocationPass(dd=3))
143
144 print_info("Generating %s..." % name)
145 bench = synth.synthesize()
146 print_info("%s Generated!" % name)
147 synth.save("%s/%s" % (DIRECTORY, name), bench=bench) # Save the benchmark
148
149
150###############################################################################
151# Example 2: loop with instructions using the MUL unit and with dependency #
152# distance of 4 #
153###############################################################################
154def example_2():
155 """ Example 2 """
156 name = "FXU-MUL"
157
158 cwrapper = microprobe.code.get_wrapper("CInfPpc")
159 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
160
161 synth.add_pass(
162 microprobe.passes.initialization.InitializeRegistersPass(value=RNDINT))
163 synth.add_pass(
164 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
165 synth.add_pass(
166 microprobe.passes.instruction.SetInstructionTypeByElementPass(
167 TARGET, [TARGET.elements['MUL_FXU0_Core0_SCM_Processor']], {}))
168 synth.add_pass(
169 microprobe.passes.register.DefaultRegisterAllocationPass(dd=4))
170
171 print_info("Generating %s..." % name)
172 bench = synth.synthesize()
173 print_info("%s Generated!" % name)
174 synth.save("%s/%s" % (DIRECTORY, name), bench=bench) # Save the benchmark
175
176
177###############################################################################
178# Example 3: loop with instructions using the ALU unit and with dependency #
179# distance of 1 #
180###############################################################################
181def example_3():
182 """ Example 3 """
183 name = "FXU-ALU"
184
185 cwrapper = microprobe.code.get_wrapper("CInfPpc")
186 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
187
188 synth.add_pass(
189 microprobe.passes.initialization.InitializeRegistersPass(value=RNDINT))
190 synth.add_pass(
191 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
192 synth.add_pass(
193 microprobe.passes.instruction.SetInstructionTypeByElementPass(
194 TARGET, [TARGET.elements['ALU_FXU0_Core0_SCM_Processor']], {}))
195 synth.add_pass(
196 microprobe.passes.register.DefaultRegisterAllocationPass(dd=1))
197
198 print_info("Generating %s..." % name)
199 bench = synth.synthesize()
200 print_info("%s Generated!" % name)
201 synth.save("%s/%s" % (DIRECTORY, name), bench=bench) # Save the benchmark
202
203
204###############################################################################
205# Example 4: loop with FMUL* instructions with different weights and with #
206# dependency distance 10 #
207###############################################################################
208def example_4():
209 """ Example 4 """
210 name = "VSU-FMUL"
211
212 profile = {}
213 profile[TARGET.instructions['FMUL_V0']] = 4
214 profile[TARGET.instructions['FMULS_V0']] = 3
215 profile[TARGET.instructions['FMULx_V0']] = 2
216 profile[TARGET.instructions['FMULSx_V0']] = 1
217
218 cwrapper = microprobe.code.get_wrapper("CInfPpc")
219 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
220
221 synth.add_pass(
222 microprobe.passes.initialization.InitializeRegistersPass(value=RNDINT))
223 synth.add_pass(
224 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
225 synth.add_pass(
226 microprobe.passes.instruction.SetInstructionTypeByProfilePass(profile))
227 synth.add_pass(
228 microprobe.passes.register.DefaultRegisterAllocationPass(dd=10))
229
230 print_info("Generating %s..." % name)
231 bench = synth.synthesize()
232 print_info("%s Generated!" % name)
233 synth.save("%s/%s" % (DIRECTORY, name), bench=bench) # Save the benchmark
234
235
236###############################################################################
237# Example 5: loop with FADD* instructions with different weights and with #
238# dependency distance 1 #
239###############################################################################
240def example_5():
241 """ Example 5 """
242 name = "VSU-FADD"
243
244 profile = {}
245 profile[TARGET.instructions['FADD_V0']] = 100
246 profile[TARGET.instructions['FADDx_V0']] = 1
247 profile[TARGET.instructions['FADDS_V0']] = 10
248 profile[TARGET.instructions['FADDSx_V0']] = 1
249
250 cwrapper = microprobe.code.get_wrapper("CInfPpc")
251 synth = microprobe.code.Synthesizer(TARGET, cwrapper())
252
253 synth.add_pass(
254 microprobe.passes.initialization.InitializeRegistersPass(value=RNDINT))
255 synth.add_pass(
256 microprobe.passes.structure.SimpleBuildingBlockPass(BENCHMARK_SIZE))
257 synth.add_pass(
258 microprobe.passes.instruction.SetInstructionTypeByProfilePass(profile))
259 synth.add_pass(
260 microprobe.passes.register.DefaultRegisterAllocationPass(dd=1))
261
262 print_info("Generating %s..." % name)
263 bench = synth.synthesize()
264 print_info("%s Generated!" % name)
265 synth.save("%s/%s" % (DIRECTORY, name), bench=bench) # Save the benchmark
266
267
268###############################################################################
269# Call the examples #
270###############################################################################
271example_1()
272example_2()
273example_3()
274example_4()
275example_5()
276exit(0)
power_v206_power7_ppc64_linux_gcc_genetic.py
Deprecated since version 0.5: Support for the PyEvolve and genetic algorithm based searches has been discontinued
The following example shows how to use the design exploration module and the genetic algorithm based searches to look for a solution. In particular, for each functional unit of the architecture and a range of IPCs (instruction per cycle), the example looks for a solution that stresses that functional unit at the given IPC. External commands (not included) are needed to evaluate the generated microbenchmarks in the target platform.
1#!/usr/bin/env python
2# Copyright 2011-2021 IBM Corporation
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8# http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""
16power_v206_power7_ppc64_linux_gcc_genetic.py
17
18Example python script to show how to generate a set of microbenchmark
19stressing a particular unit but at different IPC ratio using a genetic
20search algorithm to play with two knobs: average latency and dependency
21distance.
22
23An IPC evaluation and scoring script is required. For instance:
24
25.. code:: bash
26
27 #!/bin/bash
28 # ARGS: $1 is the target IPC
29 # $2 is the name of the generate benchnark
30 target_ipc=$1
31 source_bench=$2
32
33 # Compile the benchmark
34 gcc -O0 -mcpu=power7 -mtune=power7 -std=c99 $source_bench.c -o $source_bench
35
36 # Evaluate the ipc
37 ipc=< your preferred commands to evaluate the IPC >
38
39 # Compute the score (the closer to the target IPC the
40 score=(1/($ipc-$target_ipc))^2 | bc -l
41
42 echo $score
43
44Use the script above as a template for your own GA-based search.
45"""
46
47# Futures
48from __future__ import absolute_import, division
49
50# Built-in modules
51import datetime
52import os
53import sys
54import time as runtime
55from typing import List, Tuple
56
57# Own modules
58import microprobe.code
59import microprobe.driver.genetic
60import microprobe.passes.ilp
61import microprobe.passes.initialization
62import microprobe.passes.instruction
63import microprobe.passes.register
64import microprobe.passes.structure
65from microprobe.exceptions import MicroprobeTargetDefinitionError
66from microprobe.target import import_definition
67from microprobe.utils.cmdline import print_error, print_info, print_warning
68from microprobe.utils.misc import RNDINT
69from microprobe.utils.typeguard_decorator import typeguard_testsuite
70
71__author__ = "Ramon Bertran"
72__copyright__ = "Copyright 2011-2021 IBM Corporation"
73__credits__ = []
74__license__ = "IBM (c) 2011-2021 All rights reserved"
75__version__ = "0.5"
76__maintainer__ = "Ramon Bertran"
77__email__ = "rbertra@us.ibm.com"
78__status__ = "Development" # "Prototype", "Development", or "Production"
79
80# Benchmark size
81BENCHMARK_SIZE = 20
82
83COMMAND = None
84DIRECTORY = None
85
86# Get the target definition
87try:
88 TARGET = import_definition("power_v206-power7-ppc64_linux_gcc")
89except MicroprobeTargetDefinitionError as exc:
90 print_error("Unable to import target definition")
91 print_error("Exception message: %s" % str(exc))
92 exit(-1)
93
94
95@typeguard_testsuite
96def main():
97 """Main function."""
98
99 component_list = ["FXU", "FXU-noLSU", "FXU-LSU", "VSU", "VSU-FXU"]
100 ipcs = [float(x) / 10 for x in range(1, 41)]
101 ipcs = ipcs[5:] + ipcs[:5]
102
103 for name in component_list:
104 for ipc in ipcs:
105 generate_genetic(name, ipc)
106
107
108@typeguard_testsuite
109def generate_genetic(compname: str, ipc: float):
110 """Generate a microbenchmark stressing compname at the given ipc."""
111
112 assert COMMAND is not None, "COMMAND variable cannot be None"
113 assert DIRECTORY is not None, "DIRECTORY variable cannot be None"
114
115 comps = []
116 bcomps = []
117 any_comp: bool = False
118
119 assert TARGET.microarchitecture is not None, \
120 "Target must have a defined microarchitecture"
121
122 if compname.find("FXU") >= 0:
123 comps.append(
124 TARGET.microarchitecture.elements["FXU0_Core0_SCM_Processor"])
125
126 if compname.find("VSU") >= 0:
127 comps.append(
128 TARGET.microarchitecture.elements["VSU0_Core0_SCM_Processor"])
129
130 if len(comps) == 2:
131 any_comp = True
132 elif compname.find("noLSU") >= 0:
133 bcomps.append(
134 TARGET.microarchitecture.elements["LSU0_Core0_SCM_Processor"])
135 elif compname.find("LSU") >= 0:
136 comps.append(
137 TARGET.microarchitecture.elements["LSU_Core0_SCM_Processor"])
138
139 if (len(comps) == 1 and ipc > 2) or (len(comps) == 2 and ipc > 4):
140 return True
141
142 for elem in os.listdir(DIRECTORY):
143 if not elem.endswith(".c"):
144 continue
145 if elem.startswith("%s:IPC:%.2f:DIST" % (compname, ipc)):
146 print_info("Already generated: %s %d" % (compname, ipc))
147 return True
148
149 print_info(f"Going for IPC: {ipc} and Element: {compname}")
150
151 def generate(name: str, dist: float, latency: float):
152 """Benchmark generation function.
153
154 First argument is name, second the dependency distance and the
155 third is the average instruction latency.
156 """
157 wrapper = microprobe.code.get_wrapper("CInfPpc")
158 synth = microprobe.code.Synthesizer(TARGET, wrapper())
159 synth.add_pass(
160 microprobe.passes.initialization.InitializeRegistersPass(
161 value=RNDINT))
162 synth.add_pass(
163 microprobe.passes.structure.SimpleBuildingBlockPass(
164 BENCHMARK_SIZE))
165 synth.add_pass(
166 microprobe.passes.instruction.SetInstructionTypeByElementPass(
167 TARGET,
168 comps, {},
169 block=bcomps,
170 avelatency=latency,
171 any_comp=any_comp))
172 synth.add_pass(
173 microprobe.passes.register.DefaultRegisterAllocationPass(dd=dist))
174 bench = synth.synthesize()
175 synth.save(name, bench=bench)
176
177 # Set the genetic algorithm parameters
178 ga_params: List[Tuple[int, int, float]] = []
179 ga_params.append((0, 20, 0.05)) # Average dependency distance design space
180 ga_params.append((2, 8, 0.05)) # Average instruction latency design space
181
182 # Set up the search driver
183 driver = microprobe.driver.genetic.ExecCmdDriver(
184 generate, 20, 30, 30, f"'{COMMAND}' {ipc} ", ga_params)
185
186 starttime = runtime.time()
187 print_info("Start search...")
188 driver.run(1)
189 print_info("Search end")
190 endtime = runtime.time()
191
192 print_info("Genetic time::"
193 f"{datetime.timedelta(seconds=endtime - starttime)}")
194
195 # Check if we found a solution
196 ga_sol_params: Tuple[float, float] = driver.solution()
197 score = driver.score()
198
199 print_info(f"IPC found: {ipc}, score: {score}")
200
201 if score < 20:
202 print_warning(f"Unable to find an optimal solution with IPC: {ipc}:")
203 print_info("Generating the closest solution...")
204 generate(
205 f"{DIRECTORY}/{compname}:IPC:{ipc:.2f}:"
206 f"DIST:{ga_sol_params[0]:.2f}:LAT:{ga_sol_params[1]:.2f}-check",
207 ga_sol_params[0], ga_sol_params[1])
208 print_info("Closest solution generated")
209 else:
210 print_info("Solution found for %s and IPC %f -> dist: %f , "
211 "latency: %f " %
212 (compname, ipc, ga_sol_params[0], ga_sol_params[1]))
213 print_info("Generating solution...")
214 generate(
215 f"{DIRECTORY}/{compname}:IPC:{ipc:.2f}:"
216 f"DIST:{ga_sol_params[0]:.2f}:LAT:{ga_sol_params[1]:.2f}",
217 ga_sol_params[0], ga_sol_params[1])
218 print_info("Solution generated")
219 return True
220
221
222if __name__ == '__main__':
223 # run main if executed from the COMMAND line
224 # and the main method exists
225
226 if len(sys.argv) != 3:
227 print_info("Usage:")
228 print_info("%s output_dir eval_cmd" % (sys.argv[0]))
229 print_info("")
230 print_info("Output dir: output directory for the generated benchmarks")
231 print_info("eval_cmd: command accepting 2 parameters: the target IPC")
232 print_info(" and the filename of the generate benchmark. ")
233 print_info(" Output: the score used for the GA search. E.g.")
234 print_info(" the close the IPC of the generated benchmark to")
235 print_info(" the target IPC, the cmd should give a higher ")
236 print_info(" score. ")
237 exit(-1)
238
239 DIRECTORY = sys.argv[1]
240 COMMAND = sys.argv[2]
241
242 if not os.path.isdir(DIRECTORY):
243 print_info("Output DIRECTORY '%s' does not exists" % (DIRECTORY))
244 exit(-1)
245
246 if not os.path.isfile(COMMAND):
247 print_info("The COMMAND '%s' does not exists" % (COMMAND))
248 exit(-1)
249
250 if callable(locals().get('main')):
251 main()