Lecture 04: Understanding Performance Slides are by Mary Jane Irwin [Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005, UCB] Other handouts HW#1 To handout next time

Indeed, the cost-performance ratio of the product will depend most heavily on the implementer, just as ease of use depends most heavily on the architect. The Mythical Man-Month, Brooks, pg 46

Performance Metrics Purchasing perspective Design perspective given a collection of machines, which has the best performance ? least cost ? best cost/performance? Design perspective faced with design options, which has the best performance improvement ? Both require basis for comparison metric for evaluation Our goal is to understand what factors in the architecture contribute to overall system performance and the relative importance (and cost) of these factors Or smallest/lightest Longest battery life Most reliable/durable (in space)

Computer Performance: TIME, TIME, TIME Response Time (latency) — How long does it take for my job to run? — How long does it take to execute a job? — How long must I wait for the database query? Throughput — How many jobs can the machine run at once? — What is the average execution rate? — How much work is getting done? If we upgrade a machine with a new processor what do we increase? If we add a new machine to the lab what do we increase? Have them raise their hands when answering questions

Execution Time Elapsed Time CPU time Our focus: user CPU time counts everything (disk and memory accesses, I/O , etc.) a useful number, but often not good for comparison purposes CPU time doesn't count I/O or time spent running other programs can be broken up into system time, and user time Our focus: user CPU time time spent executing the lines of code that are "in" our program

Definition of Performance (Hennessy) Problem: machine A runs a program in 20 seconds machine B runs the same program in 25 seconds

Defining (Speed) Performance Normally interested in reducing Response time (aka execution time) – the time between the start and the completion of a task Important to individual users Thus, to maximize performance, need to minimize execution time performanceX = 1 / execution_timeX If X is n times faster than Y, then performanceX execution_timeY -------------------- = --------------------- = n performanceY execution_timeX Increasing performance requires decreasing execution time Throughput – the total amount of work done in a given time Important to data center managers Decreasing response time almost always improves throughput

Performance Factors Want to distinguish elapsed time and the time spent on our task CPU execution time (CPU time) – time the CPU spends working on a task Does not include time waiting for I/O or running other programs CPU execution time # CPU clock cycles for a program for a program = x clock cycle time or CPU execution time # CPU clock cycles for a program for a program clock rate = ------------------------------------------- Many techniques that decrease the number of clock cycles also increase the clock cycle time Can improve performance by reducing either the length of the clock cycle or the number of clock cycles required for a program

Clock Cycles Instead of reporting execution time in seconds, we often use cycles Clock “ticks” indicate when to start activities (one abstraction): cycle time = time between ticks = seconds per cycle clock rate (frequency) = cycles per second (1 Hz. = 1 cycle/sec) A 200 Mhz. clock has a cycle time time

Machine Clock Rate Clock rate (MHz, GHz) is inverse of clock cycle time (clock period) CC = 1 / CR one clock period 10 nsec clock cycle => 100 MHz clock rate 5 nsec clock cycle => 200 MHz clock rate 2 nsec clock cycle => 500 MHz clock rate 1 nsec clock cycle => 1 GHz clock rate 500 psec clock cycle => 2 GHz clock rate 250 psec clock cycle => 4 GHz clock rate 200 psec clock cycle => 5 GHz clock rate A clock cycle is the basic unit of time to execute one operation/pipeline stage/etc.

How to Improve Performance So, to improve performance (everything else being equal) you can either ________ the # of required cycles for a program, or ________ the clock cycle time or, said another way, ________ the clock rate.

How many cycles are required for a program? Could assume that # of cycles = # of instructions? 1st instruction 2nd instruction 3rd instruction 4th 5th 6th ... time This assumption is incorrect, different instructions take different amounts of time on different machines. Why? hint: remember that these are machine instructions, not lines of C code

Different numbers of cycles for different instructions time Multiplication takes more time than addition Floating point operations take longer than integer ones Accessing memory takes more time than accessing registers Important point: changing the cycle time often changes the number of cycles required for various instructions (more later)

Example Cycle time of A =1/(400x106) =1/(4x108)=2.5 nsec Our favorite program runs in 10 seconds on computer A, which has a 400 Mhz. clock. We are trying to help a computer designer build a new machine B, that will run this program in 6 seconds. The designer can use new (or perhaps more expensive) technology to substantially increase the clock rate, but has informed us that this increase will affect the rest of the CPU design, causing machine B to require 1.2 times as many clock cycles as machine A for the same program. What clock rate should we tell the designer to target?" Don't Panic, can easily work this out from basic principles Cycle time of A =1/(400x106) =1/(4x108)=2.5 nsec Program A uses10/(2.5 x10-9) =4x109 cycle Running this program in B uses 1.2x 4x109 =4.8 x 109 cycle Running this program in 6 sec then Frequency of the machine B= 4.8x109 cycle / 6 sec =0.8 x 109 cycle/sec 800MHz

Now that we understand cycles A given program will require some number of instructions (machine instructions) some number of cycles some number of seconds We have a vocabulary that relates these quantities: cycle time (seconds per cycle) clock rate (cycles per second) CPI (cycles per instruction) a floating point intensive application might have a higher CPI MIPS (millions of instructions per second) this would be higher for a program using simple instructions

Performance Performance is determined by execution time Do any of the other variables equal performance? # of cycles to execute program? # of instructions in program? # of cycles per second? average # of cycles per instruction? average # of instructions per second? Common pitfall: thinking one of the variables is indicative of performance when it really isn’t.

CPI Example Suppose we have two implementations of the same instruction set architecture (ISA). For some program, -Machine A has a clock cycle time of 10 ns. and a CPI of 2.0 -Machine B has a clock cycle time of 20 ns. and a CPI of 1.2 What machine is faster for this program, and by how much? If two machines have the same ISA which of our quantities (e.g., clock rate, CPI, execution time, # of instructions, MIPS) will always be identical?

# of Instructions Example A compiler designer is trying to decide between two code sequences for a particular machine. Based on the hardware implementation, there are three different classes of instructions: Class A, Class B, and Class C, and they require one, two, and three cycles (respectively). The first code sequence has 5 instructions: 2 of A, 1 of B, and 2 of C The second sequence has 6 instructions: 4 of A, 1 of B, and 1 of C. Which sequence will be faster? How much? What is the CPI for each sequence?

Clock Cycles per Instruction Not all instructions take the same amount of time to execute One way to think about execution time is that it equals the number of instructions executed multiplied by the average time per instruction # CPU clock cycles # Instructions Average clock cycles for a program for a program per instruction = x Clock cycles per instruction (CPI) – the average number of clock cycles each instruction takes to execute A way to compare two different implementations of the same ISA CPI for this instruction class A B C CPI 1 2 3

Effective CPI Computing the overall effective CPI is done by looking at the different types of instructions and their individual cycle counts and averaging n Overall effective CPI =  (CPIi x ICi) i = 1 Where ICi is the count (percentage) of the number of instructions of class i executed CPIi is the (average) number of clock cycles per instruction for that instruction class n is the number of instruction classes The overall effective CPI varies by instruction mix – a measure of the dynamic frequency of instructions across one or many programs

MIPS example 10million cycle x 10-8= 10x106 x 10-8 =10x10-2 =0.1 sec Two different compilers are being tested for a 100 MHz. machine with three different classes of instructions: Class A, Class B, and Class C, which require one, two, and three cycles (respectively). Both compilers are used to produce code for a large piece of software. The first compiler's code uses 5 million Class A instructions, 1 million Class B instructions, and 1 million Class C instructions. The second compiler's code uses 10 million Class A instructions, 1 million Class B instructions, and 1 million Class C instructions. Which sequence will be faster according to MIPS? Which sequence will be faster according to execution time? 10million cycle x 10-8= 10x106 x 10-8 =10x10-2 =0.1 sec รัน 7 million คำสั่ง 1 sec รัน 70 million คำสั่ง 15 million cycle x 10-8= 15 x 10 6 x 10-8 = 0.15 sec รัน 12 million คำสั่ง 1 sec รัน 80 Million คำสั่ง

Benchmarks Performance best determined by running a real application Use programs typical of expected workload Or, typical of expected class of applications e.g., compilers/editors, scientific applications, graphics, etc. Small benchmarks nice for architects and designers easy to standardize can be abused SPEC (System Performance Evaluation Cooperative) companies have agreed on a set of real program and inputs can still be abused (Intel’s “other” bug) valuable indicator of performance (and compiler technology)

THE Performance Equation Our basic performance equation is then CPU time = Instruction_count x CPI x clock_cycle or Instruction_count x CPI clock_rate CPU time = ----------------------------------------------- These equations separate the three key factors that affect performance Can measure the CPU execution time by running the program The clock rate is usually given Can measure overall instruction count by using profilers/ simulators without knowing all of the implementation details CPI varies by instruction type and ISA implementation for which we must know the implementation details Note that instruction count is dynamic – i.e., its not the number of lines in the code, but THE NUMBER OF INSTRUCTIONS EXECUTED

Determinates of CPU Performance CPU time = Instruction_count x CPI x clock_cycle Instruction_ count CPI clock_cycle Algorithm Programming language Compiler ISA Processor organization Technology For class handout

Determinates of CPU Performance CPU time = Instruction_count x CPI x clock_cycle Instruction_ count CPI clock_cycle Algorithm Programming language Compiler ISA Processor organization Technology X X X X X X X X X For lecture X X X

A Simple Example Op Freq CPIi Freq x CPIi ALU 50% 1 . Load 20% 5 Store 10% 3 Branch 2  = How much faster would the machine be if a better data cache reduced the average load time to 2 cycles? How does this compare with using branch prediction to shave a cycle off the branch time? What if two ALU instructions could be executed at once?

A Simple Example Op Freq CPIi Freq x CPIi ALU 50% 1 Load 20% 5 Store 10% 3 Branch 2  = .5 1.0 .3 .4 .5 .4 .3 .5 1.0 .3 .2 .25 1.0 .3 .4 2.2 1.6 2.0 1.95 How much faster would the machine be if a better data cache reduced the average load time to 2 cycles? How does this compare with using branch prediction to save a cycle off the branch time? What if two ALU instructions could be executed at once? CPU time new = 1.6 x IC x CC so 2.2/1.6 means 37.5% faster For lecture CPU time new = 2.0 x IC x CC so 2.2/2.0 means 10% faster CPU time new = 1.95 x IC x CC so 2.2/1.95 means 12.8% faster

Comparing and Summarizing Performance How do we summarize the performance for benchmark set with a single number? The average of execution times that is directly proportional to total execution time is the arithmetic mean (AM) AM = 1/n  Timei i = 1 n Where Timei is the execution time for the ith program of a total of n programs in the workload A smaller mean indicates a smaller average execution time and thus improved performance Guiding principle in reporting performance measurements is reproducibility – list everything another experimenter would need to duplicate the experiment (version of the operating system, compiler settings, input set used, specific computer configuration (clock rate, cache sizes and speed, memory size and speed, etc.))

SPEC Benchmarks www.spec.org Integer benchmarks FP benchmarks gzip compression wupwise Quantum chromodynamics vpr FPGA place & route swim Shallow water model gcc GNU C compiler mgrid Multigrid solver in 3D fields mcf Combinatorial optimization applu Parabolic/elliptic pde crafty Chess program mesa 3D graphics library parser Word processing program galgel Computational fluid dynamics eon Computer visualization art Image recognition (NN) perlbmk perl application equake Seismic wave propagation simulation gap Group theory interpreter facerec Facial image recognition vortex Object oriented database ammp Computational chemistry bzip2 lucas Primality testing twolf Circuit place & route fma3d Crash simulation fem sixtrack Nuclear physics accel apsi Pollutant distribution

Example SPEC Ratings

Other Performance Metrics Power consumption – especially in the embedded market where battery life is important (and passive cooling) For power-limited applications, the most important metric is energy efficiency

Summary: Evaluating ISAs Design-time metrics: Can it be implemented, in how long, at what cost? Can it be programmed? Ease of compilation? Static Metrics: How many bytes does the program occupy in memory? Dynamic Metrics: How many instructions are executed? How many bytes does the processor fetch to execute the program? How many clocks are required per instruction? How "lean" a clock is practical? Best Metric: Time to execute the program! CPI Inst. Count Cycle Time depends on the instructions set, the processor organization, and compilation techniques.

Next Lecture Next lecture MIPS non-pipelined datapath/control path review Reading assignment – PH, Chapter 5

Amdahl's Law Execution Time After Improvement = Execution Time Unaffected +( Execution Time Affected / Amount of Improvement ) Example: "Suppose a program runs in 100 seconds on a machine, with multiply responsible for 80 seconds of this time. How much do we have to improve the speed of multiplication if we want the program to run 4 times faster?" How about making it 5 times faster? Principle: Make the common case fast

Example ตัวอย่าง: ถ้า การ enhancement ทำให้ run ได้เร็วเป็น 10 เท่าจากเครื่องเดิม แต่ใช้ได้แค่ 40% ของ เวลาทั้งหมด overall speedup คืออะไร

Example ตัวอย่าง: ถ้า เราเพิ่ม speed เป็น 5 เท่าโดยไม่พิจารณา I/O แต่ว่าต้องเสีย cost เป็น 5 เท่าเช่นกัน สมมติว่า CPU ถูกใช้แค่ 50% ของเวลาทั้งหมดที่เหลื่อจะ idle รอ I/O ถ้า CPU มีราคา 1/3 ของ ราคาเครื่อง ถ้าเพิ่ม speed ของ CPU ดังกล่าวจะเป็นการลงทุนที่คุ้มหรือไม่ cost เพิ่มมากกว่าการ improve performance!!

Example Suppose we enhance a machine making all floating-point instructions run five times faster. If the execution time of some benchmark before the floating-point enhancement is 10 seconds, what will the speedup be if half of the 10 seconds is spent executing floating-point instructions? We are looking for a benchmark to show off the new floating-point unit described above, and want the overall benchmark to show a speedup of 3. One benchmark we are considering runs for 100 seconds with the old floating-point hardware. How much of the execution time would floating-point instructions have to account for in this program in order to yield our desired speedup on this benchmark?

Remember Performance is specific to a particular program/s Total execution time is a consistent summary of performance For a given architecture performance increases come from: increases in clock rate (without adverse CPI affects) improvements in processor organization that lower CPI compiler enhancements that lower CPI and/or instruction count Pitfall: expecting improvement in one aspect of a machine’s performance to affect the total performance You should not always believe everything you read! Read carefully!

CPU Performance Example

CPU Performance ตัวอย่าง สมมติว่ามี 2 ทางเลือกดังนี้ 1. CPU A : condition code จะถูกเซตโดยคำสั่งเปรียบเทียบ และจากนั้นจะต้องตามด้วย คำสั่ง branch ที่ ทด 2. CPU B คำสั่ง compare จะถูก รวมใน branch สำหรับ CPU ทั้งสอง คำสั่ง conditional branch ใช้ 2 cycle และคำสั่งอื่นๆ ใช้ 1 cycle สำหรับ CPU A 20 % ของคำสั่งทั้งหมดเป็น conditional branch ดังนั้น 20 % ของคำสั่งก็จะเป็น compare ด้วย เพราะ CPU A ไม่มีคำสั่ง compare รวมอยู่ใน branch clock cycle time จะเร็วกว่า CPU B อยู่ 25% ถามว่า CPU ใดเร็วกว่ากัน

Performance Example ก่อนอื่น หา CPI A =(0.20*2)+(0.80*1) = 1.2 เพราะว่า branch มี 20 % และใช้ 2 cycle เรารู้ว่า clock cycle time B คือ 1.25 * clock cycle time A เพราะว่า A มี cycle time เร็วกว่า B 25 % ดังนั้น เนื่องจาก CPU B ไม่ใช้ คำสั่ง compare ดังนัน 20%/80% = 25% ของคำสั่งเป็นคำสั่ง branch ใช้ 2 cycle ทำให้ 75% ของคำสั่งทั้งหมดใช้คำสั่งละ 1 cycle ดังนั้น CPI B =(0.25*2)+(0.75*1) = 1.25 และทำให้ instruction count ลดไปเป็น 0.8 * instruction count A

Performance Example ได้ว่า ทำให้ทราบว่า CPU A มี cycle time สั้นกว่าและเร็วกว่า CPU B

Performance Example ตัวอย่าง จากตัวอย่างข้างต้น ถ้า designer ต้องการแก้ไข organization ทาง hardware ทำให้ clock cycle time ลดลงเป็น 10 % ถามว่า CPU ไหนเร็วกว่ากัน ถ้าเปลี่ยน clock cycle time ให้เร็วกว่า 10 % นั่นคือ clock cycle time B คือ 1.10 * clock cycle time A ขณะที่ CPU time A ยังเหมือนเดิม ได้ ทำให้ CPU B เร็วกว่า CPU A

Operation frequency clock cycle count ALU ops 43% 1 Loads 21% 2 Performance Example ตัวอย่าง ถ้าจะเพิ่มการเปลี่ยนแปลงเข้าไปอีกเกี่ยวกับ instruction set กล่าวคือถ้าเริ่มต้นคำสั่งจะทำงานได้กับ register เท่านั้นโดยที่ มีคำสั่ง load / store เพื่อถ่ายและรับค่าจากหน่วยความจำให้ พิจารณา instruction mix ดังนี้ ถ้า 25 % ของ ALU ops เป็นคำสั่งที่ใช้ operand ที่ load เข้ามาแล้วจะไม่ได้ใช้อีก Operation frequency clock cycle count ALU ops 43% 1 Loads 21% 2 Stores 12% 2 Branches 24% 2

Performance Example ถ้ามีการเพิ่มคำสั่งว่าในประเภทของ ALU ops โดยที่ operand เป็นแบบ register-memory และมี clock cycle = 2 และถ้าใช้ชุดคำสั่งใหม่นี้จะเพื่ม clock cycle count ของคำสั่ง ประเภท branch ไป 1 แต่ไม่มีผลกับ clock cycle time ถามว่าการเปลี่ยนแปลงนี้จะเพิ่ม performance ของ CPU หรือไม่

เพิ่ม cycle count ของ branch หา CPI ของชุดคำสั่งเดิมก่อน CPI old = .43*1 + .21*1 +.12*2 + .24*2 = 1.57 ดังนั้น และหา CPI new = (.43-.25*.43))*1 + (.21-(.25*.43))*2 +.12*2 + .24*3+.25*.43*2 1-(.25*.43) = 1.703 =1.908 และ clock cycle time เท่าเดิมดังนั้น คำสั่งทาง ALU ลดไป 25 % เพิ่ม cycle count ของ branch คำสั่ง load ลดไป 25 % ของจำนวนของ คำสั่ง ALU Instruction count ลด 0.893

Cost Price and Trends

die wafer Cost Impact of time, volume, commodization, and packaging Volume : quantity (volume decreases cost.) Commodity : whether product is so common ( the product sold by multiple vendors in large volumes.)

Ratio of wafer area/die area หารเส้นรอบวงด้วยความยาวเส้นทแยงมุมของ die Cost Cost of die = cost of wafer dies per wafer * die yield dies per wafer = สัดส่วนของ die ที่ใช้ได้ Ratio of wafer area/die area หารเส้นรอบวงด้วยความยาวเส้นทแยงมุมของ die ส่วนที่ใช้ไม่ได้

Cost Number of dies per 20-cm wafer for die size 1.5 cm # die area are 1.5 * 1.5 = 2.25 dies per wafer is

Price Component Costs Direct Costs: ที่เกี่ยวข้องกับการผลิตสินค้า โดยมากจะ ประมาณ 10%-30% Gross Margin : สิ่งที่ผู้ขาย เพิ่มไป เกิดจาก Overhead อื่นๆเช่นการโฆษณา การ maintenance การทำ R&D เมื่อ รวมกับข้อ 1-2 จะได้ Average Selling Price , Gross Margin จะประมาณ 10-45 % ของ Average Selling Price List Price คือAverage Selling Price + company Discount