Some Ideas on Coverage Extendability
The biggest advantage of e regarding coverage is, in my opinion, the ability to tweak the definitions of existing coverage groups by extending them from anywhere inside the verification environment. This is particularly useful when dealing with coverage groups defined inside eVCs. For example, let's say that we have an AHB eVC that provides some very extensive coverage definitions to make sure that we rigorously verify our DUT. If that DUT is a very simple slave that can only process read transactions, we'll never reach 100% coverage unless we ignore all references to write transactions. This is easily done with just a few lines of code:
<'
extend has_coverage vgm_ahb_monitor {
cover access is also {
item direction using also ignore = (direction == WRITE);
};
};
'>
While SystemVerilog provides a comparably expressive coverage description syntax to e's (which, may be even more powerful in some respects), it doesn't allow for the same extendibility.
Let's say that we are verifying a RISC CPU that can perform simple arithmetic operations using a register file of size 8:
typedef enum { ADD, SUB, MUL, DIV } operation_e;
typedef enum { R[8] } register_e;
An instruction would contain fields for the operation to be performed, the registers where the two operands are stored and the register in which to store the result:
class instruction;
rand operation_e operation;
rand register_e op1;
rand register_e op2;
rand register_e dest;
endclass
At first glance, we might be tempted to try and cover all possible combinations of operations and registers:
class cov_collector;
covergroup cg;
coverpoint operation;
coverpoint op1;
coverpoint op2;
coverpoint dest;
cross operation, op1, op2, dest;
endgroup
// ...
endclass
Even for such a basic CPU this means covering 4 * 8 * 8 * 8 = 2048 cross bins. Adding just one more instruction would raise that number to 2560. Adding just one more register would raise that number to 2916. Adding just two more registers would raise that number to 4000. I guess it's clear that this approach won't scale when our design grows.
Maybe it isn't necessary to cover all combinations. After all, we've had to give up on the dream of traversing the entire state space of a DUT many years ago, when they started getting way too big. We could make an educated guess that it isn't important to make sure that we executed an ADD with all combinations of operands and destinations. What would be important, though, is that we've made sure that each register can be multiplexed to each of the three operation arguments:
class cov_collector;
covergroup cg;
// ...
operation_vs_op1 : cross operation, op1;
operation_vs_op2 : cross operation, op2;
operation_vs_dest : cross operation, dest;
endgroup
endclass
Some other interesting corner cases might be to make sure that we've tried to use the same register for both operands or for one of the operands and the destination, regardless of what the operation was:
class cov_collector;
covergroup cg with function sample(operation_e operation, register_e op1,
register_e op2, register_e dest
);
// ...
same_reg_both_ops : coverpoint (op1 == op2);
same_reg_op1_and_dest : coverpoint (op1 == dest);
same_reg_op2_and_dest : coverpoint (op2 == dest);
same_reg_both_ops_and_dest : coverpoint (op1 == dest && op2 == dest);
endgroup
endclass
Now that we've cut down the problem space to a more manageable size and are merrily going about verifying our design, our colleagues in marketing notice that there might be some profit to be made if we could offer a version of our CPU that only supports addition and subtraction. Since SystemVerilog coverage groups aren't extendable we can't leverage the coverage definitions from above for this other project. We'd need to define the covergroup again and specify that we want to ignore multiplications and divisions:
class no_mul_cov_collector;
covergroup cg;
coverpoint operation {
ignore_bins ignore[] = { MUL, DIV };
}
coverpoint op1;
coverpoint op2;
coverpoint dest;
operation_vs_op1 : cross operation, op1;
operation_vs_op2 : cross operation, op2;
operation_vs_dest : cross operation, dest;
same_reg_both_ops : coverpoint (op1 == op2);
same_reg_op1_and_dest : coverpoint (op1 == dest);
same_reg_op2_and_dest : coverpoint (op2 == dest);
same_reg_both_ops_and_dest : coverpoint (op1 == dest && op2 == dest);
endgroup
// ...
endclass
Those same colleagues from marketing also figure out that we could sell a slightly slower variant of our CPU that only has four registers. As before, we'd need to create another copy of the covergroup where we ignore all registers from R4 onward:
class less_regs_cov_collector;
covergroup cg;
coverpoint operation;
coverpoint op1 {
ignore_bins ignore[] = { [R4:R7] };
}
coverpoint op2 {
ignore_bins ignore[] = { [R4:R7] };
}
coverpoint dest {
ignore_bins ignore[] = { [R4:R7] };
}
operation_vs_op1 : cross operation, op1;
operation_vs_op2 : cross operation, op2;
operation_vs_dest : cross operation, dest;
same_reg_both_ops : coverpoint (op1 == op2);
same_reg_op1_and_dest : coverpoint (op1 == dest);
same_reg_op2_and_dest : coverpoint (op2 == dest);
same_reg_both_ops_and_dest : coverpoint (op1 == dest && op2 == dest);
endgroup
// ...
endclass
Now we've got three copies of essentially the same covergroup, with very small differences. If after a review we notice that we need to add a new coverage item because we missed some important aspect, we'll need to make sure that we update all three of those coverage groups. If marketing finds even more potential for stripped down CPUs (for example, without multiplication and with less registers at the same time), those new variants will only increase the number of files we need to maintain in sync.
I couldn't accept that there isn't any elegant solution to this problem, so I went digging through the LRM. The new 2012 standard added some cool new features to the coverage chapter. The one that caught my eye in particular was the with syntax for specifying coverpoints:
covergroup some_covergroup;
coverpoint some_coverpoint {
ignore_bins ignore[] = some_coverpoint with (is_ignore_bin(item));
}
endgroup
What this code snippet would do is ignore all values of the coverpoint for which the is_ignore_bin(...) function returns a 1. Let's imagine that we write the definition of the operation coverpoint in this manner. We need to find a way to switch out the implementation of the function, so that it always returns a 0 (in the general case where we don't want to ignore anything) or sometimes returns a 1 (to ignore MULs and DIVs).
There is a lot of literature on this topic (switching out method implementations) in the world of software programming. Generic programming and, more specifically, policy-based design give us the answer. This programming paradigm is based on C++ templates, which are analogous to SystemVerilog parameterized classes. We could use parameter classes to define different policies whether a bin is supposed to be ignored or not.
class cov_collector #(type POLICY);
covergroup cg;
coverpoint operation {
ignore_bins ignore[] = operation with (
POLICY::is_operation_ignore_bin(item));
}
// ...
endgroup
// ...
endclass
A policy class would need to provide an appropriate implementation of the is_operation_ignore_bin(...) function, for example to ignore multiplication and division:
class no_mul_cg_ignore_bins_policy extends cg_ignore_bins_policy;
static function bit is_operation_ignore_bin(operation_e operation);
return operation inside { MUL, DIV };
endfunction
endclass
When instantiating the coverage collector, we would select what policy to parameterize it with:
cov_collector #(no_mul_cg_ignore_bins_policy) no_mul_cov = new();
While trying out this code I got a cryptic error message regarding the use of is_operation_ignore_bin(...) inside the bin definition. Luckily, I was able to tweak the code to an equivalent form:
class cov_collector #(type POLICY);
covergroup cg;
coverpoint operation {
ignore_bins ignore[] = operation with (item inside
{ POLICY::get_operation_ignore_bins() });
}
// ...
endgroup
// ...
endclass
Now, the policy class just has to return a list containing the values we want ignore:
typedef operation_e array_of_operation_e[$];
class no_mul_cg_ignore_bins_policy extends cg_ignore_bins_policy;
static function array_of_operation_e get_operation_ignore_bins();
return '{ MUL, DIV };
endfunction
endclass
We can extend this idea to the other coverpoints as well, leading to the following definition for the covergroup:
class cov_collector #(type POLICY = cg_ignore_bins_policy);
covergroup cg;
coverpoint operation {
ignore_bins ignore[] = operation with (item inside
{ POLICY::get_operation_ignore_bins() });
}
coverpoint op1 {
ignore_bins ignore[] = op1 with (item inside
{ POLICY::get_op1_ignore_bins() });
}
coverpoint op2 {
ignore_bins ignore[] = op2 with (item inside
{ POLICY::get_op2_ignore_bins() });
}
coverpoint dest {
ignore_bins ignore[] = dest with (item inside
{ POLICY::get_dest_ignore_bins() });
}
operation_vs_op1 : cross operation, op1;
operation_vs_op2 : cross operation, op2;
operation_vs_dest : cross operation, dest;
same_reg_both_ops : coverpoint (op1 == op2);
same_reg_op1_and_dest : coverpoint (op1 == dest);
same_reg_op2_and_dest : coverpoint (op2 == dest);
same_reg_both_ops_and_dest : coverpoint (op1 == dest && op2 == dest);
endgroup
// ...
endclass
The default policy (for the fully-featured CPU) would be to not ignore anything:
class cg_ignore_bins_policy;
static function array_of_operation_e get_operation_ignore_bins();
return '{};
endfunction
static function array_of_register_e get_op1_ignore_bins();
return '{};
endfunction
static function array_of_register_e get_op2_ignore_bins();
return '{};
endfunction
static function array_of_register_e get_dest_ignore_bins();
return '{};
endfunction
endclass
Implementing new variants would just boil down to writing new policy classes. For example, for the CPU with less registers, we would have:
class less_regs_cg_ignore_bins_policy extends cg_ignore_bins_policy;
static function array_of_register_e get_op1_ignore_bins();
return '{ R4, R5, R6, R7 };
endfunction
static function array_of_register_e get_op2_ignore_bins();
return get_op1_ignore_bins();
endfunction
static function array_of_register_e get_dest_ignore_bins();
return get_op1_ignore_bins();
endfunction
endclass
We can also easily handle the CPU with less registers and no multiplier/divider by just writing a few more lines of code:
class less_regs_no_mul_cg_ignore_bins_policy extends
less_regs_cg_ignore_bins_policy;
static function array_of_operation_e get_operation_ignore_bins();
return no_mul_cg_ignore_bins_policy::get_operation_ignore_bins();
endfunction
endclass
As we already saw, selecting the appropriate coverage model is done by passing the corresponding policy as a parameter when instantiating the coverage collector:
cov_collector cov;
cov_collector #(no_mul_cg_ignore_bins_policy) no_mul_cov;
cov_collector #(less_regs_cg_ignore_bins_policy) less_regs_cov;
cov_collector #(less_regs_no_mul_cg_ignore_bins_policy) less_regs_no_mul_cov;
Using policy classes allows us to separate our coverage definitions from their parameterization. The result is that we have less code, which is also much easier to maintain because we did away with the redundancy the old-school method suffered from.
Have a look at the next post for an alternative way of using policy classes.
Comments