Interpreter handles opaque inconsistently

[parno]

This program

use vstd::prelude::*;

verus!{

fn main() { }

pub proof fn bar() {
    assert(vstd::std_specs::bits::u64_leading_zeros(0) == 64) by (compute_only);
}

}

uses u64_leading_zeros, which is defined as public and open:

verus/source/pervasive/std_specs/bits.rs

Line 655 in 2b607f7

pub open spec fn u64_leading_zeros(i: u64) -> int

Oddly, when the interpreter tries to run, when it reaches the point of trying to apply u64_leading_zeros, it does a lookup for the function's body and the lookup fails, specifically we get None here:

verus/source/vir/src/interpreter.rs

Line 1365 in 2b607f7

match ctx.fun_ssts.get(fun) {

where fun_ssts is passed into the interpreter from the state object in the ast_to_sst conversion.

If I inline the definition in the same file, there's no trouble finding and using the definition.

use vstd::prelude::*;

verus!{

fn main() { }

#[verifier::opaque]
pub open spec fn u64_leading_zeros(i: u64) -> int
    decreases i
{
    if i == 0 {
        64
    } else {
        u64_leading_zeros(i / 2) - 1
    }
}

pub proof fn bar() {
    assert(u64_leading_zeros(0) == 64) by (compute_only);
}

}
}

Is this expected behavior? Or is something going wrong in the collection of fun_ssts?

[tjhance]

A result of this is that this passes:

pub proof fn other_function() {
    reveal(vstd::std_specs::bits::u64_leading_zeros);
}

pub proof fn bar() {
    assert(vstd::std_specs::bits::u64_leading_zeros(0) == 64) by (compute_only);
}

whereas this does not:

pub proof fn bar() {
    assert(vstd::std_specs::bits::u64_leading_zeros(0) == 64) by (compute_only);
}

This is obviously unexpected behavior.

Because of the way "pruning" works, and because u64_leading_zeros is opaque, we generate SST for it iff we find that it is revealed anywhere in the module. Then as Bryan points out, the interpreter determines whether or not to unfold the definition of u64_leading_zeros based on whether or not the SST exists.

To make a fix, we need to decide: should assert-by-compute unfold opaque functions by default, or should it require
them to be revealed?

[parno]

I would tend to think that opaque is designed to control SMT-level processing, rather than proofs-by-compute. Indeed, it seems like we often use opaque for recursive functions to keep the SMT solver from getting too excited, but then those functions seem to often be the ones we can to compute on for specific concrete values. Hence, I'm tempted to say that the interpreter should unfold opaque definitions. OTOH, maybe there's value in consistently enforcing opaqueness, so that user's aren't confused by the different possible meanings. If we go that way, we should certainly do it consistently, not based on whether there's an ambient reveal elsewhere in the module.

[parno]

Based on discussions in today's Verus meeting, the consensus was that the interpreter should default to opening opaque definitions and that eventually we could add support for restricting this, via syntax along the lines of assert (...) by (compute, opaque=foo).

[parno]

Based on a discussion with @Chris-Hawblitzel, the current plan is to add assert-by-compute to the list of assertions that cause us to spinoff a separate query, and in that context we can tell the pruning phase to not remove opaque functions (or their transitive dependencies).