Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_run_refine_fullframe tolerance issues #102

Open
sk1p opened this issue Jun 25, 2024 · 0 comments
Open

test_run_refine_fullframe tolerance issues #102

sk1p opened this issue Jun 25, 2024 · 0 comments

Comments

@sk1p
Copy link
Member

sk1p commented Jun 25, 2024

          > @matbryan52 do you have an idea about the test failure? It seems that we are missing the tolerance by just a tiny bit:
E           Not equal to tolerance rtol=1e-07, atol=0.5
E           
E           Mismatched elements: 1 / 2 (50%)
E           Max absolute difference: 0.5083443
E           Max relative difference: 0.15369503
E            x: array([ 0.832502, 29.035461], dtype=float32)
E            y: array([ 0.98369 , 28.527117]

Is something going on, or do we need to tweak the tolerance?

(note that it's not related to numpy 2.0, as hdbscan installs a numpy version <2)

I have a feeling it's a tolerance issue, but it must be a rare one. The large tolerance is due to the small-ish number of peaks in the fake CBED frame and the fact that cbed_frame has to centre its peaks on integer pixels despite using floating (and random) a/b vectors:

        zero = shape / 2 + np.random.uniform(-1, 1, size=2)
        a = np.array([27.17, 0.]) + np.random.uniform(-1, 1, size=2)
        b = np.array([0., 29.19]) + np.random.uniform(-1, 1, size=2)

At a later time we could modify the testcase to be more precise, but for now I would not worry.

Originally posted by @matbryan52 in #100 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant