How to change the camera model from "SIMPLE_RADIAL" to "PINHOLE"?

[takafire2222]

How can I reconstruct one sfm model by Pinpole in the deep-image-matching?

[lcmrl]

Hi, you can go in main.py and change

    if use_pycolmap:
        from src.deep_image_matching import reconstruction

        # Define database path and camera mode
        database = output_dir / "database_pycolmap.db"

in

    if use_pycolmap:
        from src.deep_image_matching import reconstruction

        # Define database path and camera mode
        database = output_dir / "database_pycolmap.db"
        camera_mode: pycolmap.CameraMode = pycolmap.CameraMode.SINGLE # pycolmap.CameraMode.AUTO

        # Define cameras
        cam0 = pycolmap.Camera(
            model="PINHOLE",
            width=1500,
            height=1000,
            params=[
                1500,
                1500,
                750,
                500,
            ],
        )

Finally change cameras = None in cameras = [cam0]
params are fx, fy, cx, cy
Let me know if you encounter any problem

[takafire2222]

Sorry for asking again.
I rewrote as described and successfully wrote the files (database.db, database_pycolmap.db, features.h5, matches.h5, pairs.txt).
During conversion, the following error syntax appears as shown below.
I'm sorry if this is a basic question, but if you have any ideas, could you please give me some advice?

2024-01-04 18:13:28 | [INFO ] Creating an empty database...
2024-01-04 18:13:28 | [INFO ] Importing images into the database...
Traceback (most recent call last):
File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 416, in
main()
File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 362, in main
model = reconstruction.main(
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/reconstruction.py", line 145, in main
update_cameras(database, cameras)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/reconstruction.py", line 53, in update_cameras
if not all([isinstance(cam, pycolmap.Camera) for cam in cameras]):
TypeError: 'pycolmap.Camera' object is not iterable**

My corrections

if use_pycolmap:
from src.deep_image_matching import reconstruction

    # Define database path and camera mode
    database = output_dir / "database_pycolmap.db"
    camera_mode: pycolmap.CameraMode = pycolmap.CameraMode.AUTO # pycolmap.CameraMode.AUTO

    # Define cameras
    cam0 = pycolmap.Camera(
        model="PINHOLE",
        width=1500,
        height=1000,
        params=[
            1500,
            1500,
            750,
            500,
        ],
    )

)

    #cameras = [cam0] # cameras = [cam1, cam2]
    cameras = cam0

[lcmrl]

Hi, try change cameras = cam0 to cameras = [cam0] should do the work

[takafire2222]

Thank you for giving me the answer.
I was able to export the "model" without any problems and no errors.

I have about 3 questions include bug reports, so please give me some advice.

1. When I loaded it into Colmap and checked it with database management, the model name remained as SINPLE_RADIAL.
   (When I checked camera.txt, it was PINHOLE, so I think it was a bug that occurred when loading into Colmap. After that, when I performed 3D Gaussian Splatting, it was PINHOLE recognized, so I was able to perform Splatting.) However, the image was corrupted, failed. I think that This is probably the result of Splatting with SINPLE_RADIAL.)

2. When I calculated the same number of sheets as above using 4K data, there was no problem with SIMPLE_RADIAL, but when setting it to PINHOLE, the following error occurred and stopped. , unfortunately I could not output it.

100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 154.27it/s]
54%|████████████████████▉ | 188/351 [00:00<00:00, 2584.70it/s]
2024-01-04 21:26:05 | [INFO ] Creating an empty database...
2024-01-04 21:26:05 | [INFO ] Importing images into the database...
2024-01-04 21:26:10 | [INFO ] Importing features into the database...
100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 734.88it/s]
2024-01-04 21:26:10 | [INFO ] Importing matches into the database...
54%|████████████████████▉ | 188/351 [00:00<00:00, 1942.57it/s]
2024-01-04 21:26:10 | [INFO ] Running 3D reconstruction...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20240104 21:26:52.518604 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.524106 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.601553 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.607028 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.612496 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.643158 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.743846 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.749533 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
W20240104 21:26:52.754989 9171 levenberg_marquardt_strategy.cc:123] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
continue--------------------------------------------

3.When I tried to process 900 HD size movies, the following error occurred. No problem occurred with 100 sheets.
Are there any restrictions on the number of sheets that can be processed?

2024-01-04 21:32:53 | [INFO ] Number of pairs: 430128
2024-01-04 21:32:53 | [INFO ] Extracting features with superpoint...
2024-01-04 21:32:53 | [INFO ] superpoint configuration:
{'keypoint_threshold': 0.0001, 'max_keypoints': 4096, 'name': 'superpoint'}
100%|█████████████████████████████████████████| 928/928 [01:51<00:00, 8.34it/s]
2024-01-04 21:34:44 | [INFO ] Features extracted!
2024-01-04 21:34:44 | [INFO ] Matching features with lightglue...
2024-01-04 21:34:44 | [INFO ] lightglue configuration:
{'depth_confidence': -1,
'filter_threshold': 0.5,
'flash': True,
'n_layers': 9,
'name': 'lightglue',
'width_confidence': -1}
2024-01-04 21:34:44 | [INFO ] Matching features...
2024-01-04 21:34:44 | [INFO ]
0%| | 0/430128 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 416, in
main()
File "/home/denpozi/ドキュメント/deep-image-matching/./main.py", line 237, in main
match_path = img_matching.match_pairs(feature_path)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/image_matching.py", line 311, in match_pairs
matches = self._matcher.match(
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 279, in match
self._matches = self._match_by_tile(
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 363, in _match_by_tile
tile_pairs = self._tile_selection(img0, img1, method)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 513, in _tile_selection
res = self._preselction_matcher({"image0": feats0, "image1": feats1})
File "/home/denpozi/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 463, in forward
return self._forward(data)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 474, in _forward
kpts0 = normalize_keypoints(kpts0, size0).clone()
File "/home/denpozi/.local/lib/python3.10/site-packages/torch/cuda/amp/autocast_mode.py", line 118, in decorate_fwd
return fwd(*args, **kwargs)
File "/home/denpozi/ドキュメント/deep-image-matching/src/deep_image_matching/thirdparty/LightGlue/lightglue/lightglue.py", line 29, in normalize_keypoints
size = 1 + kpts.max(-2).values - kpts.min(-2).values
IndexError: max(): Expected reduction dim 1 to have non-zero size.

[lcmrl]

1. Thanks for reporting I check. There are two databases in general, database.db and database_pycolmap.db. Are you using the second one? Can I suggest in the meantime to simply import database.db in COLMAP, change the camera model in the database with the GUI and run the reconstruction with the GUI. This will give you complete control on the process
2. Have you accordenly modified the internals values for SIMPLE_RADIAL (f,cx,cy,k) and PINHOLE (fx,fy,cx,cy)?

[takafire2222]

Thank you very much for your detailed and detailed explanation.
The two problems were successfully resolved.
Next time I would like to try 900 sequence data.

[lcmrl]

The remaining problem of processing large image sets (>1000 images) still needs to be solved. I close here, and keep open issue #Issue 29. Thanks for the feedback, feel free to open other issues or contribute to the repository.