Photos with identical file names ignored during camera upload (iOS)

[szapp]

I noticed that when there are different photos that share the identical file name, only one of them is uploaded during camera uploads (on iOS).

I can reliably reproduce photos being ignored if I place a file with the same file name into the camera upload directory in my Filen drive:

Steps to reproduce

1. Snap a photo
2. Check the photo's file name in the photos app
3. Create a random file of arbitrary size and modification date but with the same name in the Filen drive in the destination of camera uploads (or a random sub-directory within)
4. Trigger camera uploads in the Filen app on the phone
5. Photo is not uploaded

Triggering "Reset uploaded assets" and waiting has no effect. As soon as the dummy file is removed, however, the upload works.

1. Repeat, but do not create file with the same name in the camera uploads directory
2. Trigger camera uploads
3. Photo is uploaded

To clarify: The event logs of the Filen account do no not list the attempt to upload, i.e. the photo does not replace the existing one, but is just completely skipped. This leads me to believe that something might be wrong in the delta-checking logic.

Why is this important?

For iOS, the file name of photos iterates from IMG_0000.heic to IMG_9999.heic. When reaching 10,000, the increment is reset and new photos are AFAIK placed in a different (internal) directory (≠ album) restarting from IMG_0000.heic. This results in photos with identical file names in the photo gallery.

So, this is an actual real-life problem; I have currently around 50,000 photos on my phone. Only a fraction of them is uploaded. I have no feasible way of finding which files are uploaded and which are skipped.

Source code

The relevant parts in the source code seem to be the lines shown below. From what I could gather, it should handle the case where two photos share the same file name by checking modification times(?) and file size(?) as well. But I couldn't find what happens if they don't match, i.e. is the second file then still uploaded but with a different file name?

Alternatively I wonder, if the modification time is not reported correctly from iOS photos or something else slips through the if-conditions (e.g. the setting of "Save all assets").

in getDeltas = async (local: CameraUploadItems, remote: CameraUploadItems)

filen-mobile/src/lib/services/cameraUpload/cameraUpload.ts

Lines 359 to 390 in 0b70ca2

	if (typeof lastModified[assetId] === "number") {
	if (convertTimestampToMs(lastModified[assetId]) !== convertTimestampToMs(local[name].lastModified)) {
	const [lastModified, lastModifiedStat, lastSize, assetURI] = await Promise.all([
	db.cameraUpload.getLastModified(local[name].asset),
	db.cameraUpload.getLastModifiedStat(local[name].asset),
	db.cameraUpload.getLastSize(local[name].asset),
	getAssetURI(local[name].asset)
	])
	const stat = await fs.stat(assetURI)
	if (
	stat.exists &&
	(convertTimestampToMs(stat.modificationTime) === convertTimestampToMs(lastModifiedStat) ||
	convertTimestampToMs(lastModified) === convertTimestampToMs(local[name].lastModified) ||
	lastSize === stat.size)
	) {
	await Promise.all([
	db.cameraUpload.setLastModified(local[name].asset, convertTimestampToMs(local[name].lastModified)),
	db.cameraUpload.setLastModifiedStat(local[name].asset, convertTimestampToMs(stat.modificationTime)),
	db.cameraUpload.setLastSize(local[name].asset, stat.size)
	])
	continue
	}
	deltas.push({
	type: "UPDATE",
	item: local[name]
	})
	}
	}

and

in upload = async (delta: Delta): Promise<void>

filen-mobile/src/lib/services/cameraUpload/cameraUpload.ts

Lines 898 to 943 in 0b70ca2

	if (
	stat.exists &&
	(convertTimestampToMs(stat.modificationTime) === convertTimestampToMs(lastModifiedStat) ||
	convertTimestampToMs(lastModified) === convertTimestampToMs(delta.item.lastModified) ||
	lastSize === stat.size)
	) {
	uploadedThisRun += 1
	storage.set("cameraUploadUploaded", currentlyUploadedCount + uploadedThisRun)
	await Promise.all([
	db.cameraUpload.setLastModified(asset, convertTimestampToMs(delta.item.lastModified)),
	db.cameraUpload.setLastModifiedStat(asset, convertTimestampToMs(stat.modificationTime)),
	db.cameraUpload.setLastSize(asset, stat.size)
	])
	return
	}
	const files = await getFiles(asset, assetURI)
	const parentFolderName = pathModule.dirname(delta.item.path)
	const parentFolderUUID = !cameraUploadAutoOrganize
	? cameraUploadFolderUUID
	: parentFolderName === "." || parentFolderName.length <= 0
	? cameraUploadFolderUUID
	: await getFileParentFolderUUID(cameraUploadFolderUUID, parentFolderName)
	for (const file of files) {
	await queueFileUpload({
	file,
	parent: parentFolderUUID,
	isCameraUpload: true
	}).catch(console.error)
	if (file.path.includes(fs.documentDirectory()) || file.path.includes(fs.cacheDirectory())) {
	await fs.unlink(file.path).catch(console.error)
	}
	}
	if (stat.exists) {
	await Promise.all([
	db.cameraUpload.setLastModified(asset, convertTimestampToMs(delta.item.lastModified)),
	db.cameraUpload.setLastModifiedStat(asset, convertTimestampToMs(stat.modificationTime)),
	db.cameraUpload.setLastSize(asset, stat.size)
	])
	}

Debug output / log

Is there a way to enable debugging logs in the mobile app to inspect what might be going on?

I am willing to help to debug this - it's just difficult without any logs.

[jonaspoxleitner]

I recently got a new iPhone and this problem also happens for me. Older photos from iCloud are automatically renamed when they are downloaded to my iPhone (from the Photos app) and new photos I took start with IMG_0000.heic again. This results in some images not uploading to Filen (as photos with the same file name already exist in the drive).