fix(RemoteStore): avoid listing all objects in remote store in empty() method

[jhamman]

Some initial exploration with the RemoteStore found that the empty() check was taking a very long time. This sped it up considerably.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[martindurant]

Do included directories count as content? The original implied yes.

[jhamman]

We only check for files in the LocalStore:

zarr-python/src/zarr/storage/local.py

Lines 101 to 111 in 5a134bf

	async def empty(self) -> bool:
	try:
	with os.scandir(self.root) as it:
	for entry in it:
	if entry.is_file():
	# stop once a file is found
	return False
	except FileNotFoundError:
	return True
	else:
	return True

[martindurant]

It cannot be guaranteed that this doesn't do a find() too... Would _ls() be enough?

There was talk of making ls() and friends in fsspec be generators, but not yet.

[jhamman]

gotcha... This was an improvement using gcsfs but I can see how it wouldn't be for others. I'll take your lead on the best path forward here. All we need to know is if there are any files in the store (ignoring empty directories).

[martindurant]

Given the very many files that can in theory be in a zarr hierarchy, maybe we can keep with this strategy for now. Having said that, in the zarr case, we do know which subdir ought to have the fewest items in it, right?

[d-v-b]

thanks for this. perennial question: what kind of performance monitoring should we have in place to flag a regression here (or anywhere)?