Triage: galaxyproject/galaxy #21869 - Migrate webdav file source plugin to fsspec

FEATURE_21869_SUMMARY.md

Triage Summary: Issue #21869 - Migrate WebDAV file source to fsspec

Top-Line Summary

Issue #21869 requests migrating Galaxy's WebDAV file source plugin from the unmaintained PyFilesystem2 (fs) backend to fsspec, using the webdav4.fsspec.WebdavFileSystem library as the replacement. This is one of five parallel migration tasks under the parent tracking issue #21832. The migration is well-scoped: the current plugin is only 99 lines, Galaxy already has a mature fsspec base class with multiple reference implementations (S3, HuggingFace, GCS, Azure), and the webdav4 library provides a direct fsspec-compatible drop-in. The recommended approach is to rewrite webdav.py to inherit from FsspecFilesSource, swap the fs.webdavfs dependency for webdav4[fsspec], handle the root path via _to_filesystem_path(), map login/password to auth tuple, and deprecate (not remove) the WebDAV-specific use_temp_files configuration. Estimated effort is 1-3 days.

Importance Assessment

Dimension	Rating	Notes
User demand	Low-Medium	No direct requests, but WebDAV serves Nextcloud/Owncloud users
Strategic value	High	Part of broader fs deprecation; unmaintained dep is growing liability
Effort estimate	Small-Medium	~1-3 days; clear patterns to follow
Risk	Low-Medium	Behavioral differences possible; use_temp_files deprecation needs care

Recommendation: Prioritize now, within the context of the #21832 migration series.

Key Questions for Group Discussion

1. Deprecation vs. removal of use_temp_files: Should we keep the config field as a no-op with a deprecation warning, or remove it outright? The former is safer for existing deployments but adds dead code.

2. Migration ordering: Should all 5 plugins (#21865-21869) be done in a single PR or individually? Individual PRs are easier to review but slower; a single PR gives a cleaner diff against _pyfilesystem2.py.

3. When to remove _pyfilesystem2.py: The old base class must remain until ALL 10 plugins listed in #21832 are migrated (not just the 5 with ready fsspec replacements). The remaining 5 (anvil, azure, basespace, onedata, rspace) don't have existing fsspec packages. Should we plan for that timeline?

4. John Chilton's concern: @jmchilton raised the alternative of forking fs instead of migrating. Has this been definitively resolved, or does it need further discussion?

Concerns

• Scope creep: This should remain a 1:1 behavioral replacement, not an opportunity to add new WebDAV features. The fsspec base class already provides improvements (caching, better pagination, search) that come "for free."

• Breaking changes: The use_temp_files / temp_path configuration exists in Galaxy's config schema and may be set by administrators. A deprecation path is needed.

• Maintenance burden: The webdav4 library (77 GitHub stars) is maintained by a single developer (skshetry). This is a risk, though it's strictly better than the completely unmaintained webdavfs. The library is also used by DVC (Data Version Control), which provides some sustainability backing.

• Testing coverage: The existing tests require a Docker WebDAV server (jmchilton/webdavdev) and are gated behind GALAXY_TEST_WEBDAV=1. This means CI may not catch regressions unless the WebDAV test environment is part of the standard test matrix.