Ensure `@source` globs with symlinks are preserved
This PR fixes an issue when working with @source and @source not that involves symlinks.
Internally our sources are mapped to a source entry where we have a base path and a pattern. You can think about this where we insert a .gitignore file in the base path for the given pattern.
However, we optimize these entries to move as many "static" parts into the base path. For example:
@source "./some/folder/here/*.html";
Is mapped to something like:
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
We then optimize it by turning it into:
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
While doing this, we also use dunce::canonicalize to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the @source directives.
So before, it could be that you have this:
@source "./some/symlinked-folder/here/*.html";
Which was mapped to this on the Rust side:
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
But was then optimized to:
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
...and we lost the symlinked-folder information. This causes issues as seen in #17985.
With this PR, we keep the symlinked information in those globs since that's what you wrote in those @source directives.
While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example:
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
Earlier I mentioned that we create .gitignore files based on these @source directives. In this case, when we're dealing with a folder, we use **/* as the contents.
Looking at the example above, we should essentially have something like this:
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
Since it's a .gitignore file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like:
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
Notice how the !except.html and **/* are flipped. When dealing with .gitignore files, the order is important.
This was caused because internally we kept a BTreeMap of BTreeSets where the map was the base path and a set of patterns. The patterns were sorted because of the BTreeSet... which is not what we want.
Test plan
- Added new tests in the scanner tests (on the Rust side)
- Added integration tests with a symlink to another folder, outside of the current folder
- Added integration tests with a symlink to another folder, inside of the current folder
- Added integration tests to ensure that the order of
@sourcefiles with a folder + file is sorted correctly. - Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]