sysfs/fixtures filenames are too long

[F21]

Trying to install github.com\prometheus\client_golang\prometheus using glide which has procfs as a dependency.

The install fails on windows because sysfs/fixtures contains subfolders with long filenames:

Cloning into 'C:\Users\user\.glide\cache\src\https-git.f4.workers.dev-prometheus-procfs'...
error: unable to create file sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_day/cache_bypass_misses: Filename too long
error: unable to create file sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_day/cache_miss_collisions: Filename too long
fatal: cannot create directory at 'sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_five_minute': Filename too long
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

ref #55 #56

[F21]

Workaround:

• Enable long paths on Windows (requires Windows 10 Anniversary Update or newer): https://superuser.com/a/1119980/97078
• Configure git to use long paths: git config --global core.longpaths true (globally) or git config core.longpaths true (per project)

[F21]

Still having issues after using the workaround:

[ERROR] Unable to export dependencies to vendor directory: chmod C:\Work\src\github.com\F21\some_project\vendor\github.com\prometheus\procfs\sysfs\fixtures.src\devices\pci0000_@colon@_00\0000_@colon@_00_@colon@_0d.0\ata4\host3\target3_@colon@_0_@colon@_0\3_@colon@_0_@colon@_0_@colon@_0\block\sdb\bcache\dirty_data: The system cannot find the path specified.

[ideaship]

65 chars: C:\Users\user\.glide\cache\src\https-git.f4.workers.dev-prometheus-procfs
192 chars: sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_day/cache_bypass_misses
194 chars: sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_day/cache_miss_collisions
180 chars: sysfs/fixtures.src/devices/pci0000_@colon@_00/0000_@colon@_00_@colon@_0d.0/ata4/host3/target3_@colon@_0_@colon@_0/3_@colon@_0_@colon@_0_@colon@_0/block/sdb/bcache/stats_five_minute
65+1+192=258
65+1+194=260
65+1+180=246
The length limitation for paths on Windows is said to be 260 characters. I don't understand why this paths are too long, even on Windows -- unless the first, absolute path is not the root from where the relative paths start.

Of course we could shorten the code used to replace colon from _@colon@_ to something shorter. Even taking into account my insufficient understanding of the problem (see above), I suppose that would fix it at least for these paths. Of course, a longer username in the path might break it again.

The chmod problem I have no idea about.

Perhaps we should consider going back to tar. My approach may be too brittle on Windows.

What are people using on Windows these days, anyway? Cygwin? The Linux subsystem for Windows?

[grobie]

Let's just use a comma inst of _@colon@_? @F21 Could you check whether that's ok?

[F21]

I think the length issue can be easily solved if you're on Windows 10. See my comment here: #57 (comment)

I tried using glide and dep and they both had the same issue with chmod. I had some discussions with @sdboyer and he reckons os.Chmod() might not be cross platform. I have a feeling that the main culprit is probably the @ causing the chmod to fail.

[F21]

@grobie If a comma works for you guys, it should be cool :) However, I am not sure whether os.Chmod() will work correctly with commas. Have you guys considered using a dash (-) or even -colon-?

[grobie]

I'd be afraid to use a dash as replacement separator, it's too common in filenames.

[F21]

How about two underscores: __colon__?

[grobie]

That's still making filenames very long. I don't think it's reasonable to expect everyone checking out the repository on Windows to have Windows 10 and able to change their filesystem configuration.

[grobie]

I meant a literal comma ,.

[F21]

Ah, I see. I think a comma should be fine as I do have files with commas in them on the filesystem and they seem to work fine.

[grobie]

Thanks so much for reporting and being so responsive @F21. I'll put something together, given I don't have access to a Windows node, would you be able to test another fix for us?

[ideaship]

Every single character is a liability going forward. There are files in /sys that already contain dashes, commas, at signs, underscores, periods, and parentheses. Are we sure that no such file will turn up in the fixtures directory at some point?

[grobie]

If you know there are commas already, that's not an option. @ideaship so let's revert and be safe and go with the tarball approach?

[F21]

Also, another liability is that the repo might not be checked out to C:/Users/someuser/....

In my case, glide or dep checks them out to C:/Users/someuser/appdata/temp/... and then copies them into my workspace, which is D:/000-Work/go/src/github.com/somecompany/someproject, which can have a very long path.

[F21]

@grobie Happy to test your fix 😄 once it's pushed.

[ideaship]

I'd say let's go back to the tar ball. It seems to be pretty much the only way to be safe.

[ideaship]

Oh, and @F21: I'd still be interested to learn what you are using as an environment on Windows (cygwin, Linux subsystem for Windows, ...).

[F21]

@ideaship I am mostly using Git Bash and occasionally just the standard shell (CMD) for most things. I did play around with Bash for Windows, but there are still some rough edges, so it's not my main shell yet.

[F21]

I just ran dep init to install my dependencies with a clean cache. I will report back once it's done.

[F21]

@ideaship @grobie Thanks for pushing the fix! I can confirm it works correctly! 😄

[grobie]

Thank you for taking the time confirm!

…

On Tue, Jun 20, 2017, 09:51 Francis Chuang ***@***.***> wrote: @ideaship <https://github.com/ideaship> @grobie <https://github.com/grobie> Thanks for pushing the fix! I can confirm it works correctly! 😄 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#57 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAANaDNRyqVQrycul6Mb49VpOyXrsKd6ks5sF3oDgaJpZM4N-91g> .

[whendream]

Workaround:

• Enable long paths on Windows (requires Windows 10 Anniversary Update or newer): https://superuser.com/a/1119980/97078
• Configure git to use long paths: git config --global core.longpaths true (globally) or git config core.longpaths true (per project)

thank you so much！！！it worked！！