how to enter the ext4 partition #579
Comments
|
Fbougron, reading different FSes in Linux requires kernel-mode modules loaded, I believe, at boot on linux. Since the NT kernel reads NTFS partitions, and LXSS stores Linux-related file properties with NT extended attributes, NTFS is the only FS available (actually also ReFS but you get the picture). To get real Ext-FS support from windows is something I don't think we can expect that from the MS team. Native Ext-FS support on Windows would be a cool feature in general, and I don't think it is really something directly linked to the Windows Subsystem for Linux (since it amounts basically to a kernel-mode filesystem driver). WSL does do the VFS (virtual filesystem) emulation, which is the interesting part of the problem anyway. The rest of it is just a matter of coding something like ext2fsd, or a driver for it. It would be cool if MS opened up the filesystem driver API for the NT Kernel so users could write drivers for the different linux filesystems though. |
|
This is why I think FUSE (#414) is so important before someone above the WSL team's pay grade (#423) declares bash.exe 'done'. MS can't reasonably do a "clean room" re-implementation of Ext4, but we can do this. With CUSE someone could do OSS or ALSA over a socket even if it's not a priority for the WSL team. User Voice is here. |
|
I mean, they could do a clean-room reimplementation of Ext4, but it would be a lot of work and relatively little market for it, since the Linix side already has interop through the ntfs-3g project. But my preference would be for MS to implement something like VFS on the Windows side with a well-documented API so OSS coders could do the hard work of a fully functional Ext4 driver for Windows. The point is though that this would have to be something implemented in directly in the NT Kernel by the Kernel team, not the WSL team (they seem to only work directly with the lxss/lxcore/lxsession/bash.exe stuff, which is my guess why they have to make inter-departmental requests to the Kernel team and the Console team for changes to be made). |
|
OSS coders can already do Ext4 on Windows without Microsoft's help, if someone felt so inclined. MS even tells you how to get started here. A VFS type layer is entirely possible too. Windows filesystems do not need to be developed by the MS NT Kernel team. It's already just a driver. |
|
@therealkenc Oh? Do the Linux FSes support all of the extended attributes and (maybe overcomplicated) security security model used by the NT kernel and implemented in NTFS? |
|
Nope. But you could implement all of the NTFS security constructs in a Linux kernel module if one was so inclined, just as you could implement all of the Linux security constructs in a Windows driver if one was so inclined. You were asking for a "a well-documented API so OSS coders could do the hard work of a fully functional Ext4 driver for Windows". If by "fully functional" you mean including Linux security constructs, then yep you'd need to implement those in a Windows driver. But you wouldn't need the MS NT Kernel team to do that. This setting aside the fact that an Ext4 filesystem on Windows would probably care about such security constructs about as much as the ntfs-3g filesystem on Linux does (that is, not that much). You'd probably mount the thing with some blanket permissions for the user that mounted it. |
|
WSL queries system-wide drives for DriveFS; letter mounts that are per-user (like some network maps, or Paragon's ext4fs driver for Windows) aren't picked up. @benhillis can you confirm that I'm accurately describing this? |
|
@segin You are mostly correct. Currently we only support fixed NTFS volumes. |
|
Another voice chiming in. Would be great to access files from my ubuntu partition via WSL. I'm still excited about WSL--please keep up the morale and hard work! |
|
I'd like to be able to do this in WSL. Thanks for all the hard work folks |
|
Would be great to work this out to make Rails 10x faster. #893 |
|
@aurimus This talk by Theodore Ts'o should probably explain why adding ext4 support is not necessarily just a matter of adding a driver: https://www.youtube.com/watch?v=2mYDFr5T4tY It turns out that getting good performance and supporting lots of features of the Ext filesystems would require MS to invest quite a lot of resources. Ts'o says in that talk that it takes 7-10 years for new filesystem drivers in the Linux kernel to become mature. MS could probably relatively quickly write a small compatibility driver (something like ntfs-3g on Linux), but to really get first-party-level-performance out of it would likely take a while (or maybe coding resources that they wouldn't want to spend on such a niche feature). Edit: But if the Microsoft team wants to prove me wrong, I'm rooting for ya. |
|
They don't have to. Ext2/3/4 File System Driver for Windows: |
|
@therealkenc so you are saying that I can see linux partitions from Bash by mounting them with your mentioned driver? |
|
@therealkenc -- I don't know the provenance of that driver; are you saying that it's as robust and well-optimized as the ext4 driver baked into the Linux kernel? |
|
There is also Paragon Extfs driver, would that do? |
|
Those file system drivers for Windows (both ext2fsd and paragon extfs) work great for mounting an EXT2/3/4 volume in Windows. But as far as I know you can't access them from Bash. |
|
@mstrelan sounds like it's true.. |
|
Just saying it's doable, same as ISO9660 or FAT32. You can't mount Ext2 partitions or CDs or DOS floppy disks in WSL Creators, of course, because the infrastructure doesn't exist. Whether you can add a "yet" to that last sentence depends on where this project goes after all the low hanging fruit bits are finished. |
|
@mstrelan @Fbougron is this true answer that could be used on WSL? https://superuser.com/questions/1190580/how-to-mount-ubuntu-home-directory-for-windows-10-wsl-bash/1204888#1204888 |
|
I'm certainly not the one to ask. I think a Samba share is a less than ideal workaround. I'm sticking with NTFS partitions for now. |
|
Native Ext4 would mean that build times could be 2-3 times quicker for us and npm installs could be up to 10 times as fast, much as they are in native Linux. Up until yesterday I exclusively used Linux as my dev OS using VS Code but as of yesterday I am now using VS Code in WSL and the only issue is the disk performance. Other than that, I find WSL is very capable, saving me from having to switch back and forth from Windows to Linux when I need one or the other. |
|
@mredbishop Ext4 doesn't have anything to do with it, really. The trouble starts from the fact that there is no *nix-compatible VFS layer in Windows, so inodes and things like Linux permissions, owner, and attributes all need to be emulated in software, which is slow. My understanding is that the WSL team is working with the NTFS team to have these features (or at least features that simplify and speed up the most expensive parts of them) implemented directly in NTFS at the driver level. A side effect of this work is that it will also probably improve interop, since if Linux-style attributes are stored directly as ordinary rather than extended attributes, they should be preserved better by Windows tools, and they should also be faster to compute, since they could be queried directly from the driver instead of read off and parsed in the LX driver from the NTFS extended attributes. Similarly, computing and caching inodes could be in the NTFS driver, which would be much faster, and perhaps even *nix-style unlink semantics could be implemented without using hacks (the current method moves the open files to a hidden directory, then the unlink happens when the open handle to the file is closed. But none of this has to do with ext4. |
The ask in the OP is for
Rather, almost none of the comments in this thread has to do with the ask. |
|
Now with ext2fsd 0.68 (NOT the latest 0.69) I can cd to the ext4 part. But I can't chroot to it. It shows Here is the PS C:\> bash
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /dev /mnt/e/dev
[sudo] password for Menci:
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /proc /mnt/e/proc
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /sys /mnt/e/sys
Menci@Menci-Surface:/mnt/c$ sudo strace chroot /mnt/e/
execve("/usr/sbin/chroot", ["chroot", "/mnt/e/"], 0x7fffea8f0df8 /* 14 vars */) = 0
brk(NULL) = 0x7fffc3f40000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
[...snip]
lstat("/mnt/e", {st_mode=S_IFDIR|0777, st_size=512, ...}) = 0
chroot("/mnt/e/") = 0
chdir("/") = 0
execve("/bin/bash", ["/bin/bash", "-i"], 0x7fffcbd002f0 /* 14 vars */) = -1 ENOEXEC (Exec format error)
execve("/bin/sh", ["/bin/sh", "/bin/bash", "-i"], 0x7fffcbd002f0 /* 14 vars */) = -1 EIO (Input/output error)
openat(AT_FDCWD, "/usr/share/locale/C.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1
[...snip]
write(2, ": Input/output error", 20: Input/output error) = 20
[...snip]
+++ exited with 126 +++
Menci@Menci-Surface:/mnt/c$ ls /mnt/e
bin cdrom etc initrd.img lib lost+found mnt proc run snap sys usr vmlinuz
boot dev home initrd.img.old lib64 media opt root sbin srv tmp var vmlinuz.old
Menci@Menci-Surface:/mnt/c$ |
|
Your In other words (simplifying), the In theory |
|
Is there documentation on what is required of an IFS to support WSL in the best way possible? |
|
No not specifically. As you know, documentation on writing a IFS that works on Windows in the "best way possible" is scant enough. I venture anything that could (in principle) be written specifically for WSL (say a blog post) would have a tiny target audience, and anything such a write-up contained today (say: "we know this doesn't work" or "we expect the driver to behave like such-and-such") would be at best a moving target. In any case, "such-and-such" amounts to: if your filesystem works exactly like That said, if you had for example FWIW accessing Dokany 1.1.0.2000 mirrorfs currently fails with |
|
Thanks for your comment. I am aware that one can solve this with trial and error, but I was just being optimistic thinking that maybe some documentation exists or is in the plans, which would probably make this an easier endeavour... |
|
There may be support for ext4 file system in future. If you've Windows 10 insider build 18262 or greater then run this command using strings from sysinternals. set X=C:\Windows\System32\lxss\LxssManager.dll
strings.exe %X% | find /i "ext4"LxssManager.dll has a string Another evidence supports my investigation. This error showed in Process Monitor while uninstalling a distro. LxssManager can't find the
|
FWIW the future came in 16176, in April 14, 2017. The volume being mounted in the OP was not a fixed NTFS volume; instead, it was a a fixed ext2fsd volume, which according to Menci mounts now. Similar to, like Ben said in the post you quote, #214 or #530. I just didn't have the heart to close the issue because it acts as a useful dupe target.
Why is it always "a number of use cases" and never one use case. [Don't answer that, I'm mumbling to myself again not you.]
More likely they need to present a static ext4 image to userspace as a ro block device and/or to satisfy some ABI I haven't thought of. That's my guess. My guess could be wrong. I think it is a better guess than your guess, tho. But to be sure, I wish you were right. |
|
Reading this thread, I have come to find it's an utter cluster.
WSL has no bearing upon `ext4` support. There's nothing magical about it
that would require it to support `ext4` - there's no `syscall` in Linux
that is strictly for `ext4`, all POSIX file system semantics are fully
supported already with NTFS, etc.
There's no reason for this feature request to happen. It's outside of the
scope of the tool, and all the tool does is translate Linux syscalls to NT
kernel syscalls.
Microsoft would be better off writing an `ext4` IFS and shipping it as part
of Windows, which again, would fall wholly outside of the scope of WSL.
A more short-term solution could be had in implementing the `FUSE` API, but
that would almost certainly restrict the `ext4` mount to being available
only to the current WSL instance (won't make a Windows drive letter or be
available to Windows programs in any way, or to other Linux distros running
at the same time. Don't try to mount the volume twice, that'll wreck it.)
In short, this is a feature request that may be difficult, if not
impossible, to properly deliver.
…On Sun, Nov 18, 2018, 12:57 therealkenc ***@***.*** wrote:
This is similar to #214 <#214> and
#530 <#530>. Currently only fixed
NTFS volumes will appear as DriveFs mounts. Definitely something that we
will be revisiting in the future.
Okay, it's the future now.
FWIW the future came in 16176
<https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-16176>,
in April 14, 2017. The volume being mounted in the OP
<#579 (comment)> was not a
fixed NTFS volume; instead, it was a a fixed ext2fsd volume, which
according to Menci mounts now
<#579 (comment)>.
Similar to, like Ben said in the post you quote, #214
<#214> or #530
<#530>. I just didn't have the
heart to close the issue because it acts as a useful dupe target.
This is blocking for a number of our use cases.
Why is it always "a number of use cases" and never one use case. [Don't
answer that, I'm mumbling to myself again not you.]
There may be support for ext4 file system in future.
LxssManager.dll has a string ext4.vhdx.
More likely they need to present a static ext4 image to userspace as a ro
block device and/or to satisfy some ABI I haven't thought of. That's my
guess. My guess could be wrong. I think it is a better guess than your
guess, tho. But to be sure, I wish you were right.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#579 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAdVxVwDOl2WNQX7BB8FeCm8UEGQa_jZks5uwa2SgaJpZM4I9r0a>
.
|
|
@segin not a cluster. We are looking for vital functionality here and it depends on WSL understanding ext4 filesystems properly. WSL is emulating a Linux environment and it cannot do that properly without speaking ext4. Use cases:
|
Replace "ext4" above with btrfs or reiserfs. Now replace that with lxfs. They all have *nix semantics, including Now, I can maybe assume what want. Maybe you want to mount a ext4 image on a loop device. [Not that I would use WSL for the purpose if it were me. WSL is demonstrably the wrong tool for the scenario right now.] But even if this is the case, the scenario that has nothing to do with the OP, which is about mounting an Windows ext2fsd drive in WSL. In short, "building multi-platform projects" doesn't have anything to do with "speaking ext4" or "speaking reiserfs" or "speaking zfs" or "speaking NFSv4". If you've got problems with |
|
@mcmordie Let me try to tell you this again because it seems you either didn't understand, or didn't like what you were told and are being arrogant about the reality of the situation. Where you wrote: "WSL understanding ext4 filesystems properly"... WSL does not understand ANY filesystem. It does not understand Where you wrote: "WSL is emulating a Linux environment and it cannot do that properly without speaking ext4." I can compile a Linux kernel, yes, a real Linux kernel, latest version straight from Where you wrote: "Mounting remote ext4 filesystems in order to have multi-platform projects building eg. Win32/Linux-ARM/Linux-x86." Remote filesystems have no bearing over what WSL can and cannot do, unless you're using iSCSI or Fibre Channel connected storage where raw block-level access of drives is performed over the network. So yes, this thread is a cluster because it's full of people asking the wrong team for something that team isn't responsible for. I wouldn't ask the President of Mexico for a change in US law and push the matter anyway, trying to argue "but it's all on Earth so they should be able to figure it out". That's basically what you're doing here. |
Nit - WSL understands lxfs and drvfs filesystems equivalently to any other diskless filesystem such as nfs or cifs or even procfs or tmpfs which lives in memory. There is no requirement for a filesystem to have an on-disk layout of any kind. WSL understands filesystems fine. [I know what you meant; I just wanted to clarify the quoted text above, which has correctness problems in boldface.]
More than one way to skin a cat, but that's a fair enough (if not exclusive) way to look at it. Mostly this issue is a cluster because no one has read the OP. Also because no one has come to the acceptance part of the grief stages - namely that MSFT isn't doing a clean-room ext4 re-implementation full stop (whatever team one thinks they should assign the task to). Because that would be silly. |
|
Hello @therealkenc I have still this issue. I have Linux partitions in the internal disk. I read this whole topic. I also read #689, #2896 and #4531. I use Paragon ExtFS for Windows and not Ext4FS. See the screenshots. The drivers
I followed the small tutorials above.
Similar to #4531, I use the Link Shell Extension to create a symlink from
I know the articles:
|
Given the green open button at the top that is not too surprising. Paragon could put their FS under a debugger and look, if they were motivated to do so. A FOSS developer could do the same with Ext3fsd, which best I can tell hasn't had a commit since early 2018. There is some recent chatter on the broader topic in #2999. An experiment to try might be to export/share that Paragon filesystem in WIndows, and them |
|
Folks, just an update to say that this is completely resolved at least for me, on WSL2. qmake, which was unable to stat filesystem entries due to a missing system call, is now working correctly. Thanks to everyone who helped make this a reality. |
How were you able to mount an ext4 partition? I'm using WSL2 and I can't access my 2nd drive's ext4 partiton (its NTFS partition is the only one I can see). |
|
Mine was a side-effect of the OP's issue-- being able to stat filesystem objects inside chroots which wasn't working because the implementation didn't fully support ext4 internally. I have also been able to mount ext4 partitions on a looper in order to prepare boot media. But mounting real hardware with ext4 partitions is another matter entirely because the host OS (Windows) would have to provide access to WSL2 to the underlying block device. |
|
If we could have access to a raw disk partition / USB device from WSL2 Linux could handle the filesystem access. Perhaps admin access of something would be required, but why not support this. Virtualbox enables this although you have to dig in the docs to figure out how. MS should be able to implement this pretty easily. |
|
We've now added support to access ext4 and other Linux file formatted physical hard disks in a WSL 2 distro. See the details here in this blog post. |
Hello
I am a new user of BASH with Windows preview insider
When i do the command "ls ", i can see all the NTFS partitions like one lettre each
Microsoft@DESKTOP-G51CU1Q:/mnt$ ls
c e f g h j k q r
But no ext4 partition!!!
however i can continue to see the contents of the EXT4 partitions on Windows10-14371 with EXT2FSD ( http://www.ext2fsd.com/ )
how to be able to also view them with BASH (mount ?)
http://www.hostingpics.net/viewer.php?id=479092ext.png
The text was updated successfully, but these errors were encountered: