Obscure issue compiling/linking Windows binaries from Linux using mingw
Apologies, I didn't include version information. This is from the docker container I am building from:
$ x86_64-w64-mingw32-gcc -v Using built-in specs. COLLECT_GCC=x86_64-w64-mingw32-gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-w64-mingw32/9.3-win32/lto-wrapper Target: x86_64-w64-mingw32 Configured with: ../../src/configure --build=x86_64-linux-gnu --prefix=/usr --includedir='/usr/include' --mandir='/usr/share/man' --infodir='/usr/share/info' --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir='/usr/lib/x86_64-linux-gnu' --libexecdir='/usr/lib/x86_64-linux-gnu' --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --enable-shared --enable-static --disable-multilib --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --libdir=/usr/lib --enable-libstdcxx-time=yes --with-tune=generic --with-headers=/usr/x86_64-w64-mingw32/include --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libgomp --enable-languages=c,c++,fortran,objc,obj-c++,ada --enable-lto --enable-threads=win32 --program-suffix=-win32 --program-prefix=x86_64-w64-mingw32- --target=x86_64-w64-mingw32 --with-as=/usr/bin/x86_64-w64-mingw32-as --with-ld=/usr/bin/x86_64-w64-mingw32-ld --enable-libatomic --enable-libstdcxx-filesystem-ts=yes --enable-dependency-tracking Thread model: win32 gcc version 9.3-win32 20200320 (GCC) $ x86_64-w64-mingw32-g++ -v Using built-in specs. COLLECT_GCC=x86_64-w64-mingw32-g++ COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-w64-mingw32/9.3-win32/lto-wrapper Target: x86_64-w64-mingw32 Configured with: ../../src/configure --build=x86_64-linux-gnu --prefix=/usr --includedir='/usr/include' --mandir='/usr/share/man' --infodir='/usr/share/info' --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir='/usr/lib/x86_64-linux-gnu' --libexecdir='/usr/lib/x86_64-linux-gnu' --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --enable-shared --enable-static --disable-multilib --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --libdir=/usr/lib --enable-libstdcxx-time=yes --with-tune=generic --with-headers=/usr/x86_64-w64-mingw32/include --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libgomp --enable-languages=c,c++,fortran,objc,obj-c++,ada --enable-lto --enable-threads=win32 --program-suffix=-win32 --program-prefix=x86_64-w64-mingw32- --target=x86_64-w64-mingw32 --with-as=/usr/bin/x86_64-w64-mingw32-as --with-ld=/usr/bin/x86_64-w64-mingw32-ld --enable-libatomic --enable-libstdcxx-filesystem-ts=yes --enable-dependency-tracking Thread model: win32 gcc version 9.3-win32 20200320 (GCC) $ x86_64-w64-mingw32-ld -v GNU ld (GNU Binutils) 2.34 $ uname -a Linux 52b13e514e20 5.6.7-100.fc30.x86_64 #1 SMP Fri Apr 24 21:54:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
keith I'm sorry, but this
COLLECT_GCC=x86_64-w64-mingw32-gcc
is not a MinGW product; indeed, that "mingw" appears in its name is a trademark infringement. As the legitimate owners of that trademark, we do not support such illegitimately distributed products.
Dammit! :) I actually pondered this as soon as I pulled the trigger on the issue. Based on the name I did think there was an affiliation.
Sorry for the noise. I shall close and move on!
Fork
Hi,
I've recently been working on providing support to build Windows Meterpreter binaries using mingw via CMake on Linux. The project source can be found here: https://github.com/OJ/metasploit-payloads/tree/cross-compile-linux/c/meterpreter
I think it's fair to say that the kind of stuff Meterpreter does is a little "non standard" compared to most C/C++ projects and so I expected to come up against some edge case issues.For the most part it has gone very well, but I stumbled on something yesterday that has taken me a solid day of debugging to get to the bottom of. I'll do my best to describe in detail and provide links to the relevant things so that the mingw developers can get themselves to the point that I was to reproduce the issue.
Meterpreter is made up of a number of components. A core component called metsrv is what's loaded first, and from there a number of other components (extensions) can be loaded on the fly. One of the extensions is called kiwi, which is made up of functionality pulled from a project called Mimikatz by Ben Delpy, the source if which can be found here: https://github.com/gentilkiwi/mimikatz
The kiwi extension makes some modifications to this project allowing it be loaded using Reflective DLL Injection. This technique was developer by Stephen Fewer, the source can be found here: https://github.com/stephenfewer/ReflectiveDLLInjection
So metsrv is loaded via RDI, it can then load kiwi via RDI. Nothing too complex so far.
The kiwi source is definitely complex. Mimikatz does crazy things with the internals of Windows via obscure APIs. In order to support various versions of Windows it has to include a few custom made libraries that are shipped with the project in the lib subfolder.
Prior to undertaking this task, we were able to build all the components of the project cleanly with VS 2013/2017/2019 and the binaries would work as expected. I was having the same experience porting things to mingw on Linux until I reached the kiwi extension, which resulted in the issue that I thought I'd log here.
Invocation of the features of the binary would result in the program crashing. After debugging, I realised that all IAT function pointers that should have been imported from advapi32.dll were set to NULL. This seemed a little strange, because all other IAT entries looked fine. Then after many hours of digging, I noticed that there was something odd going on with the Import descriptors. Please refer to the following image:
https://i.imgur.com/zXynXJp.png
This image is a screenshot from PE-Bear, a PE analysis tool, and it shows the imports for the resulting ext_server_kiwi.x64.dll binary that is generated from mingw. Things to note:
Ultimately, the root of the issue is that the import descriptor for winsta.dll is not terminted with a NULL entry. So what happens at runtime is that the DLL loader iterates through all the winsta.dll functions first, correcty resolves them using LoadLibrary and GetProcAddress, and stores the pointers in the IAT. As soon as it hits the first entry for advapi32.dll (in this case A_SHAFinal) it attempts to call GetProcAddress with an invalid module base (because it's reusing WINSTA.dll's base) and hence the call will result in NULL. The net result is that all of the advapi32.dll import entries are set to NULL.
I modified the reflective DLL injection code so that it doesn't assume that a NULL terminator is present. It will also look at the next set of import characteristics and make sure that it doesn't go past the first thunk. This change resulted in everything working fine.
This problem isn't present in the VS compilers/linkers I've used and hence I think it might be an issue specific to mingw. I do have a working solution, but I think there could be something that may impact other people if we don't get to the bottom of it. Hopefully it's specific to my project alone, however I don't think it is. Maybe it's an edge case that mingw doesn't yet cover.
I think this is a linker issue rather than a compiler issue, but that's obviously up to people with more knowledge than me to determine. I wouldn't consider this the highest priority, given that it's rather edge case and you probably haven't seen it before despite the long history of the mingw project.
If you need any more clarification on the issue, or links to sources etc, please let me know. I can provide binaries if required.
Many thanks for your time and for the great work you folks do this project.
Best regards
OJ / @TheColonial