Please include libgccjit with MinGW GCC distribution
keith A preliminary discussion, related to this request, may be found in the MinGW-Users mailing list archives; see the topic thread commencing with this post.
keith keith As noted in the referenced mailing list thread, I've chosen to explore this, with respect to the existing GCC-9.2.0 package set.
Having perused the relevant online GCC documentation, I note that I should configure with --enable-host-shared, when adding jit to the --enable-languages set. Since the principal effect of --enable-host-shared is to force generation of position independent code, within shared object libraries, this seems kind of redundant on MS-Windows, where code in DLLs is always position independent anyway; however, without --enable-host-shared, the configure script rejects the addition of jit to --enable-languages, so I configured as instructed. This immediately led into the first jit build issue: with --enable-host-shared, the generated libiberty object code is expected to reside in a libiberty/pic build directory, but no such directory is ever built for --target=mingw32!
Initially, as stated in my first response to the mail thread, I kludged around this, (on my GNU/Linux build host), by running 'cd libiberty; ln -s . pic' after the make aborted, and then restarted make. Obviously, such kludges are not satisfactory in any ultimate solution, so, drawing inspiration from this patch set, as suggested in Eli's follow-up mail, I've implemented the attached patch, to resolve this particular aspect of the issue.
keith Having resolved the libiberty issue described in my preceding comment, my libgccjit build subsequently succumbed to the compilation failures, outlined in this mail posting, viz.:
- GCC's jit sources gratuitously use dlopen(), but neglect to include dlfcn.h; jit-common.h needs a patch, to work around this.
- availability of fchmod() is gratuitously assumed (in jit-playback.c); a further patch is needed, to use chmod() instead.
The attached patches, 22-libgccjit-win32-dlfcn.patch, and 21-libgccjit-win32-fchmod.patch, respectively address each of these issues, allowing the jit code to compile, and thus taking us a step closer to a successful build.
keith keith With the previously posted series of patches, (20...22), augmenting the pre-existing series for MinGW GCC-9.2.0, the libgccjit shared library sources compile; however, they do not link, because they introduce an unresolved dependency on pthreads mutexes.
While it would be technically feasible to include -lpthread within the link operation, and so resolve this dependency, but, as noted in this mailing-list message:
The dependency on pthreads is also unfortunate, as that many times conflicts with w32api headers and creates build problems. It would be good to avoid that dependency, if possible.
It arises because the JIT context is gratuitously guarded by a POSIX mutex lock. Perhaps that could be converted to a Windows alternative, (critical section, maybe); perhaps GCC-10 already does so, because the patches you pointed me to do nothing to address this.
I think MinGW64 simply requires pthreads in all the GCC-related builds, we had our share of problems stemming from that in Emacs development, when MinGW64 users complained about that.
It should be very simple to replace a pthreads mutex lock with a critical section, if GCC 10 didn't.
the dependency on a third-party pthreads mutex implementation does seem kind of alien, on the Windows platform, when a native alternative would appear to be entirely feasible.
FWIW, a cursory examination of the GCC-10 sources suggests that the pthreads mutex dependency is still present, without change. Consequently, while not entirely trivial, I have engineered a relatively simple replacement, in terms of a native Windows CRITICAL_SECTION; it may be found in the attached 23-libgccjit-win32-no-pthreads.patch.
keith keith keith All patches, up to and including #23, are sufficient to both compile, and link, the libgccjit shared library. However:
The latter failing is corrected by the attached 25-libgccjit-mingw-link-options.patch; this modifies the link command, for the Windows build-case, to name the generated shared library as libgccjit-0.dll, while also creating the libgccjit.dll.a import library, as a linker side effect. Additionally, it modifies the installation commands, to install both the DLL, and its import library, into appropriate locations.
Although this is still WIP, AFAIU, I would like to thank you for your efforts to make this happen. Thanks!
keith keith Since all Windows code is position independent anyway, and the reason that JIT requires configuration with --enable-host-shared appears to be to ensure position independence of generated code, it seems likely that the --enable-host-shared option is not actually required for a MinGW build of libgccjit-0.dll; however, the current behaviour of GCC's configure script enforces the requirement that it be specified. The attached 26-libgccjit-mingw-host-shared.patch overrides such enforcement, in the specific case of building for a MinGW target, allowing the whole caboodle to be built in a single operation, without requiring specification of --enable-host-shared.
keith Having successfully cross-built a MinGW-GCC, inclusive of libgccjit-0.dll, but uncertain how to effectively test it, I followed Eli's suggestion from this mailing-list posting:
Is the produced libgccjit functional?
I don't know. I suspect that the testsuite will be inadequate, when cross compiling, and I don't know how to test it, otherwise.
This tutorial has a simple text program, it seems:
https://gcc.gnu.org/onlinedocs/jit/
and copied the sample program from part 1 of the libgccjit tutorial, for use as a rudimentary test program ... alas, it fails spectacularly, with a segmentation fault which will require further investigation.
keith alas, it fails spectacularly, with a segmentation fault ...
Identification of the cause is far from straightforward. The actual point of failure appears to lie deep within the bowels of Windows system DLLs, and the GDB backtrace bears no obvious relationship to the source code. Seems like a rebuild with -g -O0, followed by stepping through the code, will be necessary to locate the origin of the failure.
keith)
keith After a rebuild with -g -O0, GDB yields a more meaningful backtrace:
Fork(gdb) bt
#0 0x01765861 in pp_format (pp=0x0, text=0x22fcac)
at ../../src/gcc-9.2.0/gcc/pretty-print.c:1016
#1 0x0175bb77 in diagnostic_report_diagnostic (
context=0x238bfa0 <global_diagnostic_context>, diagnostic=0x22fcac)
at ../../src/gcc-9.2.0/gcc/diagnostic.c:1015
#2 0x0175bfe7 in diagnostic_impl (richloc=0x22fd00, opt=-1,
gmsgid=0x1eee273 <trim_filename(char const*)::this_file+211> "in %s, at %s:%d", ap=0x22fd68, kind=DK_ICE)
at ../../src/gcc-9.2.0/gcc/diagnostic.c:1159
#3 0x0175cc66 in internal_error (
gmsgid=0x1eee273 <trim_filename(char const*)::this_file+211> "in %s, at %s:%d")
at ../../src/gcc-9.2.0/gcc/diagnostic.c:1540
#4 0x0175cdbd in fancy_abort (
file=0x19aa0dc <NUM_GCC_JIT_TYPES+64> "../../src/gcc-9.2.0/gcc/jit/jit-tempdir.c", line=54,
function=0x19aa0be <NUM_GCC_JIT_TYPES+34> "make_tempdir_path_template")
at ../../src/gcc-9.2.0/gcc/diagnostic.c:1607
#5 0x00459c9f in make_tempdir_path_template ()
at ../../src/gcc-9.2.0/gcc/jit/jit-tempdir.c:54
#6 0x00459dc3 in gcc::jit::tempdir::create (this=0xd2084b0)
at ../../src/gcc-9.2.0/gcc/jit/jit-tempdir.c:90
#7 0x004553f1 in gcc::jit::playback::context::compile (this=0x22fe9c)
at ../../src/gcc-9.2.0/gcc/jit/jit-playback.c:1784
#8 0x004474da in gcc::jit::recording::context::compile (this=0xd207db0)
at ../../src/gcc-9.2.0/gcc/jit/jit-recording.c:1379
#9 0x00443e32 in gcc_jit_context_compile (ctxt=0xd207db0)
at ../../src/gcc-9.2.0/gcc/jit/libgccjit.c:2698
#10 0x0040163a in main (argc=1, argv=0xd201470)
at /z/jit/tut01-hello-world.c:103
(gdb)
The segmentation fault is due to that pp = 0x0 argument in pp_format(): it is supposed to be a (non-NULL) pointer to a pretty_printer object, and is immediately dereferenced, to acquire a pointer to the associated output buffer. Since the pretty_printer object pointer is itself NULL, which it clearly isn't expected to be, the segmentation fault is inevitable. It appears that, wherever it should have been initialized, that pretty_printer object pointer has not been; I have no idea where that initialization should have occurred, nor how to fix it.
Where I *can* identify a clear defect is at the origin of the diagnostic call, at line 54 of jit-tempdir.c, where I see
gcc_assert (tmpdir_buf[tmpdir_len - 1] == DIR_SEPARATOR);in which gcc_assert is a macro, invoking fancy_abort(), and where DIR_SEPARATOR has been explicitly defined as '/'. This follows assignment of tmpdir_buf, and tmpdir_len, within the code fragment:
/* The result of choose_tmpdir is a cached buffer within libiberty, so we must *not* free it. */ tmpdir_buf = choose_tmpdir (); /* choose_tmpdir aborts on malloc failure. */ gcc_assert (tmpdir_buf); tmpdir_len = strlen (tmpdir_buf); /* tmpdir_buf should now have a dir separator as the final byte. */ gcc_assert (tmpdir_len > 0); gcc_assert (tmpdir_buf[tmpdir_len - 1] == DIR_SEPARATOR);in which choose_tmpdir() returns a path with Windows-style '\\' directory name separators, and thus, the assertion tests an invalid assumption. This I can fix, but the potential for some other assertion triggering a segmentation fault would remain.
keith)
keith I filed GCC bug #98586, to report the segmentation fault.
keith keith Attached 27-libgccjit-invalid-assertion.patch corrects the invalid assumption, in the assertion which validates the final DIR_SEPARATOR of the temporary directory path name. With all patches applied, up to and including this one, I am able to successfully compile and link libgccjit-0.dll, and subsequently compile, link, and run the tut01-hello-world.exe example program.
I have posted the resulting libgccjit packages, to augment the GCC-9.2.0 collection in our FRS; currently published as hidden packages, you will need to access them via this direct link — they will not be visible, in the regular package list. Additionally, I have updated the source code tarball, to incorporate the additional patches, and configuration options, which are required to achieve this additional package build.
Please test.
I've succeeded in compiling and linking the hello-world test program, but I cannot run it: it looks like libgccjit-0.dll depends on libintl-8.dll, and the version of libintl you used is different from what I have here: the libintl_setlocale entry point is missing from the version I have here. (Yes, my libintl is very old, but it was never a problem until now.)
Why is libgccjit need to depend on libintl anyway? gcc.exe and cc1.exe, for example, don't depend on it. Is that dependency really needed?
Thanks again for working on this.
Btw, judging by the Info manual that comes with libgccjit, a more appropriate name for the DLL would be libgccjit-10.dll, since it implements LIBGCCJIT_ABI_10 (and the next versions of GCC advance the ABI by several more notches).
MinGW64 also calls it libgccjit-0.dll, so we could adopt the same name, regardless. But then we will need to invent ABI numbers out of thin air for future releases, which might be sub-optimal.
Reply To eliz
Btw, judging by the Info manual that comes with libgccjit, a more appropriate name for the DLL would be libgccjit-10.dll, since it implements LIBGCCJIT_ABI_10 (and the next versions of GCC advance the ABI by several more notches). MinGW64 also calls it libgccjit-0.dll, so we could adopt the same name, regardless. But then we will need to invent ABI numbers out of thin air for future releases, which might be sub-optimal.
Upstream notes on API/ABI compatibility: https://gcc.gnu.org/onlinedocs/jit/topics/compatibility.html
libgccjit hides all data layouts, and only ever adds entrypoints, so I've avoiding bumping the SONAME, using symbol versioning instead, to allow for backwards ABI compatibility. I'm not sure if that's going to work with mingw.
Reply To davidmalcolm
Reply To eliz
Btw, judging by the Info manual that comes with libgccjit, a more appropriate name for the DLL would be libgccjit-10.dll, since it implements LIBGCCJIT_ABI_10 (and the next versions of GCC advance the ABI by several more notches). MinGW64 also calls it libgccjit-0.dll, so we could adopt the same name, regardless. But then we will need to invent ABI numbers out of thin air for future releases, which might be sub-optimal.
Upstream notes on API/ABI compatibility: https://gcc.gnu.org/onlinedocs/jit/topics/compatibility.html libgccjit hides all data layouts, and only ever adds entrypoints, so I've avoiding bumping the SONAME, using symbol versioning instead, to allow for backwards ABI compatibility. I'm not sure if that's going to work with mingw.
I'm not sure I understand the documentation to which you pointed: it seems to hint that detecting the ABI needs the end-user to have objdump installed? That's not gonna happen for users of MinGW programs.
FWIW, I don't see such symbols using the objdump command described in the docs. Which doesn't surprise me.
And in any case, how would detecting the required ABI help? all you can do when you detect an incompatible ABI version (e.g., a program linked with libgccjit whose ABI level is N, but finds only lower ABI level at run time) is refuse to run. Likewise, if you try running a program which imports a symbol from a DLL, but the DLL found at run time has no such symbol, the OS loader on Windows refuses to start the program and pops up a dialog showing the offending missing function(s). And linking the program on Windows against libgccjit-NN.dll ensures that cleanly and easily. So I don't really see how checking for those symbols would help. What am I missing?
Sorry about the confusion. The information I posted is more aimed at packagers and maintainers than end-users.
For example, on my RPM-based system, I can run:
$ rpm -q --provides libgccjit libgccjit = 10.2.1-1.fc32 libgccjit(x86-64) = 10.2.1-1.fc32 libgccjit.so.0()(64bit) libgccjit.so.0(LIBGCCJIT_ABI_0)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_1)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_10)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_11)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_12)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_13)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_2)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_3)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_4)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_5)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_6)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_7)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_8)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_9)(64bit)where the versions get detected by rpmbuild and "baked into" the package metadata. Similarly, packages that use libgccjit can record exactly which ABI they were built again in their package metadata, ensuring that RPM can enforce compatibility, whilst also allowing new API/ABI entrypoints to be added without bumping the SONAME, which would require everything to be recompiled.
This relies on the dynamic loader being able to express this information and enforce it, which is the case for glibc's ld.so.
It sounds like the windows dynamic linker either doesn't have these capabilities, or has different ones, and that I've assumed too much.
Does the windows dynamic linker have any such abilities, or do we simply need to express the ABI version in the dll name, or somesuch?
Hope this makes sense
Reply To davidmalcolm
$ rpm -q --provides libgccjit libgccjit = 10.2.1-1.fc32 libgccjit(x86-64) = 10.2.1-1.fc32 libgccjit.so.0()(64bit) libgccjit.so.0(LIBGCCJIT_ABI_0)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_1)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_10)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_11)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_12)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_13)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_2)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_3)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_4)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_5)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_6)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_7)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_8)(64bit) libgccjit.so.0(LIBGCCJIT_ABI_9)(64bit) }}} where the versions get detected by rpmbuild and "baked into" the package metadata. Similarly, packages that use libgccjit can record exactly which ABI they were built again in their package metadata, ensuring that RPM can enforce compatibility, whilst also allowing new API/ABI entrypoints to be added without bumping the SONAME, which would require everything to be recompiled. This relies on the dynamic loader being able to express this information and enforce it, which is the case for glibc's ld.so. It sounds like the windows dynamic linker either doesn't have these capabilities, or has different ones, and that I've assumed too much. Does the windows dynamic linker have any such abilities, or do we simply need to express the ABI version in the dll name, or somesuch?
Like I said: the Windows program loader checks at load time that all the entry points the program imports are actually present in the DLL, and if not, it will refuse to run the program.
If all the entry points are present, but some have an incompatible ABI, then the loader will not be able to detect that. But this is largely unimportant in practice, since on Windows programs are linked against the versioned DLL names, and the name is recorded in the binary. And since symlinking a versioned DLL to an unversioned name is not practiced on Windows (because symlink support is problematic and requires privilege elevation), the problem of an incompatible ABI for the same entry points doesn't happen in practice, because the versioned DLL is being looked up explicitly.
keith Reply To eliz
Why is libgccjit need to depend on libintl anyway?
I guess, to provide internationalization of diagnostic messages.
gcc.exe and cc1.exe, for example, don't depend on it.
I'm puzzled, because that's not what I'm seeing here:
$ mingw32-ldd dist/staged/bin/gcc.exe dist/staged/bin/gcc.exe +- libiconv-2.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll +- libintl-8.dll | +- ADVAPI32.DLL | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- libiconv-2.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll +- KERNEL32.dll +- msvcrt.dll +- msvcrt.dll +- USER32.dlland:
$ mingw32-ldd `find dist/staged -name cc1.exe` dist/staged/libexec/gcc/mingw32/9.2.0/cc1.exe +- libgmp-10.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll +- libiconv-2.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll +- libintl-8.dll | +- ADVAPI32.DLL | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- libiconv-2.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll +- libisl-21.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll | +- libgmp-10.dll | +- KERNEL32.dll | +- msvcrt.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll +- KERNEL32.dll +- msvcrt.dll +- libmpc-3.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libgmp-10.dll | | +- KERNEL32.dll | | +- msvcrt.dll | | +- msvcrt.dll | | +- libgcc_s_dw2-1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- libmpfr-6.dll | +- libgmp-10.dll | | +- KERNEL32.dll | | +- msvcrt.dll | | +- msvcrt.dll | | +- libgcc_s_dw2-1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll +- libmpfr-6.dll | +- libgmp-10.dll | | +- KERNEL32.dll | | +- msvcrt.dll | | +- msvcrt.dll | | +- libgcc_s_dw2-1.dll | | +- KERNEL32.dll | | +- msvcrt.dll | +- KERNEL32.dll | +- msvcrt.dll | +- libgcc_s_dw2-1.dll | +- KERNEL32.dll | +- msvcrt.dll +- msvcrt.dll +- USER32.dllHowever, I do see that the version of gcc.exe in FRS is double the size of my staged copy, and appears to be statically linked with both libiconv.a and libintl.a, rather than with libiconv.dll.a and libintl.dll.a. I think I know why, but I'd like to review the issue; the DLL dependencies were intended.
keith Reply To eliz
Btw, judging by the Info manual that comes with libgccjit, a more appropriate name for the DLL would be libgccjit-10.dll, since it implements LIBGCCJIT_ABI_10 (and the next versions of GCC advance the ABI by several more notches). MinGW64 also calls it libgccjit-0.dll, so we could adopt the same name, regardless. But then we will need to invent ABI numbers out of thin air for future releases, which might be sub-optimal.
Our DLL version numbering scheme, like that also used by Cygwin, is based on the libtool interface version numbering conventions; like Cygwin, we compute the number, included within the DLL file name, as the libtool current version number minus the libtool age. In this case, current would appear to be 10, but, as David has indicated, ABI 10 is backwardly compatible with ABI 0, and every version between. Thus, age would also be 10, and the appropriate current - age result is, indeed, 0.
This stratagem was originally promoted by Charles (Chuck) Wilson, who contributed to both MinGW and Cygwin; his rationale was to ensure that any dependency on a specific version of a DLL could always be satisfied, through backward compatibility, by the most recent release of that specific version, but there was never any expectation, much less guarantee, that older versions of any DLL would remain forward compatible with newer applications.
Reply To keith
Reply To eliz
Btw, judging by the Info manual that comes with libgccjit, a more appropriate name for the DLL would be libgccjit-10.dll, since it implements LIBGCCJIT_ABI_10 (and the next versions of GCC advance the ABI by several more notches). MinGW64 also calls it libgccjit-0.dll, so we could adopt the same name, regardless. But then we will need to invent ABI numbers out of thin air for future releases, which might be sub-optimal.
Our DLL version numbering scheme, like that also used by Cygwin, is based on the libtool interface version numbering conventions; like Cygwin, we compute the number, included within the DLL file name, as the libtool current version number minus the libtool age. In this case, current would appear to be 10, but, as David has indicated, ABI 10 is backwardly compatible with ABI 0, and every version between. Thus, age would also be 10, and the appropriate current - age result is, indeed, 0.
I'm aware of the numbering scheme, I just wasn't sure that libtool was involved in the build and applied the scheme. From your description I deduced, perhaps incorrectly, that building libgccjit for MinGW was not yet supported, and that led me to the conclusion that perhaps the 0 part was some kind of default, not a number correctly calculated from the ABI data.
It is also a certain surprise for me to read that adding entry points is still considered to be a "compatible" ABI. I always thought that if the DLL name remains the same, then any program linked against any version with that name will be able to run with any other version of that DLL that has the same name. I guess there's something new to learn every day...
Is libgccjit indeed fully backward compatible, btw?
Reply To keith
Reply To eliz
Why is libgccjit need to depend on libintl anyway?
However, I do see that the version of gcc.exe in FRS is double the size of my staged copy, and appears to be statically linked with both libiconv.a and libintl.a, rather than with libiconv.dll.a and libintl.dll.a. I think I know why, but I'd like to review the issue; the DLL dependencies were intended.
I think statically linking against libintl is a Good Thing, especially since, as we see, the newer libintl-8.dll is not 100% compatible, as it causes the program to use libintl's version of setlocale. If using that entry point could be avoided, then a dynamic dependency on libintl wouldn't be a problem, I think (or rather hope). Otherwise, it's a bit of DLL hell, given the same name of the DLL.
keith Reply To eliz
Reply To keith
Our DLL version numbering scheme, like that also used by Cygwin, is based on the libtool interface version numbering conventions; like Cygwin, we compute the number, included within the DLL file name, as the libtool current version number minus the libtool age. In this case, current would appear to be 10, but, as David has indicated, ABI 10 is backwardly compatible with ABI 0, and every version between. Thus, age would also be 10, and the appropriate current - age result is, indeed, 0.
I'm aware of the numbering scheme, I just wasn't sure that libtool was involved in the build and applied the scheme. From your description I deduced, perhaps incorrectly, that building libgccjit for MinGW was not yet supported, and that led me to the conclusion that perhaps the 0 part was some kind of default, not a number correctly calculated from the ABI data.
In the case of libgccjit, to get infrastructure appropriate to MinGW, I did have to patch the build system. Libtool isn't used, (and wasn't for the original ELF configuration either). However, that does not prevent us from adopting the libtool versioning convention; I simply deduced, from the original soname attributes, that the appropriate current - age value would be zero, and assigned it manually.
It is also a certain surprise for me to read that adding entry points is still considered to be a "compatible" ABI. I always thought that if the DLL name remains the same, then any program linked against any version with that name will be able to run with any other version of that DLL that has the same name. I guess there's something new to learn every day...
Depends on how you define "compatible". In the MinGW ecosystem, (as it is in Cygwin's), the version number encoded within the DLL name, (equivalent to current - age in libtool parlance), represents the oldest current version of the ABI, with which a particular release of the DLL remains backwardly compatible. However, there is absolutely no guarantee of forward compatibility; the most recent release of any particularly named version of a given DLL should be fully compatible with any application requiring (and possibly working with an older release of) a similarly named DLL, but older releases of that DLL may not support a newer application.
The original reference no longer appears to be accessible, but this Wayback Machine archived reference may help to explain it.
Is libgccjit indeed fully backward compatible, btw?
I don't know for sure, but on the basis of David Malcolm's statement, (and, AIUI, he is the developer and maintainer of libgccjit), I must assume that it is.
keith)
keith Reply To eliz
I think statically linking against libintl is a Good Thing,
You are fully entitled to that opinion, but please allow me to disagree...
especially since, as we see, the newer libintl-8.dll is not 100% compatible, as it causes the program to use libintl's version of setlocale. If using that entry point could be avoided, then a dynamic dependency on libintl wouldn't be a problem, I think (or rather hope). Otherwise, it's a bit of DLL hell, given the same name of the DLL.
Are you saying that you have applications which depend on your older release of libintl-8.dll, and which will not work with MinGW's latest libintl-8.dll release? If that is indeed the case, then either your libintl-8.dll, (which I assume you built yourself), may be incompatible with any MinGW release, or GNU gettext has introduced an incompatible change, upstream, without making the necessary adjustment in the libtool versioning attributes; in the latter case, that should be considered to be an upstream bug, and should be reported as such. (Do, please, remember that the MinGW/Cygwin DLL versioning conventions promise only backward compatibility for older applications; there is no guarantee that an outdated DLL release will remain forward compatible with newer applications).
Reply To keith
Reply To eliz
I think statically linking against libintl is a Good Thing,
You are fully entitled to that opinion, but please allow me to disagree...
Can you tell why you disagree?
Are you saying that you have applications which depend on your older release of libintl-8.dll, and which will not work with MinGW's latest libintl-8.dll release?
No, that's not what I'm saying. I said "a bit of DLL hell" because I'll have to cope with 2 different DLLs with the same name.
keith Reply To eliz
Reply To keith
Reply To eliz
I think statically linking against libintl is a Good Thing,
You are fully entitled to that opinion, but please allow me to disagree...
Can you tell why you disagree?
We have umpteen different packages — each with a fairly significant build time — and each individually dependent on libintl. If a bug is identified, in libintl, and each of those umpteen dependent packages is statically linked with libintl.a, then when fixing the bug, I have to rebuild every one of those umpteen dependent packages. If they are linked with libintl-8.dll, then I have only to fix the bug, and deploy an updated libintl-8.dll, and all of the umpteen dependent packages inherit the fix, without further effort.
Additionally, although possibly a lesser advantage, but an advantage nonetheless: linking with shared libraries reduces package size, consequently reducing disk usage, and upload/download bandwidth.
Are you saying that you have applications which depend on your older release of libintl-8.dll, and which will not work with MinGW's latest libintl-8.dll release?
No, that's not what I'm saying. I said "a bit of DLL hell" because I'll have to cope with 2 different DLLs with the same name.
Why? You should need only the most recent release of libintl-8.dll, (whether yours or mine), and all of your dependent applications should be happy; no need for your own (older) version. Besides, if static linking is such a "good thing", why don't you just link your own applications statically? If you did this, why was your libintl-8.dll ever needed anyway?
Forgive me, but I simply don't understand your reasoning here.
Reply To keith
Reply To eliz
Reply To keith
Reply To eliz
I think statically linking against libintl is a Good Thing,
Are you saying that you have applications which depend on your older release of libintl-8.dll, and which will not work with MinGW's latest libintl-8.dll release?
No, that's not what I'm saying. I said "a bit of DLL hell" because I'll have to cope with 2 different DLLs with the same name.
Why? You should need only the most recent release of libintl-8.dll, (whether yours or mine), and all of your dependent applications should be happy; no need for your own (older) version. Besides, if static linking is such a "good thing", why don't you just link your own applications statically? If you did this, why was your libintl-8.dll ever needed anyway? Forgive me, but I simply don't understand your reasoning here.
I've seen libintl-8.dll versions that caused crashes on program exit, so I'm trying to use only the ones I trust. Maybe the one we have now on the MinGW site can also be trusted, but I will need to collect that experience from scratch.
IOW, I simply value my extremely stable system, and would like it to remain stable. So I try to avoid unnecessary changes to the basics, and libintl is part of that.
I hope this explains my reasoning.
Keith, thanks for all the porting work you've done.
I've now looked through the patches themselves, and suddenly realized that they conflict in places with
by Nicolas Bértolo which is on "master" for gcc 11, but not on the branches for older releases. It looks like this might do much of what your patches do. Is this patch in the code that you're working with? What GCC version are you using? Should this be backported; if so, which version?
I'm sorry for not noticing this earlier in this discussion (in my defence it was back in May, and 2020 has been a difficult year).
That said, your patches do have additional changes that appear to be necessary (in particular the mutex one). I wonder why Nicholas got it working, but you needed additional patches. Maybe he merely fixed the build, not the actually execution?
Do you have a copyright assignment in place with the FSF? According to
the FSF considers that for a patch or collection of patches that touch more than around 15 lines of code and/or text that is legally significant for copyright purposes, the FSF needs copyright papers for that contribution, which seems to be the case here (e.g. just the mutex one is > 15). Let me know if you want my help with this, or if this is an issue.
Looking at the patches themselves:
20-libiberty-mingw-host-shared.patch(971bytes)
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
21-libgccjit-win32-fchmod.patch(1012bytes)
This patch conflicts with https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=c83027f32d9cca84959c7d6a1e519a0129731501 by Nicolas Bértolo which is on "master" for gcc 11, but not on the branches for older releases. Looking at it, I think he merely hacked out the call on Windows, which may lead to a run-time failure: on WIndows, is a DLL required to be executable for it to be linkable into the process? I also wondered if there's a possible security issue here with closing the file and changing its permissions by name, but the DLL is created within a tempdir that only the user should have access to, with a random name.
22-libgccjit-win32-dlfcn.patch(532bytes)
Seems reasonable, though I wonder why Nicolas Bértolo didn't run into this.
23-libgccjit-win32-no-pthreads.patch(2KB)
> @@ -2138,12 +2173,12 @@ void
> playback::context::release_mutex ()
> {
> /* Release the big GCC mutex. */
> JIT_LOG_SCOPE (get_logger ());
> gcc_assert (active_playback_ctxt == this);
> + JIT_CONTEXT_RELEASE_LOCK (jit_mutex);
> active_playback_ctxt = NULL;
> - pthread_mutex_unlock (&jit_mutex);
> }
Usage of "active_playback_ctxt" must be guarded by the mutex, so the JIT_CONTEXT_RELEASE_LOCK needs to be after the
i.e. where the pthread_mutex_unlock call is.
Looking at the jit_mutex_initializer and the docs for https://docs.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-initializecriticalsection I see that InitializeCriticalSection needs to be called exactly once on a CRITICAL_SECTION. I see that the code does this via "static jit_mutex;" Presumably this leads to jit_mutex's ctor being called once when the DLL is loaded. Does this work correctly? (I'm always a little nervous about global ctors; older toolchains don't always implement it reliably)
24-libgccjit-full-driver-name.patch(616bytes)
Nicholas made the same change as part of c83027f32d9cca84959c7d6a1e519a0129731501.
25-libgccjit-mingw-link-options.patch(3KB)
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
26-libgccjit-mingw-host-shared.patch(2KB)
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
27-libgccjit-invalid-assertion.patch(1KB)
Nicholas completely rewrote this code for Windows as part of c83027f32d9cca84959c7d6a1e519a0129731501. Sorry again for not realizing it earlier. I'd appreciate it if you could look over gcc/jit/jit-w32.c in his patch to see if you know of a better way to securely create a temporary directory on Windows (or, indeed, if there are other issues).
Thanks again for your work on this, and again, sorry for taking so long to spot the duplication.
Reply To davidmalcolm
Keith, thanks for all the porting work you've done. I've now looked through the patches themselves, and suddenly realized that they conflict in places with https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=c83027f32d9cca84959c7d6a1e519a0129731501 by Nicolas Bértolo which is on "master" for gcc 11, but not on the branches for older releases. It looks like this might do much of what your patches do. Is this patch in the code that you're working with? What GCC version are you using? Should this be backported; if so, which version?
I'm not Keith, but let me say one thing that could answer several questions you ask: Nicolas was using MinGW64, a different flavor of GCC and runtime environment ported to native MS-Windows.
That said, your patches do have additional changes that appear to be necessary (in particular the mutex one). I wonder why Nicholas got it working, but you needed additional patches. Maybe he merely fixed the build, not the actually execution?
I believe MinGW64 uses pthreads as an integral part of their GCC port, so Nicolas didn't need this. IMNSHO, it is better to avoid gratuitous dependencies on pthreads, so I think what Keith did here is cleaner. However, if the MinGW64 port must use pthreads, then I guess both varieties will need to be supported.
22-libgccjit-win32-dlfcn.patch(532bytes) Patch to circumvent gratuitous references to undeclared <dlfcn.h> functions Seems reasonable, though I wonder why Nicolas Bértolo didn't run into this.
AFAIK, MinGW64 doesn't have the dlfcn.h header and the related emulations of Posix functions, while mingw.org's MinGW does have that.
keith Reply To davidmalcolm
I've now looked through the patches themselves, and suddenly realized that they conflict in places with https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=c83027f32d9cca84959c7d6a1e519a0129731501 by Nicolas Bértolo which is on "master" for gcc 11, but not on the branches for older releases. It looks like this might do much of what your patches do.
Quite possibly. As Eli has already pointed out, Nicolas is focussed on work for MINGW64 — actually the trademark-infringing mingw-w64 project — whereas my efforts are on behalf of the legitimate owner of the MinGW trademark. I guess it's inevitable that we would come up with some very similar changes. In any case, I'm not at all enthusiastic about the way in which Nicolas has bundled everything into just one patch; I much prefer a coherent series of granular patches, (which I can manage with Mercurial Queues), so that I can more readily adapt to variations between releases, (and to conflicts with other contributors' patches).
Is this patch in the code that you're working with?
No, I don't think so.
What GCC version are you using?
As I mentioned, in an earlier comment, the last GCC release I've published for MinGW is GCC-9.2.0; I've simply been tweaking my existing build tree for that, to explore the feasibility of incorporating libgccjit support, before I move on to GCC-10. (I tend to match my build cycles, as my available time permits, to the native compiler version on my Manjaro-Linux build machine; this is currently GCC-10.2).
Should this be backported; if so, which version?
I already have a significant collection of local patches — 28 currently, with 3 no longer required — which I apply for MinGW releases; I'm quite happy to continue with this publication strategy, so, while backporting may be useful, it's by no means imperative. I will not be looking at anything earlier than GCC-9.2.0, as a MinGW libgccjit candidate.
I'm sorry for not noticing this earlier in this discussion (in my defence it was back in May, and 2020 has been a difficult year).
Don't we all know it :-)
That said, your patches do have additional changes that appear to be necessary (in particular the mutex one). I wonder why Nicholas got it working, but you needed additional patches. Maybe he merely fixed the build, not the actually execution?
I can't speak for anything emanating from the mingw-w64 folks — they are a law on to themselves. Perhaps Nicolas was happy to introduce a dependency on a third-party (their own?) implementation of POSIX threads for Windows. I could have done likewise, but I much prefer the native solution.
Do you have a copyright assignment in place with the FSF?
I do, in respect of my contributions to the GNU troff (groff) project, but not specific to GCC.
keith Reply To davidmalcolm
Looking at the patches themselves:
20-libiberty-mingw-host-shared.patch(971bytes)
- Patch to avoid bogus libiberty/pic path references
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
Similar, but not quite the same. My change is more compact, avoids the (unnecessary) definition of an additional (new) make variable, and my accompanying comment offers an explanation as to why we don't need a PIC-specific variant of libiberty for MinGW, (which, IMO is a glaring omission from Nicolas' patch).
keith)
keith Revisiting previous comment by keith
Reply To eliz
Why [does] libgccjit need to depend on libintl anyway? gcc.exe and cc1.exe, for example, don't depend on it.
I'm puzzled, because [what I see here is that both gcc.exe and cc1.exe are dynamically linked with libintl-8.dll (and also with libiconv-2.dll), but] I do see that the version of gcc.exe in FRS is double the size of my staged copy, and appears to be statically linked with both libiconv.a and libintl.a, rather than with libiconv.dll.a and libintl.dll.a. I think I know why, but I'd like to review the issue; the DLL dependencies were intended.
This anomaly appears to result from a bug in GCC configuration scripts ... possibly originating in autoconf macros provided by GNU gettext and GNU libiconv respectively: if both libintl.dll.a and libintl.a are present, with equal precedence in the linker search path, then GCC's configure will write makefiles with libintl.a as the library reference, instead of -lintl (as it should be); likewise for libiconv.a vs. -liconv, when libiconv.dll.a is also present. This is just plain wrong ... it completely defeats the design behaviour of the GNU linker, when building for a Windows target, which stipulates that, in the presence of both libfoo.dll.a and libfoo.a, then -lfoo maps to libfoo.dll.a, unless GCC's -static option is specified, or -lfoo is resolved within the scope of the linker's -B static context; in either of these latter cases, libfoo.dll.a is ignored, and -lfoo is mapped to libfoo.a.
Curiously, if libintl.a, and libiconv.a, are removed for the linker's search path, leaving just libintl.dll.a, and libiconv.dll.a, then the GCC configuration scripts write the makefile references correctly, as -lintl and -liconv respectively.
To work around this GCC configuration bug, I have now removed libintl.a, and libiconv.a, from the linker search path directory which is used by my mingw32 cross compiler, to ensure that the respective DLLs are linked, as I've always intended. Prior to removal of these static libraries, the configuration bug was causing inadvertent static linking, and some packages were unintentionally published, with these libraries statically linked.
Reply To keith
Revisiting previous comment by keith
Reply To eliz
Why [does] libgccjit need to depend on libintl anyway? gcc.exe and cc1.exe, for example, don't depend on it.
I'm puzzled, because [what I see here is that both gcc.exe and cc1.exe are dynamically linked with libintl-8.dll (and also with libiconv-2.dll), but] I do see that the version of gcc.exe in FRS is double the size of my staged copy, and appears to be statically linked with both libiconv.a and libintl.a, rather than with libiconv.dll.a and libintl.dll.a. I think I know why, but I'd like to review the issue; the DLL dependencies were intended.
This anomaly appears to result from a bug in GCC configuration scripts ... possibly originating in autoconf macros provided by GNU gettext and GNU libiconv respectively: if both libintl.dll.a and libintl.a are present, with equal precedence in the linker search path, then GCC's configure will write makefiles with libintl.a as the library reference, instead of -lintl (as it should be); likewise for libiconv.a vs. -liconv, when libiconv.dll.a is also present. This is just plain wrong ... it completely defeats the design behaviour of the GNU linker, when building for a Windows target, which stipulates that, in the presence of both libfoo.dll.a and libfoo.a, then -lfoo maps to libfoo.dll.a, unless GCC's -static option is specified, or -lfoo is resolved within the scope of the linker's -B static context; in either of these latter cases, libfoo.dll.a is ignored, and -lfoo is mapped to libfoo.a. Curiously, if libintl.a, and libiconv.a, are removed for the linker's search path, leaving just libintl.dll.a, and libiconv.dll.a, then the GCC configuration scripts write the makefile references correctly, as -lintl and -liconv respectively. To work around this GCC configuration bug, I have now removed libintl.a, and libiconv.a, from the linker search path directory which is used by my mingw32 cross compiler, to ensure that the respective DLLs are linked, as I've always intended. Prior to removal of these static libraries, the configuration bug was causing inadvertent static linking, and some packages were unintentionally published, with these libraries statically linked.
The dependency on libiconv and libintl DLLs might be okay for GCC, but there is a larger problem here, related to those DLLs themselves. This is no longer directly related to this ticket, so I will raise that issue on the MinGW mailing list.
keith Reply To davidmalcolm
Looking at the patches themselves:
21-libgccjit-win32-fchmod.patch(1012bytes)
- Patch to correct for gratuitous misuse of fchmod()
This patch conflicts with https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=c83027f32d9cca84959c7d6a1e519a0129731501 by Nicolas Bértolo which is on "master" for gcc 11, but not on the branches for older releases.
Right. So, it wasn't in the GCC-9.2.0 tarball source, on which my patches are (and for the purposes of an immediate MinGW release, must be) based.
Looking at it, I think he merely hacked out the call on Windows, ...
That's exactly what he did. I wasn't confident enough to do that — the associated comment says:
/* Set the permissions of the copy to those of the original file,
in particular the "executable" bits. */
which doesn't give me enough confidence to dismiss bits, other than the "executable" bits, as irrelevant. If it is exclusively the "executable" bits which matter, (but the comment doesn't state this unambiguously), then the fchmod() call does become irrelevant, on Windows, because there are no such file attribute bits anyway.
which may lead to a run-time failure: on WIndows, is a DLL required to be executable for it to be linkable into the process?
No. In fact, a DLL isn't usually executable anyway; it just happens to use the same PE-coff encoding format as executables. In any case, "executability" on Windows isn't established by any "bits", which can be manipulated by chmod(), (much less fchmod(), which isn't available); it is determined, primarily, by filename extension, and in some cases, this is augmented by a magic number within the file. In the case of a DLL, the ".dll" extension identifies the file as a probable shared library, and the magic number within it confirms that it is in the PE-coff format, which is required to make it such.
I also wondered if there's a possible security issue here with closing the file and changing its permissions by name, but the DLL is created within a tempdir that only the user should have access to, with a random name.
I can't see how it would be any less secure than using fchmod() on the open file, had that been possible. In any case, if it is only the "executable" bits that matter, the whole issue becomes moot; the only bit on Windows, which chmod() can realistically manipulate is the "write" bit (and there is only one of them) — access permissions for different users, on Windows, are managed by ACLs, (and other than on WinNT derivatives, not at all), rather than by the limited set of file attribute bits
keith Reply To davidmalcolm
Looking at the patches themselves: 22-libgccjit-win32-dlfcn.patch(532bytes)
- Patch to circumvent gratuitous references to undeclared <dlfcn.h> functions
Seems reasonable, though I wonder why Nicolas Bértolo didn't run into this.
Without studying the effect of his patch in exhaustive detail, I suspect that he filtered out the <dlfcn.h> dependencies, substituting LoadLibrary() calls for dlopen(), GetProcAddress() for dlsym(), FreeLibrary() for dlclose(), and writing his own substitute for dlerror(). I didn't need to do that, because legitimate MinGW runtime library distributions have supported (mostly) POSIX-compatible emulations of dlopen(), dlsym(), dlclose(), and dlerror(), since late 2014; however, the implementation does require inclusion of <dlfcn.h>, in any translation unit using them.
keith keith keith Reply To davidmalcolm
Looking at the patches themselves:
23-libgccjit-win32-no-pthreads.patch(2KB)
- Patch to eliminate -lpthread dependency
@@ -2138,12 +2173,12 @@ void playback::context::release_mutex () { /* Release the big GCC mutex. */ JIT_LOG_SCOPE (get_logger ()); gcc_assert (active_playback_ctxt == this); + JIT_CONTEXT_RELEASE_LOCK (jit_mutex); active_playback_ctxt = NULL; - pthread_mutex_unlock (&jit_mutex); }
Usage of "active_playback_ctxt" must be guarded by the mutex, so the JIT_CONTEXT_RELEASE_LOCK needs to be after the
active_playback_ctxt = NULL;
i.e. where the pthread_mutex_unlock call is.
Okay. A careless mistake on my part; thanks for pointing it out. I've fixed it, in my local build tree, and refreshed the attached patch.
Looking at the jit_mutex_initializer and the docs for https://docs.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-initializecriticalsection I see that InitializeCriticalSection needs to be called exactly once on a CRITICAL_SECTION.
Yes ... unless DeleteCriticalSection() is called before a subsequent initialization request, (but I can't see how that might be helpful here — we need the critical section to have been initialized, before we attempt to acquire the mutex, but behaviour is undefined if we call InitializeCriticalSection() on an already initialized section, or we attempt to delete an uninitialized one).
I see that the code does this via "static jit_mutex;" Presumably this leads to jit_mutex's ctor being called once when the DLL is loaded.
Yes.
Does this work correctly?
AFAIK, it does, but it's not a feature I use very often.
(I'm always a little nervous about global ctors; older toolchains don't always implement it reliably)
I wasn't completely sure, before I created my patch, so I wrote a toy program to test the concept:
- #include <stdio.h>
- #include <winbase.h>
- #ifdef COMPILE_DLL
- extern "C" void jit_acquire_mutex( void );
- extern "C" void jit_release_mutex( void );
- static class jit_mutex_initializer
- {
- private:
- CRITICAL_SECTION mutex;
- public:
- jit_mutex_initializer();
- void lock(); void unlock();
- } jit_mutex;
- jit_mutex_initializer::jit_mutex_initializer()
- {
- InitializeCriticalSection (&mutex);
- printf ("jit_mutex initialized\n");
- }
- void jit_mutex_initializer::lock()
- {
- EnterCriticalSection (&mutex);
- printf ("jit_mutex acquired\n");
- }
- void jit_mutex_initializer::unlock()
- {
- LeaveCriticalSection (&mutex);
- printf ("jit_mutex released\n");
- }
- void jit_acquire_mutex(){ jit_mutex.lock(); }
- void jit_release_mutex(){ jit_mutex.unlock(); }
- #else
- #include <unistd.h>
- int main()
- {
- HMODULE foo_dll;
- printf ("Loading JITFOO.DLL ...\n");
- if ((foo_dll = LoadLibrary ("jitfoo.dll")) != NULL)
- {
- printf ("JITFOO.DLL loaded successfully\n");
- typedef void (*hook)(void);
- hook acquire_mutex = (hook)(GetProcAddress (foo_dll, "jit_acquire_mutex"));
- hook release_mutex = (hook)(GetProcAddress (foo_dll, "jit_release_mutex"));
- printf ("Wait for JIT_MUTEX ...\n");
- acquire_mutex(); printf ("JIT_MUTEX successfully acquired\n");
- sleep (10);
- printf ("Relinquish JIT_MUTEX ...\n");
- release_mutex(); printf ("JIT_MUTEX successfully released\n");
- }
- return 0;
- }
- #endif
$ mingw32-g++ jitfoo.cc -o ~/VirtualBox/share/jitfoo.exe $ mingw32-g++ -shared -D COMPILE_DLL jitfoo.cc -o ~/VirtualBox/share/jitfoo.dll $ ~/VirtualBox/share/jitfoo.exe Loading JITFOO.DLL ... jit_mutex initialized JITFOO.DLL loaded successfully Wait for JIT_MUTEX ... jit_mutex acquired JIT_MUTEX successfully acquired Relinquish JIT_MUTEX ... jit_mutex released JIT_MUTEX successfully released
keith Reply To davidmalcolm
Looking at the patches themselves:
24-libgccjit-full-driver-name.patch(616bytes)
- Correction for omitted $(exeext) in Makefile
Nicholas made the same change as part of c83027f32d9cca84959c7d6a1e519a0129731501.
He did, but that correction isn't in the GCC-9.2.0 tarball, so I still need my patch, for the time being. When I start working from an upstream tarball, in which the correction is already present, I will set a "guard" on my patch, so that it is skipped when my patch series is applied.
keith Reply To davidmalcolm
Looking at the patches themselves:
25-libgccjit-mingw-link-options.patch(3KB)
- Use linker options appropriate for Windows
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
Similar in effect, maybe ... streets apart, in implementation detail. I have several reservations about Nicolas' patch, in general, and specific to the scope of my 25-libgccjit-mingw-link-options.patch:
keith Expanding on previous comment by keith
Reply To davidmalcolm
Looking at the patches themselves:
25-libgccjit-mingw-link-options.patch(3KB)
- Use linker options appropriate for Windows
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.Similar in effect, maybe ...
Further to my previous comment, in relation to this patch, I notice that Nicolas introduced a new installation objective, separating the installation of libgccjit header files from the existing jit.install-common objective. While this separation may be advantageous, it is orthogonal to the objective of achieving MinGW compatibility; consequently, no such separation is specified in my patch series.
keith Reply To davidmalcolm
Looking at the patches themselves:
26-libgccjit-mingw-host-shared.patch(2KB)
- Patch to override --enable-host-shared requirement
As far as I can tell, Nicholas made similar changes as part of c83027f32d9cca84959c7d6a1e519a0129731501.
Functionally identical, yes, but I chose not to introduce white-space-only changes spanning more than a dozen lines. I would also respectively suggest that Nicolas' "but not if building for Mingw" addition to the comment is grossly unsatisfactory ... the code itself tells me that it isn't done for MinGW; the comment should explain why.
As an aside, I would also note that both Nicolas' patch, and my 26-libgccjit-mingw-host-shared.patch, touch both configure.ac, and configure. While that is appropriate, for a patch to be applied to a tarball image, in which both files must be present, I would question its relevance in a patch against an SCM repository; does it not create potential for "thrashing", in the content of a generated file, such as configure? (IMO, generated files, such as configure, do not belong in SCM, but I do understand why some developers may choose to include them).
keith Reply To davidmalcolm
Looking at the patches themselves:
27-libgccjit-invalid-assertion.patch(1KB)
- Patch to correct invalid assumption in DIR_SEPARATOR verification assertion
Nicholas completely rewrote this code for Windows as part of c83027f32d9cca84959c7d6a1e519a0129731501. Sorry again for not realizing it earlier. I'd appreciate it if you could look over gcc/jit/jit-w32.c in his patch to see if you know of a better way to securely create a temporary directory on Windows (or, indeed, if there are other issues).
Sorry, but I simply could not accept Nicolas' gcc/jit/jit-w32.c, for MinGW.
The file, itself, implements two publicly visible functions:
Given the above, when I eventually progress to working from a tarball to which Nicolas' gcc/jit/jit-w32.c patch has been applied, I will be obliged to revert it, and persist with my own local patches.
Keith, thank you for your work on porting libgccjit to MinGW. I have now quite successfully used this to build the latest version of the "native-comp" branch of GNU Emacs using mingw.org's MinGW tools.
However, there seems to be a fly in the ointment: Emacs built with the library sometimes (rarely) crashes when calling libgccjit functions. It crashes in strange ways: the C stack seems to be smashed when it does (GDB is unable to show a backtrace), and stepping through the Emacs code which involves libgccjit calls affects which of the libgccjit calls crashes(!).
So I need to put some serious debugging into this, but the fact that libgccjit-0.dll is stripped of all debug symbols gets in the way. Would it be possible for you to make the DLL with the debug symbols available for download?
Also, the GCC source distribution on the MinGW site seems to be before applying all the MinGW patches: at least the libgccjit patches seem to be not applied. is that intentional?
Reply To eliz
Keith, thank you for your work on porting libgccjit to MinGW. I have now quite successfully used this to build the latest version of the "native-comp" branch of GNU Emacs using mingw.org's MinGW tools.
And one more request: the code in jit-recording.c opens the "dump" files in the default "text" mode. This is a bad idea on Windows, since the files it needs to write may have all kinds of unusual characters, like control characters etc. I attach a trivial patch to fix that, I hope it can be applied for the next release.
keith keith Reply To eliz
Keith, thank you for your work on porting libgccjit to MinGW.
I have now quite successfully used this to build the latest version of the "native-comp" branch of GNU Emacs using mingw.org's MinGW tools.
That's good to know, thanks.
Would it be possible for you to make the DLL with the debug symbols available for download?
Sure. Uncompressed, its a 270MB monster, which exceeds the attachment size limit, so I've attached an XZ compressed copy; (originally uploaded from my build tree, before applying your subsequent file mode patch ... I've now refreshed it, to incorporate your patch).
Also, the GCC source distribution on the MinGW site seems to be before applying all the MinGW patches: at least the libgccjit patches seem to be not applied. is that intentional?
Yes, that is intentional. It's a philosophical choice, whether to distribute pristine source with patches to be applied at point of use, or to distribute already patched source; we have always adopted the former convention. FWIW, these days I use mingw-pkg to construct the packages which I contribute; that folds the patch set, together with the MinGW package specification files, into the arch/mingw32 subdirectory of the source tarball, (and also offers the mingw-pkg patch command, to apply them in the correct, alphanumerically sorted by file name, order).
keith)
Reply To keith
Would it be possible for you to make the DLL with the debug symbols available for download?
Sure. Uncompressed, its a 270MB monster, which exceeds the attachment size limit, so I've attached an XZ compressed copy; (originally uploaded from my build tree, before applying your subsequent file mode patch ... I've now updated it to incorporate your patch).
Thanks.
Also, the GCC source distribution on the MinGW site seems to be before applying all the MinGW patches: at least the libgccjit patches seem to be not applied. is that intentional?
Yes, that is intentional. It's a philosophical choice, whether to distribute pristine source with patches to be applied at point of use, or to distribute already patched source; we have always adopted the former convention. FWIW, these days I use mingw-pkg to construct the packages which I contribute; that folds the patch set, together with the MinGW package specification files, into the arch/mingw32 subdirectory of the source tarball, (and also offers the mingw-pkg patch command, to apply them in the correct, alphanumerically sorted by file name, order).
I understand. The one annoyance with this is that it makes source-level debugging problematic, unless one patches the tree.
keith)
keith keith Reply To eliz
Reply To eliz
Keith, thank you for your work on porting libgccjit to MinGW.
I have now quite successfully used this to build the latest version of the "native-comp" branch of GNU Emacs using mingw.org's MinGW tools.
And one more request: the code in jit-recording.c opens the "dump" files in the default "text" mode. This is a bad idea on Windows, since the files it needs to write may have all kinds of unusual characters, like control characters etc. I attach a trivial patch to fix that, I hope it can be applied for the next release.
Thanks. I've imported your patch into my build tree, rebuilt, and refreshed the FRS copies of gcc-9.2.0-3-mingw32-src.tar.tx and libgccjit-9.2.0-3-mingw32-dll-0.tar.xz, (together with their respective GPG signature files). I've also replaced the debugging version of libgccjit-0,dll, previously attached hereto, with an updated copy, including your patch.
keith Reply To eliz
Reply To keith
Also, the GCC source distribution on the MinGW site seems to be before applying all the MinGW patches: at least the libgccjit patches seem to be not applied. is that intentional?
Yes, that is intentional. It's a philosophical choice, whether to distribute pristine source with patches to be applied at point of use, or to distribute already patched source; we have always adopted the former convention. FWIW, these days I use mingw-pkg to construct the packages which I contribute; that folds the patch set, together with the MinGW package specification files, into the arch/mingw32 subdirectory of the source tarball, (and also offers the mingw-pkg patch command, to apply them in the correct, alphanumerically sorted by file name, order).
I understand. The one annoyance with this is that it makes source-level debugging problematic, unless one patches the tree.
Understood, but it can also be problematic the other way round. My personal preference is to make my own choice when, and to what extent, to patch the tree, so I prefer to have pristine source plus patches. Not only does that fit better with the work-flow strategy embodied within mingw-pkg, but it's more consistent with the mingw-port distribution model, (in which we distribute only the patches and specification files, then complete the tree by importing upstream source).
Please add libgccjit to the binaries included in the MinGW GCC distributions. This is required to be able to build projects that use libgccjit for JIT compilation of code. One example of this is "gccemacs", a branch of GNU Emacs development (soon to land on the master branch of Emacs) that compiles Emacs Lisp programs into native x86 code for faster runtime performance.
Thank you.