Ticket #38225

Emulate _fseeki64()/_ftelli64() API on legacy platforms.

Open Date: 2018-04-21 23:26 Last Update: 2018-12-05 05:22

5 - Medium
5 - Medium
Score: 3
100.0% (3/3)
0.0% (0/3)


Although 32-bit Windows has supported operations on files larger than 2GB, since at least WinNT4 and Win98, (and possibly earlier), Microsoft did not introduce the _fseeki64() and _ftelli64() APIs until MSVCR80.DLL, (and subsequently retrofitted it to MSVCRT.DLL, from WinVista onwards); prior to this, random access support in large files was provided through lower level APIs such as _lseeki64(), or native Windows API functions.

As noted in this mailing list query, and this subsequent follow-up thread, it is practicable to emulate the _fseeki64() and _ftelli64() APIs, such that they become available on legacy versions of Windows, which pre-date the formal introduction of these APIs by Microsoft; however, previous efforts to do so may not have been entirely robust. Hopefully, the attached patch will do a better job.

Ticket History (3/9 Histories)

2018-04-21 23:26 Updated by: keith
  • New Ticket "Emulate _fseeki64()/_ftelli64() API on legacy platforms." created
2018-04-25 01:43 Updated by: earnie

It would be extremely useful to be able to seek past the 2G limit MS has.

2018-10-10 06:04 Updated by: keith

Reply To earnie

It would be extremely useful to be able to seek past the 2G limit MS has.

It always has been possible to do so, to the extent that _lseeki64() supports it; the problem is that the effect of _lseeki64() isn't reflected in __iob context, without some additional processing.

The patch I've proposed provides fall-back implementations of _fseeki64() and _ftelli64(), based on _lseeki64() and _telli64() respectively, with the additional processing required to propagate relevant effects to the associated __iob context. Unfortunately, while these fall-back implementations should be entirely suitable for deployment on all WinNT derived platforms, they are likely to be susceptible to the "write after seek beyond EOF" issue described in Mumit Khan's mingw-fseek.c, when deployed on Win9x; however, I do have reservations about the robustness of Mumit's work-around for this issue, which may thus benefit from review, and some rework.

2018-11-26 03:15 Updated by: keith

I committed 09fb4c4, which should suffice to provide robust implementations of _fseeki64() and _ftelli64() for all WinNT derived platforms. They will also work on Win9x, but I'm leaving the ticket open for the time being, because, as I suspected, they will interact badly with fwrite() on these platforms, if the fwrite() is preceded by an _fseeki64() to an offset beyond the physical end of file.

2018-11-27 03:15 Updated by: keith

I have performed a review of Mumit Kahn's mingw-fseek.c implementation; I observe several minor issues, and at least one critical issue. The critical issue is:–

  • The fseek() wrapper sets a flag, to indicate that it has been called; this flag is then reset on the next call to the fwrite() wrapper. In neither case is there any attempt to associate the flag with any particular stream, never mind to ensure that the fwrite() is directed to the stream on which the fseek() was performed. In consequence, it is entirely possible that the flag could have been reset before the fwrite() which needs to act on it is performed.

Besides this critical issue, other non-critical issues include:–

  • Unnecessary use of (ugly) low level Windows APIs, leading to excessive complexity of the implementation.
  • Unnecessary repeated calls to the _get_osfhandle() API, within a single fwrite() wrapper call; this surely has a negative impact on performance.
  • Two fseek() wrapper functions, where one would suffice; (all fseek()-alike calls could be redirected a to single wrapper, in which a 64-bit offset argument is passed).
2018-11-27 06:38 Updated by: keith

I've attached a further patch; to resolve the issues I identified, within the Win9x fssek()/fwrite() redirector, it provides a complete reimplementation of that feature. Note that, to address the critical failing I described, in the original implementation of the redirector, it now maintains a queue of stream identifiers for which an fwrite() operation is pending, following an fseek() on the associated stream. This queue is implemented in terms of the POSIX.1-1996 linked-list queues API; as such, it is dependent on adoption of the feature request detailed on ticket #38770.

(Edited, 2018-11-27 06:40 Updated by: keith)
2018-12-05 05:22 Updated by: keith
  • Status Update from Open to Closed
  • Owner Update from (None) to keith
  • Resolution Update from None to Accepted

I committed #2c98d6a; together with previous commits #09fb4c4, and #c27255f, this completes the _fseeki64()/_ftelli64() retrofit for both WinNT and Win9x platforms, and corrects the possible misbehaviour of an immediately subsequent fwrite(), on Win9x.

This retrofit will be included within the next WSL release.

Attachment File List


Please login to add comment to this ticket » Login