Ticket #33412

Figure out what is going on with Unicode libraries

Open Date: 2014-03-08 01:11 Last Update: 2015-06-23 17:16

Open [Owner assigned]
5 - Medium
5 - Medium


It's not clear what is going on with the files in the Unicode/ section: what purpose they serve, how many of them are really source files as opposed to generated files, and what their relationship is with libunicodenames and libuninameslist. It appears at the moment some generated C files are checked into version control, and they really shouldn't be. The build system should generate them as needed. It also appears that libunicodenames and libuninameslist are separate from each other, and from FontForge, for political rather than technical reasons. At risk of taking a side in the politics, it might be good to figure out which one is actually the right one to use - if any - and stop trying to support both as optional add-ons.

Ticket History (3/4 Histories)

2014-03-08 01:11 Updated by: mskala
  • New Ticket "Figure out what is going on with Unicode libraries" created
2014-03-17 04:49 Updated by: abelcheung

For the issue of libunicodenames and libuninameslist, I think you can actually remove support of them altogether. They are only used in unicode character description popup in font view and nowhere else (except also exported in both native & python scripting languages). The only purpose of the libraries are getting annotation of a unicode code point, which is completely useless as far as Fontanvil is concerned.

There are indeed political reasons behind the libraries -- you have sticked in fontforge devel list long enough so I suppose you were already aware back at 2012. Barry Schwartz changed fontforge to use libunicodenames, but later he rage quit, and joescat reverted fontforge to use libuninameslist again. To my eyes libunicodenames is superior technically (libuninameslist is merely a long list of strings bunched in a c function), but for your purpose, Fontanvil doesn't need any unicode annotation at all.

For the utype thing, you might want to be a bit more careful though. It looks like George has redefined standard functions such as toupper/tolower to use fontforge's internal ones, as well as hard wiring some other unicode properties checking in the code. Probably you can eliminate some uses of toupper/tolower though, since they are mostly used for file name matching (which might be ok) and keyword matching in font files (which is not so ok; tolower/toupper are locale dependent).

2014-03-18 00:05 Updated by: mskala

Thanks for the comments. I'm not sure I want to remove the support for the Unicode name libraries entirely, because of the script visibility issue - even though I don't make calls to those script functions myself, removing them means that if anyone writes a script that uses the Unicode libraries, that script won't run on FontAnvil. And "don't break scripts!" is a big part of the point of the exercise. However, it may well be that what's needed to support the scripting features is much less than the whole library, and in that case, bundling just the necessary code might be a good option.

George wrote a lot of low-level string-handling code, like many different versions of "strcpy" and "strlen" to handle many different internal encodings (C strings, counted-array strings, wchar_t, explicit UTF-8) and so on. Much of that is dead code once the GUI is removed, and I've been taking it out a little at a time. The locale-dependency of toupper/tolower is a bit worrying, but we already have what looks like it may be locale dependency in the "encoding" logic. FontForge doesn't actually define most of its own encodings; instead it dynamically reverse-engineers iconv at run time, so if you set the encoding of a font to let's say "KOI8-R", what that means depends on what your iconv thinks it means, and I don't know if we can count on that being the same for everybody. I am not thrilled with this design choice, but not sure I want to make an effort to change it. The toupper/tolower stuff seems related.

2015-06-23 17:16 Updated by: mskala

Attachment File List

No attachments


You are not logged in. I you are not logged in, your comment will be treated as an anonymous post. » Login