ɑժѵҽղԵմɾҽՏ íղ մղíϲօժҽ
The Not-So-Shared Experience in Second Life
On June 20th, 2017 the Unicode foundation updated the international standard for universal character mapping to version 10. Unicode 10.0 adds 8,518 characters, for a total of 136,690 characters. These additions include 4 new scripts, for a total of 139 scripts, as well as 56 new emoji characters.
If you are a regular user of Second Life, then you�ve very likely seen these Unicode characters in use for things like announcements, custom names, and various decoration and fancy text for profiles.
But there is a bit of a problem.
While the Unicode standard is now up to version 10.0.0, a majority of Second Life viewers (including the default viewer) doesn�t really support a majority of the Unicode standard characters. So much for that �shared experience�, right?
The Unicode set comes in quite handy when you want to add a visual element to your displays. For instance, there are a number of Unicode characters for icons and emojis that, while useful in a frivolous manner for text decoration, are very useful as universal symbols to denote common tasks.
For instance, if you wanted to use the age-old icon for �Save�, you�d use the floppy disk icon, right?
What if you wanted an icon for �Favorite�? Well, the obvious symbol would be an icon of a heart, like this: ?
If all you saw with those two characters was [] for each, then you�re out of date for your font and/or Operating System. As far as I can tell, most modern operating systems and programs can see more recent Unicode characters by default.
The problem is that with Second Life, the Unicode support is all over the map at best. The default viewer (and most third party viewers) simply do not support a wide range of Unicode characters. That diskette icon from before would show up as a tofu � which is the name of the empty block symbol �[]� you see whenever the font you are using doesn�t have the symbol being used.
You�ve probably seen a lot of those �tofu� blocks in Second Life as you wander around and see people�s custom names, gestures and whatever. This is your viewer failing to support a wider range of Unicode characters which are standard internationally. More importantly, it�s your viewer not including fonts that support those Unicode characters.
The default viewer for Second Life comes with a few fonts installed, and they all fail the wider scope of Unicode support. Third party viewers like Singularity use Dejavu in a reduced format as the default font set, and doesn�t support a lot of Unicode characters up to version 7, which is actually from 2010 (7 years old!).
I won�t even get into the recently released Unicode 10 standard. That�s probably out of the question entirely, but it isn�t too much to ask for consistent support for a four year old Unicode character set.
Comparative Testing
While a majority of the Second Life viewers more or less fail a more comprehensive Unicode character test, the one viewer that actually fares better than all of them (including the default viewer) happens to be Firestorm.
As a test, the ONYX Stereo Receiver in Second Life uses Unicode icons on the menus in order to better differentiate. Keeping in mind that we didn�t use any Unicode past version 7 to be �safe� � we assumed characters that have been around for 4 years would be supported without a problem by now.
Unfortunately, this isn�t the case.
Here�s the visual comparison -
Default SL Viewer
One thing you�ll notice among the viewers that fail the Unicode test is that they all effectively crap out around Unicode 3 or 4 support in their included font(s). With Second Life Default viewer, you�d think Linden Lab would have the foresight to support a much wider set of characters in their included font.
Singularity Viewer
Next on the list for testing was Singularity Viewer, the third party viewer that I recall being the staple for Phoenix and older viewer style holdouts. Here, again, you see it fails the Unicode test on the ONYX menu.
Further research finds that Singularity uses Dejavu (much like Firestorm) but nothing else� or whatever else it has in the Fonts folder still doesn�t support a newer Unicode standard, so there are no fallbacks.
Black Dragon Viewer
So maybe I�m choosing �crappy� third party viewers, then? The low end of the spectrum, and that�s why they don�t handle Unicode so well? I wish that were the case, really� as below, you�ll see that even Black Dragon viewer fails the Unicode test.
Firestorm Viewer
And then there is Firestorm.
Notice anything different? Well, what do ya know� all the Unicode characters in the menu show up fine in Firestorm.
What�s Different?
When you look inside the Firestorm Fonts folder, you notice that there are more fonts in there than you�d find in Singularity or the Default Viewer, and while Singularity and the Default viewer also uses the Dejavu Sans font like Firestorm, all this tells us is that DejaVu isn�t the solution.
So what font is responsible for adding extensive Unicode support in the Firestorm viewer?
I�d like to say our secret sauce just might be thanks to Google.
See, I noticed in the Firestorm distribution that there includes the Noto family of fonts along with the other fonts in the folder. Noto (short for No Tofu) is a font family designed by Google to support extensive Unicode characters.
It is a free font and (from what I can see) doesn�t include any real restrictions for bundling it in your applications, and is actually encouraged. Which is probably why it shows up as part of Firestorm, in turn enabling all Firestorm users to see extended Unicode characters by default.
Specifically included in the Firestorm Fonts folder are:
- NotoMono-Regular.ttf
- NotoSans-Bold.ttf
- NotoSans-BoldItalic.ttf
- NotoSansCombined-Regular.ttf
- NotoSans-Italic.ttf
The way that the viewers work in the XML files which declare the fonts in use is � you name a main font, and then you name off some �fallback� fonts when the main font doesn�t have a particular character to display (like with Unicode or foreign languages). So when the main font �fails� to find a character, it goes to the other fallback fonts listed to see if the character is in there. At least that�s how they used to work� and now (apparently) only Firestorm kept that ability.
As far as I can tell � Only Firestorm seems to have included Noto (and a few others) in their Font folder, and therefore can render much more Unicode than everyone else. Of course, they declared Noto in the XML file as a fallback font.
Which begs the question:
If it�s really this simple to update Unicode support for Second Life viewers, then why is Firestorm the only viewer that seems to have done so?
I�m actually quite surprised that I haven�t seen any of the other third party viewers do this, nor even Linden Lab with the �official� viewer. This has got to be (hands down) the quickest �shared experience� fix they can all do.
Just include Noto in your viewers.
Until then, we�re stuck answering a design question when creating items in Second Life:
Do I use Unicode icon characters in my text and menus, do I omit the Unicode characters, do I include a toggle button to show/hide them, or do I say �To hell with it�� and just build with Unicode characters knowing a majority of people use Firestorm?
This will be the question until everyone catches up and includes better Unicode font support in their next viewer releases, catching up to Firestorm.
Until then, I�m probably just going to use the Unicode characters in menus whether you can see them or not. It�s a simple fix, and you (the users) should get on your respective viewer developers to include better support for it ASAP.
Yes� even Linden Lab.
Addendum
After a bit of tinkering/testing, I had the idea that maybe one could simply copy/paste the Firestorm Fonts folder to the other viewer Fonts folder, XML files and all for the fallbacks� but it seems the other viewers just ignore it all.
This is curious to me as there was a time when the Default viewer used the XML setup to declare the fonts and fallbacks. As a result, I believe two things are going on:
- The Default viewer (and other 3rd party Viewers) no longer have the XML functionality built in, or it�s disabled hard coded.
- Firestorm kept the original XML functionality and put it to good use.
Which now begs the question:
If the third party viewers are open source, including Firestorm, then why is this functionality only in Firestorm? Even more, why would a function that was in the older Default viewers be taken out, if it allowed for such an obvious standards and compatibility fix with little effort?
This is quite possibly one of the easiest �Shared Experience� fixes to include, and (before Linden Lab thinks of it), no that doesn�t mean make Firestorm remove the functionality. That would be the most bone-headed thing Linden Lab could do.
As a result of the other viewers ignoring the XML files, this means that (unfortunately) you will have to petition your favorite viewer developer to put the functionality back in and include the same font set as Firestorm to fix the Unicode compatibility for your particular viewer of choice (including Linden Lab).
Because apparently, they�d all have to include the XML function back into their viewers and include the fonts in the folders of their own accord.
My recommendation, then, is the following:
If you�re not using Firestorm, go to your respective viewer JIRA and file it as a feature request. In the year 2017, there is little reason a modern application shouldn�t be up to date with Unicode compatibility. We should all be raising a little bit of hell over this one�
So go get�em, tiger!
download file now