Subject: Site update: unicode support for movie titles and vehicle makes/models
20/11/2023 @ 19:56:52: antp: Site update: unicode support for movie titles and vehicle makes/models
Feature in development: I'm switching to unicode for the whole site, allowing to use non-latin characters for titles and a good support of exotic accents for makes & models.
I'm still busy changing things, but it should arrive in the next days/week if all goes well.

For the movie titles I was planning to have two main title fields:
- Original title: we would enter the real original title, in its original alphabet
- International title: the title used internationally, as replacement of the original one when too complex (mostly for non-latin titles)
And then, like before, the list of alternative titles where we can enter the country-specific titles, alternative ones, the transliterations, etc.

The difference from before would be that as main title we would for example have "사랑의 불시착 " instead of "Sarang-ui Bulsichak", as these transliterated are not so useful for people not speaking the language (I think).

So I would always display the international title beside the original one, having the movie displayed as "사랑의 불시착 (Crash Landing on You)" instead of "Sarang-ui Bulsichak" as currently.
It adds a nice exotic touch to the site :grin: And easier for everyone to also have the international title displayed. The downside is that it makes longer titles.

What do you think?
21/11/2023 @ 13:42:03: rjluna2: (future) Site update: unicode support for movie titles and vehicle makes/models
Nice addition :smile:

I see you are trying to expand the data field upon existing data field in the database.
21/11/2023 @ 14:54:44: Parzival49: (future) Site update: unicode support for movie titles and vehicle makes/models
I agree!
Love the idea about display the international title beside the original one
But which titles to show in Last completed movie pages corner? Would it look messy as some titles are so long and some are in non-English language?
22/11/2023 @ 18:36:16: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
Good question. Indeed for that list only one should be chosen. I guess we could use the original one. Anyway when you see it in the list, the transliterated one (as currently) is usually not more explicit than the one in non-latin alphabet.
24/11/2023 @ 20:33:18: JB007: (future) Site update: unicode support for movie titles and vehicle makes/models
I find this a very good idea!
06/12/2023 @ 06:31:19: Jnglmpera: (future) Site update: unicode support for movie titles and vehicle makes/models
Very late to the party, but it's definitely a good addition for non-Latin writers like myself and the other Asian user/movie submitters :grin:
06/12/2023 @ 06:43:27: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
It took more time than expected, but it is nearly finished, I'll probably put that on the live site around this weekend or next week.
15/12/2023 @ 19:57:26: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
All is ready, I'll try to deploy the update on the site at some point this weekend, I don't know when exactly yet.
For doing the update I'll have to lock all changes on the site, and keep it locked at least a few hours after to be sure that there is no big bug.
I'll try to put a notice some time before locking everything.
15/12/2023 @ 21:23:52: dhill_cb7: (future) Site update: unicode support for movie titles and vehicle makes/models
Thanks for the heads up :smile:
15/12/2023 @ 23:44:52: Baube: (future) Site update: unicode support for movie titles and vehicle makes/models
noted :smile:
16/12/2023 @ 04:38:55: Parzival49: (future) Site update: unicode support for movie titles and vehicle makes/models
Is it possible for admins or non-admins to change those transliterated title before the site update to new one?
16/12/2023 @ 09:41:15: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
:confused: change how? Better wait for the update so they will be properly handled when entering them in the new version. It will be a long manual task to add the non-latin titles, but it is not something that has to be done quickly, it will be done from time to time.
If non-admins want to help, I'll also give an explanation on how to submit me a list of changes for mass-update (to avoid posting comments on each movie).

Also there are buttons for admins to easily switch original<->alternative and international<->alternative titles, I'll explain it here later.
16/12/2023 @ 12:52:14: RushCars24ID: (future) Site update: unicode support for movie titles and vehicle makes/models
Finally! That title idea change I'm really waiting for. About time.

...as these transliterated are not so useful for people not speaking the language (I think).

Honesty, I disagreed with this because what if those, including myself, are very average/bad at reading those non-latin characters? I feel like that's not necessary to get rid of them, it's fine to keep them. How about an optional setting option where we can restore those kind of transliterated titles back by choice? So we can switch between non-latin character and transliterated titles anytime, with either of those two in the latter will still shown by hovering our cursor pointing to the title.
16/12/2023 @ 12:54:34: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
Maybe but it would have been useful to suggest it last month :tongue:
Transliterated titles won't be gone, they would be in the alternative titles.
I'll see what I can do, I could add a specific field for the transliterated title, so the site knows which one it is from the alternative titles.
And then improve the display later.
16/12/2023 @ 14:56:38: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
Update will start soon.
If you notice errors in the next hour, it is normal, until all changes are applied there may be errors displayed for a short period.
16/12/2023 @ 15:44:38: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
All is converted. Please check now that all seems correctly converted, browse a little the site in various movie.
I case there is some big issue in the data I can still revert back to the old database.
I'll unlock the site probably in the next hours.
Please report here any error that you notice.

First "signs" of the switch to unicode is that now special characters should be correctly displayed in links, and makes with these characters are correctly handled.
e.g. İşbora is now correctly displayed with the makes starting by "I" in https://www.imcdb.org/makes.php rather than being at the top of the list.

For users who had special non-ASCII characters in their passwords (e.g. accents), you may need to reset your password. You can put the same as the old one, but as it is not stored in the database (only a hash is stored) I can't convert these (the hash seems computed differently from a unicode string).
Once the site will be unlocked, if you encounter a problem for resetting your password, do not hesitate to contact me.
16/12/2023 @ 16:05:42: antp: (future) Site update: unicode support for movie titles and vehicle makes/models
Note for admins: when the site will be unlocked, you can now easily switch between original, international and alternative titles with new buttons.

Here is an example on how easily update titles if you wish.
For this Korean movie, this is how it appears with the data from the old version of the site:

https://i.imgur.com/5oVGTiq.png

Here is how it should be modified:

https://i.imgur.com/Tf7Tybi.png

- The international title (first additional title in the old version) can be promoted to international title by clicking "Move to international title"
- Add the title in Korean alphabet as alternative title, click "Update", then click "Switch with original title"
- Now you can select "Transliterated title" as title type for the old original title.
- The Japanese title in its original form can be added as well
16/12/2023 @ 16:28:19: dhill_cb7: Site update: unicode support for movie titles and vehicle makes/models
Very nice addition antp! Can’t wait to see how we can utilize this new feature!
16/12/2023 @ 17:03:46: dsl: Site update: unicode support for movie titles and vehicle makes/models
First "signs" of the switch to unicode is that now special characters should be correctly displayed in links, and makes with these characters are correctly handled. e.g. İşbora is now correctly displayed with the makes starting by "I" in https://www.imcdb.org/makes.php rather than being at the top of the list.


Just dabbled a bit quickly in the makes list because I once posted an Åkerman and wanted to see where it now sits - and it looks as if it's now neatly in line among Ak(xxxx..) neighbours. But at the bottom of the list after the Z(xxx...)s dry up, there's a couple of lonely orphans shivering in the snow - Øveraasen and Đuro Đaković.
16/12/2023 @ 17:13:26: antp: Site update: unicode support for movie titles and vehicle makes/models
I suppose that it is because Ø is sorted after Z in language that use it, but not always for Å.
I don't know for Đ though, it should be between D and E apparently.
It is quite difficult to define a perfect international sort order, here we rely on what MariaDB does for the "utf8mb4_general_ci" character encoding :smile:
I should check if utf8mb4_unicode_ci" or even "utf8mb4_unicode_520_ci" works better, I missed that one, it is supposed to be better with various language but a little more complex for the server.

The advantage of now using the "utf8mb4" version over the classic "utf8" (or even the "iso-8859-1" used on the site before the update) is that we can already use emojis in makes & titles. This may be useful some day :grin:
These can also be used in comments, I don't recall if it was working before, but now it seems to work fine.
On Windows you can enter emojis via Win+; shortcut.
... 
Back