Internationalizing JavaScript Applications Norbert Lindenberg. Norbert Lindenberg All rights reserved.

Transcription

1 Internationalizing JavaScript Applications Norbert Lindenberg Norbert Lindenberg All rights reserved.

2 Agenda Unicode support Collation Number and date/time formatting Localizable resources Message construction 2

3 JavaScript is ECMAScript Language ECMAScript Internationalization API Browser: DOM, Navigator, XMLHttpRequest Server: Node.js Platforms: Firefox OS, Windows 8, Phonegap Libraries: jquery, Dojo, YUI, GWT, Node modules, etc. 3

4 ECMAScript Language Specification Developed by Ecma TC 39 Language syntax and semantics Core API: Object, String, Array, RegExp,... Edition 5.1 current Edition 6 expected December

5 ECMAScript Internationalization API Specification Developed by Ecma TC 39 + experts API: Collator, NumberFormat, DateTimeFormat Edition 1 approved December 2012 Chrome, Opera, Explorer, Windows shipped; Firefox, Node.js coming Edition 2 expected December

6 Unicode

7 Unicode support All text in UTF-16 internally UTF-8 well supported for transport Need to identify charset in <script> tags, Content-Type headers Need to use encodeuricomponent for path and query string components 7

8 Occupy Wall Street.

9 Supplementary characters Characters above U+FFFF Emoji, rare CJK, ancient scripts, musical symbols,... 2 code units in UTF-16 9

10 Today: UCS-2 or UTF-16? UCS-2: Regular expressions String comparison Case conversion UTF-16: Source text conversion URI handling 10

11 Today: UCS-2 or UTF-16? UCS-2: Regular expressions String comparison Case conversion UTF-16: Source text conversion URI handling DOM, text input, text rendering, XMLHttpRequest 11

12 ECMAScript 6: UTF-16 Case conversion for full Unicode Full Unicode in identifiers String accessors for code points But: no change to low-level string comparison Planned: New Unicode mode in regular expressions 12

13 Regular expressions RegExp in ES5 doesn t have much Unicode support No support for Unicode character properties No support for supplementary characters 13

14 Regular expressions CSet (inimino): Character classes with supplementary characters XRegExp (Steven Levithan and Mathias Bynens): Unicode categories and properties with supplementary characters 14

15 Unicode normalization Makes strings be equal that users perceive as equal (more or less) ä = a U+00E4 = U+0061 U+0308 = U+AE40 = U+1100 U+1175 U+11B7 15

16 Unicode normalization ECMAScript 5 assumes normalization happens where needed Reality: applications have to do it ECMAScript 6: String.prototype.normalize " ".normalize("nfd") "\u1100\u1175\u11b7" Libraries available, but not up to date: unorm (Matsuza) Richard Ishida s normalizer 16

17 .

18 .

19 Internationalized domain names Unicode at user interface ASCII under the hood. = xn--1lq90ic7fzpc.xn--fiqs8s Main steps: normalization (as discussed) punycode (Mathias Bynens has latest) 19

20 Collation

21 Collation (sorting) Old: String.prototype.localeCompare Only string argument New: Intl.Collator locales options Fixed: String.prototype.localeCompare With locales and options arguments 21

22 Locales BCP 47 language tags Language, script, country codes es, en-au, zh-hans-cn Unicode locale extension de-u-co-phonebk Preference lists [ mr, hi, en-in ] 22

23 Locale negotiation BCP 47 Lookup [ es-gt, es-mx ] es-gt, es, es-mx Best fit implementation defined [ es-gt, es-mx ] es-gt, es-mx, es Unicode extension handled separately 23

24 Collator extensions co: collation phonebook, pinyin,... kf: case first upper, lower kn: numeric sorting 24

25 Collator options localematcher: lookup, best fit usage: sort, search sensitivity: base, accent, case, variant ignorepunctuation numeric, casefirst 25

26 Non-ECMAScript Nothing good found (some for Latin only) Collation is hard Knowledge of full Unicode character set Big tables Send lists that need alphabetic sorting to server 26

27 Number formatting 27

28 Number formatting Old: Number.prototype.toLocaleString No arguments New: Intl.NumberFormat locales options Fixed: Number.prototype.toLocaleString With locales and options arguments 28

29 NumberFormat extensions nu: numbering system 29

30 NumberFormat options localematcher: lookup, best fit style: decimal, currency, percent currency: ISO 4217 currency code currencydisplay: symbol, code, name minimum/maximum digits usegrouping 30

31 Non-ECMAScript % ๙ #, Globalize Dojo Closure Windows s ilib : currency formatting. %: percent formatting. ๙: numbering systems. #: digit settings.,: grouping separator option. : supported locales. 31

32 Date and time formatting

33 Date and time formatting Old: Date.prototype.toLocale[ Date Time]String No arguments New: Intl.DateTimeFormat locales options Fixed: Date.prototype.toLocale[ Date Time]String With locales and options arguments 33

34 DateTimeFormat extensions ca: calendar nu: numbering system 34

35 DateTimeFormat options localematcher: lookup, best fit timezone: UTC hour12 weekday, era, year, month, day, hour, minute, second, timezonename: components formatmatcher: basic, best fit 35

36 Non-ECMAScript ca tz ๙ Globalize Dojo Closure Windows 8? -?? Moment YUI ca: calendars. tz: time zones. ๙: numbering systems. : supported locales. 36

37 Resources

38 Localizable resources ECMAScript doesn t have an I/O system, therefore no standard resource loading Developers have invented many different mechanisms 38

39 Representation At runtime, use object as key-value map JSON good for transfer; not as source Localizers need comments Existing source formats have problems Java properties files are not UTF-8 gettext.po files encoding is unspecified YUI uses its own YRB format 39

40 How dynamic would you like it? Making resources available to application Inject strings/objects into JavaScript code Bundle resources with JavaScript code Load resource bundles at runtime 40

41 Injecting strings into JavaScript Server knows locale when generating JavaScript and HTML Inject strings or objects directly where needed E.g., JavaServer Pages Standard Tag Library: <fmt:bundle basename="messages > <script> alert("<fmt:message key='hello'/>"); </script> </fmt:bundle> 41

42 Injecting strings into JavaScript Problems: Mixes multiple programming languages Can introduce syntax errors through localization 42

43 Bundling resources with JavaScript Server knows locale when serving JS Bundles resources with JavaScript Convert to JavaScript/JSON Concatenate with other JavaScript Resources: var MyResources = {HELLO: " "}; Code: alert(myresources.hello); 43

44 Loading resources at runtime Locale not known until runtime Request resources at runtime Using XMLHttpRequest By creating script tag Resources in JSON or module format 44

45 Loading resources at runtime Cross-domain support? Not with XMLHttpRequest Possible with script tag Synchronous access? More convenient programming model Can lock up browser BCP 47 support? Many loaders assume aa-aa format 45

46 Access to resources in libraries Dojo Loading at runtime, synchronous GWT Injecting resources (Constant) Bundling resources (with HTML, Dictionary) YUI Bundling resources via module loader 46

47 Message construction Photo Den Widhana

48 Message construction Substitution {user} went to {city}. {user} {city} 48

49 Message construction Plurals {user} est allé à {city}. {user1} et {user2} sont allés à {city}. 1-6 forms depending on language {number, plural {one {...} few {...} many {...}}} 49

50 Message construction Gender {user} est allé à {city}. {user} est allée à {city}. 1-4 forms depending on language {gender, select {female {...} male {...} unknown {...}}} 50

51 Message construction {gender, select { female {num, plural { one {{user1} est allée à {city}.} other {{user1} et {user2} sont allées à {city}.}}} male {num, plural { one {{user1} est allé à {city}.} other {{user1} et {user2} sont allés à {city}.}}} }} 51

52 Message construction Google has MessageFormat for Closure environment Alex Sexton provided standalone version Mozilla has even more ambitious L20n library 52

53 Summary ECMAScript Internationalization API provides core functionality ecmascript-internationalization-api/ Libraries provide more internationalization support than you may think javascript-internationalization/ 53