OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: i18n (RE: MVP Discussion) - An Updated Proposal


Hi, All,

Thank you very much for all the discussions.
I have come up with an updated proposal, which
I believe is much more acceptable for many,
while meeting most of the requirements I have seen.
Please let me start a new thread on this issue 
with the details on the updated proposal at the end 
of this message.

The difference from the last one is...
I realized that the text itself can serve as its text_id and 
that we can do away with text_id. (This realization came 
to me during my sleep last night around 3:30 am JST.)
If we use Base64-encoded MD5 hash for the string in UTF-8, 
it would be like:

-----
{
  ...
  title: {"en": "Dridex Campaign - Botnet 121"},
   ...
  translations: {
    {"text_ref": "QcsyoNdNXQf1Nis+ZvJFyQ==",
     "ja": "Dridex キャンペーン - ボットネット 121"}, 
    ...
}

It is only additional 7 bytes (without white spaces) for each text field
if there is no translation. It is self-contained so it is very unlikely this 
impacts other parts of STIX and other standards.
Any hashing algorithm works as long as it is reasonably collision-free and 
we make the hashing algorithm a part of standard 
so everyone know which it is referring.
(By the way, I used https://quickhash.com/ to calculate the hash.)
------

Regards,

Ryu

------------------------------------------------------------
Internationalization - An Updated Proposal
------------------------------------------------------------
 
- Always give the language code as the keyword for every text field
  (So that anyone can give translations to the field later, knowing 
  which language it is in.)

- Always give "text_ref" and the language code as the keyword for every translation.
  Use Base64-encoded MD5 hash of the original text in UTF-8 for the "text_ref" value to 
  refer the original text.

- One can provide the translation for one of translated texts other than the original text.
  (Example: A CTI text field created in Japanese, then it is given an English translation.
   Then German and French translations are produced based on the English translation.)

-----
- Pattern A - Translation given inside the same original package
-----

{
  "type": "package",
  ...
  "campaigns": [
    {
      "type": "campaign",
      "id": "campaign--a1201df6-c352-4a81-9c7c-5a6f896a4e31",
     "revision": 1,
      "spec_version": "stix-2.0",
      "created_at": "2015-12-03T13:13Z",
      "created_by_ref": "identity--69a17e1b-bb45-4657-9a9d-96db3faccdde",
      "title": {"en": "Dridex Campaign - Botnet 121"}, 
      "descriptions": {"en": "Dridex-based campaign leveraging Botnet 121"}, 
      "intended_effects": [
        {"value": "theft-identity-theft"}
      ],
      "status": "Ongoing"
   }
  ],
  "translations": [
    {"text_ref: "QcsyoNdNXQf1Nis+ZvJFyQ==", 
     "ja": "Dridex キャンペーン - ボットネット 121"}, 
    {"text_ref": "6EZdQR9lgOi2fXePJaeCNA==", 
     "ja": "ボットネット 121 を活用する Dridex を元にしたキャンペーン"}, 
    {"text_ref": "QcsyoNdNXQf1Nis+ZvJFyQ==", 
     "de": "Some German Title"}, 
    {"text_ref": "6EZdQR9lgOi2fXePJaeCNA==", 
     "de": "Some German Description"}
  ]
  ...
}
 
-----
- Pattern B - Translation given by a third-party in some external database
-----

{
  "translations": [
    {"text_ref": "QcsyoNdNXQf1Nis+ZvJFyQ==", 
     "es": "Some Spanish Title"}, 
    {"text_ref": "6EZdQR9lgOi2fXePJaeCNA==", 
     "es": "Some Spanish Description"}, 
    {"text_ref": "QcsyoNdNXQf1Nis+ZvJFyQ==", 
     "fr": "Some French Title"}, 
    {"text_ref": "6EZdQR9lgOi2fXePJaeCNA==", 
     "fr": "Some French Description"}
  ]
}
-----

------------------------------
Notes - Simple, minimal, coherent, consistent, self-contained, context-free, future-proofed
------------------------------

- Only seven additional bytes (without white spaces) for each text field.

- As it is refers to the text itself, it does not break if there is 
  revisions of the objects as long as the text stays the same. 

- As its scope is limited to text-fields and therefore it is self-contained:

  - It is very unlikely this impacts other parts of STIX and other standards. 

  - There will be very little (if not "no") considerations necessary 
    for future standard developments/changes. 

  - It would be easy to implement as the same and context-free codes can 
    handle any text field. 

- There is only one way to express text fields and translations

- Resources spent for translation will not be wasted as long as the text stays same.

  - Even if someone else reuses the same text, its translations are still applicable.
 
------------------------------
Internationalization Use Cases
------------------------------

CN: Chinese
DE: German
EN: English
FR: French
JA: Japanese

------------------------------
(1) Providing text fields in multiple languages simultaneously at the time of creation.
------------------------------

  [ja/en (in case of Japan), en/fr/de (in case of EU countries), etc.]

This is the most likely use case (for me). The original CTI has titles/descriptions in 
multiple languages from the start. When you create a CTI file, you include 
both English and Japanese titles/descriptions for major objects in it
so that non-Japanese speaking people can at least find out what it is at the top level.

Or another use case is the CTI provider in Japan writes a CTI file with its
title in English and description in Japanese. This is because many Japanese
can read short English titles, but many Japanese have difficulties to understand
long and detailed descriptions in English. 

------------------------------
(2) CTI Database Receiving CTI from Multiple CTI Sources in Different Languages
------------------------------

This is a case where you receive CTI from a English CTI source and 
another CTI source in Japanese. 
You put all CTI into MongoDB or some other No-SQL Database and 
would like to do mix and match. I would like the CTI Database still 
can track the language code of textual fields.

------------------------------
(3) EN CTI received by a Japanese entity, which provides EN translation
  (Or vice versa, JA CTI received by a US entity, which provides EN translation
------------------------------

  A Japanese entity receives CTI information pieces in English.
  The entity determines some of them are important/critical
  and worth translating them into Japanese, add descriptions in Japanese
  and redistribute them to other Japanese entities (if redistribution is allowed).
  The CTIM (CTI Management System) of a receiving party displays
  the Japanese description whenever possible, while allowing access to
  the original English descriptions."

  Work Flow:
  1. Company 1 in EN creates an Indicator and TTP and shares them to Company 2 in JP.  
    It is important to note that the flow may be direct or may be through a series of brokers and other entities.  
    1. This Indicator and TTP has a producer of Company 1 and a version of 1
  2. Company 2 builds a translated version of the TTP and Indicator and releases it.
    1. This new Indicator and TTP has a producer of Company 2 and a version of 2.  
    2. It is unrealistic to think that Company 2 can or will share the translated object back to Company 1 and that if Company 1 gets the translated object that they will do anything with it.  Their legal departments will probably prohibit accepting 3rd party translations and then using them in their offerings.

------------------------------
(4) An English CTI report describing attacks against Japanese entities in EN  
------------------------------

  An English report on Cyber Attacks on Japan.
  There are filenames of lure attachments in Japanese (original/real) and their
  translations in English.  Another similar report in English might have an email title along with 
  its translation in English next to it. That report also has a Windows pathname 
  in Chinese (not Japanese) found in a binary along with its translation in English.

  These Japanese texts can be found in descriptions, not just 

  [Ex. Original File Name (JA): "医療費通知", Translated File Name (EN): "Medical expenses notice"]

  Note: This should probably be okay as long as the standards require use of UTF-8 for encoding.

------------------------------
(5) Email subject/body, supposed to be in JP, but includes CN characters (by mistake of the attackers)
------------------------------

  This can happen due to Chinese/Japanese/Korean sharing Unicode characters
  (CJK characters - https://en.wikipedia.org/wiki/CJK_characters.)

  This can be a very important clue as to the attackers.

  Note: This should probably be okay as long as the standards require use of UTF-8 for encoding.

------------------------------
(6) CTI translation service
------------------------------

  A CTI translation service provider keeps translations to target languages of text fields 
  from publicly available and/or commercial/private CTI sources.
  The service is available through some kind of online API.
  Consumers of this translation service will use this service to translate text fields
  in their CTI system through the API provided by the translation service provider.

------------------------------
(7) CTI provider
------------------------------

  A CTI provider (in English) plans to penetrate the Japanese and other APAC markets
  and needs a standard way to add translations of their text fields.
  The CTI provider gives its customer a CTI package with all the translations in it
  or a CTI package with translations to the languages of user's choosing.  

------------------------------------------------------------



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]