Merge pull request #49 from ARMmbed/devel_retarget

Retargeting

Author: PrzemekWirkus

.gitignore

@@ -4,6 +4,7 @@ __pycache__/
 
 # mbed-ls related files
 .mbedls-mock
+mbedls.json
 
 # C extensions
 *.so

README.md

@@ -11,6 +11,9 @@
   * [Exporting mbedls output to JSON](#exporting-mbedls-output-to-json)
 * [Porting instructions](#porting-instructions)
   * [mbed-ls auto-detection approach for Ubuntu](#mbed-ls-auto-detection-approach-for-ubuntu)
+* [Retarget mbed-ls autodetection results](#retarget-mbed-ls-autodetection-results)
+  * [mbedls.json file properties](#mbedlsjson-file-properties)
+  * [Example of retargeting](#example-of-retargeting)
 * [Mocking new or existing target to custom platform name](#mocking-new-or-existing-target-to-custom-platform-name)
   * [Mock command line examples](#mock-command-line-examples)
   * [Mocking example with Freescale K64F platform](#mocking-example-with-freescale-k64f-platform)
@@ -328,6 +331,86 @@ $ mbedls
 +---------------------+-------------------+-------------------+----------------------------------------+
 ```
 
+# Retarget mbed-ls autodetection results
+
+User can create file ```mbedls.json``` in given directory. ```mbedls.json``` file should contain JSON formatted data which will redefine mbed's parameters returned by mbed-ls. ```mbed-ls``` will automatically read ```mbedls.json``` file and alter auto-detection result.
+File should be placed in directory where we want to alter mbed-ls behavior.
+
+* Note: This feature in implicitly ON.
+* Note: This feature can be turned off with command line switch ```--skip-retarget```.
+
+## mbedls.json file properties
+* If file ```mbedls.json``` exists will be implicitly used to retarget results.
+* If file ```mbedls.json``` exists and flag ```--skip-retarget``` is set, there will be no retarget.
+* If file ```mbedls.json``` doesn't exist flag ```--skip-retarget``` has no effect.
+
+## Example of retargeting
+In this example we will replace serial port name during Freescale's K64F auto-detection:
+```
+$ mbedls
++--------------+---------------------+------------+------------+-------------------------------------------------+
+|platform_name |platform_name_unique |mount_point |serial_port |target_id                                        |
++--------------+---------------------+------------+------------+-------------------------------------------------+
+|K64F          |K64F[0]              |F:          |COM9        |0240022648cb1e77000000000000000000000000b512e3cf |
++--------------+---------------------+------------+------------+-------------------------------------------------+
+```
+
+Our device is detected on port ```COM9``` and MSD is mounted on ```F:```. We can check more details using ```--json``` switch:
+```
+$ mbedls --json
+[
+    {
+        "mount_point": "F:",
+        "platform_name": "K64F",
+        "platform_name_unique": "K64F[0]",
+        "serial_port": "COM9",
+        "target_id": "0240022648cb1e77000000000000000000000000b512e3cf",
+        "target_id_mbed_htm": "0240022648cb1e77000000000000000000000000b512e3cf",
+        "target_id_usb_id": "0240022648cb1e77000000000000000000000000b512e3cf"
+    }
+]
+```
+
+We must understand that ```mbed-ls``` stores information about mbed devices in dictionaries.
+The same information can be presented as dictionary where its keys are ```target_id``` and value is a mbed auto-detection data.
+
+```
+$ mbedls --json-by-target-id
+{
+    "0240022648cb1e77000000000000000000000000b512e3cf": {
+        "mount_point": "F:",
+        "platform_name": "K64F",
+        "platform_name_unique": "K64F[0]",
+        "serial_port": "COM9",
+        "target_id": "0240022648cb1e77000000000000000000000000b512e3cf",
+        "target_id_mbed_htm": "0240022648cb1e77000000000000000000000000b512e3cf",
+        "target_id_usb_id": "0240022648cb1e77000000000000000000000000b512e3cf"
+    }
+}
+```
+
+Let's say we want change ```serial_port```'s value to other COM port. For example we are using other serial port (e.g. while debugging) on our device as standard output.
+To do so we would have to create a new file called ```mbedls.json``` in directory where want to use this modification. File content could look like this: a JSON file where keys are ```target_id```'s and values are dictionaries with new values:
+
+```
+$ cat mbedls.ls
+{
+    "0240022648cb1e77000000000000000000000000b512e3cf" : {
+        "serial_port" : "MyComPort01"
+    }
+}
+```
+
+Now, when we issue ```mbedls``` command in this directory our auto-detection data will be replaced:
+```
+$ mbedls
++--------------+---------------------+------------+------------+-------------------------------------------------+
+|platform_name |platform_name_unique |mount_point |serial_port |target_id                                        |
++--------------+---------------------+------------+------------+-------------------------------------------------+
+|K64F          |K64F[0]              |F:          |MyComPort01 |0240022648cb1e77000000000000000000000000b512e3cf |
++--------------+---------------------+------------+------------+-------------------------------------------------+
+```
+
 # Mocking new or existing target to custom platform name
 Command line switch ```--mock``` provide simple manufacturers ID masking with new platform name.
 Users should be able to add locally new ```MID``` -> ```platform_name``` mapping when e.g. prototyping.

mbed_lstools/lstools_base.py

@@ -29,6 +29,7 @@ def __init__(self):
         #extra flags
         self.DEBUG_FLAG = False     # Used to enable debug code / prints
         self.ERRORLEVEL_FLAG = 0    # Used to return success code to environment
+        self.retarget_data = {}          # Used to retarget mbed-enabled platform properties
 
         # If there is a local mocking data use it and add / override manufacture_ids
         mock_ids = self.mock_read()
@@ -169,30 +170,71 @@ def __init__(self):
     }
 
     MOCK_FILE_NAME = '.mbedls-mock'
+    RETARGET_FILE_NAME = 'mbedls.json'
 
     def mock_read(self):
         """! Load mocking data from local file
         @return Curent mocking configuration (dictionary)
         """
         if os.path.isfile(self.MOCK_FILE_NAME):
             if self.DEBUG_FLAG:
-                self.debug(self.mock_read.__name__, "reading %s"% self.MOCK_FILE_NAME)
-            with open(self.MOCK_FILE_NAME, "r") as f:
-                return json.load(f)
+                self.debug(self.mock_read.__name__, "reading mock file %s"% self.MOCK_FILE_NAME)
+            try:
+                with open(self.MOCK_FILE_NAME, "r") as f:
+                    return json.load(f)
+            except IOError as e:
+                self.err("reading file '%s' failed: %s"% (os.path.abspath(self.MOCK_FILE_NAME),
+                    str(e)))
+            except ValueError as e:
+                self.err("reading file '%s' content failed: %s"% (os.path.abspath(self.MOCK_FILE_NAME),
+                    str(e)))
         return {}
 
     def mock_write(self, mock_ids):
-        # Write current mocking structure
+        """! Write current mocking structure
+        @param mock_ids JSON mock data to dump to file
+        """
         if self.DEBUG_FLAG:
             self.debug(self.mock_write.__name__, "writing %s"% self.MOCK_FILE_NAME)
-        with open(self.MOCK_FILE_NAME, "w") as f:
-            f.write(json.dumps(mock_ids, indent=4))
+        try:
+            with open(self.MOCK_FILE_NAME, "w") as f:
+                f.write(json.dumps(mock_ids, indent=4))
+        except IOError as e:
+            self.err("reading file '%s' failed: %s"% (os.path.abspath(self.MOCK_FILE_NAME),
+                str(e)))
+        except ValueError as e:
+            self.err("reading file '%s' content failed: %s"% (os.path.abspath(self.MOCK_FILE_NAME),
+                str(e)))
+
+    def retarget_read(self):
+        """! Load retarget data from local file
+        @return Curent retarget configuration (dictionary)
+        """
+        if os.path.isfile(self.RETARGET_FILE_NAME):
+            if self.DEBUG_FLAG:
+                self.debug(self.retarget_read.__name__, "reading retarget file %s"% self.RETARGET_FILE_NAME)
+            try:
+                with open(self.RETARGET_FILE_NAME, "r") as f:
+                    return json.load(f)
+            except IOError as e:
+                self.err("reading file '%s' failed: %s"% (os.path.abspath(self.RETARGET_FILE_NAME),
+                    str(e)))
+            except ValueError as e:
+                self.err("reading file '%s' content failed: %s"% (os.path.abspath(self.RETARGET_FILE_NAME),
+                    str(e)))
+        return {}
+
+    def retarget(self):
+        """! Enable retargeting
+        """
+        self.retarget_data = self.retarget_read()
+        return self.retarget_data
 
     def mock_manufacture_ids(self, mid, platform_name, oper='+'):
         """! Replace (or add if manufacture id doesn't exist) entry in self.manufacture_ids
-            @param oper '+' add new mock / override existing entry
-                        '-' remove mid from mocking entry
-            @return Mocked structure (json format)
+        @param oper '+' add new mock / override existing entry
+                    '-' remove mid from mocking entry
+        @return Mocked structure (json format)
         """
         mock_ids = self.mock_read()
 
@@ -236,9 +278,7 @@ def list_mbeds(self):
 
     def list_mbeds_ext(self):
         """! Function adds extra information for each mbed device
-
         @return Returns list of mbed devices plus extended data like 'platform_name_unique'
-
         @details Get information about mbeds with extended parameters/info included
         """
         platform_names = {} # Count existing platforms and assign unique number
@@ -252,13 +292,22 @@ def list_mbeds_ext(self):
                 platform_names[platform_name] += 1
             # Assign normalized, unique string at the end of target name: TARGET_NAME[x] where x is an ordinal integer
             mbeds[i]['platform_name_unique'] = "%s[%d]" % (platform_name, platform_names[platform_name])
+
+            # Retarget values from retarget (mbedls.json) file
+            if self.retarget_data and 'target_id' in val:
+                target_id = val['target_id']
+                if target_id in self.retarget_data:
+                    mbeds[i].update(self.retarget_data[target_id])
+                    if self.DEBUG_FLAG:
+                        self.debug(self.list_mbeds_ext.__name__, ("retargeting", target_id, mbed[i]))
+
             if self.DEBUG_FLAG:
                 self.debug(self.list_mbeds_ext.__name__, (mbeds[i]['platform_name_unique'], val['target_id']))
         return mbeds
 
     def list_platforms(self):
-        """ Useful if you just want to know which platforms are currently available on the system
-            @return List of (unique values) available platforms
+        """! Useful if you just want to know which platforms are currently available on the system
+        @return List of (unique values) available platforms
         """
         result = []
         mbeds = self.list_mbeds()
@@ -269,8 +318,8 @@ def list_platforms(self):
         return result
 
     def list_platforms_ext(self):
-        """ Useful if you just want to know how many platforms of each type are currently available on the system
-            @return Dict of platform: platform_count
+        """! Useful if you just want to know how many platforms of each type are currently available on the system
+        @return Dict of platform: platform_count
         """
         result = {}
         mbeds = self.list_mbeds()
@@ -283,12 +332,9 @@ def list_platforms_ext(self):
         return result
 
     def list_mbeds_by_targetid(self):
-        """ Get information about mbeds with extended parameters/info included
-
-            @return Returns dictionary where keys are TargetIDs and values are mbed structures
-
-            @details Ordered by target id (key: target_id).
-
+        """! Get information about mbeds with extended parameters/info included
+        @return Returns dictionary where keys are TargetIDs and values are mbed structures
+        @details Ordered by target id (key: target_id).
         """
         result = {}
         mbed_list = self.list_mbeds_ext()
@@ -299,27 +345,23 @@ def list_mbeds_by_targetid(self):
 
     # Private part, methods used to drive interface functions
     def load_mbed_description(self, file_name):
-        """ Loads JSON file with mbeds' description (mapping between target id and platform name)
+        """! Load JSON file with mbeds' description (mapping between target id and platform name)
             Sets self.manufacture_ids with mapping between manufacturers' ids and platform name.
         """
         #self.manufacture_ids = {}   # TODO: load this values from file
         pass
 
     def err(self, text):
         """! Prints error messages
-
         @param text Text to be included in error message
-
         @details Function prints directly on console
         """
         print 'error: %s'% text
 
     def debug(self, name, text):
         """! Prints error messages
-
         @param name Called function name
         @param text Text to be included in debug message
-
         @details Function prints directly on console
         """
         print 'debug @%s.%s: %s'% (self.__class__.__name__, name, text)
@@ -333,12 +375,10 @@ def __str__(self):
 
     def get_string(self, border=False, header=True, padding_width=0, sortby='platform_name'):
         """! Printing with some sql table like decorators
-
         @param border Table border visibility
         @param header Table header visibility
         @param padding_width Table padding
         @param sortby Column used to sort results
-
         @return Returns string which can be printed on console
         """
         from prettytable import PrettyTable
@@ -367,7 +407,6 @@ def get_string(self, border=False, header=True, padding_width=0, sortby='platfor
 
     def get_json_data_from_file(self, json_spec_filename, verbose=False):
         """! Loads from file JSON formatted string to data structure
-
         @return None if JSON can be loaded
         """
         result = None
@@ -386,9 +425,7 @@ def get_json_data_from_file(self, json_spec_filename, verbose=False):
 
     def get_mbed_htm_target_id(self, mount_point):
         """! Function scans mbed.htm to get information about TargetID.
-
         @return Function returns targetID, in case of failure returns None.
-
         @details Note: This function should be improved to scan variety of boards' mbed.htm files
         """
         result = None
@@ -409,7 +446,6 @@ def get_mbed_htm_target_id(self, mount_point):
 
     def scan_html_line_for_target_id(self, line):
         """! Scan if given line contains target id encoded in URL.
-
         @return Returns None when no target_id string in line
         """
         # Detecting modern mbed.htm file format