Merge pull request #676 from HowJMay/hornet

feat: Allow HORNET/SSL connection

Author: howjmay

README.md

@@ -19,7 +19,7 @@ use scenarios such as MAM and [TangleID](https://tangleid.github.io/).
 
 At the moment, it is not feasible to host fully-functioned full nodes on Raspberry Pi class
 Arm devices, but Raspberry Pi 3 is known to be capable to execute `Tangle-accelerator`
-without problems. Since it is written in C/C++ with [entangled](https://github.com/iotaledger/entangled),
+without problems. Since it is written in C/C++ with [iota.c](https://github.com/iotaledger/iota.c),
 both footprint and startup time are behaved pretty well.
 
 ## Architecture
@@ -90,27 +90,30 @@ Tangle-accelerator is built and launched through Bazel, it also requires Redis t
 
 ## Build from Source
 
-Before running tangle-accelerator, please edit binding address/port of accelerator instance, IRI, and redis server in `accelerator/config.h` unless they are all localhost and/or you don't want to provide external connection. With dependency of [entangled](https://github.com/iotaledger/entangled), IRI address doesn't support https at the moment. Here are some configurations and command you might need to change and use:
+Before running tangle-accelerator, please edit binding address/port of accelerator instance, IOTA full node, and redis server in `accelerator/config.h` unless they are all localhost and/or you don't want to provide external connection. With dependency of [iota.c](https://github.com/iotaledger/iota.c), IOTA full node address doesn't support https at the moment. Here are some configurations and command you might need to change and use:
 
-* `TA_HOST`: binding address of accelerator instance
-* `TA_PORT`: port of accelerator instance
-* `IRI_HOST`: binding address of IRI
-* `IRI_PORT`: port of IRI
-* `HTTP_THREADS`: Determine thread pool size to process HTTP connections.
-* `quiet`: Turn off logging message
+* `ta_host`: Binding address of accelerator instance.
+* `ta_port`: Port of accelerator instance.
+* `node_host`: Binding address of IOTA full node which includes IRI and Hornet or other community implementation.
+* `node_port`: Port of IOTA full node.
+* `http_threads`: Determine thread pool size to process HTTP connections.
+* `quiet`: Turn off logging message.
 
 ```
 $ make && bazel run //accelerator
 ```
+
 ### Building Options
+
 Tangle-accelerator supports several different build time options.
 
 * Docker images
 * MQTT connectivity
 * External database
 * Debug Mode
 
-    Debug mode enables tangle-accelerator to display extra `debug` logs.
+Debug mode enables tangle-accelerator to display extra `debug` logs.
+
 ```
 bazel run --define build_type=debug //accelerator
 ```
@@ -160,13 +163,13 @@ clang-format can be installed by command:
 ## Usage
 `Tangle-accelerator` currently supports two categories of APIs
 * Direct API: check [wiki page](https://github.com/DLTcollab/tangle-accelerator/wiki) for details.
-* Proxy API to IRI core functionalities
+* Proxy API to IOTA core functionalities
 
-### IRI Proxy API
-`tangle-accelerator` allows the use of IRI core APIs. The calling process does not have to be aware of the destination machine running IRI. With the exactly same format of IRI API, `tangle-accelerator` would help users forward the request to IRI and forward the response back to users.
-We support two way to forward Proxy APIs to IRI:
-1. Bypass Proxy APIs directly to IRI.
-2. Process the Proxy APIs, then transmit them to IRI.
+### Full Node Proxy API
+`tangle-accelerator` allows the use of IOTA core APIs. The calling process does not have to be aware of the destination machine running IOTA full node. With the exactly same format of IOTA core APIs, `tangle-accelerator` would help users forward the request to IOTA full node and forward the response back to users.
+We support two way to forward Proxy APIs to IOTA full node:
+1. Bypass Proxy APIs directly to IOTA full node.
+2. Process the Proxy APIs, then transmit them to IOTA full node.
 
 The user can choose which way they want with CLI argument `--proxy_passthrough`.
 All the Proxy APIs are supported with the first way.

WORKSPACE

@@ -15,13 +15,13 @@ git_repository(
 
 git_repository(
     name = "iota.c",
-    commit = "fd9d67989dc5161117b39a9a3351afcfb118bcad",
+    commit = "3f1c255bd6e7ccc19bd527a83fa4593d342cad32",
     remote = "https://github.com/iotaledger/iota.c.git",
 )
 
 git_repository(
     name = "org_iota_common",
-    commit = "c644b7715662dc7b73191ae964f4ede06ee6c975",
+    commit = "cf649803757abf48432d4fa60e9f27ac119bae5f",
     remote = "https://github.com/iotaledger/iota_common.git",
 )
 

accelerator/cli_info.h

@@ -22,14 +22,14 @@ extern "C" {
  */
 
 typedef enum ta_cli_arg_value_e {
-  /** TA */
+  /** tangle-accelerator */
   TA_HOST_CLI = 127,
   TA_PORT_CLI,
 
-  /** IRI */
-  IRI_HOST_CLI,
-  IRI_PORT_CLI,
-  IRI_ADDRESS_CLI,
+  /** IOTA full node */
+  NODE_HOST_CLI,
+  NODE_PORT_CLI,
+  NODE_ADDRESS_CLI,
 
   /** MQTT */
   MQTT_HOST_CLI,
@@ -58,6 +58,8 @@ typedef enum ta_cli_arg_value_e {
 
   /** LOGGER */
   QUIET,
+
+  CA_PEM,
 } ta_cli_arg_value_t;
 
 static struct ta_cli_argument_s {
@@ -73,22 +75,23 @@ static struct ta_cli_argument_s {
     {"ta_port", required_argument, NULL, TA_PORT_CLI, "TA listening port"},
     {"http_threads", required_argument, NULL, HTTP_THREADS_CLI,
      "Determine thread pool size to process HTTP connections."},
-    {"iri_host", required_argument, NULL, IRI_HOST_CLI, "IRI listening host"},
-    {"iri_port", required_argument, NULL, IRI_PORT_CLI, "IRI listening port"},
+    {"node_host", required_argument, NULL, NODE_HOST_CLI, "IOTA full node listening host"},
+    {"node_port", required_argument, NULL, NODE_PORT_CLI, "IOTA full node listening port"},
+    {"CA_PEM", required_argument, NULL, CA_PEM, "The path to CA PEM file"},
     {"mqtt_host", required_argument, NULL, MQTT_HOST_CLI, "MQTT listening host"},
     {"mqtt_root", required_argument, NULL, MQTT_ROOT_CLI, "MQTT listening topic root"},
-    {"iri_address", required_argument, NULL, IRI_ADDRESS_CLI, " List of IRI listening URL"},
+    {"node_address", required_argument, NULL, NODE_ADDRESS_CLI, " List of IOTA full node listening URL"},
     {"redis_host", required_argument, NULL, REDIS_HOST_CLI, "Redis server listening host"},
     {"redis_port", required_argument, NULL, REDIS_PORT_CLI, "Redis server listening port"},
     {"db_host", required_argument, NULL, DB_HOST_CLI, "DB server listening host"},
-    {"milestone_depth", optional_argument, NULL, MILESTONE_DEPTH_CLI, "IRI milestone depth"},
+    {"milestone_depth", optional_argument, NULL, MILESTONE_DEPTH_CLI, "IOTA full node milestone depth"},
     {"mwm", optional_argument, NULL, MWM_CLI, "minimum weight magnitude"},
     {"seed", optional_argument, NULL, SEED_CLI, "IOTA seed"},
     {"cache", required_argument, NULL, CACHE, "Enable/Disable cache server. It defaults to off"},
     {"config", required_argument, NULL, CONF_CLI, "Read configuration file"},
-    {"proxy_passthrough", no_argument, NULL, PROXY_API, "Pass proxy API directly to IRI without processing"},
+    {"proxy_passthrough", no_argument, NULL, PROXY_API, "Pass proxy API directly to IOTA full node without processing"},
     {"health_track_period", no_argument, NULL, HEALTH_TRACK_PERIOD,
-     "The period for checking IRI host connection status"},
+     "The period for checking IOTA full node host connection status"},
     {"no-gtta", no_argument, NULL, NO_GTTA, "Disable getTransactionToConfirm (gTTA) when sending transaction"},
     {"buffer_list", required_argument, NULL, BUFFER_LIST, "Set the value of `buffer_list_name`"},
     {"done_list", required_argument, NULL, DONE_LIST, "Set the value of `done_list_name`"},

accelerator/config.c

@@ -13,6 +13,28 @@
 #include "yaml.h"
 
 #define CONFIG_LOGGER "config"
+#define CA_PEM_LEN 1208
+#define DEFAULT_CA_PEM                                                   \
+  "-----BEGIN CERTIFICATE-----\r\n"                                      \
+  "MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF\r\n" \
+  "ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6\r\n" \
+  "b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL\r\n" \
+  "MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv\r\n" \
+  "b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj\r\n" \
+  "ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM\r\n" \
+  "9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw\r\n" \
+  "IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6\r\n" \
+  "VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L\r\n" \
+  "93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm\r\n" \
+  "jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC\r\n" \
+  "AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA\r\n" \
+  "A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI\r\n" \
+  "U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs\r\n" \
+  "N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv\r\n" \
+  "o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU\r\n" \
+  "5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy\r\n" \
+  "rqXRfboQnoZsG4q5WTP468SQvvG5\r\n"                                     \
+  "-----END CERTIFICATE-----\r\n"
 
 static logger_id_t logger_id;
 
@@ -46,6 +68,7 @@ status_t cli_core_set(ta_core_t* const core, int key, char* const value) {
   iota_config_t* const iota_conf = &core->iota_conf;
   ta_cache_t* const cache = &core->cache;
   iota_client_service_t* const iota_service = &core->iota_service;
+  FILE* file = NULL;
   char* conf_file = core->conf_file;
 #ifdef DB_ENABLE
   db_client_service_t* const db_service = &core->db_service;
@@ -83,17 +106,17 @@ status_t cli_core_set(ta_core_t* const core, int key, char* const value) {
       }
       break;
 
-    // IRI configuration
-    case IRI_HOST_CLI:
+    // IOTA full node configuration
+    case NODE_HOST_CLI:
       idx = 0;
-      for (char* p = strtok(value, ","); p && idx < MAX_IRI_LIST_ELEMENTS; p = strtok(NULL, ","), idx++) {
+      for (char* p = strtok(value, ","); p && idx < MAX_NODE_LIST_ELEMENTS; p = strtok(NULL, ","), idx++) {
         ta_conf->iota_host_list[idx] = p;
       }
       strncpy(iota_service->http.host, ta_conf->iota_host_list[0], HOST_MAX_LEN);
       break;
-    case IRI_PORT_CLI:
+    case NODE_PORT_CLI:
       idx = 0;
-      for (char* p = strtok(value, ","); p && idx < MAX_IRI_LIST_ELEMENTS; p = strtok(NULL, ","), idx++) {
+      for (char* p = strtok(value, ","); p && idx < MAX_NODE_LIST_ELEMENTS; p = strtok(NULL, ","), idx++) {
         strtol_temp = strtol(p, &strtol_p, 10);
         if (strtol_p != p && errno != ERANGE && strtol_temp >= 0 && strtol_temp <= USHRT_MAX) {
           ta_conf->iota_port_list[idx] = (uint16_t)strtol_temp;
@@ -103,6 +126,19 @@ status_t cli_core_set(ta_core_t* const core, int key, char* const value) {
       }
       iota_service->http.port = ta_conf->iota_port_list[0];
       break;
+
+    case CA_PEM:
+      if ((file = fopen(value, "r")) == NULL) {
+        /* The specified configuration file does not exist */
+        ta_log_error("%s\n", ta_error_to_string(SC_CONF_FOPEN_ERROR));
+        return SC_CONF_FOPEN_ERROR;
+      }
+      char* temp_ca_pem = (char*)malloc(sizeof(char) * (CA_PEM_LEN + 1));
+      fread(temp_ca_pem, CA_PEM_LEN, 1, file);
+      iota_service->http.ca_pem = temp_ca_pem;
+      fclose(file);
+      break;
+
     case HEALTH_TRACK_PERIOD:
       strtol_temp = strtol(value, NULL, 10);
       if (strtol_p != value && errno != ERANGE && strtol_temp >= INT_MIN && strtol_temp <= INT_MAX) {
@@ -215,14 +251,15 @@ status_t cli_core_set(ta_core_t* const core, int key, char* const value) {
   return SC_OK;
 }
 
-status_t ta_set_iota_client_service(iota_client_service_t* service, char const* host, uint16_t port) {
+status_t ta_set_iota_client_service(iota_client_service_t* service, char const* host, uint16_t port,
+                                    char const* const ca_pem) {
   strncpy(service->http.path, "/", CONTENT_TYPE_MAX_LEN);
   strncpy(service->http.content_type, "application/json", CONTENT_TYPE_MAX_LEN);
   strncpy(service->http.accept, "application/json", CONTENT_TYPE_MAX_LEN);
   strncpy(service->http.host, host, HOST_MAX_LEN);
   service->http.port = port;
   service->http.api_version = 1;
-  service->http.ca_pem = NULL;
+  service->http.ca_pem = ca_pem ? ca_pem : DEFAULT_CA_PEM;
   service->serializer_type = SR_JSON;
   init_json_serializer(&service->serializer);
 
@@ -248,8 +285,8 @@ status_t ta_core_default_init(ta_core_t* const core) {
   ta_conf->host = TA_HOST;
   ta_conf->port = TA_PORT;
   memset(ta_conf->iota_host_list, 0, sizeof(ta_conf->iota_host_list));
-  for (int i = 0; i < MAX_IRI_LIST_ELEMENTS; i++) {
-    ta_conf->iota_port_list[i] = IRI_PORT;
+  for (int i = 0; i < MAX_NODE_LIST_ELEMENTS; i++) {
+    ta_conf->iota_port_list[i] = NODE_PORT;
   }
   ta_conf->http_tpool_size = DEFAULT_HTTP_TPOOL_SIZE;
   ta_conf->proxy_passthrough = false;
@@ -267,20 +304,21 @@ status_t ta_core_default_init(ta_core_t* const core) {
   cache->done_list_name = DONE_LIST_NAME;
   cache->capacity = CACHE_MAX_CAPACITY;
 
-  ta_log_info("Initializing IRI configuration\n");
+  ta_log_info("Initializing IOTA full node configuration\n");
   iota_conf->milestone_depth = MILESTONE_DEPTH;
   iota_conf->mwm = MWM;
   iota_conf->seed = SEED;
   char mam_file_path[] = MAM_FILE_PREFIX;
   mkstemp(mam_file_path);
   iota_conf->mam_file_path = strdup(mam_file_path);
 
-  ta_log_info("Initializing IRI connection\n");
+  ta_log_info("Initializing IOTA full node connection\n");
   strncpy(iota_service->http.path, "/", CONTENT_TYPE_MAX_LEN);
   strncpy(iota_service->http.content_type, "application/json", CONTENT_TYPE_MAX_LEN);
   strncpy(iota_service->http.accept, "application/json", CONTENT_TYPE_MAX_LEN);
-  strncpy(iota_service->http.host, IRI_HOST, HOST_MAX_LEN);
-  iota_service->http.port = IRI_PORT;
+  strncpy(iota_service->http.host, NODE_HOST, HOST_MAX_LEN);
+  iota_service->http.ca_pem = DEFAULT_CA_PEM;
+  iota_service->http.port = NODE_PORT;
   iota_service->http.api_version = 1;
   iota_service->serializer_type = SR_JSON;
 #ifdef DB_ENABLE
@@ -440,7 +478,7 @@ status_t ta_core_set(ta_core_t* core) {
   db_client_service_t* const db_service = &core->db_service;
 #endif
   if (iota_client_service_init(iota_service)) {
-    ta_log_error("Initializing IRI connection failed!\n");
+    ta_log_error("Initializing IOTA full node connection failed!\n");
     ret = SC_OOM;
     goto exit;
   }
@@ -466,7 +504,7 @@ status_t ta_core_set(ta_core_t* core) {
 }
 
 void ta_core_destroy(ta_core_t* const core) {
-  ta_log_info("Destroying IRI connection\n");
+  ta_log_info("Destroying IOTA full node connection\n");
 #ifdef DB_ENABLE
   ta_log_info("Destroying DB connection\n");
   db_client_service_free(&core->db_service);

accelerator/config.h

@@ -49,9 +49,9 @@ extern "C" {
 #define DEFAULT_HTTP_TPOOL_SIZE 4 /**< Thread number of MHD thread pool */
 #define MAX_HTTP_TPOOL_SIZE \
   (get_nprocs_conf() - get_nthds_per_phys_proc()) /** < Preserve at least one physical processor */
-#define IRI_HOST "localhost"
-#define IRI_PORT 14265
-#define MAX_IRI_LIST_ELEMENTS 5
+#define NODE_HOST "localhost"
+#define NODE_PORT 443
+#define MAX_NODE_LIST_ELEMENTS 5
 #define DB_HOST "localhost"
 #define MILESTONE_DEPTH 3
 #define MWM 14
@@ -73,12 +73,12 @@ extern "C" {
 
 /** struct type of accelerator configuration */
 typedef struct ta_config_s {
-  char* version;                                  /**< Binding version of tangle-accelerator */
-  char* host;                                     /**< Binding address of tangle-accelerator */
-  int port;                                       /**< Binding port of tangle-accelerator */
-  char* iota_host_list[MAX_IRI_LIST_ELEMENTS];    /**< List of binding hosts of IOTA services */
-  uint16_t iota_port_list[MAX_IRI_LIST_ELEMENTS]; /**< List of binding ports of IOTA services */
-  int health_track_period;                        /**< The period for checking IRI host connection status */
+  char* version;                                   /**< Binding version of tangle-accelerator */
+  char* host;                                      /**< Binding address of tangle-accelerator */
+  int port;                                        /**< Binding port of tangle-accelerator */
+  char* iota_host_list[MAX_NODE_LIST_ELEMENTS];    /**< List of binding hosts of IOTA services */
+  uint16_t iota_port_list[MAX_NODE_LIST_ELEMENTS]; /**< List of binding ports of IOTA services */
+  int health_track_period;                         /**< The period for checking full node connection status */
 #ifdef MQTT_ENABLE
   char* mqtt_host;       /**< Address of MQTT broker host */
   char* mqtt_topic_root; /**< The topic root of MQTT topic */
@@ -196,12 +196,14 @@ void ta_core_destroy(ta_core_t* const core);
  * @param service[in] IOTA client service
  * @param host[in] host of connecting service
  * @param port[in] port of connecting service
+ * @param ca_pem[in] CA PEM path
  *
  * @return
  * - SC_OK on success
  * - non-zero on error
  */
-status_t ta_set_iota_client_service(iota_client_service_t* service, char const* host, uint16_t port);
+status_t ta_set_iota_client_service(iota_client_service_t* service, char const* host, uint16_t port,
+                                    char const* const ca_pem);
 
 #ifdef __cplusplus
 }

accelerator/core/apis.c

@@ -281,7 +281,8 @@ status_t api_send_transfer(const ta_core_t* const core, const iota_client_servic
   }
 
   // return transaction object
-  if (hash243_queue_push(&txn_obj_req->hashes, hash243_queue_peek(res->hash))) {
+  if (hash243_queue_count(res->hash) == 0 || hash243_queue_push(&txn_obj_req->hashes, hash243_queue_peek(res->hash))) {
+    ret = SC_CCLIENT_FAILED_RESPONSE;
     ta_log_error("%s\n", "hash243_queue_push failed");
     goto done;
   }
@@ -342,30 +343,30 @@ status_t api_send_trytes(const ta_config_t* const info, const iota_config_t* con
   return ret;
 }
 
-status_t api_get_iri_status(const iota_client_service_t* const service, char** json_result) {
+status_t api_get_node_status(const iota_client_service_t* const service, char** json_result) {
   status_t ret = SC_OK;
 
-  ret = ta_get_iri_status(service);
+  ret = ta_get_node_status(service);
   switch (ret) {
     /*
      * The values of each status_t are listed as the following. Not listed status code are unexpected errors which
      * would cause TA return error.
      *
-     * SC_CCLIENT_FAILED_RESPONSE: Can't connect to IRI host
-     * SC_CORE_IRI_UNSYNC: IRI host is not at the latest milestone
-     * SC_OK: IRI host works fine.
+     * SC_CCLIENT_FAILED_RESPONSE: Can't connect to IOTA full node
+     * SC_CORE_NODE_UNSYNC: IOTA full node is not at the latest milestone
+     * SC_OK: IOTA full node works fine.
      **/
     case SC_CCLIENT_FAILED_RESPONSE:
-    case SC_CORE_IRI_UNSYNC:
+    case SC_CORE_NODE_UNSYNC:
     case SC_OK:
-      ret = get_iri_status_res_serialize(ret, json_result);
+      ret = get_node_status_res_serialize(ret, json_result);
       if (ret) {
         ta_log_error("failed to serialize. status code: %d\n", ret);
       }
       break;
 
     default:
-      ta_log_error("check IRI connection failed. status code: %d\n", ret);
+      ta_log_error("check IOTA full node connection failed. status code: %d\n", ret);
       break;
   }