Support JSON error response bodies

configs/body_factory/default/.body_factory_info

@@ -3,14 +3,22 @@
 # The .body_factory_info file contains descriptive information
 # about the error pages in this directory.
 #
-# Currently, .body_factory_info contains information which
-# indicates the character set and natural language of the error
-# pages in this directory.  For example, to describe Korean
-# web pages encoded in the iso-2022-kr character set, you might
-# add these lines to .body_factory_info file:
+# Supported directives:
 #
-#	Content-Language: kr
+#   Content-Language  Natural language of the error pages (default: en)
+#   Content-Charset   Character encoding (default: utf-8)
+#   Content-Type      Complete MIME type for the response
+#
+# For example, to describe Korean web pages encoded in the
+# iso-2022-kr character set, you might add these lines:
+#
+#	Content-Language: ko-KR
 #	Content-Charset: iso-2022-kr
 #
-# If this file is empty, or only contains comments, the default is
-# assumed: English text in the standard utf-8 character set.
+# To serve plain text error pages instead of HTML:
+#
+#	Content-Type: text/plain
+#
+# If this file is empty, or only contains comments, the defaults are
+# assumed: English text/html in the utf-8 character set. An explicit
+# Content-Type value is used exactly as configured.

doc/admin-guide/monitoring/error-messages.en.rst

@@ -108,6 +108,47 @@ it would be used instead of ``cache#read_error`` if there is no ``apache_cache#r
 The text for an error message is processed as if it were a :ref:`admin-logging-fields` which
 enables customization by values present in the transaction for which the error occurred.
 
+.. _body-factory-info:
+
+Template Set Metadata
+---------------------
+
+Each template set directory must contain a ``.body_factory_info`` file for the template set to be
+loaded. This file controls the ``Content-Type``, ``Content-Language``, and character set of the
+HTTP response headers sent with error pages.
+
+The following directives are supported:
+
+``Content-Language``
+   The natural language of the error pages. This value is sent in the ``Content-Language`` HTTP
+   response header. Default: ``en``.
+
+``Content-Charset``
+   The character encoding of the error pages. When ``Content-Type`` is not explicitly configured,
+   this value is appended to the default ``text/html`` type as a ``charset`` parameter. Default:
+   ``utf-8``.
+
+``Content-Type``
+   The complete MIME type for the error response. The value is used exactly as configured, so include
+   a ``charset`` parameter here when one is desired. Default: ``text/html; charset=utf-8``.
+
+For example, to serve plain text error pages in English::
+
+   Content-Language: en
+   Content-Charset: utf-8
+   Content-Type: text/plain
+
+This would produce the response header ``Content-Type: text/plain``. To declare the character set as
+well, configure ``Content-Type: text/plain; charset=utf-8``.
+
+To describe Korean error pages encoded in the ``iso-2022-kr`` character set::
+
+   Content-Language: ko-KR
+   Content-Charset: iso-2022-kr
+
+If the file is empty or contains only comments, the defaults are used: English ``text/html`` in
+the ``utf-8`` character set. If the file is absent, the entire template set directory is skipped.
+
 The following table lists the hard-coded Traffic Server HTTP messages,
 with corresponding HTTP response codes and customizable files.
 
@@ -269,4 +310,3 @@ with corresponding HTTP response codes and customizable files.
    Could not process this request because the request uri
    was too long ..
    ``request#uri_len_too_long``
-

doc/admin-guide/plugins/header_rewrite.en.rst

@@ -1175,30 +1175,68 @@ set-body
 ~~~~~~~~
 ::
 
-  set-body <text>
+  set-body <text> [<content-type>]
 
-Sets the body to ``<text>``. Can also be used to delete a body with ``""``. This is only useful when overriding the origin status, i.e.
-intercepting/pre-empting a request so that you can override the body from the body-factory with your own.
+Sets the body to ``<text>``. If ``<content-type>`` is supplied, it is used for
+the response instead of the default ``text/html``. Can also be used to delete
+a body with ``""``. This is only useful when overriding the origin status,
+i.e. intercepting/pre-empting a request so that you can override the body from
+the body-factory with your own.
+
+Quoted values support escaped quotes and the ``\n``, ``\r``, and ``\t``
+control characters. For example, a JSON error response can be configured as::
+
+   cond %{REMAP_PSEUDO_HOOK}
+      set-status 400
+      set-body "{\"error\": \"bad request\"}\n" "application/problem+json"
+
+set-body-from-file
+~~~~~~~~~~~~~~~~~~
+::
+
+  set-body-from-file <path> [<content-type>]
+
+Loads the file at ``<path>`` when the rule configuration is loaded and uses
+its exact contents as the response body. Relative paths are resolved from the
+directory containing the rule file. If ``<content-type>`` is omitted, the
+default ``text/html`` is used. Reload the header rewrite configuration after
+changing the body file.
+
+For example::
+
+   cond %{REMAP_PSEUDO_HOOK}
+      set-status 403
+      set-body-from-file errors/forbidden.json "application/json"
 
 set-body-from
 ~~~~~~~~~~~~~
 ::
 
-  set-body-from <URL>
+  set-body-from <URL> [<content-type>]
 
 Will call ``<URL>`` (see URL in `URL Parts`_) to retrieve a custom error response
 and set the body with the result. Triggering this rule on an OK transaction will
 send a 500 status code to the client with the desired response. If this is triggered
 on any error status code, that original status code will be sent to the client.
+By default, the fetched response's ``Content-Type`` is also used. The optional
+``<content-type>`` overrides that value.
 
 .. note::
-    This config should only be set using READ_RESPONSE_HDR_HOOK
+    This operator is supported only with ``READ_RESPONSE_HDR_HOOK`` because
+    its fetch suspends an active transaction. To generate a literal response
+    at remap time, use ``set-body`` instead.
 
 An example config would look like::
 
    cond %{READ_RESPONSE_HDR_HOOK}
       set-body-from http://www.example.com/second
 
+To create a JSON error during remap without contacting another endpoint::
+
+   cond %{REMAP_PSEUDO_HOOK}
+      set-status 400
+      set-body "{\"error\": \"bad request\"}\n" "application/problem+json"
+
 Where ``http://www.example.com/second`` is the destination to retrieve the custom response from.
 This can be enabled per-mapping or globally.
 Ensure there is a remap rule for the second endpoint as well!

include/proxy/http/HttpBodyFactory.h

@@ -121,6 +121,7 @@ class HttpBodySetRawData
   char                          *set_name;
   char                          *content_language;
   char                          *content_charset;
+  char                          *content_type;
   std::unique_ptr<TemplateTable> table_of_pages;
 };
 
@@ -215,7 +216,7 @@ class HttpBodyFactory
 private:
   char *fabricate(StrList *acpt_language_list, StrList *acpt_charset_list, const char *type, HttpTransact::State *context,
                   int64_t *resulting_buffer_length, const char **content_language_return, const char **content_charset_return,
-                  const char **set_return = nullptr);
+                  const char **content_type_return, const char **set_return = nullptr);
 
   const char       *determine_set_by_language(StrList *acpt_language_list, StrList *acpt_charset_list);
   const char       *determine_set_by_host(HttpTransact::State *context);

plugins/header_rewrite/factory.cc

@@ -73,6 +73,8 @@ operator_factory(const std::string &op)
     o = new OperatorSetDebug();
   } else if (op == "set-body") {
     o = new OperatorSetBody();
+  } else if (op == "set-body-from-file") {
+    o = new OperatorSetBodyFromFile();
   } else if (op == "set-http-cntl") {
     o = new OperatorSetHttpCntl();
   } else if (op == "set-plugin-cntl") {

plugins/header_rewrite/operators.cc

@@ -23,6 +23,8 @@
 #include <cstring>
 #include <algorithm>
 #include <iomanip>
+#include <system_error>
+#include <utility>
 
 #include "records/RecCore.h"
 #include "ts/ts.h"
@@ -40,10 +42,16 @@ const unsigned int LOCAL_IP_ADDRESS = 0x0100007f;
 const unsigned int MAX_SIZE         = 256;
 const int          LOCAL_PORT       = 8080;
 
+struct SetBodyFromData {
+  TSHttpTxn   http_txn;
+  std::string content_type;
+};
+
 int
 handleFetchEvents(TSCont cont, TSEvent event, void *edata)
 {
-  TSHttpTxn http_txn = static_cast<TSHttpTxn>(TSContDataGet(cont));
+  auto     *fetch_data = static_cast<SetBodyFromData *>(TSContDataGet(cont));
+  TSHttpTxn http_txn   = fetch_data->http_txn;
 
   switch (static_cast<int>(event)) {
   case OperatorSetBodyFrom::TS_EVENT_FETCHSM_SUCCESS: {
@@ -58,7 +66,21 @@ handleFetchEvents(TSCont cont, TSEvent event, void *edata)
 
       TSHttpHdrTypeSet(hdr_buf, hdr_loc, TS_HTTP_TYPE_RESPONSE);
       if (TSHttpHdrParseResp(parser, hdr_buf, hdr_loc, &data_start, data_end) == TS_PARSE_DONE) {
-        TSHttpTxnErrorBodySet(http_txn, TSstrdup(data_start), (data_end - data_start), nullptr);
+        char *content_type = nullptr;
+
+        if (!fetch_data->content_type.empty()) {
+          content_type = TSstrdup(fetch_data->content_type.c_str());
+        } else if (TSMLoc field_loc = TSMimeHdrFieldFind(hdr_buf, hdr_loc, TS_MIME_FIELD_CONTENT_TYPE, TS_MIME_LEN_CONTENT_TYPE);
+                   field_loc != TS_NULL_MLOC) {
+          int         value_len = 0;
+          const char *value     = TSMimeHdrFieldValueStringGet(hdr_buf, hdr_loc, field_loc, -1, &value_len);
+
+          if (value != nullptr && value_len > 0) {
+            content_type = TSstrndup(value, value_len);
+          }
+          TSHandleMLocRelease(hdr_buf, hdr_loc, field_loc);
+        }
+        TSHttpTxnErrorBodySet(http_txn, TSstrdup(data_start), data_end - data_start, content_type);
       } else {
         TSWarning("[%s] Unable to parse set-custom-body fetch response", __FUNCTION__);
       }
@@ -79,6 +101,8 @@ handleFetchEvents(TSCont cont, TSEvent event, void *edata)
     TSHttpTxnReenable(http_txn, TS_EVENT_HTTP_CONTINUE);
   } break;
   case TS_EVENT_HTTP_TXN_CLOSE: {
+    delete fetch_data;
+    TSContDataSet(cont, nullptr);
     TSContDestroy(cont);
     TSHttpTxnReenable(http_txn, TS_EVENT_HTTP_CONTINUE);
   } break;
@@ -806,8 +830,10 @@ void
 OperatorSetBody::initialize(Parser &p)
 {
   Operator::initialize(p);
-  // we want the arg since body only takes one value
   _value.set_value(p.get_arg(), this);
+  _content_type.set_value(p.get_value(), this);
+  require_resources(RSRC_SERVER_RESPONSE_HEADERS);
+  require_resources(RSRC_RESPONSE_STATUS);
 }
 
 void
@@ -821,13 +847,75 @@ bool
 OperatorSetBody::exec(const Resources &res) const
 {
   std::string value;
+  std::string content_type;
 
   _value.append_value(value, res);
+  _content_type.append_value(content_type, res);
   char *msg = nullptr;
   if (!value.empty()) {
     msg = TSstrdup(value.c_str());
   }
-  TSHttpTxnErrorBodySet(res.state.txnp, msg, value.size(), nullptr);
+  TSHttpTxnErrorBodySet(res.state.txnp, msg, value.size(), content_type.empty() ? nullptr : TSstrdup(content_type.c_str()));
+  return true;
+}
+
+// OperatorSetBodyFromFile
+void
+OperatorSetBodyFromFile::initialize(Parser &p)
+{
+  Operator::initialize(p);
+  _content_type.set_value(p.get_value(), this);
+  require_resources(RSRC_SERVER_RESPONSE_HEADERS);
+  require_resources(RSRC_RESPONSE_STATUS);
+
+  swoc::file::path path{p.get_arg()};
+
+  if (path.is_relative()) {
+    swoc::file::path base{TSConfigDirGet()};
+
+    if (has_config_location()) {
+      base = swoc::file::path{get_config_filename()};
+      if (base.is_relative()) {
+        base = swoc::file::path{TSConfigDirGet()} / base;
+      }
+      base = base.parent_path();
+    }
+    path = base / path;
+  }
+
+  std::error_code ec;
+
+  _body = swoc::file::load(path, ec);
+  if (ec) {
+    TSError("[%s] unable to load body file '%s': %s", PLUGIN_NAME, path.c_str(), ec.message().c_str());
+    return;
+  }
+  _loaded = true;
+}
+
+void
+OperatorSetBodyFromFile::initialize_hooks()
+{
+  add_allowed_hook(TS_REMAP_PSEUDO_HOOK);
+  add_allowed_hook(TS_HTTP_SEND_RESPONSE_HDR_HOOK);
+}
+
+bool
+OperatorSetBodyFromFile::exec(const Resources &res) const
+{
+  if (!_loaded) {
+    return true;
+  }
+
+  std::string content_type;
+  char       *body = nullptr;
+
+  _content_type.append_value(content_type, res);
+  if (!_body.empty()) {
+    body = static_cast<char *>(TSmalloc(_body.size()));
+    std::memcpy(body, _body.data(), _body.size());
+  }
+  TSHttpTxnErrorBodySet(res.state.txnp, body, _body.size(), content_type.empty() ? nullptr : TSstrdup(content_type.c_str()));
   return true;
 }
 
@@ -1349,8 +1437,8 @@ void
 OperatorSetBodyFrom::initialize(Parser &p)
 {
   Operator::initialize(p);
-  // we want the arg since body only takes one value
   _value.set_value(p.get_arg(), this);
+  _content_type.set_value(p.get_value(), this);
   require_resources(RSRC_SERVER_RESPONSE_HEADERS);
   require_resources(RSRC_RESPONSE_STATUS);
 }
@@ -1371,29 +1459,30 @@ OperatorSetBodyFrom::exec(const Resources &res) const
     return true;
   }
 
-  char req_buf[MAX_SIZE];
-  int  req_buf_size = 0;
-  if (createRequestString(_value.get_value(), req_buf, &req_buf_size) == TS_SUCCESS) {
+  std::string url;
+  std::string content_type;
+  char        req_buf[MAX_SIZE];
+  int         req_buf_size = 0;
+
+  _value.append_value(url, res);
+  _content_type.append_value(content_type, res);
+  if (createRequestString(url, req_buf, &req_buf_size) == TS_SUCCESS) {
     TSCont fetchCont = TSContCreate(handleFetchEvents, TSMutexCreate());
-    TSContDataSet(fetchCont, static_cast<void *>(res.state.txnp));
+    TSContDataSet(fetchCont, new SetBodyFromData{res.state.txnp, std::move(content_type)});
 
     TSHttpTxnHookAdd(res.state.txnp, TS_HTTP_TXN_CLOSE_HOOK, fetchCont);
 
     TSFetchEvent event_ids;
-    event_ids.success_event_id = TS_EVENT_FETCHSM_SUCCESS;
-    event_ids.failure_event_id = TS_EVENT_FETCHSM_FAILURE;
-    event_ids.timeout_event_id = TS_EVENT_FETCHSM_TIMEOUT;
+    event_ids.success_event_id = OperatorSetBodyFrom::TS_EVENT_FETCHSM_SUCCESS;
+    event_ids.failure_event_id = OperatorSetBodyFrom::TS_EVENT_FETCHSM_FAILURE;
+    event_ids.timeout_event_id = OperatorSetBodyFrom::TS_EVENT_FETCHSM_TIMEOUT;
 
-    struct sockaddr_in addr;
+    struct sockaddr_in addr {
+    };
     addr.sin_family      = AF_INET;
     addr.sin_addr.s_addr = LOCAL_IP_ADDRESS;
     addr.sin_port        = LOCAL_PORT;
-    TSFetchUrl(static_cast<const char *>(req_buf), req_buf_size, reinterpret_cast<struct sockaddr const *>(&addr), fetchCont,
-               AFTER_BODY, event_ids);
-
-    // Forces original status code in event TSHttpTxnErrorBodySet changed
-    // the code or another condition was set conflicting with this one.
-    // Set here because res is the only structure that contains the original status code.
+    TSFetchUrl(req_buf, req_buf_size, reinterpret_cast<struct sockaddr const *>(&addr), fetchCont, AFTER_BODY, event_ids);
     TSHttpTxnStatusSet(res.state.txnp, res.resp_status, PLUGIN_NAME);
   } else {
     TSError(PLUGIN_NAME, "OperatorSetBodyFrom:exec:: Could not create request");