Go gorilla / handlers

Time:2021-7-29

brief introduction

In the previous article, we introduced the routing management library in the gorilla web development kitgorilla/muxAt the end of the article, we introduce how to use middleware to deal with general logic. In daily go web development, developers encounter many of the same middleware requirements, gorilla / handlers (hereinafter referred to ashandlers)Collected some commonly used middleware. Let’s have a look~

The previous articles have introduced a lot about middleware. I won’t repeat it here.handlersThe middleware provided by the library can be used in the standard librarynet/httpAnd all supporthttp.HandlerInterface framework. becausegorilla/muxAlso supporthttp.HandlerInterface, so it can also be connected withhandlersLibrary.This is the benefit of standards compatibility

Project initialization & installation

The code in this article uses go modules.

Create directory and initialize:

$ mkdir gorilla/handlers && cd gorilla/handlers
$ go mod init github.com/darjun/go-daily-lib/gorilla/handlers

installgorilla/handlersLibrary:

$ go get -u github.com/gorilla/handlers

The following describes each middleware and the corresponding source code in turn.

journal

handlersTwo logging middleware are provided:

  • LoggingHandler: Based on ApacheCommon Log FormatRecord HTTP request log in log format;
  • CombinedLoggingHandler: Based on ApacheCombined Log FormatThe log format records HTTP request logs. Apache and nginx use this log format by default.

There is little difference between the two log formats,Common Log FormatThe format is as follows:

%h %l %u %t "%r" %>s %b

Each indicator has the following meanings:

  • %h: IP address or host name of the client;
  • %lRFC 1413Client ID defined by the client on the client machineidentdProgram generation. If it does not exist, the field is-
  • %u: authenticated user name. If it does not exist, the field is-
  • %t: time in the formatday/month/year:hour:minute:second zone, where:

    • day: 2 digits;
    • monthMonth abbreviation, 3 letters, such as:Jan
    • year: 4 digits;
    • hour: 2 digits;
    • minute: 2 digits;
    • second: 2 digits;
    • zone+or-Followed by 4 digits;
    • For example:21/Jul/2021:06:27:33 +0800
  • %r: contains HTTP request line information, for exampleGET /index.html HTTP/1.1
  • %>s: status code sent by the server to the client, for example:200
  • %b: response length in bytes.

Combined Log FormatThe format is as follows:

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"

Visible comparisonCommon Log FormatJust more:

  • %{Referer}i: in HTTP headerRefererInformation;
  • %{User-Agent}i: in HTTP headerUser-AgentInformation.

For middleware, we can make it work globally, that is, all processors, or make it work only for some processors. If you want to be effective for all processors, you can callUse()method. If you only need to act on a specific processor, wrap the processor with middleware during registration:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Handle("/", handlers.LoggingHandler(os.Stdout, http.HandlerFunc(index)))
  r.Handle("/greeting", handlers.CombinedLoggingHandler(os.Stdout, greeting("dj")))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

In the above codeLoggingHandlerWorks only on handler functionsindexCombinedLoggingHandlerProcessor onlygreeting("dj")

Run the code and access it through the browserlocalhost:8080andlocalhost:8080/greeting

::1 - - [21/Jul/2021:06:39:45 +0800] "GET / HTTP/1.1" 200 12
::1 - - [21/Jul/2021:06:39:54 +0800] "GET /greeting HTTP/1.1" 200 11 "" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"

Comparing the indicators analyzed earlier, it is easy to see the various parts.

because*mux.RouterYesUse()Method acceptance type isMiddlewareFuncMiddleware:

type MiddlewareFunc func(http.Handler) http.Handler

andhandlers.LoggingHandler/CombinedLoggingHandlerNot satisfied, so it needs another layer of packaging before it can be passed toUse()method:

func Logging(handler http.Handler) http.Handler {
  return handlers.CombinedLoggingHandler(os.Stdout, handler)
}

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

in additionhandlersIt also providesCustomLoggingHandler, we can use it to define our own logging middleware:

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler

The most criticalLogFormatterType definition:

type LogFormatterParams struct {
  Request    *http.Request
  URL        url.URL
  TimeStamp  time.Time
  StatusCode int
  Size       int
}

type LogFormatter func(writer io.Writer, params LogFormatterParams)

We implement a simpleLogFormatter, record time + request line + response code:

func myLogFormatter(writer io.Writer, params handlers.LogFormatterParams) {
  var buf bytes.Buffer
  buf.WriteString(time.Now().Format("2006-01-02 15:04:05 -0700"))
  buf.WriteString(fmt.Sprintf(` "%s %s %s" `, params.Request.Method, params.URL.Path, params.Request.Proto))
  buf.WriteString(strconv.Itoa(params.StatusCode))
  buf.WriteByte('\n')

  writer.Write(buf.Bytes())
}

func Logging(handler http.Handler) http.Handler {
  return handlers.CustomLoggingHandler(os.Stdout, handler, myLogFormatter)
}

use:

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The log recorded now is in the following format:

2021-07-21 07:03:18 +0800 "GET /greeting/ HTTP/1.1" 200

Looking at the source code, we can findLoggingHandler/CombinedLoggingHandler/CustomLoggingHandlerAre based on the bottomloggingHandlerThe difference isLoggingHandlerPredefinedwriteLogAsLogFormatterCombinedLoggingHandlerPredefinedwriteCombinedLogAsLogFormatter, andCustomLoggingHandlerUse our own definitionLogFormatter

func CombinedLoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeCombinedLog}
}

func LoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeLog}
}

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler {
  return loggingHandler{out, h, f}
}

PredefinedwriteLog/writeCombinedLogThe implementation is as follows:

func writeLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, '\n')
  writer.Write(buf)
}

func writeCombinedLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, ` "`...)
  buf = appendQuoted(buf, params.Request.Referer())
  buf = append(buf, `" "`...)
  buf = appendQuoted(buf, params.Request.UserAgent())
  buf = append(buf, '"', '\n')
  writer.Write(buf)
}

They are all based onbuildCommonLogLineConstruction basic information,writeCombinedLogAlso called separatelyhttp.Request.Referer()andhttp.Request.UserAgentGot itRefererandUser-AgentInformation.

loggingHandlerIt is defined as follows:

type loggingHandler struct {
  writer    io.Writer
  handler   http.Handler
  formatter LogFormatter
}

loggingHandlerThe implementation has a clever place: in order to record the response code and response size, a type is definedresponseLoggerOriginal packaginghttp.ResponseWriter, record information when writing:

type responseLogger struct {
  w      http.ResponseWriter
  status int
  size   int
}

func (l *responseLogger) Write(b []byte) (int, error) {
  size, err := l.w.Write(b)
  l.size += size
  return size, err
}

func (l *responseLogger) WriteHeader(s int) {
  l.w.WriteHeader(s)
  l.status = s
}

func (l *responseLogger) Status() int {
  return l.status
}

func (l *responseLogger) Size() int {
  return l.size
}

loggingHandlerKey methods ofServeHTTP()

func (h loggingHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  t := time.Now()
  logger, w := makeLogger(w)
  url := *req.URL

  h.handler.ServeHTTP(w, req)
  if req.MultipartForm != nil {
    req.MultipartForm.RemoveAll()
  }

  params := LogFormatterParams{
    Request:    req,
    URL:        url,
    TimeStamp:  t,
    StatusCode: logger.Status(),
    Size:       logger.Size(),
  }

  h.formatter(h.writer, params)
}

structureLogFormatterParamsObject, calling the correspondingLogFormatterFunction.

compress

If there are in the client requestAccept-EncodingIn the header, the server can compress the response using the algorithm indicated in the header to save network traffic.handlers.CompressHandlerThe middleware enables compression. One moreCompressHandlerLevelYou can specify the compression level. actuallyCompressHandlerIs to usegzip.DefaultCompressionInvokedCompressHandlerLevel

func CompressHandler(h http.Handler) http.Handler {
  return CompressHandlerLevel(h, gzip.DefaultCompression)
}

Look at the code:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CompressHandler)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Run, requestlocalhost:8080, you can see that the response adopts gzip compression through the network tab of chrome developer tool:

Go gorilla / handlers

Ignore some details,CompressHandlerLevelThe function code is as follows:

func CompressHandlerLevel(h http.Handler, level int) http.Handler {
  const (
    gzipEncoding  = "gzip"
    flateEncoding = "deflate"
  )

  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    var encoding string
    for _, curEnc := range strings.Split(r.Header.Get(acceptEncoding), ",") {
      curEnc = strings.TrimSpace(curEnc)
      if curEnc == gzipEncoding || curEnc == flateEncoding {
        encoding = curEnc
        break
      }
    }

    if encoding == "" {
      h.ServeHTTP(w, r)
      return
    }

    if r.Header.Get("Upgrade") != "" {
      h.ServeHTTP(w, r)
      return
    }

    var encWriter io.WriteCloser
    if encoding == gzipEncoding {
      encWriter, _ = gzip.NewWriterLevel(w, level)
    } else if encoding == flateEncoding {
      encWriter, _ = flate.NewWriter(w, level)
    }
    defer encWriter.Close()

    w.Header().Set("Content-Encoding", encoding)
    r.Header.Del(acceptEncoding)

    cw := &compressResponseWriter{
      w:          w,
      compressor: encWriter,
    }

    w = httpsnoop.Wrap(w, httpsnoop.Hooks{
      Write: func(httpsnoop.WriteFunc) httpsnoop.WriteFunc {
        return cw.Write
      },
      WriteHeader: func(httpsnoop.WriteHeaderFunc) httpsnoop.WriteHeaderFunc {
        return cw.WriteHeader
      },
      Flush: func(httpsnoop.FlushFunc) httpsnoop.FlushFunc {
        return cw.Flush
      },
      ReadFrom: func(rff httpsnoop.ReadFromFunc) httpsnoop.ReadFromFunc {
        return cw.ReadFrom
      },
    })

    h.ServeHTTP(w, r)
  })
}

Request fromAccept-EncodingGet the compression algorithm indicated by the client in the header. If the client does not specify it, or there is an error in the request headerUpgrade, no compression. Otherwise, it is compressed. According to the identified compression algorithm, create the correspondinggziporflateYesio.WriterImplementation object.

Like the previous logging middleware, in order to compress the written content, new types are addedcompressResponseWriterencapsulationhttp.ResponseWriter, rewriteWrite()Method to pass the written byte stream into the previously createdio.WriterAchieve compression:

type compressResponseWriter struct {
  compressor io.Writer
  w          http.ResponseWriter
}

func (cw *compressResponseWriter) Write(b []byte) (int, error) {
  h := cw.w.Header()
  if h.Get("Content-Type") == "" {
    h.Set("Content-Type", http.DetectContentType(b))
  }
  h.Del("Content-Length")

  return cw.compressor.Write(b)
}

Content type

We can passhandler.ContentTypeHandlerSpecifies the of the requestContent-TypeMust be in the type we give, only forPOST/PUT/PATCHThe method takes effect. For example, we restrict that login requests must passapplication/x-www-form-urlencodedSent as:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Methods("GET").Path("/login").HandlerFunc(login)
  r.Methods("POST").Path("/login").
    Handler(handlers.ContentTypeHandler(http.HandlerFunc(dologin), "application/x-www-form-urlencoded"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

So, just ask/loginYesContent-Typenoapplication/x-www-form-urlencoded415 error will be returned. Ask us to make a mistake on purpose and then look at our performance:

Unsupported content type "application/x-www-form-urlencoded"; expected one of ["application/x-www-from-urlencoded"]

ContentTypeHandlerThe implementation of is very simple:

func ContentTypeHandler(h http.Handler, contentTypes ...string) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    if !(r.Method == "PUT" || r.Method == "POST" || r.Method == "PATCH") {
      h.ServeHTTP(w, r)
      return
    }

    for _, ct := range contentTypes {
      if isContentType(r.Header, ct) {
        h.ServeHTTP(w, r)
        return
      }
    }
    http.Error(w, fmt.Sprintf("Unsupported content type %q; expected one of %q", r.Header.Get("Content-Type"), contentTypes), http.StatusUnsupportedMediaType)
  })
}

Is to readContent-TypeFirst, judge whether it is in the type we specify.

Method processor

In the above example, we register the path/loginYesGETandPOSTMethod treatmentr.Methods("GET").Path("/login").HandlerFunc(login)This lengthy way of writing.handlers.MethodHandlerThis can be simplified:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Handle("/login", handlers.MethodHandler{
    "GET":  http.HandlerFunc(login),
    "POST": http.HandlerFunc(dologin),
  })

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

MethodHandlerThe bottom is amap[string]http.HandlerType, itsServeHTTP()Method calls different processing according to the requested method:

type MethodHandler map[string]http.Handler

func (h MethodHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  if handler, ok := h[req.Method]; ok {
    handler.ServeHTTP(w, req)
  } else {
    allow := []string{}
    for k := range h {
      allow = append(allow, k)
    }
    sort.Strings(allow)
    w.Header().Set("Allow", strings.Join(allow, ", "))
    if req.Method == "OPTIONS" {
      w.WriteHeader(http.StatusOK)
    } else {
      http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
    }
  }
}

Method, if not registered, returns405 Method Not Allowed。 Except for one method,OPTIONS。 The method passesAllowWhich methods are supported by the first return.

redirect

handlers.CanonicalHostRedirect the request to the specified domain name and the response code. It is useful when the same server corresponds to multiple domain names:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "hello world")
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CanonicalHost("http://www.gorillatoolkit.org", 302))
  r.HandleFunc("/", index)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The above redirects all requests to 302http://www.gorillatoolkit.org

CanonicalHostThe implementation of is also very simple:

func CanonicalHost(domain string, code int) func(h http.Handler) http.Handler {
  fn := func(h http.Handler) http.Handler {
    return canonical{h, domain, code}
  }

  return fn
}

Key typescanonical

type canonical struct {
  h      http.Handler
  domain string
  code   int
}

Core approach:

func (c canonical) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  dest, err := url.Parse(c.domain)
  if err != nil {
    c.h.ServeHTTP(w, r)
    return
  }

  if dest.Scheme == "" || dest.Host == "" {
    c.h.ServeHTTP(w, r)
    return
  }

  if !strings.EqualFold(cleanHost(r.Host), dest.Host) {
    dest := dest.Scheme + "://" + dest.Host + r.URL.Path
    if r.URL.RawQuery != "" {
      dest += "?" + r.URL.RawQuery
    }
    http.Redirect(w, r, dest, c.code)
    return
  }

  c.h.ServeHTTP(w, r)
}

It can be seen from the source code that the domain name is illegal or does not forward the request without specifying the protocol (scheme) or domain name (host).

Recovery

We realized it ourselves beforePanicRecoverMiddleware to avoid panic during request processing.handlersProvides aRecoveryHandlerCan be used directly:

func PANIC(w http.ResponseWriter, r *http.Request) {
  panic(errors.New("unexpected error"))
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.RecoveryHandler(handlers.PrintRecoveryStack(true)))
  r.HandleFunc("/", PANIC)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

optionPrintRecoveryStackOutput stack information when representing panic.

RecoveryHandlerThe implementation of is basically the same as that we wrote before:

type recoveryHandler struct {
  handler    http.Handler
  logger     RecoveryHandlerLogger
  printStack bool
}

func (h recoveryHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  defer func() {
    if err := recover(); err != nil {
      w.WriteHeader(http.StatusInternalServerError)
      h.log(err)
    }
  }()

  h.handler.ServeHTTP(w, req)
}

summary

There are many open source go web middleware implementations on GitHub, which can be used directly to avoid repeated wheel building.handlersVery lightweight and easy to work with standard librariesnet/httpAnd gorilla routing LibrarymuxCombined use.

If you find a fun and easy-to-use go language library, you are welcome to submit an issue on GitHub, the daily library of go

reference resources

  1. gorilla/handlers GitHub:github.com/gorilla/handlers
  2. Go one library a day GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to my WeChat official account, GoUpUp, learn together and make progress together.

Go gorilla / handlers