[go] go get auto agent


Original link: https://blog.thinkeridea.com/201903/go/go_get_proxy.html

Recently, it was found that many people in the technology exchange group were asking about the failure of go get wall outsourcing, and they gave many solutions:

  • From GitHub’s code base clone
  • Set the goproxy environment variable to configure the proxy, for example: goproxy = https://goproxy.io
  • Configure command-line proxy, httpsproxy environment variable
  • Use go mod replace
  • Using tools like gopm
  • ……

Go’s community is very active. Gopher’s enthusiasm for go in China will not be reduced due to the existence of the wall. It can be seen from the community’s thinking of so many wall climbing schemes.

The above methods are all feasible, but there are always some unsatisfactory ones. The community has been looking for a better way. I have been using the automatic proxy method to obtain the packages outside the wall, which can support all the go native pull package operations, such as go get, go mod, DEP, godep, glide and other methods. It only needs to be configured once, as long as the command is run with a prefix before any native command. Yes, it’s very efficient.


Actual operation

The tool class does not talk about the principle first. The students who want to get the method directly can see this part. The students who want to understand the principle can see the principle part later.

You need to prepare the following tools:

  • An HTTP proxy
  • Git
  • Set SSH for GitHub account
  • Set SSH for other git related services (such as self built gitlab)
  • An environment that can run shell (GIT bash can be used for Linux, MAC and windows)

Specific steps

  • First, set the website that needs no proxy through GIT. Take GitHub as an example, executegit config --global [email protected]:.insteadof https://github.com/From HTTPS to SSH protocol, the HTTPS proxy we set will not work on SSH protocol. If there is a self built service, just change the address.
  • Create a new script (proxy) and change the proxy address to your own, as follows:

    #!/usr/bin/env bash 
    #If your system does not have bash, or / usr / bin / Env, please modify the previous line of instructions for your environment
    Export HTTP? Proxy = Proxy address needs to be changed to its own
    export https_proxy=
    export ftp_proxy=
    exec ${@:1}
  • Set the executable permission for the shell script, and then put it under the path environment variable path.
  • testproxy curl https://www.google.comandcurl https://www.google.comThe first command can get results, the second command can’t.
  • testproxy go get -v golang.org/x/text/…The package can be downloaded normally. Any other pull package command can be added.proxyPrefix execution, such asproxy dep ensure -v

As of now, you have configured onego getIn the environment of automatic agent, when the instruction of wall climbing operation is run, addproxyThat’s enough. This method is not only applicable togo get, which can be used by any command-line agent that requires it.

Principle chapter

The actual principle is simple, and it’s also a coincidence to find this method. Before going to the pit, I often used linux. At that time, there were some requirements that required the command line to climb the wall and found three environment variables.http_proxyhttps_proxyftp_proxyHowever, global settings slow down many requests. If you temporarily set a window, you need to remember that the window has a proxy set, and the cost of switching the window is relatively high. Later, according to the characteristics of the shell, any script has its own independent environment variables, so you can use a script to set the proxy environment variables.exec ${@:1}You can execute the instructions behind the script, that is, the instructions we actually need to run. In this way, you can add the script prefix before you need the commands of the agent, and the single line command agent is simply configured.

When I use go in the early stage, when I encounter a package that cannot be downloaded, I willgo getFront plusproxyInstruction, but I found that the efficiency of pulling GitHub package is very low, and it’s very fast to visit GitHub in China. It’s also a coincidence. At that time, our company’s go project was migrated to GitHub. All projects were private projects. A colleague provided a git HTTPS to SSH protocol operation.git config --global [email protected]….:.insteadof https://…./, this operation let me see a hidden benefit. The previous proxy only proxy HTTPS and can’t proxy SSH protocol. Then use this instruction to convert all websites that don’t need proxy to SSH protocol, and then addproxyFunctiongo getIt has become an automatic agent. It is really efficient after trying. It has been used up to now.


Author of this paper: Qi Yin (thinkeridea)

Link to this article: https://blog.thinkeridea.com/201903/go/go_get_proxy.html

Copyright notice: except for the special notice, all articles in this blog are licensed under CC by 4.0 CN agreement. Reprint please indicate the source!