A security warning repair & Analysis of GitHub project dependency

Time:2021-10-21

I write this article because I think I can supplement the marginal knowledge in the process of solving problems & the method of learning open source projects. Let’s take a look at your GitHub project. If there is a safety warning, you can refer to the idea of this article to practice together~

summary

Repair sectionHow to passpage & command lineMethod (GIT rebase) to handle security warnings

Analysis sectionAnalyze the main problems solved by this version upgrade of Axios (v0.21.0 = > v0.21.1), and try to solve the problems by TDD.

Zero, cause

I went into GitHub’s question brushing warehouse to find the following safety warning:

A security warning repair & Analysis of GitHub project dependency

There are “potential security vulnerabilities”. Open it to have a look:

A security warning repair & Analysis of GitHub project dependency

It’s actually a minor version upgrade of Axios vulnerability repair. As I just read the source code of Axios version 0.21.0 last week, I certainly want to know about it (and then this article)

1、 Repair

Maybe the order of the article is a little strange. After all, general problems are “analyzed” first and then “repaired”. However, the security vulnerability repair methods relied on by the project are the same, and more than half of the later analysis process may only be suitable for students in the front-end direction, so the “repair” link is put in front~

Let’s talk about the repair method. Generally, it is to manually repair the dependent version and submit it to the main branch or the current branch (please determine to modify the branch according to the release process of your project). However, after GitHub detects CVE (common vulnerabilities & exposures), this time cve-2020-28168, it will generally create a repair branch based on the main branch

A security warning repair & Analysis of GitHub project dependency

Now that the official has helped us to change it, there is no need to do it by ourselves. Direct git rebase one wave of penetration (if the main branch has made submission restrictions or the process has access control, just honestly merge). You can deal with the problem through the web page or command line:

P. S. even if my branch has just submitted before, it is OK, as shown in the figure below

A security warning repair & Analysis of GitHub project dependency

1. Web page operation mode

Tip: if you don’t have 200% confidence in the operation of the command line, please select this. The specification is easy to operate and there are prompts for errors

If you want to solve the problem on the web page, go to the pull requests tab of the project, and select the merge method after confirmation:

A security warning repair & Analysis of GitHub project dependency

Here I choose rebase adn merge, which will be merged into the code as the latest submission (and the branch that GitHub automatically generates security warnings can be deleted directly, which is convenient and fast). The results are as follows:

A security warning repair & Analysis of GitHub project dependency

2. Command line mode

Latest information of GIT fetch # pull project
Git pull origin master # ensures that the main branch is up to date
git checkout -b tmp origin/dependabot/npm_ and_ Yarn / axios-0.21.1 # pull "GitHub auto repair branch" to local TMP branch
Git rebase master TMP # rebase: execute on the TMP branch, tail the updated contents of TMP to the master, and store the results in the TMP branch
Git push origin TMP: the master # takes TMP as the master and submits it to the master branch of the remote warehouse
git push origin --delete dependabot/npm_ and_ Yarn / axios-0.21.1 # manually delete GitHub auto repair branch

A security warning repair & Analysis of GitHub project dependency

After completion, the result is consistent with that of the web page. After completion, enter the warehouse page and find that the security vulnerability warning prompt has disappeared (scattered flowers ✿✿ヽ (° ▽ °) ノ✿

A security warning repair & Analysis of GitHub project dependency

2、 Analysis

Git operation teaching aboveRepair partIt’s over for the time being. Now let’s look back and see what happened to Axios that led to the safety warning. Let’s go to the pull request tab:

A security warning repair & Analysis of GitHub project dependency

Release notes shows the update of v0.21.1 relative to v0.21.0, whereInternal and tests sectionThey are all test case fixes, which have little to do with security warnings, so let’s focus onFixes and functionality section, item 1Hotfix: Prevent SSRFIt is the information of safety warning.

From the following link #3410 points, you can understand the cause, discussion process and treatment method of this repair. Next, we will analyze the security warning from the initial issue.

0. Pre knowledge: what is “follow redirects”

Tip: I should have started analyzing the issue directly, but this knowledge point may affect the analysis and reading of the issue, so I’ll come here first

I’ll start a server program that returns to 302 in the local 8080. Let’s make up for itbrowserBuilt in HTTP module of node.jsAxios in node.js environmentWhat results will be obtained by accessing the localhost: 8080 link? The program is as follows:

const axios = require('axios')
const http = require('http')

http.createServer(function (req, res) { 
    res.writeHead(302, {location: 'http://example.com'})
    res.end()
}).listen(8080) 

A security warning repair & Analysis of GitHub project dependency

Now the answer is:

  • Browser: enter the target website provided by redirection http://example.com

A security warning repair & Analysis of GitHub project dependency

Based on MDN:

302: the status code indicates that the requested URI resource path has changed temporarily and may continue to change. Therefore, the client must continue to use the URI when accessing it in the future. The new URL will be displayed at the end of the responseLocation:Found in header field

Coupled with observing the process of network requests, it can be deducedbrowserThe request process of is shown in the figure below, in which “request process 2” isThe browser automatically implements “follow redirects”

A security warning repair & Analysis of GitHub project dependency

  • Built in HTTP module of node.js: get 302 status code, indicating that the built-in HTTP module does not have“Follow redirects”Capability, only the first request process is executed, that is, the 302 status code and redirection address are obtained:

    A security warning repair & Analysis of GitHub project dependency

  • Axios in node.js environment: andbrowserThe final result is the same as the status code 200, and the web page content can be obtained

    A security warning repair & Analysis of GitHub project dependency

So why is it the sameNode.js environmentaxiosandBuilt in HTTP moduleHow does it behave differently for redirection? Search for redirects in the Axios GitHub document, and thenRequest configurationThe following information can be found in this section:

A security warning repair & Analysis of GitHub project dependency

Axios is on by default“Follow redirects”Ability (by the way, how to add follow redirects to Axios in 2015), and can changemaxRedirectsField“Follow redirects”Open or close the. When 302 status code is encountered, if it is enabled“Follow redirects”It will get the redirect address and continue to jump. If it is not turned on, it will return the result directly

Now putmaxRedirectsSet the field to 0 and run the program again. You can see that Axios handles redirection in the same way as nodejs:

A security warning repair & Analysis of GitHub project dependency

Understand what is“Follow redirects”After that, let’s take a look at the problems encountered by the elder brother who mentioned issue~

1. Initial issue analysis

Let’s first look at the cause of this safety warning:

  • Link: issue3369 – the request to reschedule back was not passed through the proxy
  • Note: the elder brother who raised the issue proposed that he use the Axios with “proxy configuration” to access. Since the request passes through the proxy server and the proxy server always returns 302, the operation phenomenon expected by the elder brother is:

    • 1) For the first request, because of the configuration, the 302 status code is obtained through the proxy server address localhost: 8080. Because Axios is enabled by default“Follow redirects”Therefore, the results will not be printed for the time being, and Axios will automatically try to access the destination address reset backward http://example.com ;
    • 2) However, due to the existence of agent configuration, the second access will still access localhost: 8080 and obtain 302 status code again;
    • 3) Repeat the above two steps until the maximum number of follow redirects modes of Axios is reached, and then an error is reported;

Now let’s take a look at the complete reproduction code provided by the elder brother of issue:

const axios = require('axios')
const http = require('http')

const PROXY_PORT = 8080
let count = 0

// A fake proxy server
http.createServer(function (req, res) {
      Count + + // (the counting logic I added) accumulate the number of requests
    res.writeHead(302, {location: 'http://example.com'})
    res.end()
  }).listen(PROXY_PORT)

//Compared with our previous case, an "Axios request with proxy configuration" is added
axios({
  method: "get",
  url: " http://www.google.com/ ", // any link written will be replaced by a proxy object
  Proxy: {// key parts
    host: "localhost",
    port: PROXY_PORT,
  },
})
.then((r) => {
  Console.log (count) // (print I added) number of times to access the proxy server for successful printing
  Console.log (r.status) // (the print I added) is used to observe the returned status code
  console.log(r.data)
})
.catch(e => {
  Console.log (count) // number of times to access the proxy server for printing failure
  console.error(e)
})

Different from the 302 error results we expect, the execution effect of using Axios v0.21.0 is as follows:

A security warning repair & Analysis of GitHub project dependency

The phenomenon is:Enter the proxy server once, and then the status code of Axios request result is 200, and the content is http://example.com Page content for.

According to the phenomenon, the execution process is: the first request successfully entered the proxy server and accumulated a count (and only this time), but the request in step 2 bypassed the proxy in the Axios configuration and directly accessed the redirect address http://example.com.

So now the problem is obvious. When 302 returns, you shouldCarry agent configurationThe request was reissued, but Axios v0.21.0 was lost when the request was reissuedAgent configuration, so the thing to do is to study Axios“Follow redirects”Why was it lostAgent configuration.

2. Problem location & formulation of repair scheme

The problem is the nodejs environment, so it is natural to find the location of nodejs redirection request configuration in the Axios source code.

Go to the / lib / default.js location of the Axios project, and the following code gives the Axios implementationCross platform network request capability, it will automatically determine the running platform and use different platform logic to implement network requests:

function getDefaultAdapter() {
  var adapter;
  if (typeof XMLHttpRequest !== 'undefined') {
    adapter = require('./adapters/xhr'); //  Browser environment go here
  } else if (typeof process !== 'undefined' && Object.prototype.toString.call(process) === '[object process]') {
    adapter = require('./adapters/http'); //  The node.js environment goes here
  }
  return adapter;
}

Next, after entering / lib / adapters / http.js, it is found that there is only one function that receives the config parameter and returns a promise:

A security warning repair & Analysis of GitHub project dependency

The request logic implementation should all be in this function, and then directly search config.proxy to find itAgent logic, find relevant codes and get the following analysis:

A security warning repair & Analysis of GitHub project dependency

The httpfollow in line 171 here comes from the follow redirects module. The official description of the module is:

Drop-in replacement for Node’s http and https modules that automatically follows redirects.

In other words, the module has the capability of built-in HTTP & HTTPS module and“Follow redirects”Capability (default)“Follow redirects”Up to 21 times, i.emaxRedirectsProperties).

After knowing this, I want to fix it“Missing agent configuration”To solve this problem, you need to know how to use the follow redirects module. Find the official demo here:

const url = require('url');
const { http, https } = require('follow-redirects');

const options = url.parse('http://bit.ly/900913');
options.maxRedirects = 10;
options.beforeRedirect = (options, { headers }) => {
  //Adjust options during redirection
  if (options.hostname === "example.com") {
    options.auth = "user:password";
  }
};
http.request(options);

So thisoptions.beforeRedirectThis is what we are looking for. It passes in options before executing the request and runs the function body to modify the options. Therefore, beforeredirect needs to be added in / lib / adapters / http.js of Axios project to redirect the originalAgent configurationAdd options

4. Repair the issue by TDD

In order to test the repair effect, we pull the source code of Axios v0.21.0 (the commit is 94ca24b5b23f343769a15f325693246e07c177d2) locally, copy the code of Axios v0.21.1 test case / test / unit / region / snyk-js-axios-1038255.js, create a folder with the same name in the project and paste it. The following is the code and analysis of the test case:

// https://snyk.io/vuln/SNYK-JS-AXIOS-1038255
// https://github.com/axios/axios/issues/3407
// https://github.com/axios/axios/issues/3369

const axios = require('../../../index');
const http = require('http');
const assert = require('assert');

const PROXY_ PORT = 4777; //  Proxy server port
const EVIL_ PORT = 4666; //  Redirect the port of the location address. If the code logic is correct, it should not enter the port

describe('Server-Side Request Forgery (SSRF)', () => {
  let fail = false;
  let proxy;
  let server;
  let location;
  beforeEach(() => {
    server = http.createServer(function (req, res) {
      fail = true;
      res.end('rm -rf /');
    }).listen(EVIL_PORT);
    proxy = http.createServer(function (req, res) {
      //When the first request reaches the proxy server, req.url is http://www.google.com/ , go back to the logic of 302
      //The second request is sent by the "follow redirects" capability of Axios. The URL should be http://localhost:4666
      if (req.url === 'http://localhost:' + EVIL_PORT + '/') {
        return res.end(JSON.stringify({
          msg: 'Protected',
          Headers: req.headers, // return the request header
        }));
      }
      Res.writehead (302, {location}) // the status code 302 and 302 are returned after the first request http://localhost:4666
      res.end()
    }).listen(PROXY_PORT);
  });
  afterEach(() => {
    server.close();
    proxy.close();
  });
  it('obeys proxy settings when following redirects', async () => {
    location = 'http://localhost:' + EVIL_PORT;
    let response = await axios({
      method: "get",
      url: "http://www.google.com/",
      proxy: {
        host: "localhost",
        port: PROXY_PORT,
        auth: {
          username: 'sam',
          password: 'password',
        }
      },
    });

    assert.strictEqual(fail, false);
    assert.strictEqual(response.data.msg, 'Protected');
    assert.strictEqual(response.data.headers.host, 'localhost:' + EVIL_PORT);
    assert.strictEqual(response.data.headers['proxy-authorization'], 'Basic ' + Buffer.from('sam:password').toString('base64'));

    return response;
  });
});

Then use it locallynpm testperhapsyarn testRun test, the results are as follows:

A security warning repair & Analysis of GitHub project dependency

It is true that the new test case exploded. Because the corresponding capabilities are not implemented in Axios, no problems are prompted. Now let’s verify and write the repair Code:

The problem isAgent configurationMissing, so go to / lib / adapters / http.js to find the proxy related code (lines 148 ~ 152):

A security warning repair & Analysis of GitHub project dependency

Copy a copy to beforeredirect (the Axios project has lint check, so change the options to tmpoptions) so that the agent will not be lost~

if (proxy) {
  options.beforeRedirect = function(tmpOption) {
    tmpOption.hostname = proxy.host;
    tmpOption.host = proxy.host;
    tmpOption.headers.host = parsed.hostname + (parsed.port ? ':' + parsed.port : '');
    tmpOption.port = proxy.port;
    tmpOption.path = protocol + '//' + parsed.hostname + (parsed.port ? ':' + parsed.port : '') + options.path;
  };
} 

Add the above code to line 160 of / lib / adapters / http.js and use it againnpm testperhapsyarn testThe running test was carried out successfully this time“Follow redirects”Requested and reported an error due to exceeding the maximum number of times:

A security warning repair & Analysis of GitHub project dependency

The reason why it didn’t achieve the desired effect was that it didn’t“Follow redirects”Request to configure the correct request target link. Original test case expectations:

  • When the first request reaches the proxy server, req.url is http://www.google.com/ , go back to the logic of 302 and“Follow redirects”Target link set to http://localhost:4666
  • The second request is sent by the “follow redirects” capability of Axios. Req.url should be http://localhost:4666

However, the actual situation is that the target link is not updated after the first request 302 is returned, so you still need to read the following redirects moduleoptions.beforeRedirectCall location of (index. JS under the root directory):

A security warning repair & Analysis of GitHub project dependency

You can see the call at line 357options.beforeRedirectOptions & responsedetails containing the redirection response body is passed in, so we get the target configuration from the response, and fill in the path and headers.host:

if (proxy) {
  Options. Beforeredirect = function (tmpoption, response) {// response corresponds to the responsedetails of the redirect response body
    //Hostname (host) & port is the domain name & port of the server to which the request is sent. The proxy configuration remains unchanged
    //Path & headers.host is the target path & target host. When 302 is encountered, you should read the location and refill it
    tmpOption.hostname = proxy.host;
    tmpOption.host = proxy.host;
    tmpOption.port = proxy.port;
    // tmpOption.headers.host = parsed.hostname + (parsed.port ? ':' + parsed.port : '');
    // tmpOption.path = protocol + '//' + parsed.hostname + (parsed.port ? ':' + parsed.port : '') + options.path;
    var redirectInfo = url.parse(response.headers.location);
    tmpOption.path = redirectInfo.href; //  Redirect link
    tmpOption.headers.host = redirectInfo.host; //  Redirected target host
  };
}

Run the test again and successfully pass:

A security warning repair & Analysis of GitHub project dependency

5. Compare official problem solutions

The problem has been solved and passed the test case, but there may be omissions, so it must be compared with the official repair for verification, so as to meet the learning closed loop.

You can see #3410 related submissions:

A security warning repair & Analysis of GitHub project dependency

This is also a TDD process. First, write test cases to reproduce the issue, then fix the problem, and then refactor the code.

Of course, you can also directly click the last file changed tab to directly see what code has been modified as a whole.

It can be seen that the official processing method is basically the same as the above repair method except for the reconstruction part, but one point aroused my interest:

A security warning repair & Analysis of GitHub project dependency

options.beforeRedirectIn the method body, only one redirection is used to complete the assignment of redirection.header.host, and we used the second parameter above. Here, I choose to continue to look in the follow redirects module. Sure enough, I found it in redirectablerequest.prototype_ This logic was found in processresponse:

A security warning repair & Analysis of GitHub project dependency

Therefore, the repair code just now can no longer use the second parameter response, and there is no need to re resolve the location:

if (proxy) {
  options.beforeRedirect = function(tmpOption) {
    var redirectHost = tmpOption.host; //  Take it out first to prevent it from being covered
    tmpOption.hostname = proxy.host;
    tmpOption.host = proxy.host;
    tmpOption.port = proxy.port;
    tmpOption.path = tmpOption.href; //  Redirected destination path
    tmpOption.headers.host = redirectHost; //  Redirected target host
  };
}

Run the test again and successfully pass:

A security warning repair & Analysis of GitHub project dependency

OK, here the analysis of #3410 is completed ~ for the second problem, protocol not parsed when setting proxy config from env vars #3070, you can also try to parse and learn in this way~

Flower spreading ✿✿ヽ (° ▽ °) ノ✿

Welcome to make bricks. I think it’s OK. You’re also welcome to praise the collection~
Xinkaigong: “adventure tan without dream” welcome to pay attention (search nodreame ~)
The journey is continuing

Recommended Today

Swift advanced (XV) extension

The extension in swift is somewhat similar to the category in OC Extension can beenumeration、structural morphology、class、agreementAdd new features□ you can add methods, calculation attributes, subscripts, (convenient) initializers, nested types, protocols, etc What extensions can’t do:□ original functions cannot be overwritten□ you cannot add storage attributes or add attribute observers to existing attributes□ cannot add parent […]