MyBatis source code study notes (1) first encounter

Time:2022-9-22

The opening of this series of articles "When we talk about looking at the source code, what are we looking at" started in October last year, and today I started to fill in the pits of this series. MyBatis is the first ORM framework I came into contact with, and it is also the ORM framework I am most familiar with. I have been using it for a long time. Today, I will try to see the internal structure of MyBatis. If you don't know MyBatis yet, you can read "Pretending to be Xiaobai's Re-learning MyBatis (1)" first.

So how to look at the source code?

I downloaded the source code of MyBatis and looked at it aimlessly? Will this be lost in the source code? I remember that when I first arrived at my current company, I looked at the code one by one, and then I felt a headache, and I didn't understand what to do in the end. After reflecting for a while, you should actually pay attention to the macro process, that is, what function does this code achieve. These codes are all to achieve this function. You don’t need to look at each method line by line, but look at the method as a unit. This method is from the whole Let's see what kind of things have been done, and don't have to pay too much attention to the internal implementation details. Looking at the code in this way, you probably have a good idea. Also in MyBatis, this is also the first code I have studied carefully, so in the first article of the MyBatis series, we first look at its implementation from a macro perspective, and slowly fill in its details in the later process. The main line of this article is how the addition, deletion, modification, and query statements we wrote in xml are executed.

After referring to a lot of MyBatis source code information, the overall architecture of MyBatis can be divided into three layers:

  • Interface layer: SqlSession is the core interface that we usually interact with MyBatis (including the SqlSessionTemplte used by the subsequent integration of SpringFramework)
  • Core layer: The method of SqlSession execution, the bottom layer needs to go through configuration file parsing, SQL parsing, parameter mapping, SQL execution, result set mapping when executing SQL, and there are extension plug-ins interspersed in it.
  • Support layer: The function realization of the core layer is based on the coordination of various modules at the bottom layer.

Build the MyBatis environment

The environment for building MyBatis has been discussed in "Pretending to be Xiaobai's Heavy Learning MyBatis (1)", here is just a brief talk:

  • Introduce Maven dependencies
<dependency>
  <groupId>org.mybatis</groupId>
  <artifactId>mybatis</artifactId>
  <version>3.5.6</version>
 </dependency>
  <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.47</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.2.5</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.30</version>
        </dependency>
  • then a table
CREATE TABLE `student`  (
  `id` int(11) NOT NULL COMMENT 'Unique ID',
  `name` varchar(255) ,
  `number` varchar(255) ,
  `money` int(255) NULL DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4;
  • Come to a MyBatis configuration file
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
        PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
        "http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
    <!--加载配置文件-->
    <properties resource="jdbc.properties"/>
    <!--指定默认环境, 一般情况下,我们有三套环境,dev 开发 ,uat 测试 ,prod 生产 -->
    <environments default="development">
        <environment id="development">
            <!-- 设置事务管理器的管理方式  -->
            <transactionManager type="JDBC"/>
            <!-- 设置数据源连接的关联方式为数据池  -->
            <dataSource type="POOLED">
                <property name="driver" value="com.mysql.jdbc.Driver"/>
                <property name="url" value="jdbc:mysql://localhost:3306/studydatabase?characterEncoding=utf-8"/>
                <property name="username" value="root"/>
                <property name="password" value="root"/>
            </dataSource>
        </environment>
    </environments>
    <mappers>
        <!--设置扫描的xml,org/example/mybatis是包的全类名,StudentMapper.xml会讲-->

    <mappers>
        <!--设置扫描的xml,org/example/mybatis是包的全类名,这个BlogMapper.xml会讲
         <package name = "org.example.mybatis"/> <!-- 包下批量引入 单个注册 -->
          <mapper resource="org/example/mybatis/StudentMapper.xml"/> 
    </mappers>
    </mappers>
</configuration>
  • Come to a Student class
public class Student {
    private Long id;
    private String name;
    private String number;
    private String money;
    // omit get set function
}
  • Come to a Mapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper
        PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
        "http://mybatis.org/dtd/mybatis-3-mapper.dtd">

<mapper namespace = "org.example.mybatis.StudentMapper">
    <select id = "selectStudent" resultType = "org.example.mybatis.Student">
        SELECT * FROM STUDENT
    </select>
</mapper>
  • come up with an interface
public interface StudentMapper {
    List<Student> selectStudent();
}
  • log configuration file
log4j.rootCategory=debug, CONSOLE

# Set the enterprise logger category to FATAL and its only appender to CONSOLE.
log4j.logger.org.apache.axis.enterprise=FATAL, CONSOLE

# CONSOLE is set to be a ConsoleAppender using a PatternLayout.
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding=UTF-8
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %-6r [%15.15t] %-5p %30.30
  • Start your inquiry journey
public class MyBatisDemo {
    public static void main(String[] args) throws Exception {
        Reader reader = Resources.getResourceAsReader("conf.xml");
        SqlSessionFactory sqlSessionFactory = new SqlSessionFactoryBuilder().build(reader);
        SqlSession sqlSession = sqlSessionFactory.openSession();
        StudentMapper studentMapper = sqlSession.getMapper(StudentMapper.class);
        List<Student> studentList = studentMapper.selectStudent();
        studentList.forEach(System.out::println);
    }
}

After execution, you can see the following output in the console:

Let's start with the SQL execution journey

Analysis of the execution process

The above execution process can be roughly divided into three steps:

  • Parse the configuration file and build the SqlSessionFactory
  • Get SqlSession through SqlSessionFactory, and then get proxy class
  • Execute the method of the proxy class

Parse the configuration file

Parsing configuration files is performed through the build method of SqlSessionFactoryBuilder, which has several overloads:

Reader points to the conf file, environment is the environment, and properties are used for conf to get values ​​from other properties. Our configuration file is an xml, so XmlConfigBuilder is ultimately an encapsulation of the configuration file. Here we don't pay attention to how the XmlBuilder is constructed. Let's look down. After building the Xml object, call the parse method to convert it into the MyBatis Configuration object:

// parseConfiguration This method is used to take the value of the xml tag and set it to the Configuration
public Configuration parse() {
    if (parsed) {
      throw new BuilderException("Each XMLConfigBuilder can only be used once.");
    }
    parsed = true;
    parseConfiguration(parser.evalNode("/configuration"));
    return configuration;
  }
// The process of getting tags, XML-&gt;Configuration
private void parseConfiguration(XNode root) {
    try {
      // issue #117 read properties first
      propertiesElement(root.evalNode("properties"));
      Properties settings = settingsAsProperties(root.evalNode("settings"));
      loadCustomVfs(settings);
      loadCustomLogImpl(settings);
      typeAliasesElement(root.evalNode("typeAliases"));
      pluginElement(root.evalNode("plugins"));
      objectFactoryElement(root.evalNode("objectFactory"));
      objectWrapperFactoryElement(root.evalNode("objectWrapperFactory"));
      reflectorFactoryElement(root.evalNode("reflectorFactory"));
      settingsElement(settings); 
      // read it after objectFactory and objectWrapperFactory issue #631
      environmentsElement(root.evalNode("environments"));
      databaseIdProviderElement(root.evalNode("databaseIdProvider"));
      typeHandlerElement(root.evalNode("typeHandlers"));
      mapperElement(root.evalNode(&quot;mappers&quot;)); // Get the mapper method,
    } catch (Exception e) {
      throw new BuilderException("Error parsing SQL Mapper Configuration. Cause: " + e, e);
    }
  }
  • Configuration overview

  • mapperElement

​ Note that the theme of this article is to focus on how the sql we write in the xml tag is executed, so here we focus on the mapperElement method of parseConfiguration. From the name, we roughly infer that this method loads the mapper.xml file. Let's click in and take a look:

// parent is the mappers tag
private void mapperElement(XNode parent) throws Exception {
    if (parent != null) {
      for (XNode child : parent.getChildren()) { // Traverse the nodes under mappers
        if (&quot;package&quot;.equals(child.getName())) { // If it is a package tag, import it in batches
          String mapperPackage = child.getStringAttribute("name");
          configuration.addMappers(mapperPackage);
        } else {
          String resource = child.getStringAttribute(&quot;resource&quot;); // This time we look at the single introduction method
          String url = child.getStringAttribute("url");
          String mapperClass = child.getStringAttribute("class");
          if (resource != null && url == null && mapperClass == null) {
            ErrorContext.instance().resource(resource);
            InputStream inputStream = Resources.getResourceAsStream(resource); // Load the XML in the specified folder
            XMLMapperBuilder mapperParser = new XMLMapperBuilder(inputStream, configuration, resource, configuration.getSqlFragments()); 
            mapperParser.parse(); // Map the label value in mapper to MyBatis object
          } else if (resource == null && url != null && mapperClass == null) {
            ErrorContext.instance().resource(url);
            InputStream inputStream = Resources.getUrlAsStream(url);
            XMLMapperBuilder mapperParser = new XMLMapperBuilder(inputStream, configuration, url, configuration.getSqlFragments());
            mapperParser.parse(); // Let's take a look at the implementation of the parse method
          } else if (resource == null && url == null && mapperClass != null) {
            Class<?> mapperInterface = Resources.classForName(mapperClass);
            configuration.addMapper(mapperInterface);
          } else {
            throw new BuilderException("A mapper element may only specify a url, resource or class, but not more than one.");
          }
        }
      }
    }
  }

The parent parameter is the mappers label, we can verify this by debugging:

public void parse() {
  if (!configuration.isResourceLoaded(resource)) {
    configurationElement(parser.evalNode("/mapper"));
    configuration.addLoadedResource(resource);
    bindMapperForNamespace();
  }

  parsePendingResultMaps();
  parsePendingCacheRefs();
  parsePendingStatements();
}

At the time of introduction, Xi'an judged whether the xml has been loaded, and then parsed the tags such as additions, deletions, changes, and checks under the mapper tag. We can see this in configurationElement.

private void configurationElement(XNode context) {
    try {
      String namespace = context.getStringAttribute("namespace");
      if (namespace == null || namespace.isEmpty()) {
        throw new BuilderException("Mapper's namespace cannot be empty");
      }
      builderAssistant.setCurrentNamespace(namespace);
      cacheRefElement(context.evalNode("cache-ref"));
      cacheElement(context.evalNode("cache"));
      parameterMapElement(context.evalNodes("/mapper/parameterMap"));
      resultMapElements(context.evalNodes("/mapper/resultMap"));
      sqlElement(context.evalNodes("/mapper/sql"));
      buildStatementFromContext(context.evalNodes(&quot;select|insert|update|delete&quot;)); //This method parses the label
    } catch (Exception e) {
      throw new BuilderException("Error parsing Mapper XML. The XML location is '" + resource + "'. Cause: " + e, e);
    }
  }
private void buildStatementFromContext(List<XNode> list) {
  if (configuration.getDatabaseId() != null) { // dataBaseId is used to indicate under which database the tag is executed
    buildStatementFromContext(list, configuration.getDatabaseId());
  }
  buildStatementFromContext(list, null);
}

The parseStatementNode method is relatively long. In the end, it parses the attributes of select, insert, update, and delete of Mapper.xml, and passes the parsed attributes to the builderAssistant.addMappedStatement() method. This method has slightly more parameters. Let's take a screenshot:

At this point, we basically end the process of building the configuration. We can think that at this step, the Mybatis configuration file and Mapper.xml have been basically parsed.

Get SqlSession object

SqlSession is an interface with two main implementation classes:

What we built in the first step is actually the DefaultSqlSessionFactory:

public SqlSessionFactory build(Configuration config) {
  return new DefaultSqlSessionFactory(config);
}

In fact, openSession is also executed by DefaultSqlSessionFactory. Let's take a look at what is roughly done in the process of openSession:

@Override
public SqlSession openSession() {
  return openSessionFromDataSource(configuration.getDefaultExecutorType(), null, false);
}

Pay attention to this getDefaultExecutorType, this fact is the SQL executor of the core layer in the MyBatis layer, let's look down at openSessionFromDataSource:

// level isolation level, whether autoCommit automatically commits
// ExecutorType is an enumeration value: SIMPLE, REUSE, BATCH
private SqlSession openSessionFromDataSource(ExecutorType execType, TransactionIsolationLevel level, boolean autoCommit) {
    Transaction tx = null;
    try {
      final Environment environment = configuration.getEnvironment();
      final TransactionFactory transactionFactory = getTransactionFactoryFromEnvironment(environment);
      tx = transactionFactory.newTransaction(environment.getDataSource(), level, autoCommit);
      // Return an executor, let's look at the newExecutor method
      final Executor executor = configuration.newExecutor(tx, execType);
      // Finally construct the SqlSession  
      return new DefaultSqlSession(configuration, executor, autoCommit);
    } catch (Exception e) {
      closeTransaction(tx); // may have fetched a connection so lets call close()
      throw ExceptionFactory.wrapException("Error opening session.  Cause: " + e, e);
    } finally {
      ErrorContext.instance().reset();
    }
  }
public Executor newExecutor(Transaction transaction, ExecutorType executorType) {
  executorType = executorType == null ? defaultExecutorType : executorType;
  executorType = executorType == null ? ExecutorType.SIMPLE : executorType;
  Executor executor;
  if (ExecutorType.BATCH == executorType) {
    executor = new BatchExecutor(this, transaction);
  } else if (ExecutorType.REUSE == executorType) {
    executor = new ReuseExecutor(this, transaction);
  } else {
    executor = new SimpleExecutor(this, transaction);
  }
  // The above is to generate the corresponding executor according to the executorType
  // If caching is enabled, wrap its executor as another form of executor
  if (cacheEnabled) {
    executor = new CachingExecutor(executor);
  }
  // interceptorChain is an interceptor chain
  // Add the executor to the interceptor chain to enhance, which is actually the plug-in development of MyBatis.
  // It is also an application of the decorator pattern, which will be discussed later.
  executor = (Executor) interceptorChain.pluginAll(executor);
  return executor;
}

Perform CRUD

Then let's see how the methods in our interface are executed,

In fact, when StudentMapper executes the selectStudent method, it should enter the object corresponding to the proxy. We enter the next step. In fact, we enter the invoke method. This invoke method actually rewrites the InvocationHandler method. InvocationHandler is a dynamic proxy interface provided by JDK. The delegated method actually goes to the invoke method, which is implemented as follows:

public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
    try {
      if (Object.class.equals(method.getDeclaringClass())) {
        return method.invoke(this, args);
      } else {
         // This method will cache the method. If it should be in the cache, there is no need to generate it again. The methodCache inside is ConcurrentHashMap
         // Finally, the MapperMethod object will be returned to call the invoke method.
        // My final MethodInvoker here is PlainMethodInvoker
        return cachedInvoker(method).invoke(proxy, method, args, sqlSession); 
      }
    } catch (Throwable t) {
      throw ExceptionUtil.unwrapThrowable(t);
    }
  }

The final invoke method is shown below:

@Override
public Object invoke(Object proxy, Method method, Object[] args, SqlSession sqlSession) throws Throwable {
  //This execute is too long, and the next operation is performed according to the label type. Here we put a screenshot 
  return mapperMethod.execute(sqlSession, args);
}

Let's follow the execution of the executeForMany method:

private <E> Object executeForMany(SqlSession sqlSession, Object[] args) {
  List<E> result;
  Object param = method.convertArgsToSqlCommandParam(args);
   // default pagination
    if (method.hasRowBounds()) {
    RowBounds rowBounds = method.extractRowBounds(args);
    result = sqlSession.selectList(command.getName(), param, rowBounds);
  } else {
    // Will go under the selectList of DefaultSqlSession
    result = sqlSession.selectList(command.getName(), param);
  }
  // issue #510 Collections & arrays support
  // convert result  
  if (!method.getReturnType().isAssignableFrom(result.getClass())) {
    if (method.getReturnType().isArray()) {
      return convertToArray(result);
    } else {
      return convertToDeclaredCollection(sqlSession.getConfiguration(), result);
    }
  }
  return result;
}
  public <E> List<E> selectList(String statement, Object parameter) {
    return this.selectList(statement, parameter, RowBounds.DEFAULT);
  }
// 
@Override
  public <E> List<E> selectList(String statement, Object parameter, RowBounds rowBounds) {
    try {
      // This statement is a method reference: org.example.mybatis.StudentMapper.selectStudent
      // Through this key, the constructed MappedStatement can be obtained from the configuration
      MappedStatement ms = configuration.getMappedStatement(statement);
      // The query will determine whether the result is in the cache, we have not introduced the cache
      // The queryFromDatabase method in the query that will eventually go.
      // queryFromDatabase will call the doQuery method
      return executor.query(ms, wrapCollection(parameter), rowBounds, Executor.NO_RESULT_HANDLER);
    } catch (Exception e) {
      throw ExceptionFactory.wrapException("Error querying database.  Cause: " + e, e);
    } finally {
      ErrorContext.instance().reset();
    }
  }

Here we focus on the doQuery method:

@Override
  public <E> List<E> doQuery(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler, BoundSql boundSql) throws SQLException {
    // Here we can actually see that MyBatis is ready to call JDBC
    // Statement is in JDBC
    Statement stmt = null;
    try {
      Configuration configuration = ms.getConfiguration();
      // Process the SQL in the tag according to the parameter
      StatementHandler handler = configuration.newStatementHandler(wrapper, ms, parameter, rowBounds, resultHandler, boundSql);
      // Generate Statement that executes SQL
      stmt = prepareStatement(handler, ms.getStatementLog());
      // Then call the query method. It will eventually go to the query method of PreparedStatementHandler  
      return handler.query(stmt, resultHandler);
    } finally {
      closeStatement(stmt);
    }
  }
// Finally execute the SQL
 public <E> List<E> query(Statement statement, ResultHandler resultHandler) throws SQLException {
    PreparedStatement ps = (PreparedStatement) statement;
    ps.execute();
    return resultSetHandler.handleResultSets(ps);
  }

PreparedStatement is JDBC, and it has begun to call JDBC to execute SQL. resultSetHandler is the handler for processing JDBC results.

Here we roughly sort out the Handlers encountered above:

  • StatementHandler: statement handler
  • ResultSetHandler: result handler, if there is a result handler, there will be a parameter handler
  • ParameterHandler: parameter handler,

in conclusion

In MyBatis, the question of how the SQL statement we wrote in the xml file is executed has now been answered:

  • The query statements and attributes in xml will be pre-loaded into the Configuration, and there are MappedStatements in the Configuration, which is a Map, and the key is the id of the tag.
  • When we execute the corresponding Mapper, we must first execute the acquisition of the Session. In this process, we will pass through the interceptor of MyBatis. We can choose to enhance MyBatis in this process.
  • When the method corresponding to the interface is called, the method of the proxy class is actually called. The proxy class will first process the parameters, obtain the MappedStatement according to the method signature, and then convert it to JDBC for processing.

Now we have a general understanding of the execution process that MyBatis already has. Maybe some methods are not too detailed, because talking about those details is not very helpful to the macro execution process.

References

  • MyBatis Video Tutorial (Advanced) Video Yanqun https://www.bilibili.com/vide…
  • Play MyBatis: In-depth Analysis and Customizationhttps://juejin.cn/book/694491…