[Kails] An open source project of nodejs similar to Rails based on Koa2

Time:2019-6-5

This article was first published in Blog of Embbnux. For reprinting, please indicate the source of the original text and keep the link of the original text.
https://www.embbnux.com/2016/09/04/kails_with_koa2_like_ruby_on_rails

Recently, we have studied the next Koa2 framework and loved the idea of middleware. But found that it is too concise, only basic functions, although it can easily take a variety of services, but can adapt to the rapid development of the website framework is still a little far away. Referring to the general framework of Rails, a website framework kails is built. With Postgres and redis, MVC framework, front-end webpack and react front-end isomorphic website development framework are implemented. This paper mainly introduces various technology stacks and ideas in the construction of kails.

Koa comes from the creative team of express. it mainly uses the generators feature of ES6 to implement a new framework based on Middleware idea. but unlike express, koa does not want to provide a framework to meet the basic website development, but more like a basic functional module. to meet the needs of the website, it still needs to introduce many functional modules. So depending on the size of the selection, there are various KOA projects, and kails can be seen by name as a koa project similar to Ruby on Rails.

Project address: https://github.com/embbnux/kails

The main catalogue structure is as follows

├── app.js
├── assets
│   ├── images
│   ├── javascripts
│   └── stylesheets
├── config
│   ├── config.js
│   ├── development.js
│   ├── test.js
│   ├── production.js
│   └── webpack.config.js
│   ├── webpack
├── routes
├── models
├── controllers
├── views
├── db
│   └── migrations
├── helpers
├── index.js
├── package.json
├── public
└── test

First step ES6 support

Kails chooses koa2 as the core framework. koa2 uses ES7 async and await functions. node can not run after harmony is opened, so Babel and other language transformation tools should be used to support it.

Babel 6 configuration file

.babelrc

{
  "presets": [
    "es2015",
    "stage-0",
    "react"
  ]
}

Use Babel to load the entire function at the entrance to support ES6

require('babel-core/register')
require('babel-polyfill')
require('./app.js')

II. Core file app.js

App.js is the core file, the introduction and use of koa2 middleware is mainly here, where various middleware and configurations will be introduced, specific functions will be introduced later slowly.

Below is part of the content, see the details of the GitHub warehouse

import Koa from 'koa'
import session from 'koa-generic-session'
import csrf from 'koa-csrf'
import views from 'koa-views'
import convert from 'koa-convert'
import json from 'koa-json'
import bodyParser from 'koa-bodyparser'

import config from './config/config'
import router from './routes/index'
import koaRedis from 'koa-redis'
import models from './models/index'

const redisStore = koaRedis({
  url: config.redisUrl
})

const app = new Koa()

app.keys = [config.secretKeyBase]

app.use(convert(session({
  store: redisStore,
  prefix: 'kails:sess:',
  key: 'kails.sid'
})))

app.use(bodyParser())
app.use(convert(json()))
app.use(convert(logger()))

// not serve static when deploy
if(config.serveStatic){
  app.use(convert(require('koa-static')(__dirname + '/public')))
}

//views with pug
app.use(views('./views', { extension: 'pug' }))

// csrf
app.use(convert(csrf()))

app.use(router.routes(), router.allowedMethods())

app.listen(config.port)
export default app

3. MVC Framework Construction

Website architecture is also multi-layered and practical with mvc, which can satisfy many scenarios of website development, logic and complexity can be added to a service layer, where routing is distributed based on koa-router, thus implementing MVC layering.

Routes/index.js files are used to automatically load other files in their directories. Each file is responsible for routing distribution under the corresponding routing header, as follows

routes/index.js

import fs from 'fs'
import path from 'path'
import Router from 'koa-router'

const basename = path.basename(module.filename)
const router = Router()

fs
  .readdirSync(__dirname)
  .filter(function(file) {
    return (file.indexOf('.') !== 0) && (file !== basename) && (file.slice(-3) === '.js')
  })
  .forEach(function(file) {
    let route = require(path.join(__dirname, file))
    router.use(route.routes(), route.allowedMethods())
  })

export default router

The routing file is mainly responsible for distributing the corresponding requests to the corresponding controllers, and the routing is mainly based on restful cells.

routes/articles.js

import Router from 'koa-router'
import articles from '../controllers/articles'

const router = Router({
  prefix: '/articles'
})
router.get('/new', articles.checkLogin, articles.newArticle)
router.get('/:id', articles.show)
router.put('/:id', articles.checkLogin, articles.checkArticleOwner, articles.checkParamsBody, articles.update)
router.get('/:id/edit', articles.checkLogin, articles.checkArticleOwner, articles.edit)
router.post('/', articles.checkLogin, articles.checkParamsBody, articles.create)

// for require auto in index.js
module.exports = router

In the model layer, the base database Postgres are docked with ORM based on Sequelize, and the migration function of database is realized by sequelize-cli.

user.js

import bcrypt from 'bcrypt'

export default function(sequelize, DataTypes) {
  const User = sequelize.define('User', {
    id: {
      type: DataTypes.INTEGER,
      primaryKey: true,
      autoIncrement: true
    },
    name: {
      type: DataTypes.STRING,
      validate: {
        notEmpty: true,
        len: [1, 50]
      }
    },
    email: {
      type: DataTypes.STRING,
      validate: {
        notEmpty: true,
        isEmail: true
      }
    },
    passwordDigest: {
      type: DataTypes.STRING,
      field: 'password_digest',
      validate: {
        notEmpty: true,
        len: [8, 128]
      }
    },
    password: {
      type: DataTypes.VIRTUAL,
      allowNull: false,
      validate: {
        notEmpty: true
      }
    },
    passwordConfirmation: {
      type: DataTypes.VIRTUAL
    }
  },{
    underscored: true,
    tableName: 'users',
    indexes: [{ unique: true, fields: ['email'] }],
    classMethods: {
      associate: function(models) {
        User.hasMany(models.Article, { foreignKey: 'user_id' })
      }
    },
    instanceMethods: {
      authenticate: function(value) {
        if (bcrypt.compareSync(value, this.passwordDigest)){
          return this
        }
        else{
          return false
        }
      }
    }
  })
  function hasSecurePassword(user, options, callback) {
    if (user.password != user.passwordConfirmation) {
      throw new Error('Password confirmation doesn\'t match Password')
    }
    bcrypt.hash(user.get('password'), 10, function(err, hash) {
      if (err) return callback(err)
      user.set('passwordDigest', hash)
      return callback(null, options)
    })
  }
  User.beforeCreate(function(user, options, callback) {
    user.email = user.email.toLowerCase()
    if (user.password){
      hasSecurePassword(user, options, callback)
    }
    else{
      return callback(null, options)
    }
  })
  User.beforeUpdate(function(user, options, callback) {
    user.email = user.email.toLowerCase()
    if (user.password){
      hasSecurePassword(user, options, callback)
    }
    else{
      return callback(null, options)
    }
  })
  return User
}

IV. Development, Testing and Online Environment

Website development, testing and deployment will have different environments, so different configurations are needed. Here I mainly divide the development, test and production environments. When using these environments, different configurations are loaded automatically based on NODE_ENV variables.

Implementation code:

config/config.js

var _ = require('lodash');
var development = require('./development');
var test = require('./test');
var production = require('./production');

var env = process.env.NODE_ENV || 'development';
var configs = {
  development: development,
  test: test,
  production: production
};
var defaultConfig = {
  env: env
};

var config = _.merge(defaultConfig, configs[env]);

module.exports = config;

Configuration of production environment:

config/production.js

const port = Number.parseInt(process.env.PORT, 10) || 5000
module.exports = {
  port: port,
  hostName: process.env.HOST_NAME_PRO,
  serveStatic: process.env.SERVE_STATIC_PRO || false,
  assetHost: process.env.ASSET_HOST_PRO,
  redisUrl: process.env.REDIS_URL_PRO,
  secretKeyBase: process.env.SECRET_KEY_BASE
};

5. Optimizing Code Using Middleware

Koa is built on the idea of middleware, which can not be separated from natural code. Here we introduce several middleware applications.

CurrtUser injection

CurrtUser is used to acquire the current logged-in user, which is very important in the user system of the website.

app.use(async (ctx, next) => {
  let currentUser = null
  if(ctx.session.userId){
    currentUser = await models.User.findById(ctx.session.userId)
  }
  ctx.state = {
    currentUser: currentUser,
    isUserSignIn: (currentUser != null)
  }
  await next()
})

In the future middleware, the current user can be obtained through ctx. state. currentUser.

Optimize the controller code

For example, edit and update in the controller of an article need to find the current article object, and also need to validate permissions, which is the same. In order to avoid code duplication, middleware can also be used here.

controllers/articles.js

async function edit(ctx, next) {
  const locals = {
    Title:'Editor',
    nav: 'article'
  }
  await ctx.render('articles/edit', locals)
}

async function update(ctx, next) {
  let article = ctx.state.article
  article = await article.update(ctx.state.articleParams)
  ctx.redirect('/articles/' + article.id)
  return
}

async function checkLogin(ctx, next) {
  if(!ctx.state.isUserSignIn){
    ctx.status = 302
    ctx.redirect('/')
    return
  }
  await next()
}

async function checkArticleOwner(ctx, next) {
  const currentUser = ctx.state.currentUser
  const article = await models.Article.findOne({
    where: {
      id: ctx.params.id,
      userId: currentUser.id
    }
  })
  if(article == null){
    ctx.redirect('/')
    return
  }
  ctx.state.article = article
  await next()
}

Application of Middleware in Routing

router.put('/:id', articles.checkLogin, articles.checkArticleOwner, articles.update)
router.get('/:id/edit', articles.checkLogin, articles.checkArticleOwner, articles.edit)

This is equivalent to implementing rails’before_action function

6. Webpack Configuration of Static Resources

Before the separation of front-end and back-end is realized, there must be front-end code in engineering code. Now webpack is a well-known tool for front-end modular programming. Here we use it to do the function of assets pipeline in rails. Here we introduce the basic configuration.

config/webpack/base.js

var webpack = require('webpack');
var path = require('path');
var publicPath = path.resolve(__dirname, '../', '../', 'public', 'assets');
var ManifestPlugin = require('webpack-manifest-plugin');
var assetHost = require('../config').assetHost;
var ExtractTextPlugin = require('extract-text-webpack-plugin');

module.exports = {
  context: path.resolve(__dirname, '../', '../'),
  entry: {
    application: './assets/javascripts/application.js',
    articles: './assets/javascripts/articles.js',
    editor: './assets/javascripts/editor.js'
  },
  module: {
    loaders: [{
      test: /\.jsx?$/,
      exclude: /node_modules/,
      loader: ['babel-loader'],
      query: {
        presets: ['react', 'es2015']
      }
    },{
      test: /\.coffee$/,
      exclude: /node_modules/,
      loader: 'coffee-loader'
    },
    {
      test: /\.(woff|woff2|eot|ttf|otf)\??.*$/,
      loader: 'url-loader?limit=8192&name=[name].[ext]'
    },
    {
      test: /\.(jpe?g|png|gif|svg)\??.*$/,
      loader: 'url-loader?limit=8192&name=[name].[ext]'
    },
    {
      test: /\.css$/,
      loader: ExtractTextPlugin.extract("style-loader", "css-loader")
    },
    {
      test: /\.scss$/,
      loader: ExtractTextPlugin.extract('style', 'css!sass')
    }]
  },
  resolve: {
    extensions: ['', '.js', '.jsx', '.coffee', '.json']
  },
  output: {
    path: publicPath,
    publicPath: assetHost + '/assets/',
    filename: '[name]_bundle.js'
  },
  plugins: [
    new webpack.ProvidePlugin({
      $: 'jquery',
      jQuery: 'jquery'
    }),
    // new webpack.HotModuleReplacementPlugin(),
    new ManifestPlugin({
      fileName: 'kails_manifest.json'
    })
  ]
};

7. Front-end and back-end isomorphism of react

The advantage of node is that the V8 engine can run as long as it is js, so the rendering DOM function of react can also be rendered in the back end. It can realize the isomorphism of front and back end of react, which is beneficial to SEO and is more friendly to the content of user’s first screen.

I won’t talk about reacting at the front end. Here’s how to achieve it in Koa.

import React from 'react'
import { renderToString } from 'react-dom/server'
async function index(ctx, next) {
  const prerenderHtml = await renderToString(
    <Articles articles={ articles } />
  )
}

VIII. Testing and Lint

Testing and lint are naturally an indispensable part of Engineering in the development process, where kails uses mocha for testing, and lint uses eslint for testing.

.eslintrc

{
  "parser": "babel-eslint",
  "root": true,
  "rules": {
    "new-cap": 0,
    "strict": 0,
    "no-underscore-dangle": 0,
    "no-use-before-define": 1,
    "eol-last": 1,
    "indent": [2, 2, { "SwitchCase": 0 }],
    "quotes": [2, "single"],
    "linebreak-style": [2, "unix"],
    "semi": [1, "never"],
    "no-console": 1,
    "no-unused-vars": [1, {
      "argsIgnorePattern": "_",
      "varsIgnorePattern": "^debug$|^assert$|^withTransaction$"
    }]
  },
  "env": {
    "browser": true,
    "es6": true,
    "node": true,
    "mocha": true
  },
  "extends": "eslint:recommended"
}

9. console

If you have used rails, you should know that rails has a rails console, which can enter the environment of the website in the form of command line. It’s very convenient. Here, it’s based on repl.

if (process.argv[2] && process.argv[2][0] == 'c') {
  const repl = require('repl')
  global.models = models
  repl.start({
    prompt: '> ',
    useGlobal: true
  }).on('exit', () => { process.exit() })
}
else {
  app.listen(config.port)
}

10. PM2 deployment

After development, it is natural to deploy on-line, where PM2 is used to manage:

NODE_ENV=production ./node_modules/.bin/pm2 start index.js -i 2 --name "kails" --max-memory-restart 300M --merge-logs --log-date-format="YYYY-MM-DD HH:mm Z" --output="log/production.log"

NPM scripts

Some common commands have more parameters and are longer. You can use NPM scripts to make aliases for these commands.

{
  "scripts": {
    "console": "node index.js console",
    "start": "./node_modules/.bin/nodemon index.js & node_modules/.bin/webpack --config config/webpack.config.js --progress --colors --watch",
    "app": "node index.js",
    "pm2": "NODE_ENV=production ./node_modules/.bin/pm2 start index.js -i 2 --name \"kails\" --max-memory-restart 300M --merge-logs --log-date-format=\"YYYY-MM-DD HH:mm Z\" --output=\"log/production.log\"",
    "pm2:restart": "NODE_ENV=production ./node_modules/.bin/pm2 restart \"kails\"",
    "pm2:stop": "NODE_ENV=production ./node_modules/.bin/pm2 stop \"kails\"",
    "pm2:monit": "NODE_ENV=production ./node_modules/.bin/pm2 monit \"kails\"",
    "pm2:logs": "NODE_ENV=production ./node_modules/.bin/pm2 logs \"kails\"",
    "test": "NODE_ENV=test ./node_modules/.bin/mocha --compilers js:babel-core/register --recursive --harmony --require babel-polyfill",
    "assets_build": "node_modules/.bin/webpack --config config/webpack.config.js",
    "assets_compile": "NODE_ENV=production node_modules/.bin/webpack --config config/webpack.config.js -p",
    "webpack_dev": "node_modules/.bin/webpack --config config/webpack.config.js --progress --colors --watch",
    "lint": "eslint . --ext .js",
    "db:migrate": "node_modules/.bin/sequelize db:migrate",
    "db:rollback": "node_modules/.bin/sequelize db:migrate:undo",
    "create:migration": "node_modules/.bin/sequelize migration:create"
  }
}

In this way, there will be more commands:

npm install
npm run db:migrate
NODE_ENV=test npm run db:migrate
# run for development, it start app and webpack dev server
npm run start
# run the app
npm run app
# run the lint
npm run lint
# run test
npm run test
# deploy
npm run assets_compile
NODE_ENV=production npm run db:migrate
npm run pm2

XII. Further

At present, kails implements basic blog functions, including basic privilege verification and markdown editing. Now we can think of a further step:

  • Performance optimization to speed up response

  • Dockerfile simplifies deployment

  • On-line code precompilation

Welcome pull request: https://github.com/embbnux/kails

Recommended Today

Custom factory class

using System; using System.Collections; using System.Collections.Generic; using System.Linq; using System.Reflection; using System.Text; namespace LYZB.ApiConfig { /// ///Custom factory class /// // generic public abstract class AssesUtil where T : class,new() { /// ///Cache collection /// private static Hashtable hash = Hashtable.Synchronized(new Hashtable()); /// ///Execution instance method /// ///Method name ///Parameter type ///Parameter entity /// public […]