Open source low code platform development practice 2: build a low code backend based on ER diagram from 0

Time:2021-10-21

The front and rear ends are separated!

When I first knew this, I was confused.

Front end all go out to do spa, SEOs agree?

Later, SSR came.

He said, “the SEOs agreed!”

Anyone’s opposition is useless. Times have changed.

All kinds of spas have come, as well as all kinds of small programs wearing the same clothes as spas.

Do something for them? So rxmodels was born. As a back-end that does not want to be abandoned, it hopes to serve the front-end in a more convenient way.

By the way, share how to design and make, which may have some reference significance. Even if there is something unreasonable, someone will kindly point it out.

Keeping open, giving and receiving will happen at the same time, which is a process of two-way benefits.

What is rxmodels?

An open source, universal, low code backend.

With rxmodels, you can customize a back-end out of the box by just drawing an ER diagram. It provides permission management with granularity accurate to fields, and provides expression support for instance level permission management.

The main modules include: graphical entity and relationship management interface (Rx models client), general JSON format data operation interface service (Rx models), front-end call auxiliary hooks Library (rxmodels SWR), etc.

Rxmodels is implemented based on typescript, nestjs, typeorm and antv X6.

Typescript’s strong typing support can solve some errors at compile time. With strong typing support, IDE can automatically introduce dependencies, improve development efficiency and save time.

The target execution code after typescript compilation is JS, a runtime interpretation language. This feature gives rxmodels dynamic publishing entities and hot loadinginstructionsAbility. Users can useinstructionsImplement business logic and extend general JSON data interface. More usage scenarios have been added to rxmodels.

Nest JS helps to organize the code and make it have a good architecture.

Typeorm is a lightweight ORM library that maps object models to relational databases. It can “separate entity definitions”, pass in JSON description to build the database, and provide object-oriented query support for the database. Thanks to this feature, the graphical business model can be transformed into a database model, and rxmodels can be completed with only a small amount of code.

Antv X6 has relatively comprehensive functions. It supports embedding react components in nodes. Using this personality to draw ER diagrams, the effect is very good. If you have time later, you can write another article on how to use antv X6 to draw ER diagram.

If you want to follow this article and work out this project step by step, you’d better learn the technology stack mentioned in this section in advance.

Rxmodels targeting

Mainly for small and medium-sized projects.

Why dare not serve big projects?

I dare not. The author is an amateur programmer and has no experience in large projects.

Combing data and data mapping

Let’s take a look at the demonstration first, and intuitively know what the project looks like:Rxmodels demo

Meta Data Define

Metadata is used to describe the data of business entity model. A part of metadata is transformed into typeorm entity definition, and then the database is generated; Another part of the metadata business model is graphical information, such as the size and location of entities, the location and shape of relationships, etc.

Metadata to be converted into typeorm entity definitions include:

import { ColumnMeta } from "./column-meta";

/**
*Entity type enumeration. At present, only ordinary entities and enumerated entities are supported,
*Enumeration entities are similar to syntax sugar and do not map to the database,
*The fields of enumeration type are mapped to the database as string type
*/
export enum EntityType{
  NORMAL = "Normal",
  ENUM = "Enum",
}

/**
*Entity metadata
*/
export interface EntityMeta{
  /**Unique identification*/
  uuid: string;

  /**Entity name*/
  name: string;

  /**Table name. If tablename is not set, the entity name will be converted into snake naming method and used as the table name*/
  tableName?: string;

  /**Entity type*/
  entityType?: EntityType|"";

  /**Field metadata list*/
  columns: ColumnMeta[];

  /**Enumeration value JSON, used by enumeration type entities, does not participate in database mapping*/
  enumValues?: any;
}
/**
*Field types, enumerations. The current version only supports these types, which can be extended later
*/
export enum ColumnType{

  /**Number type*/
  Number = 'Number',

  /**Boolean type*/
  Boolean = 'Boolean',

  /**String type*/  
  String = 'String',

  /**Date type*/  
  Date = 'Date',

  /**JSON type*/
  SimpleJson = 'simple-json',

  /**Array type*/
  SimpleArray = 'simple-array',

  /**Enumeration type*/
  Enum = 'Enum'
}

/**
*Field metadata, basically corresponding to typeorm column
*/
export interface ColumnMeta{

  /**Unique identification*/
  uuid: string;

  /**Field name*/
  name: string;

  /**Field type*/
  type: ColumnType;

  /**Primary key*/
  primary?: boolean;

  /**Auto generate*/
  generated?: boolean;

  /**Can it be blank*/
  nullable?: boolean;

  /**Field defaults*/
  default?: any;

  /**Is it unique*/
  unique?: boolean;

  /**Is this the creation date*/
  createDate?: boolean;

  /**Is this the update date*/
  updateDate?: boolean;

  /**Whether it is the deletion date, the soft deletion function is used*/
  deleteDate?: boolean;

  /**
   *Whether it can be selected during query. If this is false, it will be hidden during query.
   *The password field uses it
   */
  select?: boolean;

  /**Length*/
  length?: string | number;

  /**Used when the entity is an enumerated type*/
  enumEnityUuid?:string;

  /**
   *================== the following properties correspond to typeorm, but are not enabled
   */
  width?: number;
  version?: boolean;
  readonly?: boolean;  
  comment?: string;
  precision?: number;
  scale?: number;
}
/**
 *Relationship type
 */
export enum RelationType {
  ONE_TO_ONE = 'one-to-one',
  ONE_TO_MANY = 'one-to-many',
  MANY_TO_ONE = 'many-to-one',
  MANY_TO_MANY = 'many-to-many',
}

/**
 *Relational metadata
 */
export interface RelationMeta {
  /**Unique identification*/
  uuid: string;

  /**Relationship type */  
  relationType: RelationType;

  /**Source entity ID of the relationship*/  
  sourceId: string;

  /**Relationship target entity ID*/  
  targetId: string;

  /**Relationship properties on the source entity*/  
  roleOnSource: string;

  /**Relationship properties on target entity*/    
  roleOnTarget: string;

  /**Entity ID with relationship, corresponding to jointable or joincolumn of typeorm*/
  ownerId?: string;
}

Metadata that do not need to be converted to typeorm entity definitions include:

/**
 *Package metadata
 */
export interface PackageMeta{
  /**ID, primary key*/
  id?: number;

  /**Unique identification*/
  uuid: string;

  /**Package name*/
  name: string;

  /**Entity list*/
  entities?: EntityMeta[];

  /**ER diagram list*/
  diagrams?: DiagramMeta[];

  /**Relationship list*/
  relations?: RelationMeta[];
}
import { X6EdgeMeta } from "./x6-edge-meta";
import { X6NodeMeta } from "./x6-node-meta";

/**
 *Er entity data
 */
export interface DiagramMeta {
  /**Unique identification*/
  uuid: string;

  /**ER diagram name*/
  name: string;

  /**Node*/
  nodes: X6NodeMeta[];

  /**Connection of relationship*/
  edges: X6EdgeMeta[];
}
export interface X6NodeMeta{
  /**Corresponding entity ID UUID*/
  id: string;
  /**Node X coordinate*/
  x?: number;
  /**Node y coordinate*/
  y?: number;
  /**Node width*/
  width?: number;
  /**Node height*/
  height?: number;
}
import { Point } from "@antv/x6";

export type RolePosition = {
  distance: number,
  offset: number,
  angle: number,
}
export interface X6EdgeMeta{
  /**Correspondence UUID*/
  id: string;

  /**Break point data*/
  vertices?: Point.PointLike[];

  /**Source Relationship Attribute location label location*/
  roleOnSourcePosition?: RolePosition;

  /**Target Relationship Attribute location label location*/
  roleOnTargetPosition?: RolePosition;
}

Rxmodels has a back-end service that builds databases based on this data.

Rxmodels has a front-end management interface to manage and produce this data.

Server RX models

The core of the whole project is built based on nestjs. Typeorm needs to be installed. Only ordinary typeorm core projects need to be installed, and nestjs encapsulated version does not need to be installed.

nest new rx-models

cd rx-models

npm install npm install typeorm

This is only the key installation. Other libraries are not listed one by one.

The specific project has been completed, code address:https://github.com/rxdrag/rx-models

The first version undertakes the task of technology exploration, and only MySQL is enough.

Universal JSON interface

Design a set of interfaces and specify the interface semantics, just like graphql. The advantage of this is that there is no need for interface documents and no need to define interface versions.

The interface takes JSON as the parameter and returns JSON data, which can be called JSON interface.

Query interface

Interface Description:

url: /get/jsonstring...
method: get
Return value:{
  data:any,
  pagination?:{
    pageSize: number,
    pageIndex: number,
    totalCount: number
  }
}

The URL length is 2048 bytes, which is enough to pass a query string. In the query interface, you can put JSON query parameters in the URL and use the get method to query data.

One obvious advantage of putting JSON query parameters in the URL is that the client can cache query results based on the URL, such as usingSWR Library

A special point to pay attention to is URL transcoding. Otherwise, like will be used when querying%This will cause back-end errors. Therefore, it is necessary to write a set of query SDK for the client and encapsulate these transcoding operations.

Query interface example

By passing in the entity name, you can query the instance of the entity. For example, to query all posts, you can write:

{
  "entity": "Post"
}

To queryid = 1The article reads as follows:

{
  "entity": "Post",
  "id": 1
}

Sort the articles by title and date, and write this:

{
  "entity": "Post",
  "@orderBy": {
    "title": "ASC",
    "updatedAt": "DESC"
  }
}

You only need to query the title field of the article, which reads as follows:

{
  "entity": "Post",
  "@select": ["title"]
}

You can also write this:

{
  "entity @select(title)": "Post"
}

Take only one record:

{
  "entity": "Post",
  "@getOne": true
}

Or:

{
  "entity @getOne": "Post"
}

Only check the articles with the word “water” in the title:

{
  "entity": "Post",
  "Title @ like": "% water%"
}

You also need more complex queries and embedded SQL like expressions:

{
  "entity": "Post",
  "@ where": "name% like '% wind%' and..."
}

There are too many data. Page by page, 25 entries per page, record the first page:

{
  "entity": "Post",
  "@paginate": [25, 0]
}

Or:

{
  "entity @paginate(25, 0)": "Post"
}

Relation query, picture relation medias with article:

{
  "entity": "Post",
  "medias": {}
}

Relationship nesting:

{
  "entity": "Post",
  "medias": {
    "owner":{}
  }
}

Add a condition to the relationship:

{
  "entity": "Post",
  "medias": {
    "Name @ like": "% scenery%"
  }
}

Only take the first 5 of the relationship

{
  "entity": "Post",
  "medias @count(5)": {}
}

Smart, you can make further design changes to the interface in this direction.

@After the symbol, it is calledinstructions

By putting business logic in instructions, the interface can be extended very flexibly. For example, add a copyright notice at the bottom of the article content to define a copyright@addCopyRightInstruction:

{
  "entity": "Post",
  "@addCopyRight": "content"
}

Or:

{
  "entity @addCopyRight(content)": "Post"
}

Does the instruction look like a plug-in?

Since it is a plug-in, give it the ability of hot loading!

By uploading the third-party instruction code through the management interface, the instruction can be inserted into the system.

The first version does not support the command upload function, but this capability has been reserved in the architecture design, but the supporting interface has not been done.

Post interface

Interface Description:

url: /post
method: post
Parameter: JSON
Return value: the object whose operation succeeded

Pass in JSON data through the post method.

The post interface is expected to have the ability to pass in a group of object combinations (or object trees with relational constraints) and directly synchronize this group of objects to the database.

If the ID field is provided to the object, the existing object is updated. If the ID field is not provided, a new object is created.

Post interface example

Upload an article with Picture Association, which can be written as follows:

{
  "Post": {
    "Title": "gently, I'm leaving",
    "content": "...",
    //Author Association ID
    "author": 1,
    //Picture Association ID
    "medias":[3, 5, 6 ...]
  }
}

You can also import more than one article at a time

{
  "Post": [
    {
      "id": 1,
      "Title": "gently, I'm leaving",
      "Content": "the content has changed...",
      "author": 1,
      "medias":[3, 5, 6 ...]
    },
    {
      "Title": "as if I came gently",
      "content": "...",
      "author": 1,
      "medias": [6, 7, 8 ...]
    }
  ]
}

The first article has an ID field to update the database. The second article has no ID field to create a new one.

You can also pass in instances of multiple entities. Similarly, you can also pass in instances of post and media at the same time:

{
  "Post": [
    {
      ...
    },
    {
      ...
    }
  ],
  "Media": [
    {
      ...
    }
  ]
}

The association can be passed in together. If an article is associated with a seometa object, the seometa will be created together when the article is created:

{
  "Post": {
    "Title": "gently, I'm leaving",
    "content": "...",
    "author": 1,
    "medias":[3, 5, 6 ...],
    "seoMeta":{
      "Title": "Poetry Interpretation: gently, I'm gone | Poetry Interpretation network",
      "descript": "...",
      "Keywords": "poetry, interpretation, Poetry Interpretation"
    }
  }
}

Passing this parameter will create two objects at the same time and establish an association between them.

This association can be deleted under normal circumstances. It can be written as follows:

{
  "Post": {
    "Title": "gently, I'm leaving",
    "content": "...",
    "author": 1,
    "medias":[3, 5, 6 ...],
    "seoMeta":null
  }
}

Saving an article in this way will delete the association with seometa, but the object of seometa has not been deleted. Other articles don’t need this seometa. If you don’t actively delete it, a piece of garbage data will be generated in the database.

When saving an article, add one@cascadeInstruction can solve this problem:

{
  "Post @cascade(medias)": {
    "Title": "gently, I'm leaving",
    "content": "...",
    "author": 1,
    "medias":[3, 5, 6 ...],
    "seoMeta":null
  }
}

@cascadeThe directive cascades and deletes the seometa object associated with it.

Can this instruction be written on an associated attribute like this?

{
  "Post": {
    "Title": "gently, I'm leaving",
    "content": "...",
    "author": 1,
    "medias @cascade":[3, 5, 6 ...],
    "seoMeta":null
  }
}

It’s best not to write like this. The client won’t be very convenient to use.

User defined instructions can extend the post interface. For example, to add a mail sending service, you can develop one@sendEmailInstruction:

{
  "Post @sendEmail(title, content, [email protected])": {
    "Title": "gently, I'm leaving",
    "content": "...",
    "author": 1,
    "medias @cascade":[3, 5, 6 ...],
  }
}

Suppose that every time the article is saved successfully, the sendemail instruction will send the title and content to the specified mailbox.

Update interface

Interface Description:

url: /update
method: post
Parameter: JSON
Return value: the object whose operation succeeded

postThe interface already has the update function. Why do you need to do another oneupdateInterface?

Sometimes, you need the ability to batch modify one or more fields, such as marking a specified message as read.

In order to deal with such a scenario, aupdateInterface. If you want to update the status of all articles to published:

{
  "Post": {
    "status": "published",
    "@ids":[3, 5, 6 ...],
  }
}

Based on security considerations, the interface does not provide conditional instructions, only@idsInstructions (Legacy reasons, demo version does not need @ symbol, write directlyidsOn the line, which will be modified later).

Delete interface

Interface Description:

url: /delete
method: post
Parameter: JSON
Return value: deleted object

The delete interface, like the update interface, does not provide conditional instructions and only accepts IDs or ID arrays.

To delete an article, just write:

{
  "Post": [3, 5, ...]
}

Like update, such deletion will not delete objects related to articles. Instructions are required for cascading deletion@cascade

Cascade delete seometa, which reads:

{
  "Post @cascade(seoMeta)": [3, 5, ...]
}

Upload interface

url: /upload
method: post
Parameter: formdata
headers: {"Content-Type": "multipart/form-data;boundary=..."}
Return value: rxmedia object is generated after successful upload

Rxmodels is best to provide online file management services, which can be combined with third-party object management services, such as Tencent cloud, Alibaba cloud and qiniu.

The first version does not integrate with third-party object management. Files exist locally, and file types only support pictures.

Use the entity rxmedia to manage these uploaded files. The client creates formdata and sets the following parameters:

{
   "entity": "RxMedia",
   "file": ...,
   "Name": "file name"
   }

After the introduction of all JSON interfaces, the next step is how to implement and use these interfaces.

Before continuing, let’s talk about why JSON is used instead of other ways.

Why not use OData

When I started this project, I didn’t know about OData.

I simply checked some information and said that only when open data (open data to other organizations) is needed, it is necessary to design restful API according to OData protocol.

If the data is not open to other organizations, the introduction of OData increases the complexity. You need to develop an OData parameter parsing engine.

OData has been out for a long time and is not very popular. It is not as well known as graphql later.

Why not use graphql?

I tried, but it didn’t work.

A person who does open source projects can only access the existing open source ecology. It is impossible for a person to do anything.

To use graphql, you can only use existing open source libraries. Most of the existing mainstream graphql open source libraries are based on code generation. As mentioned in the previous article, I don’t want to be a low code project based on code generation.

Another reason is that the target orientation is small and medium-sized projects. Graphql has two problems for these small and medium-sized projects: 1. It is a little cumbersome; 2. The learning cost of users is high.

Some small projects have only three or five pages, and a lightweight small back-end is pulled up in a short time. There is no need to use graphql.

The learning cost of graphql is not low, and some users of small and medium-sized projects are unwilling to pay these learning costs.

Combining these factors, the first version of the interface did not use graphql.

What should I do if I use graphql?

When communicating with some friends, some friends still love graphql. And after several years of development, the popularity of graphql slowly began to come up.

If you use graphql to do a similar project, what do you need to do?

You need to develop a set of graphql server. This server is similar to hasura. You can’t use code generation mechanism and dynamic operation mechanism. Hasura compiles GQL into SQL. You can choose to do this or not. As long as you can pull out the objects according to the GQL query requirements without the compilation process.

Under the framework of graphql, full consideration should be given to authority management, business logic expansion and hot loading. This requires a deeper understanding of graphql.

If you want to make a low code front-end, you also need to make a special front-end framework. Graphql front-end libraries such as Apollo are not suitable for making a low code front-end. Because the low code front-end needs dynamic type binding, this requirement is not particularly ideal to fit with these front-end libraries.

Each item requires a lot of time and energy. It is not a work that can be completed by one person. It needs a team.

Or one day, the author also wants to make such an attempt.

But it may not be successful. Graphql itself does not represent anything. If it can bring real benefits to users, it is the reason to be selected.

Login authentication interface

Use JWT authentication mechanism to implement two login related interfaces.

url: /auth/login
method: post
Parameters:{
  username: string,
  password: string
}
Return value: JWT token
url: /auth/me
method: get
Return value: current login user, rxuser type

There is nothing difficult to implement these two interfaces. Just follow the nestjs document.

Metadata Store

The metadata produced by the client in the form of ER diagram is stored in the database and an entityRxPackageThat’s enough:

export interface RxPackage {
  /*ID database primary key*/
  id: number;

  /**Unique identifier UUID, which is useful when metadata is shared between different projects*/
  uuid: string;

  /**Package name*/
  name: string;

  /**All entity metadata of the package is stored in the database in JSON*/
  entities: any;

  /**All ER diagrams of the package are stored in the database in JSON*/
  diagrams?: any;

  /**All relationships of the package are stored in the database in JSON*/
  relations?: any;
}

After data mapping, all contents of a package seen in the interface correspond torx_packageA data record of a table.

How is this data used?

We add a publishing function to the package. If the package is published, make a JSON file according to the database record and put it in the schemas directory. The file name is${uuid}.json

When the server creates a typeorm connection, it hot loads these JSON files and parses them into typeorm entity definition data.

Application installation interface

The ultimate goal of rxmodels is to release a code package that users can install through a graphical interface without touching the code.

Two Page Wizard to complete the installation. The interface is required:

url: install
method: post
Parameters:{
  /**Database type*/
  type: string;

  /**Host of database*/
  host: string;

  /**Database port*/
  port: string;

  /**Database schema name*/
  database: string;

  /**Data login user*/
  username: string;

  /**Database login password*/
  password: string;

  /**Super administrator login*/
  admin: string;

  /**Super administrator password*/
  adminPassword: string;

  /**Create demo account*/
  withDemo: boolean;
}

You also need an interface to query whether it has been installed:

url: /is-installed
method: get
Return value:{
  installed: boolean
}

As long as these interfaces are completed, the back-end functions will be realized. Come on!

architecture design

Thanks to nestjs’s elegant framework, the whole back-end service can be divided into the following modules:

  • auth, ordinary nestjs module, which implements the login verification interface. This module is very simple and will not be introduced separately later.

  • package-manage, metadata management and publishing module.

  • install, ordinary nestjs module, which realizes the installation function.

  • schema, the common nest JS module manages the system metadata and converts the metadata in the previously defined format into an entity definition acceptable to typeorm. The core code isSchemaService

  • typeorm, the encapsulation of typeorm provides a connection with metadata definition. The core code isTypeOrmService, the module does not have a controller.

  • magic, the core module of the project, the general JSON interface implementation module.

  • directive, the instruction definition module defines the basic classes used for instruction functions, hot loads instructions, and provides instruction retrieval services.

  • directives, all instructions implement classes, and the system hot loads all instructions from this directory.

  • magic-meta, the data formats used to parse JSON parameters are mainly modulesmagic, becausedirectiveModules also use these data. In order to avoid circular dependence between modules, this part of data is extracted as a single module, and the two modules depend on this module at the same time.

  • entity-interface, the system seed data type interface is mainly used for type recognition of typescript compiler. The code export function of the client directly copies the exported files. The client will copy the same code for use.

Package manage

Provide an interfacepublishPackages。 Publish the metadata passed in by parameters to the system and synchronize it to the database mode:

  • It is a package and a file placed in the root directoryschemasUnder the directory, the file name is the name of the packageuuid+. JSON suffix.

  • Inform the typeorm module to re create the database connection and synchronize the database at the same time.

Install module install

There is a seed file in the moduleinstall.seed.json, there are some preset entities in the system. The format is the metadata format defined above. These data are uniformly organized inSystemIn the bag.

When the client is not finished, it writes a TS file for debugging. After the client is finished, it directly uses the package export function to export a JSON file to replace the handwritten TS file. It is equivalent to the basic data part. It can be bootstrapped.

The core code of this module isInstallServiceIn, it is completed step by step:

  • Write the database configuration information from the client into the dbconfig.json file in the root directory.

  • holdinstall.seed.jsonThe predefined packages in the file are released. Call the above directlypublishPackagesRealize the publishing function.

Metadata management module schema

This module provides a controller namedSchemaController。 Provide a get interface/published-schema, which is used to obtain published metadata information.

These published metadata information can be used by the permission setting module of the client, because it is meaningful to set permissions for only the published module. The low code visual editing front end can also use this information for drop-down data binding.

Core classSchemaService, it also provides more functions:

  • from/schemasDirectory, load the published metadata.

  • These metadata are organized into a list + tree structure to provide query services by name and UUID.

  • Parse metadata into entity definition JSON acceptable to typeorm.

Encapsulate typeorm

Writing an ORM library by herself requires a lot of work and has to use ready-made ones. Typeorm is a good choice. First, she is like a young girl, beautiful and energetic. Second, she’s not as fat as PRISMA.

In order to cater to the existing tyeorm, some places have to make compromises. This low code back-end project is an ideal way to build an ORM library and realize the functions completely according to their own needs. In that way, it may feel like a childhood sweetheart, but it needs a team, not one person.

Since it is a person, then feel at ease to do what a person can do.

Typeorm has only one entry that can pass in entity definitions, that iscreateConnection。 Before calling this function, you need to parse the metadata and separate the entity definition. Of this moduleTypeOrmServiceThe management of these connections depends on the schema moduleSchemaService

adoptTypeOrmServiceYou can restart the current connection (close and recreate) to update the database definition. When creating a connection, use the install moduledbconfig.jsonGet database configuration file. Note that typeormormconfig.jsonThe file is not used.

Magic module

In the magic module, whether querying or updating, the operations implemented by each interface are in a complete transaction.

Should the query interface also be included in a transaction?

Yes, because sometimes the query may contain some instructions for simple operation of the database. For example, when querying an article, increase its reading times by 1.

The operations of adding, deleting, checking and modifying magic module are restricted by permission, and its core moduleMagicInstanceServicePass it to the instruction, and the instruction code can safely use its interface to operate the database without paying attention to the permission problem.

MagicInstanceService

MagicInstanceServiceIt’s an interfaceMagicServiceImplementation of. Interface definition:

import { QueryResult } from 'src/magic-meta/query/query-result';
import { RxUser } from 'src/entity-interface/RxUser';

export interface MagicService {
  me: RxUser;

  query(json: any): Promise;

  post(json: any): Promise;

  delete(json: any): Promise;

  update(json: any): Promise;
}

The controller of magic module directly calls this class to implement the interface defined above.

AbilityService

The permission management class is used to query the permission configuration of the entity and field of the current login user.

query

/magic/queryDirectory, implementation/get/json...Interface code.

MagicQueryIs the core code to realize the query business logic. It usesMagicQueryParserParse the incoming JSON parameters into a data tree and separate the relevant instructions. Data structure defined in/magic-meta/querycatalogue The amount of code is too large to analyze one by one. Read it by yourself. If you have any questions, you can contact the author.

What needs special attention isparseWhereSqlFunction. This function is responsible for parsing statements in SQL where format, and uses the open source librarysql-where-parser

It is placed in this directory because the magic module needs it, and the direct module also needs it. In order to avoid the circular dependency of the module, it is extracted into this directory independently.

/magic/query/traverserThe directory stores some traversers for processing the parsed tree data.

MagicQueryUsing typeormQueryBuilderBuild query. Key points:

  • Using the directive moduleQueryDirectiveServiceGets the instruction processing class. Instruction processing classes can: 1. BuildQueryBuilderCondition statements used, 2. Filter query results.

  • fromAbilityServiceGet the permission configuration and modify it according to the permission configurationQueryBuilder, filter the fields in the query results according to the permission configuration.

  • The query statements used by querybuilder are divided into two parts: 1. Statements that affect the number of query results, such as take instruction and paginate instruction. These instructions are only the result of the number of instructions to be intercepted; 2. Other query statements that do not have this effect. Because when paging, you need to return a total number of records. First check the database with the second type of query statement to obtain the total number of records, and then add the first type of query statement to obtain the query results.

post

/magic/postDirectory, implementation/postInterface code.

MagicPostClass is the core code to implement business logic. It usesMagicPostParserParse the incoming JSON parameters into a data tree and separate the relevant instructions. Data structure defined in/magic-meta/postcatalogue It can:

  • Recursively save associated objects, which can be nested infinitely in theory.

  • according toAbilityServiceDo permission check.

  • Using the directive modulePostDirectiveServiceGet the instruction processing class. The instruction handler will be called before and after the instance is saved. Please refer to the code for details.

update

/magic/updateDirectory, implementation/updateInterface code.

Simple function and simple code.

delete

/magic/deleteDirectory, implementation/deleteInterface code.

Simple function and simple code.

upload

/magic/uploadDirectory, implementation/uploadInterface code.

At present, the function of upload is relatively simple. You can add some cutting instructions and other functions later.

Directive module

Instruction service module. Hot load instructions and provide query services for these instructions.

This module is also relatively simple. The required statement is used for hot loading.

As for the back-end, there is nothing to say about other modules. They are very simple. Just look at the code directly.

Client RX models client

A client is needed to manage production and metadata, test the general data query interface, set entity permissions, installation, etc. Create a common react project that supports typescript.

npx create-react-app rx-models-client--template typescript

This project has been completed. On GitHub, the code address is:https://github.com/rxdrag/rx-models-client

The amount of code is a little too much. I’ll explain it all here. I can’t put it down. You can only pick the key points. If you have any questions to communicate, please contact the author.

ER diagram – Graphical business model

This module is the core of the client. It looks scary, but it’s not difficult at all. cataloguesrc/components/entity-boardThe following is all the codes of the module.

Thanks to antv X6, the production of this module is much simpler than expected.

X6 acts as a view layer. It is only responsible for rendering entity graphics and relationship connections, and returning some user interaction events. It is used to undo and redo the operation history function. It is not used in this project. You can only write it all yourself.

Mobx also plays a very important role in this module. It manages all States and undertakes some business logic. For low code and drag and drop projects, mobx is really very easy to use and worthy of recommendation.

Define mobx observable data

Each of the metadata defined above corresponds to a mobx observable class and a root index class. These data contain each other to form a tree structuresrc/components/entity-board/storeDirectory.

  • EntityBoardStore, the root node in the tree structure is also the overall status data of the module. It records the following information:
export class EntityBoardStore{
  /**
   *Is there any modification for unsaved prompt
   */
  changed = false;

  /**
   *All packages
   */
  packages: PackageStore[];

  /**
   *ER diagram currently being opened
   */
  openedDiagram?: DiagramStore;

  /**
   *X6 graph object currently in use
   */
  graph?: Graph;

  /**
   *The relationship on the toolbar is pressed to record the specific type
   */
  pressedLineType?: RelationType;

  /**
   *It is in the state of dragging the mouse to draw a line
   */
  drawingLine: LineAction | undefined;

  /**
   *Selected node
   */
  selectedElement: SelectedNode;

  /**
   *Command mode, undo list
   */
  undoList: Array = [];

  /**
   *Command mode, redo list
   */
  redoList: Array = [];

  /**
   *When the constructor passes in the package metadata, it will be automatically parsed into a mobx observable tree
   */
  constructor(packageMetas:PackageMeta[]) {
    this.packages = packageMetas.map(
      packageMeta=> new PackageStore(packageMeta,this)
    );
    makeAutoObservable(this);
  }
  
  /**
   *There is no need to expand a large number of later set methods
   */
  ...

}
  • PackageStore, the tree is completely consistent with the packagemeta defined above. The difference is that all meta related are replaced by store related:
export class PackageStore{
  id?: number;
  uuid: string;
  name: string;
  entities: EntityStore[] = [];
  diagrams: DiagramStore[] = [];
  relations: RelationStore[] = [];
  status: PackageStatus;
  
  constructor(meta:PackageMeta, public rootStore: EntityBoardStore){
    this.id = meta.id;
    this.uuid = meta?.uuid;
    this.name = meta?.name;
    this.entities = meta?.entities?.map(
      meta=>new EntityStore(meta, this.rootStore, this)
    )||[];
    this.diagrams = meta?.diagrams?.map(
      meta=>new DiagramStore(meta, this.rootStore, this)
    )||[];
    this.relations = meta?.relations?.map(
      meta=>new RelationStore(meta, this)
    )||[];
    this.status = meta.status;
    makeAutoObservable(this)
  }

  /**
   *Omit set method
   */
  ...

  
  /**
   *Finally, it provides a method to reverse convert the store into metadata for sending data to the back end
   */
  toMeta(): PackageMeta {
    return {
      id: this.id,
      uuid: this.uuid,
      name: this.name,
      entities: this.entities.map(entity=>entity.toMeta()),
      diagrams: this.diagrams.map(diagram=>diagram.toMeta()),
      relations: this.relations.map(relation=>relation.toMeta()),
      status: this.status,
    }
  }
}

And so onEntityStoreColumnStoreRelationStoreandDiagramStore

Previously definedX6NodeMetaandX6EdgeMetaThere is no need to make the corresponding store class, because it is impossible to update the X6 view through the mobx mechanism, and this work should be done in other ways.

DiagramStoreIt mainly provides data for displaying ER diagram. Add two methods to it:

export type NodeConfig = X6NodeMeta & {data: EntityNodeData};
export type EdgeConfig = X6EdgeMeta & RelationMeta;

export class DiagramStore {
  ...

  /**
   *Obtain all nodes of the current Er graph and use mobx update mechanism,
   *As long as the data changes, the view calling this method will be updated automatically,
   *The parameter only indicates the currently selected node or whether a connection is required,
   *These states affect the view and can be passed directly to each node here
   */
  getNodes(
    selectedId:string|undefined, 
    isPressedRelation:boolean|undefined
  ): NodeConfig[]

  /**
   *Obtain all the connections of the current Er graph and use the mobx update mechanism,
   *Whenever the data changes, the view that calls the method is automatically updated
   */
  getAndMakeEdges(): EdgeConfig[]

}

How to use mobx observable data

Use the context of react to pass the store data defined above to the sub components.

Define context:

export const EnityContext = createContext({} as EntityBoardStore);
export const EntityStoreProvider = EnityContext.Provider;
export const useEntityBoardStore = (): EntityBoardStore => useContext(EnityContext);

Create context:

...
const [modelStore, setModelStore] = useState(new EntityBoardStore([]));

...
  return (
    
      ...
    
  )

When using, it is called directly in the sub componentconst rootStore = useEntityBoardStore()You can get the data.

Tree Editor

Using Mui’s tree control + mobx object, the code is not complex. If you are interested, look through it, leave a message or contact the author if you have any questions.

How to use antv x6

X6 supports embedding react components in nodes and defining a componentEntityViewJust embed it. X6 related codes are in this directory:

src/componets/entity-board/grahp-canvas

The business logic is split into many react hooks:

  • useEdgeChange, the processing relationship line is dragged

  • useEdgeLineDraw, handle drawing line moving

  • useEdgeSelect, the processing relationship line is selected

  • useEdgesShow, rendering relationships, including updates

  • useGraphCreate, create a grpah object for X6

  • useNodeAdd, handle the action of dragging into a node

  • useNodeChange, the processing entity node is dragged or resized

  • useNodeSelect, the processing node is selected

  • useNodesShow, render solid nodes, including updates

Undo, redo

Undo and Redo are related not only to the ER diagram, but also to the entire store tree. That is to say, the revocation and redo mechanism of X6 is useless and can only be redone by itself.

Fortunately, the command mode in the design mode is relatively simple. Defining some commands and defining positive and negative operations can be easily completed. The implementation code is in:

src/componets/entity-board/command

Global status Appstore

According to the above method, mobx is used to build a global status management class Appstore to manage the status of the whole application, such as pop-up operation success prompt, pop-up error message, etc.

Code insrc/storeDirectory.

Interface test

Code insrc/components/api-boardDirectory.

Very simple, a module, the code should be easy to understand. Usedrxmodels-swrLibrary, just refer to its documentation directly.

JSON input control, encapsulated by Monaco’s react:react-monaco-editor, it’s very easy to use. Installation is a little troublesome. It needs to be installedreact-app-rewired

Monaco is not proficient. If you are proficient later, you can add the following functions, such as input prompt and code verification.

Authority management

Code insrc/components/auth-boardDirectory.

This module mainly focuses on the organization of back-end data and interface definition. There is little front-end code based onrxmodels-swrLibrary complete.

Permission definitions support expressions, which are similar to SQL statements and have built-in variables$meRefers to the currently logged in user.

The SQL expression needs to be verified during front-end input, so the open source library is also introducedsql-where-parser

Installation, login

Installation code insrc/components/installDirectory.

The login page issrc/components/login.tsx

The code can be seen at a glance.

Postscript

This article is quite long, but I’m not sure whether I have made it clear. If there are any questions, leave a message or contact the author.

After the demonstration can run, it has risked being kicked and sent it in several QQ groups. I received a lot of feedback. Thank you very much for your warm-hearted friends.

Rxmodels, finally took the first step

First contact with the front end

Rxmodels is coming, and we are moving towards the front end enthusiastically.

The front end frowned and said, “stay away, you’re not what we want.”

Rxmodels said, “I will change and grow. One day in the future, we will be the best partners.”

Next article

Build a visual low code front end from 0. It is estimated that it will take some time to reconstruct the front end first.