Build a simple but not simple configuration center for micro services

Time:2021-3-11

Without hesitation, the modern rapid development of the Internet has created batch after batch of network celebrities, which have greatly spawned a wave of traffic on a specific platform, but it is a chicken feather for programmers. Whether it is operation and maintenance or development, they will worry about server crash and program down every day. I still miss those stand-alone structures in the past, and I’m even a little envious of those programmers who do applications with little access to the intranet. They don’t have to work overtime, they don’t have to worry, and they don’t have to buy Bawang shampoo every year.

When it comes to stand-alone architecture, you can’t afford it in Internet applications. You can doubt your life because of the impact of traffic peak. Upgrading a cluster with a single machine brings not only technical challenges, but also the complexity of configuration while resisting the traffic peak and catering to the business. This is also my topic today: configuration management. In the stand-alone era, no matter what language it is, Java or C #, a configuration file is enough. With the birth of the so-called micro service, which seems to be able to solve all problems, more complex configuration problems are introduced: service information, various parameters of service, configuration update problems, etc. As you can imagine, if you have 100 servers in your service, it is a painful thing to modify a configuration item and update it one by one by using a single framework. The traditional configuration file method can no longer meet the requirements of developers for configuration management

  1. Security. If the configuration information is released with the code, it is easy to cause configuration leakage.
  2. Real time. To modify the configuration, the traditional stand-alone architecture must restart the service to take effect.
  3. limitations. It can’t support dynamic adjustment, like the most common log switch function.
  4. Environmental differentiation. It’s difficult to distinguish production, development and test environment by traditional configuration file.
  5. Configuration modification record problem. It is difficult to track the modification records of this configuration file in the static configuration file mode.

In view of the above problems, some companies use database record configuration to solve the problem. It’s not that they can’t, but the database can’t solve the fundamental problem. Take a very simple example: how can the client be notified in real time when there is the latest record modification? Although other solutions can be used to solve the problem, the database based method is not the best.

Whether the configuration center adopts database or other methods, the most essential starting point is to see the specific needs of the business and the corresponding human and material resources that can be invested. It’s not that Apollo like Ctrip is bad, but whether such a huge configuration system is suitable for your company and your business. In many cases, the company’s business development in a certain stage is short, flat and fast, there is no need and no time to invest energy to study a huge system, after the business develops slowly, it can be iterated slowly, this is the process of system architecture upgrading, those who need to establish Taobao level architecture at the beginning of the business will eventually get everyone tired.

Having said so much, I just want to build a simple and landing configuration center. To ensure the normal operation of the configuration center, there are several points to be considered at the beginning of the design

Need a reliable, strong consistency of storage to support

After many times of technical research, I finally chose etcd, not because I like the most hot spots, but because etcd is just corresponding to my needs in terms of scene and function,

Etcd is an open source project launched by the coreos team in June 2013. Its goal is to build a highly available distributed key value database. In etcd, raft protocol is used as the consistency algorithm, and etcd is implemented based on go language.

  1. Simple: the installation and configuration is simple, and the HTTP API is provided for interaction, and the use is also very simple
  2. Security: support SSL certificate verification
  3. Fast: according to the official benchmark data, single instance supports 2K + reads per second
  4. Reliable: adopt raft algorithm to realize the availability and consistency of distributed system data
  5. Watch mechanism: etcd supports the watch mechanism for a key or a group of keys. Once the data changes, the client will be notified in real time.

Need to support multi environment configuration

Although many stored display parameters do not have parameters like environment, they all provide functions like directory storage, just like the file directory of windows, which provides us with strong adaptability for custom functions. For example, our storage a application development environment can be like this: A / dev /, which represents the development environment configuration of a application.

When the configuration item changes, the client needs to be informed in real time

Etcd based on the first point naturally supports the watch mechanism, so it’s a good way to notify the client in real time when the configuration item changes. Even if the notification fails, we can also customize the time to delay the update of the configuration. See the following code later.

Easy to use

For users, the only business interface provided by the configuration center is to obtain the configuration of a key, where the key can be a collection of parameters such as application + environment. In order to assist tracking, you can expose the handling events when the program is abnormal, just like the following program:

/// <summary>
    ///Configuration center, client, application, sub application, environment, version, grayscale
    /// </summary>
    public interface IConfigCenterClient
    {
        /// <summary>
        ///Event when configuration information changes. Parameter: key / new value / action (put / delete). It is the event of etcd / consumer watch. If the value of each key changes, it will trigger, and each key will trigger one
        /// </summary>
        event Action<string, string, string> ConfigValueChangedEvent;

        /// <summary>
        ///Event when an exception occurs
        /// </summary>
        event Action<Exception> ErrorOccurredEvent;

        /// <summary>
        ///Get the corresponding configuration
        /// </summary>        
        ///< param name = "configkey" > configuration name < / param >
        ///< param name = "version" > version number < / param >
        /// <returns></returns>
        string GetConfig(string configKey);
       
    }

After abstracting this operation interface, as for how to implement it, it can be based on etcd, consul, or even dB. It can be seen how correct interface oriented programming is.

Need to be able to continue to work in case of network failure, etc

In Internet applications, there is always a truth: the network is unreliable. As a core system of the company, configuration center should provide services as much as possible. However, preventive measures should also be taken to prevent the applicable party from being affected during the short period when the configuration center is not available. For using the client side, since the server side can not guarantee high availability, it needs to be operated locally: the obtained configuration information can be stored locally and persisted with the watch mechanism. In this way, when the configuration center network is not available, try to ensure that the client program is available. As for the way of local storage, it doesn’t matter, even if the text file, or sqllite can.

High performance

One of the most significant business characteristics of configuration center is that it does not change frequently, but the client uses it frequently. So we can load the configuration information into the memory, and the data in the memory will change with the change of the watch mechanism, so that the memory data and the server data are highly consistent.

I’m sure you’re tired of all the above. Anyone can talk about theory, and the landing code is the truth.

Client configuration center interface
/// <summary>
    ///Configuration center, client, application, sub application, environment, version, grayscale
    /// </summary>
    public interface IConfigCenterClient
    {
        /// <summary>
        ///Event when configuration information changes. Parameter: key / new value / action (put / delete). It is the event of etcd / consumer watch. If the value of each key changes, it will trigger, and each key will trigger one
        /// </summary>
        event Action<string, string, string> ConfigValueChangedEvent;

        /// <summary>
        ///Event when an exception occurs
        /// </summary>
        event Action<Exception> ErrorOccurredEvent;

        /// <summary>
        ///Get the corresponding configuration
        /// </summary>        
        ///< param name = "configkey" > configuration name < / param >
        ///< param name = "version" > version number < / param >
        /// <returns></returns>
        string GetConfig(string configKey);
       
    }
     /// <summary>
    ///Definition of environment
    /// </summary>
    public enum EnvEnum
    {
        /// <summary>
        ///Local environment
        /// </summary>
        Local=1,

        /// <summary>
        ///Development environment
        /// </summary>
        DEV,

       /// <summary>
       ///Simulation environment
       /// </summary>
        UAT,

        /// <summary>
        ///Production environment
        /// </summary>
        PRO
    }
    public class ConfigCenterETCDProvider : ConfigCenterBaseProvider, IConfigCenterClient
    {
      
        //Notification events for configuration changes
        public event Action<string, string, string> ConfigValueChangedEvent;
        //An event when an exception occurs
        public override event Action<Exception> ErrorOccurredEvent;

        //Configuration of etcd
        private ConfigCenterETCDOption option { get; set; }

        private EtcdClient client = null;

        internal ConfigCenterETCDProvider(ConfigCenterBaseOption _option) : base(_option)
        {
            if (_option == null)
            {
                throw new Exception("config is null!");
            }
            option = _option as ConfigCenterETCDOption;
            if (option == null)
            {
                throw new Exception("option type is wrong!");
            }
            if (string.IsNullOrWhiteSpace(option.ConnectionString))
            {
                //Default address
                option.ConnectionString = "http://127.0.0.1";
            }
          
            client = new EtcdClient(option.ConnectionString, option.Port, option.Username, option.Password, option.CaCert, option.ClientCert, option.ClientKey, option.PublicRootCa);
            
            client.WatchRange(WatchPath, ETCDWatchEvent, exceptionHandler: e =>
            {
                ErrorOccurredEvent?.Invoke(e);
                Thread.Sleep(1000);
            });

            //Read local file initializes read only once           
            InitConfigMapFromLocal();
        }

        #Interface method of region implementation
        //Set the value of a configuration
        public bool SetConfig(string configKey, string configValue)
        {
            string key = FillConfigKey(configKey);
            try
            {
                var putRet = client.Put(key, configValue);

            }
            catch (Exception e)
            {
                ErrorOccurredEvent?.Invoke(e);
                return false;
            }
            return true;

        }

        //Delete the value of a configuration
        public bool DeleteConfig(string configKey)
        {
            string key = FillConfigKey(configKey);
            try
            {
                client.Delete(key);
            }
            catch (Exception e)
            {
                ErrorOccurredEvent?.Invoke(e);
                return false;
            }
            return true;
        }
        #endregion

        #Region base class method
        //Etcd organization key
        protected override string FillConfigKey(string configKey)
        {
            return $"{WatchPath}/{configKey}";
        }
        //Etcd get remote key
        protected override (bool isSuccess, string value) GetValueFromServer(string configKey)
        {
            try
            {
                var value = client.GetVal(configKey);
                return (true, value);
            }
            catch (Exception e)
            {
                ErrorOccurredEvent?.Invoke(e);
                return (false, null);
            }
        }

        #endregion

        #Region private method
        //Events that monitor changes in value
        private void ETCDWatchEvent(WatchEvent[] response)
        {
            lock (ConfigCenterBaseProvider.ObjLock)
            {
                foreach (WatchEvent e in response)
                {
                    if (e.Type == Mvccpb.Event.Types.EventType.Delete)
                    {
                        if (ConfigMap.ContainsKey(e.Key))
                        {
                            ConfigMap.Remove(e.Key);
                        }
                    }
                    else
                    {
                        ConfigMap[e.Key] = e.Value;                       
                    }

                    Console.WriteLine(JsonConvert.SerializeObject(ConfigMap));
                    if (ConfigValueChangedEvent != null)
                    {
                        ConfigValueChangedEvent.Invoke(e.Key, e.Value, e.Type.ToString());
                    }

                }
                if (response != null && response.Any())
                {
                    //Write the latest value to the local file
                    FlushLocalFileFromMap();
                }

            }

        }


        #endregion
    }

In view of the code, there is no longer a small buddy interested in adding WeChat dishes or official account “architect’s path of practice” to the “configuration center”. The code is based on the complete ETCD simplified version of the configuration center.

Build a simple but not simple configuration center for micro services

More wonderful articles

Build a simple but not simple configuration center for micro services