Using C + + template display instantiation to solve the separation of template function declaration and Implementation

Time:2020-11-13

Background of the problem

Before starting the text, do some background bedding, so that readers can understand my engineering requirements. My project is a client message distribution center. After connecting to the message background, the background will push some messages to me from time to time, and then I will forward them to other desktop products of this computer for display. In order to ensure that messages can be pushed to the client, the background adopts a repeated push strategy. That is to say, every time I reconnect to the background, the background will push all the messages within a period of time to me, regardless of whether these messages have been pushed before. If I push the messages directly to the product without processing, it may cause repeated display of the same message Question. For this reason, after receiving messages, I will save them in a container in the process. When new messages arrive, I will check whether the message has been received in this container. If so, it will not be forwarded.

1 namespace GCM {
 2     class server_msg_t 
 3     {
 4     public:
 5         void dump(char const* prompt); 
 6     
 7         std::string appname; 
 8         std::string uid; 
 9         std::string msgid; 
10         time_t recv_first = 0; 
11         time_t recv_last = 0; 
12         int recv_cnt = 0; 
13     };
14     
15     class WorkEngine
16     {
17     public:
18         WorkEngine();
19         ~WorkEngine();
20     
21     private:
22         // to avoid server push duplicate messages to same client.
23         // note this instance is only accessed when single connection to server arrives message, so no lock needed..
24         std::vector m_svrmsgs;
25     };
26 }

 

Above is the simplified code, M_ Svrmsgs member stores all background messages received, server_ msg_ T represents a background message. AppName and uid are used to locate which instance of which product is sent; msgid is used to uniquely identify a message; recv_ first、recv_ last、recv_ CNT represents the first time, last time and repeated receiving times of the message. Now, a very practical problem is that I need to serialize these messages to permanent storage so that the information can still be stored after the process is restarted. Here I use the SQLite database, and the related code is encapsulated in the member function of workengine. It is easy to think of a function declaration method as follows:

1 namespace GCM {
 2     class server_msg_t 
 3     {
 4     public:
 5         void dump(char const* prompt); 
 6     
 7         std::string appname; 
 8         std::string uid; 
 9         std::string msgid; 
10         time_t recv_first = 0; 
11         time_t recv_last = 0; 
12         int recv_cnt = 0; 
13     };
14     
15     class WorkEngine
16     {
17     public:
18         WorkEngine();
19         ~WorkEngine();
20     
21     protected:
22         int db_store_server_msg (std::vector const& vec); 
23         int db_fetch_server_msg (std::vector & vec);
24     
25     private:
26         // to avoid server push duplicate messages to same client.
27         // note this instance is only accessed when single connection to server arrives message, so no lock needed..
28         std::vector m_svrmsgs;
29     };
30 }
31

 

As line 22-23 shows, use STD:: vector directlyThis container is used as a parameter (some people may think that I am making a redundant use of the function to access M_ Svrmsgs members are not enough. Why should they be passed through parameters? This example may not be obvious, but there are some cases where the container exists as a local variable rather than a member variable. Here, some simplifications are made for the purpose of illustration). But I think it’s too rigid to write like this. In case I change the container later, is it necessary to change it here? Perhaps it is the generic algorithm read too much, always feel that this writing is not “universal”. However, if it is written as follows, the soup should not be changed

int db_store_server_msg (std::vector::iterator beg, std::vector::iterator end);

 

Refer to the STD:: copy algorithm in the standard library and modify it. The result is as follows:

template 
int db_store_server_msg(InputIterator beg, InputIterator end);

 

Member function template, member template function, or template member function I don’t know. Anyway, it’s member function + template function. If it is implemented, it can be written as follows:

1 namespace GCM {
 2     template 
 3     int WorkEngine::db_store_server_msg(InputIterator beg, InputIterator end)
 4     {
 5         int ret = 0, rowid = 0; 
 6         qtl::sqlite::database db(SQLITE_TIMEOUT);
 7     
 8         try
 9         {
10             db.open(get_db_path().c_str(), NULL);
11             writeInfoLog("open db for store server msg OK");
12     
13             db.begin_transaction();
14     
15             for (auto it = beg; it != end; ++it)
16             {
17                 // 1th, insert or update user info
18                 rowid = db.insert_direct("replace into server_msg (appname, uid, msgid, first_recv, last_recv, count) values (?, ?, ?, ?, ?, ?);", 
19                     it->appname, it->uid, it->msgid, it->recv_first, it->recv_last, it->recv_cnt);
20     
21                 ret++; 
22             }
23     
24             db.commit();
25             db.close();
26             writeInfoLog("replace into %d records", ret); 
27         }
28         catch (qtl::sqlite::error &e)
29         {
30             writeInfoLog("manipute db for store server msg error: %s", e.what());
31             db.rollback();
32             db.close();
33             return -1;
34         }
35     
36         return ret; 
37     }
38 }

 

As you can see, the core code is to traverse the iterator interval (line 15). The caller is also very concise:

db_store_server_msg(m_svrmsgs.begin(), m_svrmsgs.end()); 

 

It seems to have been completed in one line, and there is no difficulty to speak of. So what does this article want to illustrate? Don’t worry, the real difficulty is recovering data from the database. First of all, we can’t use iterators directly, because we need to insert elements into the container. Iterators can only traverse the elements, and it doesn’t help at all. But I believe that readers must have read such code:

1 int main (void)
 2 {
 3     int arr[] = { 1, 3, 5, 7, 11 }; 
 4     std::vector vec; 
 5     std::copy (arr, arr + sizeof (arr) / sizeof (int), std::back_inserter(vec)); 
 6     for (auto it = vec.begin (); it != vec.end (); ++ it) 
 7         printf ("%d\n", *it); 
 8 
 9     return 0; 
10 }

 

In order to insert elements at the end of the container, the standard library algorithm uses back_ Inserter. So naturally I thought, can we declare back here_ How about using the inserter as the input parameter? For example, this is the following:

template 
int db_fetch_server_msg(OutputIterator it);

 

The template implementation is as follows:

1 namespace GCM {
 2     template 
 3     int WorkEngine::db_fetch_server_msg(OutputIterator it)
 4     {
 5         int ret = 0;
 6         qtl::sqlite::database db(SQLITE_TIMEOUT);
 7     
 8         try
 9         {
10             db.open(get_db_path().c_str(), NULL);
11             writeInfoLog("open db for fetch server msg OK");
12     
13             db.query("select appname, uid, msgid, first_recv, last_recv, count from server_msg", 
14                 [&ret, &it](std::string const& appname, std::string const& uid, std::string const& msgid, time_t first_recv, time_t last_recv, int count) {
15                     server_msg_t sm; 
16                     sm.appname = appname; 
17                     sm.uid = uid; 
18                     sm.msgid = msgid; 
19                     sm.recv_first = first_recv; 
20                     sm.recv_last = last_recv; 
21                     sm.recv_cnt = count; 
22                     *it = sm; 
23                     ++ret; 
24             }); 
25     
26             db.close();
27             writeInfoLog("query %d records", ret);
28         }
29         catch (qtl::sqlite::error &e)
30         {
31             writeInfoLog("manipute db for store server msg error: %s", e.what());
32             db.close();
33             return -1;
34         }
35     
36         return ret;
37     }
38 }

 

In fact, the core is a pair of back_ Inserter assignment statement (line 22). The caller also does this in one line:

db_fetch_server_msg (std::back_inserter(m_svrmsgs)); 

 

Separation of template declaration and template implementation

The above code can be compiled normally, but only if the template implementation and template call are in the same file. Considering that there was a lot of logic before this class, I decided to move the database related content to a new file (engine)_ db.cpp )To reduce the amount of code in a single file. The adjusted file structure is as follows:

+Engine. H: workengine declaration
+ engine.cpp : workengine implementation (including engine. H)
+ engine_ db.cpp :WorkEngine::db_ XXX template implementation (including engine. H)

 

Recompile, reported a link error:

1> workengine.obj  : error LNK2001: unresolved external symbol "protected: int"__ thiscall GCM::WorkEngine::db_ fetch_ server_ msg > > >(class std::back_ insert_ iterator > >)" (??$db_ fetch_ server_ [email protected]? $back_ insert_ [email protected]? [email protected]_ msg_ [email protected]@@V ? [email protected]_ msg_ [email protected]@@@[email protected]@@[email protected]@@[email protected]@@[email protected]@@IAEHV ?$back_ insert_ [email protected]? [email protected]_ msg_ [email protected]@@V ? [email protected]_ msg_ [email protected]@@@[email protected]@@[email protected]@@[email protected]@@Z )

 

This is obviously caused by the failure to find the corresponding link when calling the template. In this case, you need to use “template display instantiation” in engine_ db.cpp File to force the template to generate the corresponding code entity, and engine.cpp The call point in the. You need to add the following two lines of code at the beginning of the file:

using namespace GCM;template int WorkEngine::db_fetch_server_msg > >(std::back_insert >);

 

Pay attention to the syntax of template member function display instantiation. I specially checked CPP primer in the following format:

template return_type CLASS::member_func<type1, type2, ……> (type1, type2, ……);

 

Corresponding to the above statement, STD:: back is used_ insert>Instead of the original output iterator type, tell the compiler to display the generation of such a function template instance. Note that the same type should be written twice, one is function template parameter, the other is function parameter. However, this display instantiation syntax failed to compile

1>engine_ db.cpp (15) : error c2061: syntax error: identifier "back"_ inserter”
1>engine_ db.cpp (15): error C2974: 'GCM::WorkEngine::db_ fetch_ server_ MSG ': template is not a valid parameter for' outputtilter ', expected type
1> F: gdpclient, SRC, GCM, gcmsvc, workengine. H (137): see "GCM:: workengine:: DB"_ fetch_ server_ MSG "
1>engine_ db.cpp (15) : error c3190: int GCM:: workengine:: dB with supplied template parameters_ fetch_ server_ MSG (void) "is not an explicit instantiation of any member function of" GCM:: workengine "
1>engine_ db.cpp (15) : error c2945: explicit instantiation does not refer to template class specialization

 

It’s hard to understand. I went out for a circle and took a breath of fresh air, and my head suddenly came to light: there was a long chain of link errors before, and the type in that was used directly, which should be able to be compiled! You can do what you say, so you have the following long list of instantiation statements:

template int GCM::WorkEngine::db_fetch_server_msg > > >(class std::back_insert_iterator > >)

 

What’s too much is that it has passed the compilation! Take a closer look at this long list of type statements. It seems that the vector is just expanded. I’ll use the “condensed” vector to declare again. What’s the change

template int GCM::WorkEngine::db_fetch_server_msg > >(std::back_insert_iterator >);

 

It passed. It’s just back_ insert_ Iterator instead of back_ Just insert it, back_ insert_ What kind of ghost is iterator? View back_ The definition of inserter is as follows:

1 template inline back_insert_iterator back_inserter(_Container& _Cont)
2 {    // return a back_insert_iterator
3     return (_STD back_insert_iterator(_Cont));
4 }

 

Looks like back_ Inserter is a return back_ insert_ Template function of iterator type, and STD:: make_ Pair (a, b) and STD:: pairThe relationship is very similar, because what you want here is a type, so you can’t send back directly_ The inserter function declares the instantiation of the display. Well, up to now, we have replaced the clumsy container parameters with one inserter or two iterator parameters, and split the declaration, call and implementation into three different files, which is perfect. The drawback is that the template display instantiation still has some wordiness. Here, typedef is used to define the type to be instantiated, so that the above statement can be modified more clearly

typedef std::back_insert_iterator > inserter_t;
template int WorkEngine::db_fetch_server_msg(inserter_t);

 

Similarly, for DB_ store_ server_ MSG was modified in the same way

typedef std::vector ::iterator iterator_t;
template int WorkEngine::db_store_server_msg(iterator_t, iterator_t);

 

Is this more perfect?

Use map instead of vector

In the process of use, it is found that map can be used to query whether the message is already in the container more quickly and conveniently. Therefore, we decided to change the definition of message container as follows:

std::map m_servmsgs;

 

The value part of the map remains the same as before, and the added key part is msgid. After this change, the traversal should use “it > second.” instead of “it – >”; when inserting elements, you need to use “* it = STD:: make”_ pair ( sm.msgid , SM) “instead of” * it = SM “. After the above modification, I found that the program still failed to compile. After some investigation, it was found that it was back_ The inserter does not fit into the map container. Because of back_ The back corresponding to the inserter_ insert_ Iterator calls the container’s push in the = operator_ The back interface, which is only supported by several containers such as vector, list and deque, is not supported by map. What to do? Fortunately, a kind-hearted person has already written the mapinserter map_ inserter:

1 #pragma once
 2 
 3 namespace std
 4 {
 5     template >
 6     class map_inserter {
 7 
 8     public:
 9         typedef std::map map_type;
10         typedef typename map_type::value_type value_type;
11 
12     private:
13         map_type &m_;
14 
15     public:
16         map_inserter(map_type &_m)
17             : m_(_m)
18         {}
19 
20     public:
21         template 
22         class map_inserter_helper {
23         public:
24             typedef map_inserter mi_type;
25             typedef typename mi_type::map_type map_type;
26             typedef typename mi_type::value_type value_type;
27 
28             map_inserter_helper(map_type &_m)
29                 :m_(_m)
30             {}
31 
32             const value_type & operator= (const value_type & v) {
33                 m_[v.first] = v.second;
34                 return v;
35             }
36         private:
37             map_type &m_;
38         };
39 
40         typedef map_inserter_helper<_Key, _Value, _Compare> mi_helper_type;
41         mi_helper_type operator* () {
42             return mi_helper_type(m_);
43         }
44 
45         map_inserter &operator++() {
46             return *this;
47         }
48 
49         map_inserter &operator++(int) {
50             return *this;
51         }
52 
53     };
54 
55     template 
56     map_inserter map_insert(std::map &m) {
57         return map_inserter(m);
58     }
59 };

 

I copied this code from the Internet. Please refer to the following link for details: inserter implementation of STD:: map. Unfortunately, this code is “disabled”. I don’t know whether it was the author who stole the link or didn’t input it completely. There are some congenital grammatical defects in this code, which makes it even unable to be compiled. In the process of my unremitting “brain toning”, the missing part has been supplemented by highlighted parts, which can be enjoyed directly by all the guests~

It should be noted that the most technical missing occurs in a reference character of line 37. If this is not added, it can be compiled, but during the running process, the inserter cannot insert elements into the map, which will result in an empty map after reading from the database. I’ve been trying to find the original text of this article, but I can’t find it. I’m very sorry to find such an error in the process of Internet communication (although I don’t understand it very well, you can’t pit me)

Well, to get back to the point, we have a map_ After inserter, we can state as follows:

typedef std::map_inserter > inserter_t;
template int WorkEngine::db_fetch_server_msg(inserter_t);

 

For this map_ To implement the inserter, we need to pass the three template parameters of the map, not the map itself. I am not sure whether it is a progress or a retrogression. Anyway, this map_ The inserter is a little strange. It is not encapsulated into a map_ insert_ iterator + map_ There are still some differences between the form of the inserter and the implementation level of the standard library. Let’s have a look. The caller also needs to make some fine-tuning:

db_fetch_server_msg(std::map_inserter >(m_svrmsgs));

 

Fortunately, we have not implemented the standard of inserter_ Type T can be rewritten as follows:

db_fetch_server_msg(inserter_t(m_svrmsgs));

 

It’s much simpler. Now let’s look at the file composition of the project:

+ map_ inserter.hpp : map_ Inserter declaration + implementation
+Engine. H: workengine declaration (including map_ inserter.hpp )
+ engine.cpp : workengine implementation (including engine. H)
+ engine_ db.cpp :WorkEngine::db_ XXX template implementation (including engine. H)
……

 

Here, in order to reduce the complexity, map_ Inserters are shared in the header file, similar to the use of standard library header files.

Use common template functions instead of class member template functions

At the end of this paper, we look back at the two member template functions in the above example, and find that they are not used in other members of the class. In fact, they can be called independently as two ordinary template functions. For example, it is changed to the following:

1 namespace GCM {
 2     class server_msg_t 
 3     {
 4     public:
 5         void dump(char const* prompt); 
 6     
 7         std::string appname; 
 8         std::string uid; 
 9         std::string msgid; 
10         time_t recv_first = 0; 
11         time_t recv_last = 0; 
12         int recv_cnt = 0; 
13     };
14     
15     class WorkEngine
16     {
17     public:
18         WorkEngine();
19         ~WorkEngine();
20         
21     private:
22         // to avoid server push duplicate messages to same client.
23         // note this instance is only accessed when single connection to server arrives message, so no lock needed..
24         std::vector m_svrmsgs;
25     };
26 
27     template <class InputIterator>
28     int db_store_server_msg(InputIterator beg, InputIterator end); 
29     template <class OutputIterator>
30     int db_fetch_server_msg(OutputIterator it);
31     
32     typedef std::map string, server_msg_t>::iterator iterator_t;
33     typedef std::map_inserterstring, server_msg_t, std::lessstring> > inserter_t;
34 }

 

Move template function declaration from class to out of class (line 27-30) and modify engine_ db.cpp To define and display instantiation statements of two classes in, remove the class restriction (workengine::):

template int db_fetch_server_msg(inserter_t);
template int db_store_server_msg(iterator_t, iterator_t);

 

No modification is required at the call. Error is reported when compiling again

1>engine_ db.cpp (16): warning C4667: “int GCM::db_ fetch_ server_ msg(GCM::inserter_ t) ": no function template is defined to match the mandatory instantiation
1>engine_ db.cpp (17): warning C4667: “int GCM::db_ store_ server_ msg(GCM::iterator_ t,GCM::iterator_ t) ": no function template is defined to match the mandatory instantiation
1> Creating library F: gdpcclient / SRC / GCM / release\ gcmsvc.lib  And object F: gdpcclient / SRC / GCM / release\ gcmsvc.exp
1> workengine.obj  : error LNK2001: unresolved external symbol 'Int'__ cdecl GCM::db_ fetch_ server_ msg,class std::allocator >,class GCM::server_ msg_ t,struct std::less,class std::allocator > > > >(class std::map_ inserter,class std::allocator >,class GCM::server_ msg_ t,struct std::less,class std::allocator > > >)" (??$db_ fetch_ server_ [email protected]? $map_ [email protected]? $basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@Vserver_ msg_ [email protected]@@U ? [email protected]? $basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@@[email protected]@[email protected]@@[email protected]@YAHV ?$map_ [email protected]? $basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@Vserver_ msg_ [email protected]@@U ? [email protected]? $basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@@[email protected]@[email protected]@@Z )
1> workengine.obj  : error LNK2001: unresolved external symbol 'Int'__ cdecl GCM::db_ store_ server_ msg,class std::allocator > const ,class GCM::server_ msg_ t> > > > >(class std::_ Tree_ iterator,class std::allocator > const ,class GCM::server_ msg_ t> > > >,class std::_ Tree_ iterator,class std::allocator > const ,class GCM::server_ msg_ t> > > >)" (??$db_ store_ server_ [email protected]? $5 Tree_ [email protected]? $5 Tree_ [email protected]? $5 Tree_ simple_ [email protected]? [email protected]$$CBV?$basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@Vserver_ msg_ [email protected]@@@[email protected]@@[email protected]@@[email protected]@@[email protected]@@[email protected]@YAHV $? Tree_ [email protected]? $5 Tree_ [email protected]? $5 Tree_ simple_ [email protected]? [email protected]$$CBV?$basic_ [email protected]? $char_ [email protected]@[email protected]@V ? [email protected]@[email protected]@[email protected]@Vserver_ msg_ [email protected]@@@[email protected]@@[email protected]@@[email protected]@@[email protected]@[email protected] )

 

The first two warnings are because after the member function is changed into a common function, the display instantiation needs to be placed behind the function implementation. We can adjust these two statements to the end of the file. For the latter two link errors, I was puzzled. Later, I used a very simple test template function to test and found that it was the namespace that caused the problem. We need to add a namespace qualification (GCM::) before the definition and display of each function instantiation statement

template int GCM::db_fetch_server_msg(inserter_t);
template int GCM::db_store_server_msg(iterator_t, iterator_t);

 

We can see that there is a big difference between the class member template function and the ordinary template function, because the class itself is also a kind of namespace, and its appearance simplifies the addressing of member functions.

epilogue

In fact, this paper describes a general method of reading containers through iterators and inserting container elements through inserters. This method is much more elegant than directly passing on the container itself. Although it can not achieve 100% seamless container switching, it also provides great flexibility. In particular, it also studies how to declare and implement the template functions in different files, so as to achieve the purpose of decoupling code, which has strong practicability. Of course, only template instantiation is used here. If different types of templates need to be implemented with different functions, you may also encounter template specialization syntax (including full specialization and partial specialization), and the complexity will increase. No further exploration is made here.

reference resources

[1] . C + + 11 lambda expression

[2] Inserter implementation of. STD:: Map

[3] Separation of declaration and implementation of C + + template class (template instantiation)

[4] Compiling method of C + + function template

[5] . C + + function template declaration is separated from definition

[6] Function template instantiation and materialization of C + + template

[7] . C + + function template instantiation and materialization

[8] The implicit instantiation, display instantiation, implicit call, display call and template specialization of C + + template

[9] . C + + template function declaration and definition separation

[10] . C + + template programming: how to separate declaration and definition of non generic template functions

Recommended Today

Method of hiding version number and web page cache time in nginx

Nginx Optimization — Hide version number and page cache time Configure nginx to hide version number In the production environment, we need to hide the version number of nginx to avoid security Leakage of loopholes View method Use Fiddler I to check the nginx version number on the Windows client Using “curl – I URL” […]