Deferred caller(or delegator.. whatever you name it) implementation with variadic template

Deferred caller, delegator implementation with variadic template

Recently I started playing with a small toy project – tiny web server with C++. The first requirement I wanted to implement was the ‘static type routing handler’ as shown from the crow project.

Which means, users should be able to register its routing handlers as the following manner:

In order to do that, two things come up to the list:

  1. constexpr string parser to determine the parameter type (e.g <int>, <string>…)
  2. being able to register lambda which can have variadic template arguments

Regarding #2 issue, people would agree it should have been called ‘delegator’ or ‘deferred caller’. Its concept is basically same. Accepts a functor, saves it and calls it later. Then, how can I save functor which can have variadic number and types without using variant?

I got a hint from the Codeproject article, and implemented as follows. The routing_delegator structure is a container owns a pointer to the routing_record interface pointer.

‘to(F&& f)’ is lambda subscribing function. If user tries to pass a lambda to the function, you can figure out the lambda’s parameters by specifying operator() functor signature to a specialized template below. (refer to this SO link)

The typed_routing_recored is the template class, and lambda signature will be deduced, and the typed figured out all will be saved to the std::function.

How about calling the saved functors? The routing_record is an interface and declares one abstract function – which is ‘handle(void*)’. As you might guess from its name, void* is the key point to enable accept variadic sized, typed arguments.

If user calls ‘handle’ with parameters, routing_delegator::handle(Args&&… args) wconverterts those parameters into tuple and passes it to the routing_record::handle(void*) with its address

And finally child class’s handle(void*) implementation will casts into tuple again and delivers it to the actual std::function functor.

You can check out the all code here, https://github.com/heejune/tinyweb-cppserver/blob/develop/router.h

Please be warned, it’s still in development and all experimental code. Thanks!

Heejune.

Sieve of Eratosthenes in C++

Updated June 8th, 2017. This post was introduced to the Reddit /r/cpp soon after I posted and immediately got a feedback saying “both semantically wrong as well as poorly optimized” which is quite true. 🙂  So I’d like to recommend visiting the Reddit link and consider the comments before preceding or even better, refer to the better example introduced here.

Let’s talk about prime numbers. As an application/system software developer, it’s not often to deal with prime numbers in daily working environments. For me, I came across it while solving the Project Euler’s questions.

There’re a lot of good reads on the net describing what prime numbers are, how to get the numbers. So instead of a basic introduction, I’d like to talk about few tricks which can improve the performance of getting prime numbers.

Let’s first talk about the very basic approach to get prime numbers to compare with optimized version later.

You can check whether it’s a prime number or not when a number is given by following:

You’ll also get the Nth prime number based on the isPrime() function above by incrementing the counter when it returns true.

It’s self-descriptive and easy.

Can we improve the performance? Sure! There’s a ‘Sieve of Eratosthenes‘ algorithm, and basically, it does pre-calculate all the non-prime numbers when it found a new prime number by removing all multiples.

Here’s the implementation of the algorithm and its example.

And Lastly, can we improve the Sieve of Eratosthenes algorithm better? Yeah, it’s possible. I first saw the optimized implementation from one of the answers here: http://qa.geeksforgeeks.org/3090/how-to-find-nth-prime-number It’s tricky to understand at first glance in fact. So that I rewrote a little bit and here’s the result:

Comments are still Koreans and will be translated into English soon. Few tricks used to improve the performance are like following:

#1. Check against odd numbers only from line 26 – if (j & 1

#2. Getting the odd number index when an odd number n is given: (n – 3) >> 1

If I changed the unit into ns…

D:\workspace\playground\SieveOfEratosthenes\Release>SieveOfEratosthenes.exe –benchmark_format=json

{

“context”: {

“date”: “06/05/17 01:06:32”,

“num_cpus”: 8,

“mhz_per_cpu”: 2592,

“cpu_scaling_enabled”: false,

“library_build_type”: “release”

},

“benchmarks”: [

{

“name”: “BM_get_nth_prime_without_sieve/10001”,

“iterations”: 160,

“real_time”: 4618235,

“cpu_time”: 4589844,

“time_unit”: “ns”

},

{

“name”: “BM_get_nth_prime_with_basic_sieve/10001”,

“iterations”: 373,

“real_time”: 1854873,

“cpu_time”: 1843164,

“time_unit”: “ns”

},

{

“name”: “BM_get_nth_prime_with_optimized_sieve/10001”,

“iterations”: 1120,

“real_time”: 642190,

“cpu_time”: 655692,

“time_unit”: “ns”

}

]

}

You can access the sample here: https://github.com/heejune/SieveOfEratosthenes

It’ll require the google benchmark and few modifications to correctly link and search headers and libraries to build.

Thanks,

Heejune

Visual Studio 2017 – Why can’t I see the local and auto variables during UWP app debugging?

I’d like to share my short episode regarding UWP app debugging which happened today. Recent days I’ve developed a WinRT dynamic library which is based on cppwinrt. Most of time, I tested it with an unittest project and everything went fine. However, when I used the WinRT dll from actual .exe UWP application then very strange problem showed up.

As you see the below screenshots, the Visual Studio 2017 debugger shows nothing about local variables, auto and even watch window variables during the UWP app debugging.

Moreover, if I tried to type specific variables into Watch window or immediate window, the debugger shows ‘Internal error in the C# compiler‘ error message!

Later I noticed that the strange problem starts just right after the following output messages come up from Output window.

‘Example.exe’ (CoreCLR: CoreCLR_UWP_Domain): Loaded ‘Anonymously Hosted DynamicMethods Assembly’.

‘Example.exe’ (Win32): Loaded ‘C:\src\Example\build\Win32\Debug\AppX\Example.winmd’. Module was built without symbols.

‘Example.exe’ (CoreCLR: CoreCLR_UWP_Domain): Loaded ‘C:\src\Example\build\Win32\Debug\AppX\Example.winmd’. Module was built without symbols.

How just loading .winmd file could affect the weird debugger problem? Then I realized I used the same name(‘Example’) for the application and the WinRT module both. Which means I created the WinRT dll component as ‘Example.DLL’ and along with ‘Example.winmd’ as its descriptive winmd file name, of course. But I also named the target UWP application as ‘Example’.

So while debugging, the debugger tried to load the winmd file for it’s target WinRT component which was being debugged and suddenly the namespace seemed clashed within between the app and debugee DLL.

So, I changed the name of the WinRT component as ‘Example.Library.dll’ and problem solved!

Thanks,

Heejune

Lessons learend (from failuere) while I developing a log server for Windows

Wrote this post in Jan 2015 as a quick draft, and remained as it is still today. I thought it’s not that useful so tried to delete it, but changed mind and released here. Hope it helps.

Recently I developed a logging server based on C#/asp.net on Windows 2008 R2 server. The server primarily does receiving log files from clients and write into database and also drop files to that server. After I released it to production, I met some unexpected issues.

1. Huge(99.9%) memory consumption issue

The logging server writes huge excessive IO to the disk and it instantly caused the os SYSTEM process to consume almost all physical memories. I searched online and analyzed it through the Sysinternals’ RAMMap. It turns out the OS’s system cache takes all the memories. You can find out details from following links:

http://www.microsoft.com/en-us/download/details.aspx?id=9258 Windows Dynamic Cache Service

Sysinternals http://technet.microsoft.com/en-us/sysinternals/bb897561

KB article http://support.microsoft.com/kb/976618/

[Korean] http://snoopybox.co.kr/1635

Lesson learned: Reduce the file IO as much as you can!

 

http://blogs.msdn.com/b/ntdebugging/archive/2009/02/06/microsoft-windows-dynamic-cache-service.aspx

http://blogs.msdn.com/b/ntdebugging/archive/2007/11/27/too-much-cache.aspx

 

2. MongoDB

  • Don’t forget to use the indexing
  • You can figure it out whether it uses the index or not by using ‘Explain’

 

3. Deleting files from disk will take longer than you expect

  • My initial design approach was complete wrong and failure. The server used to drop huge number of files into the physical disk, and just deleting the very small portion of files required actually even hours.

Lesson learned: Does your server collect a number of files? Store it into the database directly.

Thanks,

Heejune

Beginning the coroutine with Visual Studio 2015 Update 3 Part 2

As we discussed through Part I, co_wait keyword requires sort of “resumable thing”. So what if we want to use co_await with time delta?

For example, you can see the code which the co_await takes std::chrono::duration in order to suspend for the specified time from here: https://github.com/Microsoft/cppwinrt/blob/master/10.0.14393.0/Samples/JustCoroutines/Main.cpp#L31

The answer is the fact that vc++ compiler allows co_await to be overloaded. So, we can overload the co_await keyword and returns the “resumable thing” within the overloading implementation.

See the code here. https://github.com/Microsoft/cppwinrt/blob/master/10.0.14393.0/winrt/base.h#L8897

If you follow the resume_after(), then you’ll clearly see it actually returns the instance of resumable thing struct which contain implementations of await_ready, await_suspend, and await_resume.

https://github.com/Microsoft/cppwinrt/blob/master/10.0.14393.0/winrt/base.h#L8643

Let’s test with another test sample here. https://github.com/heejune/wrl-cppwinrt-sample/blob/async-support/SampleLib.Shared/DemoCore.cpp#L81

In this sample, I called the winrt’s co_await as the above sample illustrated.

And if I step into the co_await… we hit the overloaded co_await.

part_ii_1

Step in again, then finally it creates resume_after struct instance and calls the await_suspend.

part_ii_2

part_ii_3

It spawns timer thread with specified time, and finally resume_after::callback static callback is being called.

part_ii_3-4png

The callback will finally call resume().

part_ii_4

And the resume() transfers its context into where the co_await being called.

part_ii_5

And lastly, the coroutine function hits the co_return keyword after looping 5 times,

part_ii_6

And reached the end.

Next time, I’d like to talk about another co_await adapter ‘await_adapter’ which makes winrt IAsync.. types possible to be awaitable. Thanks.

Heejune

How to upload a file from slack using SlackClient?

Slack provides an official python client module which you can easily work with.

It’s simple and easy to use. For example, sending a text line to a specific channel is simple enough to call through just one API – api_call(chat.postMessage, channel=channel, text=msg, attachments=attachment, as_user=True)

You can also upload a file to a channel. However, when I was searching some info how to achieve that at the first time, the answer described in the SlackClient’s Github issue was actually wrong. So I tested and added a snippet. Please check the issue page.

Anyway, I also shared the snippet at the public gist. The ‘upload_file’ method takes a long text as the ‘content’ param and upload it as a file. You’ll also upload a file directory from disk or memory if you modify it slightly.

Thanks,

Heejune

Building WinRT component with WRL(non C++/CX) and cppwinrt

The cppwinrt project focused to consume Microsoft provided OS winrt components and it doesn’t support building a winrt component(yet) although they mentioned it will be supported later.

Even though it’s not technically supported yet, we still can create a winrt component with WRL(pure C++ and non C++/CX) and still can get some benefits from using cppwinrt within WRL component.

For example, I created a WinRT sample component with WRL and exports IAsyncAction async methods from it. Inside the method, I utilized the cppwinrt provided coroutine method which easily can produce IAsync operations. See the following:

https://github.com/heejune/wrl-cppwinrt-sample/blob/async-support/SampleLib.Shared/DemoCore.cpp

There’s a DemoCore::GetCppwinrtDataasync method defined as this:

It just (detached and) returns the object which also returned from GetAsyncOp private method. GetAsyncOp is a coroutine function which also creates an instance of winrt::Windows::Foundation::IAsyncOperation. And that’s the winrt implementation of IAsyncOperation.

You can download the source project and test it from https://github.com/heejune/wrl-cppwinrt-sample/tree/async-support

Thanks,

Heejune