Threading improvements

[wraitii]

There are issues with our implementation: we use "naked" std::thread instead of cleverer solutions. This, notably, means we fail to detect exceptions happening in the threads (see ​https://code.wildfiregames.com/rP22649#45341 )

[25656] implements a task manager. Now not every task wich is run concurently does need to create it's own std::thread.

​Phab:D4956 does implement exception handling for tasks in the task manager.

These tasks use the task manager:

• map generator
• pathfinder
• texture converter

These tasks should be adapted to use the task manager:

• upnp

A task in the task manager should never 'wait' (for a condition_variable; for a mutex; for some time...) that would put the whole thread to sleep. Instead it would be better to run another task in the meantime and resume the original task later. ​Phab:D3877 or ​Phab:D4907 does extend the task manager to make that possible. These tasks rely on such a feature:

• net client (net session) ​Phab:D3877
• net server
• sound
• user reporter ​Phab:D4426

[Stan]

​https://akrzemi1.wordpress.com/2012/11/14/not-using-stdthread/

[wraitii]

Net client was threaded in A24, A25 will hopefully thread the pathfinder.

[wraitii]

In 25656:

Implement a global task manager using a pool of worker threads

Tasks are simple callables (e.g. lambdas), and can be pushed with 2 priority levels. Pushing a task returns a future.
Futures can be waited on, can return results, and can be cancelled deterministically. Futures can also not be waited on.

This gives 'hardware concurrency - 1' threads to maximize CPU usage in a work-stealing workflow.

Reviewed by: vladislavbelov

Refs #5874

Differential Revision: ​https://code.wildfiregames.com/D3848

[Stan]

Unlikely to get done for A26

[jprahman]

@wraitii @stanislas69 what sorts of remaining improvements do we see here? Have some interest in looking at this. Have a fair bit of experience with system programming/threading from my day job.

[wraitii]

Replying to jprahman:

@wraitii @stanislas69 what sorts of remaining improvements do we see here? Have some interest in looking at this. Have a fair bit of experience with system programming/threading from my day job.

Well, plenty of systems aren't threaded and _could_ be threaded.If you check the last commits of ​https://github.com/wraitii/0ad/commits/__THREADPOOL, you'll see some ways to thread some systems, such as sound.

More generally, threading is interesting in general as a speedup, though there are limitations on what you can do in 0 A.D.

[phosit]

Add link to differentials

[Freagarach]

Pushing back.

[phosit]

Rewrite the ticket. Task manager wasn't mentioned and i described the problem some tasks have wich are not jet converted. I also added a link two new diffs.

[phosit]

user reporter aslo relyes on a repeating task.

[phosit]

The wrong diff was linked

[phosit]

Add missing upnp.

[phosit]

In 27944:

Put the CMapGeneratorWorker completely inside the task

The return-slot provided by the Future is used for synchronisation.

Refs: #5874

Comments By: @Stan, @vladislavbelov, @wraitii

Differential Revision: ​https://code.wildfiregames.com/D5001

[phosit]

I forgot to refference this ticket in the commit [27984].