properly wait for all threads to exit when
running the destructor

Signed-off-by: Luca Fulchir <luker@fenrirproject.org>