Summary: It's no secret the job market recovery has been sluggish, but the culprits might surprise you.